From kim.barrett at oracle.com Thu Jan 1 07:42:46 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 1 Jan 2015 02:42:46 -0500 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se>, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> Message-ID: On Dec 31, 2014, at 6:12 AM, Erik ?sterlund wrote: > >> The rest of these comments only matter if, contrary to my suggestion >> above, some additional infrastructure is thought necessary. >> > > This is what I originally thought too and hence the current solution. But voices said that using macros for this is, I quote, "that bad". > >> [?] >> >> This scales with the number of AtomicPlatform definitions, rather than >> the number of specialized functions as happens with the existing code. > > So your proposal is to still use ugly macros but move it to the class level instead, even though at the moment there is really only one member function that needs it. That feels a bit meh to be honest. IMO if we want to do this right, why go half the way and still rely on ugly macros? Please lay off with the perjorative "ugly macros". The preprocessor is part of C++, and isn't going away. It's a tool that can be used, or abused, and I think this is a fine use. To me, "ugly" is adding 800+ lines of code / tests / comments to eliminate a dozen lines of flag macro definitions and associated #ifdef's. If that additional infrastructure were amortized across more use-cases then it might be more appealing. I might even have some additional use-cases for something along that line, though not in its present form. (More on that below.) >> I think the name "IsBaseOfAndDerived" would be better as >> "IsBaseAndDerived?. [?] >> >> The current implementation of IsBaseOfAndDerived (and therefore >> IsBaseOf) may (silently) provide incorrect answers when called with >> incomplete types. [?] >> >> The actual usage in this proposed change-set even relies on this >> flawed behavior of these metafunctions, since AtomicPlatform isn't >> always defined. [?] >> > > So what you are saying here is that I should rename IsBaseOfAndDerived to IsBaseAndDerived to more closely resemble other external libraries we do not use. But if I do that my behaviour is "incorrect" because it is not identical to that of the external library. And therefore my change would not work if it was made "correctly?. The proposed IsBaseOfAndDerived and IsBaseOf are being described and placed as general utilities. Their names are strongly reminiscent of existing facilities in other libraries, including the (C++11) standard library. Conforming to these external standards will avoid surprises, especially when the proposed facilities are even described as being the same (except for a noted corner case that is both hard to deal with in a portable manner and not actually very interesting, so I'm not complaining about that) in the review request email. Additionally, I think quietly providing a potentially incorrect answer in the case of incomplete types renders the offered code quite broken for use as a general utility. I think the requirement for complete types by the well-known / standard facilities is well justified. > On the contrary this is a well tested feature on all our supported compilers and I did not intend to engineer it to be identical to some external library if it only causes trouble rather than helping. This is our own class and it can do what we want it to do and be called what we want it to be called, and I think that is fine as long as it is well tested, which it is. At least this is my opinion. Anyone else agree? Problem is, I think it's well tested code to solve the wrong problem. I suggest that what we're really looking for is not a base/derived relationship between a pair of classes, but that we actually want to determine whether a platform-specific class exists. As a result, I think there is more infrastructure in this change set than is actually needed. The present implementation of SelectBaseClass uses IsBaseOf, but doesn't need a lot of the functionality of that metafunction. There is no actual need for a Base/Derived relationship between the (possibly not defined) platform-specific class and the default platform-generic class, so the use of IsBaseOf is an unnecessary restriction that only serendipitously tests for defined or not, due to the previously discussed defect in this particular implementation. I also think the name SelectBaseClass is perhaps overly general. I might be able to use a similar approach in some code I've been working on as a background task. And while writing this reply I've thought of another place that might benefit from something along those lines. Thinking about the code under review in that context, I think some changes would make these other possible use-cases easier and clearer. I believe what is actually needed is a metatfunction something like: /** * Metafunction whose nested type member is Default if Specific is * incomplete, otherwise Specific. * * Specific and Default must both be non-cv qualified class types, and * must not be the same type. * * If Specific is incomplete at the point where this metafunction is * invoked, it must never be defined elsewhere in the program. */ template struct SelectPlatformBase; [Note: I'm not wedded to that name.] Now, it turns out to be hard / impossible to write a generic is_complete metafunction, due to ODR. Whether a type is complete can vary between translation units, and even within a translation unit. Having is_complete::value have different values depending on where it is instantiated is an ODR violation. (One can make progress by using anonymous namespaces and macro wrappers with __LINE__ concatenation, but there is still the fundamental question of whether is_complete really even makes semantic sense.) However, we don't need a fully general is_complete utility. We have a much more restricted use-case, where we want to select an optionally defined platform-specific class if it exists, and otherwise fall back to a more generic class. And we're not using this in arbitrary contexts, but only to select a platform-specialized implementation if such exists. Here's a lightly tested implementation of SelectPlatformBase: // --- SelectPlatformBase template struct EnableIf { typedef T type; }; template struct EnableIf { }; template struct If { typedef Then type; }; template struct If { typedef Else type; }; // Metafunction whose nested value member is true if T is defined // (complete), and false if T is incomplete. T must be a non-cv // qualified class type. If T is incomplete at the point where this // metafunction is invoked, it must never be defined elsewhere in the // program. template class IsClassDefined { typedef char yes[1]; typedef char no[2]; template static typename EnableIf::type & check(U*); static no& check(...); public: static const bool value = sizeof(check((T*)0)) == sizeof(yes); }; template struct SelectPlatformBase { typedef typename If< IsClassDefined::value, Specific, Default>::type type; }; // --- end SelectPlatformBase That's ~30 lines of code (+ tests and comments TBD) to do precisely what we need, replacing ~100 lines (+ tests and comments) that have various issues. (If and EnableIf should probably be hoisted out as separate utilities.) We don't actually need SelectPlatformBase, and could instead just directly use the If and IsClassDefined metafunctions, but giving this functionality a name might be clearer. However, while I think there are other use cases for this specific utility, I still think the proposed change set as a whole is adding a lot of code just to avoid a a few lines of macro usage. That seems like a poor tradeoff to me. From kim.barrett at oracle.com Thu Jan 1 07:43:48 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 1 Jan 2015 02:43:48 -0500 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> Message-ID: On Dec 17, 2014, at 11:38 AM, Erik ?sterlund wrote: > > RFE: https://bugs.openjdk.java.net/browse/JDK-8067790 > Webrev: http://cr.openjdk.java.net/~jwilhelm/8067790/webrev.00/ A couple more comments: ------------------------------------------------------------------------------ Note that once the fix for 8067306 is done, the STATIC_ASSERT tests can be moved to namespace/global scope and no longer need to be wrapped up in a dummy "test" function. ------------------------------------------------------------------------------ src/cpu/x86/vm/stubGenerator_x86_64.cpp Fixing the names in the "Support for ..." comments is fine, but they should really be fixed, not partially updated. Specifically, "atomic::" => "Atomic::" should be applied throughout. From erik.osterlund at lnu.se Thu Jan 1 14:07:22 2015 From: erik.osterlund at lnu.se (=?Windows-1252?Q?Erik_=D6sterlund?=) Date: Thu, 1 Jan 2015 14:07:22 +0000 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <,<8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> Message-ID: <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> Hi Kim, > On 01 Jan 2015, at 07:42, Kim Barrett wrote: > > On Dec 31, 2014, at 6:12 AM, Erik ?sterlund wrote: >> >>> The rest of these comments only matter if, contrary to my suggestion >>> above, some additional infrastructure is thought necessary. >>> >> >> This is what I originally thought too and hence the current solution. But voices said that using macros for this is, I quote, "that bad". >> >>> [?] >>> >>> This scales with the number of AtomicPlatform definitions, rather than >>> the number of specialized functions as happens with the existing code. >> >> So your proposal is to still use ugly macros but move it to the class level instead, even though at the moment there is really only one member function that needs it. That feels a bit meh to be honest. IMO if we want to do this right, why go half the way and still rely on ugly macros? > > Please lay off with the perjorative "ugly macros". The preprocessor > is part of C++, and isn't going away. It's a tool that can be used, > or abused, and I think this is a fine use. To me, "ugly" is adding > 800+ lines of code / tests / comments to eliminate a dozen lines of > flag macro definitions and associated #ifdef?s. Well people are entitled to have different opinions of course. As I already mentioned, I did this because other reviewers (as well as me) found it ugly to use macros for conditionally specializing, which is the whole motivation of doing this, and was well understood from the beginning. But I?m fine with leaving it as macros if this is in general preferred and opinions have suddenly changed in this matter. Personally I don?t see how the number of rows of code define uglyness. Reusability, testing, documentation and well definedness in contracts/behaviours all lead to more lines of code, but I don?t think that equals uglyness. So while I disagree more code is uglier because it?s more, it?s all up to opinion and it?s quite pointless to discuss; you are entitled to think that is ugly. > If that additional infrastructure were amortized across more use-cases > then it might be more appealing. I might even have some additional > use-cases for something along that line, though not in its present > form. (More on that below.) Yeah I specifically had in mind the wish to make a split between commercial extensions to open source code actually. This pattern allows easy optional class inheritance injection as a general design pattern to achieve this. And for my own purposes the ability to define asymmetric dekker synchronization to elide storeload fences when the platform (e.g. TSO) supports it but otherwise revert to a generalized implementation (using storeload fences). This is why I think a clean reusable strategy is almost always better because it is widely applicable, even if there would not be obvious other examples where it is useful. > >>> I think the name "IsBaseOfAndDerived" would be better as >>> "IsBaseAndDerived?. [?] >>> >>> The current implementation of IsBaseOfAndDerived (and therefore >>> IsBaseOf) may (silently) provide incorrect answers when called with >>> incomplete types. [?] >>> >>> The actual usage in this proposed change-set even relies on this >>> flawed behavior of these metafunctions, since AtomicPlatform isn't >>> always defined. [?] >>> >> >> So what you are saying here is that I should rename IsBaseOfAndDerived to IsBaseAndDerived to more closely resemble other external libraries we do not use. But if I do that my behaviour is "incorrect" because it is not identical to that of the external library. And therefore my change would not work if it was made "correctly?. > > The proposed IsBaseOfAndDerived and IsBaseOf are being described and > placed as general utilities. Their names are strongly reminiscent of > existing facilities in other libraries, including the (C++11) standard > library. Conforming to these external standards will avoid surprises, > especially when the proposed facilities are even described as being > the same (except for a noted corner case that is both hard to deal > with in a portable manner and not actually very interesting, so I'm > not complaining about that) in the review request email. > > Additionally, I think quietly providing a potentially incorrect answer > in the case of incomplete types renders the offered code quite broken > for use as a general utility. I think the requirement for complete > types by the well-known / standard facilities is well justified. > While I disagree it is universally incorrect (as long as it is well defined and tested) to not have the exact same requirement on completeness on the parameters as C++11/boost, who also already have different behaviour in this exact regard as pointed out by yourself, I can try to fix it if it bugs you, but it will make the infrastructure even larger which seems to be a concern. >> On the contrary this is a well tested feature on all our supported compilers and I did not intend to engineer it to be identical to some external library if it only causes trouble rather than helping. This is our own class and it can do what we want it to do and be called what we want it to be called, and I think that is fine as long as it is well tested, which it is. At least this is my opinion. Anyone else agree? > > Problem is, I think it's well tested code to solve the wrong problem. > > I suggest that what we're really looking for is not a base/derived > relationship between a pair of classes, but that we actually want to > determine whether a platform-specific class exists. > > As a result, I think there is more infrastructure in this change set > than is actually needed. The present implementation of > SelectBaseClass uses IsBaseOf, but doesn't need a lot of the > functionality of that metafunction. There is no actual need for a > Base/Derived relationship between the (possibly not defined) > platform-specific class and the default platform-generic class, so the > use of IsBaseOf is an unnecessary restriction that only > serendipitously tests for defined or not, due to the previously > discussed defect in this particular implementation. I also think the > name SelectBaseClass is perhaps overly general. The main idea of SelectBaseClass is to inject an optional class into the class hierarchy for specialization of generalized behaviour. It is absolutely critical (and intended) that this class is constrained to be a derived class of the fall-back class - it is not an artifact nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, A must be able to rely on the superclass being at least C-like so that the behaviour of the superclass can be used reliably and transparently regardless of the existence of the optional B. If this contract would not hold, I think it could lead to some really weird and unreliable code which should not be encouraged (at least not according to the Liskov substitution principle etc). So what I need in the contract is a check if the optional class is defined and if it fits in the inheritance hierarchy: they are both important IMO. Yes it could have been less code. From the beginning I had a single trait class that checked for both the inheritance and whether the class is defined or not but you said you wanted the behaviour to more closely match that of third party libraries. This is why there is now a lot more code to more closely (but still not exactly) match that behaviour. > I might be able to use a similar approach in some code I've been > working on as a background task. And while writing this reply I've > thought of another place that might benefit from something along those > lines. Thinking about the code under review in that context, I think > some changes would make these other possible use-cases easier and > clearer. > > I believe what is actually needed is a metatfunction something like: > > /** > * Metafunction whose nested type member is Default if Specific is > * incomplete, otherwise Specific. > * > * Specific and Default must both be non-cv qualified class types, and > * must not be the same type. > * > * If Specific is incomplete at the point where this metafunction is > * invoked, it must never be defined elsewhere in the program. > */ > template > struct SelectPlatformBase; > > [Note: I'm not wedded to that name.] > > Now, it turns out to be hard / impossible to write a generic > is_complete metafunction, due to ODR. Whether a type is complete > can vary between translation units, and even within a translation > unit. Having is_complete::value have different values depending on > where it is instantiated is an ODR violation. (One can make progress > by using anonymous namespaces and macro wrappers with __LINE__ > concatenation, but there is still the fundamental question of whether > is_complete really even makes semantic sense.) Yes and that makes me wonder if we care enough about the completeness semantics being the same as third party libraries (who are already inconsistent) if we do not need them to be, and on the contrary benefit from them not to be. > However, we don't need a fully general is_complete utility. We have a > much more restricted use-case, where we want to select an optionally > defined platform-specific class if it exists, and otherwise fall back > to a more generic class. And we're not using this in arbitrary > contexts, but only to select a platform-specialized implementation if > such exists. > > Here's a lightly tested implementation of SelectPlatformBase: > > // --- SelectPlatformBase > > template > struct EnableIf { typedef T type; }; > > template > struct EnableIf { }; > > template > struct If { typedef Then type; }; > > template > struct If { typedef Else type; }; > > // Metafunction whose nested value member is true if T is defined > // (complete), and false if T is incomplete. T must be a non-cv > // qualified class type. If T is incomplete at the point where this > // metafunction is invoked, it must never be defined elsewhere in the > // program. > template > class IsClassDefined { > typedef char yes[1]; > typedef char no[2]; > > template > static typename EnableIf::type & check(U*); > static no& check(...); > > public: > static const bool value = sizeof(check((T*)0)) == sizeof(yes); > }; > > template > struct SelectPlatformBase { > typedef typename If< > IsClassDefined::value, Specific, Default>::type type; > }; > > // --- end SelectPlatformBase > > That's ~30 lines of code (+ tests and comments TBD) to do precisely > what we need, replacing ~100 lines (+ tests and comments) that have > various issues. (If and EnableIf should probably be hoisted out as > separate utilities.) We don't actually need SelectPlatformBase, and > could instead just directly use the If and IsClassDefined > metafunctions, but giving this functionality a name might be clearer. As stated, I think it?s dangerous to allow conditional inheritance where there is no inheritance constraint on the optional class. Therefore I do not think this is better, although I agree it is smaller. I agree that our use case here is smaller. If the amount of code is the problem and less code (and hence fewer reusable components) is preferred, then the original implementation with a simple IsBaseOfAndDerived does the trick (without checking types/cv qualifiers) and could be done in just a few lines, and this is what I originally did before you told me to expand it to more closely resemble external libraries: template class IsBaseOfAndDerived { typedef char yes[1]; typedef char no[2]; template static yes &check(Derived*, T); static no &check(Base*, int); template struct IsBaseOfHost { operator B*() const; operator D*(); }; public: static const bool value = sizeof(check(IsBaseOfHost(), int())) == sizeof(yes); }; It checks the inheritance and existence of the Derived class which is exactly the constraints I need. If you do not want to expose it publicly and remove the reusability because it does not have identical semantics as third party libraries regarding the requirements of the Derived type to be complete, it could be an internal class of SelectBaseClass. I?m not going to value the reusability vs LoC, I leave that decision to you. > However, while I think there are other use cases for this specific > utility, I still think the proposed change set as a whole is adding a > lot of code just to avoid a a few lines of macro usage. That seems > like a poor tradeoff to me. Okay. In the end, we have to make a choice - what is more important, that the traits resemble some external library or code size. Because last time I proposed a small change and you wanted it to be closer to C++11 behaviour and I made it closer to that at the expense of more code. Now you are saying I have to make it even tighter to C++11 (or boost?), but also don?t want more lines of code which I believe is impossible given that I want to constrain both the class hierarchy to be reliable and check the existence of the optional class. This brings me to a few architectural questions which I may have (strong) opinions about but are not up to me to decide. 1. Do we want to replace the macros with template metaprogramming? 2 reviewers (and me) liked the metaprogramming approach and it was implemented because they (like me) did not want the macros. But you have a different opinion. I can?t judge who is right and wrong here. 2. Do we want to minimize the code size at the cost of reusability by making contracts of dependent components weaker and putting them as private classes in the SelectBaseClass? 3. If we do want template metaprogramming and want reusable components, the question is, is it important that the specification of completeness (which in general is almost impossible to implement correctly in a portable way as you pointed out) of the template arguments in IsBaseOf is identical as C++11/boost even though no code needs it to be and on the contrary some code benefits from it not to be? In that case which one of them do we, because they are already different anyway? There is a tradeoff between code size and complexity vs copying either one of the external libraries regarding requirements of completeness of template parameters. Thanks, /Erik From kim.barrett at oracle.com Thu Jan 1 18:32:37 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 1 Jan 2015 13:32:37 -0500 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se>, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> Message-ID: <1755B922-8C7E-4F13-8997-52C785C769F1@oracle.com> On Jan 1, 2015, at 2:42 AM, Kim Barrett wrote: > > // Metafunction whose nested value member is true if T is defined > // (complete), and false if T is incomplete. T must be a non-cv > // qualified class type. If T is incomplete at the point where this > // metafunction is invoked, it must never be defined elsewhere in the > // program. > template > class IsClassDefined { > typedef char yes[1]; > typedef char no[2]; > > template > static typename EnableIf::type & check(U*); Change "sizeof(U) == sizeof(U)? => "sizeof(U) != 0?. > static no& check(...); > > public: > static const bool value = sizeof(check((T*)0)) == sizeof(yes); > }; > Further (web) research indicates that perhaps some compilers implement sizeof of an incomplete type as returning 0, rather than as an error. Strictly speaking, applying sizeof to an incomplete type results in undefined behavior, but this technique is in quite wide-spread use. From aph at redhat.com Fri Jan 2 14:19:33 2015 From: aph at redhat.com (Andrew Haley) Date: Fri, 02 Jan 2015 14:19:33 +0000 Subject: 8068053: AARCH64: C1 and C2 compilers Message-ID: <54A6A8F5.4040907@redhat.com> http://cr.openjdk.java.net/~aph/aarch64-8068053-2/ Thanks, Andrew. From aph at redhat.com Fri Jan 2 14:20:35 2015 From: aph at redhat.com (Andrew Haley) Date: Fri, 02 Jan 2015 14:20:35 +0000 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime Message-ID: <54A6A933.8000204@redhat.com> http://cr.openjdk.java.net/~aph/aarch64-8068054-1/ Thanks, Andrew. From aph at redhat.com Fri Jan 2 14:21:43 2015 From: aph at redhat.com (Andrew Haley) Date: Fri, 02 Jan 2015 14:21:43 +0000 Subject: 8068055: AARCH64: os_cpu/ Message-ID: <54A6A977.4080405@redhat.com> http://cr.openjdk.java.net/~aph/aarch64-8068055-1/ Thanks, Andrew. From coleen.phillimore at oracle.com Fri Jan 2 18:30:50 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 02 Jan 2015 13:30:50 -0500 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54A6A933.8000204@redhat.com> References: <54A6A933.8000204@redhat.com> Message-ID: <54A6E3DA.40504@oracle.com> Hi Andrew, I had a short look at this code and spot-checked some functions in the interpreter. In all these files, were any of the functions added in a different place than in the x86 ports or is this consistent with the other ports? +// Method entry for java.lang.ref.Reference.get. +address InterpreterGenerator::generate_Reference_get_entry(void) { + return NULL; +} Why isn't this implemented? I think this needs to have code to support the G1 garbage collector. I don't know this architecture, but I think you can use my code review nonetheless. Thanks, Coleen On 1/2/15, 9:20 AM, Andrew Haley wrote: > http://cr.openjdk.java.net/~aph/aarch64-8068054-1/ > > Thanks, > Andrew. > From aph at redhat.com Fri Jan 2 18:45:28 2015 From: aph at redhat.com (Andrew Haley) Date: Fri, 02 Jan 2015 18:45:28 +0000 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54A6E3DA.40504@oracle.com> References: <54A6A933.8000204@redhat.com> <54A6E3DA.40504@oracle.com> Message-ID: <54A6E748.3070009@redhat.com> Hi, On 01/02/2015 06:30 PM, Coleen Phillimore wrote: > > Hi Andrew, I had a short look at this code and spot-checked some > functions in the interpreter. In all these files, were any of the > functions added in a different place than in the x86 ports or is this > consistent with the other ports? Gosh. I had no idea that consistency with any other port would be any kind of an issue; I never even considered the question, but it's similar to x86. > +// Method entry for java.lang.ref.Reference.get. > +address InterpreterGenerator::generate_Reference_get_entry(void) { > + return NULL; > +} > > Why isn't this implemented? I think this needs to have code to support > the G1 garbage collector. I don't know this architecture, but I think > you can use my code review nonetheless. Okay. I think G1 works, but I don't know about this issue. I'll have a look. Thanks, Andrew. From coleen.phillimore at oracle.com Fri Jan 2 19:01:10 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 02 Jan 2015 14:01:10 -0500 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54A6E748.3070009@redhat.com> References: <54A6A933.8000204@redhat.com> <54A6E3DA.40504@oracle.com> <54A6E748.3070009@redhat.com> Message-ID: <54A6EAF6.106@oracle.com> On 1/2/15, 1:45 PM, Andrew Haley wrote: > Hi, > > On 01/02/2015 06:30 PM, Coleen Phillimore wrote: >> Hi Andrew, I had a short look at this code and spot-checked some >> functions in the interpreter. In all these files, were any of the >> functions added in a different place than in the x86 ports or is this >> consistent with the other ports? > Gosh. I had no idea that consistency with any other port would be any > kind of an issue; I never even considered the question, but it's > similar to x86. The reason I asked is that if I have to add something to one platform, it would be nice to find it in the same place for another, that's all. > >> +// Method entry for java.lang.ref.Reference.get. >> +address InterpreterGenerator::generate_Reference_get_entry(void) { >> + return NULL; >> +} >> >> Why isn't this implemented? I think this needs to have code to support >> the G1 garbage collector. I don't know this architecture, but I think >> you can use my code review nonetheless. > Okay. I think G1 works, but I don't know about this issue. I'll have > a look. You may not need this code; it may be an optimization. The hg history and bug number it points to would say. Coleen > Thanks, > Andrew. From serguei.spitsyn at oracle.com Fri Jan 2 22:23:11 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 02 Jan 2015 14:23:11 -0800 Subject: 2-nd round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54A44562.3090207@oracle.com> References: <54A43584.6050707@oracle.com> <54A44300.4010801@oracle.com> <54A44562.3090207@oracle.com> Message-ID: <54A71A4F.4050308@oracle.com> Coleen, Please, find new open webrev here: http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.2/ I've added a separate helper class in the VM_RedefineClasses and new method in the klassItable klass. TBD: still need to run the SVC tests listed in the first email. I'm on vacation from today till Jan 9 but will try to reply on emails in non-active mode. Thanks, Serguei On 12/31/14 10:50 AM, serguei.spitsyn at oracle.com wrote: > Colleen, > > Thank you for a quick review. > You are right, this is a good catch! > I need to rework my fix as it is incorrect in general. > > > On 12/31/14 10:40 AM, Coleen Phillimore wrote: >> >> The adjustment should be in klassItable::adjust_method_entries() not >> the checking function. I don't think >> check_no_old_or_obsolete_entries is called in product mode. >> >> I still don't see how the first klassItable::adjust_method_entries() >> missed this function. > > The Unsafe::throw_illegal_access is added to other itables, not to the > one that belongs to the class Unsafe. > > Thanks, > Serguei > > >> >> Coleen >> >> On 12/31/14, 12:42 PM, serguei.spitsyn at oracle.com wrote: >>> Please, review the fix for: >>> https://bugs.openjdk.java.net/browse/JDK-8068162 >>> >>> >>> Open webrev: >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8068162-JVMTI-old.1/ >>> >>> >>> >>> Summary: >>> >>> The sun.misc.Unsafe:throwIllegalAccessError() method is used in >>> place of a default >>> interface method in the itable if a default method was not >>> defined in the interface. >>> >>> This approach creates a problem when the class sun.misc.Unsafe is >>> retransformed. >>> The Method* pointer to the old (redefined) method in the itable >>> triggers an assert >>> (see the hs_err log in the bug report). >>> >>> The fix is to replace the old method in the itable with the >>> latest method version. >>> >>> >>> Testing: >>> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >>> >>> >>> Thanks, >>> Serguei >>> >> > From coleen.phillimore at oracle.com Fri Jan 2 23:04:30 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 02 Jan 2015 18:04:30 -0500 Subject: 2-nd round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54A71A4F.4050308@oracle.com> References: <54A43584.6050707@oracle.com> <54A44300.4010801@oracle.com> <54A44562.3090207@oracle.com> <54A71A4F.4050308@oracle.com> Message-ID: <54A723FE.2080501@oracle.com> Hi Serguei, I don't see why you don't add a || the_class_oop == unsafe to the if statement I pointed out in the bug. Then you don't have to walk the CLDG classes again. Have a nice vacation! Thanks, Coleen On 1/2/15, 5:23 PM, serguei.spitsyn at oracle.com wrote: > Coleen, > > Please, find new open webrev here: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.2/ > > I've added a separate helper class in the VM_RedefineClasses and new > method in the klassItable klass. > TBD: still need to run the SVC tests listed in the first email. > > I'm on vacation from today till Jan 9 but will try to reply on emails > in non-active mode. > > Thanks, > Serguei > > > On 12/31/14 10:50 AM, serguei.spitsyn at oracle.com wrote: >> Colleen, >> >> Thank you for a quick review. >> You are right, this is a good catch! >> I need to rework my fix as it is incorrect in general. >> >> >> On 12/31/14 10:40 AM, Coleen Phillimore wrote: >>> >>> The adjustment should be in klassItable::adjust_method_entries() not >>> the checking function. I don't think >>> check_no_old_or_obsolete_entries is called in product mode. >>> >>> I still don't see how the first klassItable::adjust_method_entries() >>> missed this function. >> >> The Unsafe::throw_illegal_access is added to other itables, not to >> the one that belongs to the class Unsafe. >> >> Thanks, >> Serguei >> >> >>> >>> Coleen >>> >>> On 12/31/14, 12:42 PM, serguei.spitsyn at oracle.com wrote: >>>> Please, review the fix for: >>>> https://bugs.openjdk.java.net/browse/JDK-8068162 >>>> >>>> >>>> Open webrev: >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8068162-JVMTI-old.1/ >>>> >>>> >>>> >>>> Summary: >>>> >>>> The sun.misc.Unsafe:throwIllegalAccessError() method is used in >>>> place of a default >>>> interface method in the itable if a default method was not >>>> defined in the interface. >>>> >>>> This approach creates a problem when the class sun.misc.Unsafe >>>> is retransformed. >>>> The Method* pointer to the old (redefined) method in the itable >>>> triggers an assert >>>> (see the hs_err log in the bug report). >>>> >>>> The fix is to replace the old method in the itable with the >>>> latest method version. >>>> >>>> >>>> Testing: >>>> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >>> >> > From serguei.spitsyn at oracle.com Sat Jan 3 00:13:19 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 02 Jan 2015 16:13:19 -0800 Subject: 2-nd round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54A723FE.2080501@oracle.com> References: <54A43584.6050707@oracle.com> <54A44300.4010801@oracle.com> <54A44562.3090207@oracle.com> <54A71A4F.4050308@oracle.com> <54A723FE.2080501@oracle.com> Message-ID: <54A7341F.50105@oracle.com> Hi Coleen, Thank you for the quick response! On 1/2/15 3:04 PM, Coleen Phillimore wrote: > > Hi Serguei, > > I don't see why you don't add a || the_class_oop == unsafe to the if > statement I pointed out in the bug. Then you don't have to walk the > CLDG classes again. Do you mean something like in the webrev below ? http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.3/ I like this! > > Have a nice vacation! Thanks! Serguei > > Thanks, > Coleen > > On 1/2/15, 5:23 PM, serguei.spitsyn at oracle.com wrote: >> Coleen, >> >> Please, find new open webrev here: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.2/ >> >> I've added a separate helper class in the VM_RedefineClasses and new >> method in the klassItable klass. >> TBD: still need to run the SVC tests listed in the first email. >> >> I'm on vacation from today till Jan 9 but will try to reply on emails >> in non-active mode. >> >> Thanks, >> Serguei >> >> >> On 12/31/14 10:50 AM, serguei.spitsyn at oracle.com wrote: >>> Colleen, >>> >>> Thank you for a quick review. >>> You are right, this is a good catch! >>> I need to rework my fix as it is incorrect in general. >>> >>> >>> On 12/31/14 10:40 AM, Coleen Phillimore wrote: >>>> >>>> The adjustment should be in klassItable::adjust_method_entries() >>>> not the checking function. I don't think >>>> check_no_old_or_obsolete_entries is called in product mode. >>>> >>>> I still don't see how the first >>>> klassItable::adjust_method_entries() missed this function. >>> >>> The Unsafe::throw_illegal_access is added to other itables, not to >>> the one that belongs to the class Unsafe. >>> >>> Thanks, >>> Serguei >>> >>> >>>> >>>> Coleen >>>> >>>> On 12/31/14, 12:42 PM, serguei.spitsyn at oracle.com wrote: >>>>> Please, review the fix for: >>>>> https://bugs.openjdk.java.net/browse/JDK-8068162 >>>>> >>>>> >>>>> Open webrev: >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8068162-JVMTI-old.1/ >>>>> >>>>> >>>>> >>>>> Summary: >>>>> >>>>> The sun.misc.Unsafe:throwIllegalAccessError() method is used in >>>>> place of a default >>>>> interface method in the itable if a default method was not >>>>> defined in the interface. >>>>> >>>>> This approach creates a problem when the class sun.misc.Unsafe >>>>> is retransformed. >>>>> The Method* pointer to the old (redefined) method in the itable >>>>> triggers an assert >>>>> (see the hs_err log in the bug report). >>>>> >>>>> The fix is to replace the old method in the itable with the >>>>> latest method version. >>>>> >>>>> >>>>> Testing: >>>>> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>> >>> >> > From edward.nevill at linaro.org Sat Jan 3 20:16:53 2015 From: edward.nevill at linaro.org (Edward Nevill) Date: Sat, 03 Jan 2015 20:16:53 +0000 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54A6EAF6.106@oracle.com> References: <54A6A933.8000204@redhat.com> <54A6E3DA.40504@oracle.com> <54A6E748.3070009@redhat.com> <54A6EAF6.106@oracle.com> Message-ID: <54A84E35.2000403@linaro.org> Hi, On 02/01/15 19:01, Coleen Phillimore wrote: > > On 1/2/15, 1:45 PM, Andrew Haley wrote: >> >> On 01/02/2015 06:30 PM, Coleen Phillimore wrote: >>> +// Method entry for java.lang.ref.Reference.get. >>> +address InterpreterGenerator::generate_Reference_get_entry(void) { >>> + return NULL; >>> +} >>> >>> Why isn't this implemented? I think this needs to have code to support >>> the G1 garbage collector. I don't know this architecture, but I think >>> you can use my code review nonetheless. >> Okay. I think G1 works, but I don't know about this issue. I'll have >> a look. > > You may not need this code; it may be an optimization. The hg history > and bug number it points to would say. I think it is necessary. See http://mail.openjdk.java.net/pipermail/aarch64-port-dev/2014-October/001503.html All the best, Ed. From david.holmes at oracle.com Sun Jan 4 23:56:57 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 05 Jan 2015 09:56:57 +1000 Subject: 8068055: AARCH64: os_cpu/ In-Reply-To: <54A6A977.4080405@redhat.com> References: <54A6A977.4080405@redhat.com> Message-ID: <54A9D349.1000504@oracle.com> Hi Andrew, On 3/01/2015 12:21 AM, Andrew Haley wrote: > http://cr.openjdk.java.net/~aph/aarch64-8068055-1/ Just curious - where are things like __sync_lock_test_and_set and __atomic_thread_fence defined? Are these external library routines that are part of your toolset? Thanks, David > Thanks, > Andrew. > From david.holmes at oracle.com Mon Jan 5 02:21:48 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 05 Jan 2015 12:21:48 +1000 Subject: RFR(S) 8067331: Zero: Atomic::xchg and Atomic::xchg_ptr need full memory barrier In-Reply-To: <1418656152.3297.44.camel@localhost.localdomain> References: <1418656152.3297.44.camel@localhost.localdomain> Message-ID: <54A9F53C.5040507@oracle.com> Hi Severin, Sorry this slipped through the cracks before the holidays. On 16/12/2014 1:09 AM, Severin Gehwolf wrote: > Hi, > > Could someone please review and sponsor the following change? > > bug: https://bugs.openjdk.java.net/browse/JDK-8067331 > webrev: > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8067331/webrev.02/ > > The issue is that all atomic operations are expected to be full memory > barriers, but two implementations of the Zero variant JVM - Atomic::xchg > and Atomic::xchg_ptr - use GCC's built-in __sync_lock_test_and_set which > is an acquire barrier only. The fix adds full memory barriers manually > for those. > > Thoughts? Looks completely consistent with what I just saw in the aarch64 port code. I can sponsor this but we still need a second review. Thanks, David > Cheers, > Severin > From david.holmes at oracle.com Mon Jan 5 04:34:03 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 05 Jan 2015 14:34:03 +1000 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> Message-ID: <54AA143B.2080900@oracle.com> Sorry for the top-post but just wanted to reset the context a little here. Originally [1] I made the comment: "It's fine to have generic shared approaches but there really needs to be a way to allow platform specific "overrides"." For the actual original RFR I said [2]: "Can we pause and give some more thought to a clean mechanism for allowing a shared implementation if desired with the ability to override if desired. I really do not like to see CPU specific ifdefs being added to shared code. (And I would also not like to see all platforms being forced to reimplement this natively). I'm not saying we will find a simple solution, but it would be nice if we could get a few folk to think about it before proceeding with the ifdefs :)" Erik then proposed three alternative approaches [3] and the simple variant was chosen [4] and presented for review. However Paul Hohensee also made comments about an inheritance-based approach and Erik floated the first template-based variant [5] and there was some discussion with Paul and Kim. But we then ended up with the simple solution, leaving an inheritance-based solution for future work [6] (which is what is what is being discussed now). This template-based meta-programming is not something I have any familiarity with. My initial thought is this seems overkill for the situation we have with the Atomic class - but as I said I'm ignorant of the technique being used here. David ----- [1] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015292.html [2] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015303.html [3] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015362.html [4] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015401.html [5] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-October/015586.html [6] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-November/015889.html On 2/01/2015 12:07 AM, Erik ?sterlund wrote: > Hi Kim, > >> On 01 Jan 2015, at 07:42, Kim Barrett wrote: >> >> On Dec 31, 2014, at 6:12 AM, Erik ?sterlund wrote: >>> >>>> The rest of these comments only matter if, contrary to my suggestion >>>> above, some additional infrastructure is thought necessary. >>>> >>> >>> This is what I originally thought too and hence the current solution. But voices said that using macros for this is, I quote, "that bad". >>> >>>> [?] >>>> >>>> This scales with the number of AtomicPlatform definitions, rather than >>>> the number of specialized functions as happens with the existing code. >>> >>> So your proposal is to still use ugly macros but move it to the class level instead, even though at the moment there is really only one member function that needs it. That feels a bit meh to be honest. IMO if we want to do this right, why go half the way and still rely on ugly macros? >> >> Please lay off with the perjorative "ugly macros". The preprocessor >> is part of C++, and isn't going away. It's a tool that can be used, >> or abused, and I think this is a fine use. To me, "ugly" is adding >> 800+ lines of code / tests / comments to eliminate a dozen lines of >> flag macro definitions and associated #ifdef?s. > > Well people are entitled to have different opinions of course. As I already mentioned, I did this because other reviewers (as well as me) found it ugly to use macros for conditionally specializing, which is the whole motivation of doing this, and was well understood from the beginning. But I?m fine with leaving it as macros if this is in general preferred and opinions have suddenly changed in this matter. > > Personally I don?t see how the number of rows of code define uglyness. Reusability, testing, documentation and well definedness in contracts/behaviours all lead to more lines of code, but I don?t think that equals uglyness. So while I disagree more code is uglier because it?s more, it?s all up to opinion and it?s quite pointless to discuss; you are entitled to think that is ugly. > >> If that additional infrastructure were amortized across more use-cases >> then it might be more appealing. I might even have some additional >> use-cases for something along that line, though not in its present >> form. (More on that below.) > > Yeah I specifically had in mind the wish to make a split between commercial extensions to open source code actually. This pattern allows easy optional class inheritance injection as a general design pattern to achieve this. And for my own purposes the ability to define asymmetric dekker synchronization to elide storeload fences when the platform (e.g. TSO) supports it but otherwise revert to a generalized implementation (using storeload fences). This is why I think a clean reusable strategy is almost always better because it is widely applicable, even if there would not be obvious other examples where it is useful. > >> >>>> I think the name "IsBaseOfAndDerived" would be better as >>>> "IsBaseAndDerived?. [?] >>>> >>>> The current implementation of IsBaseOfAndDerived (and therefore >>>> IsBaseOf) may (silently) provide incorrect answers when called with >>>> incomplete types. [?] >>>> >>>> The actual usage in this proposed change-set even relies on this >>>> flawed behavior of these metafunctions, since AtomicPlatform isn't >>>> always defined. [?] >>>> >>> >>> So what you are saying here is that I should rename IsBaseOfAndDerived to IsBaseAndDerived to more closely resemble other external libraries we do not use. But if I do that my behaviour is "incorrect" because it is not identical to that of the external library. And therefore my change would not work if it was made "correctly?. >> >> The proposed IsBaseOfAndDerived and IsBaseOf are being described and >> placed as general utilities. Their names are strongly reminiscent of >> existing facilities in other libraries, including the (C++11) standard >> library. Conforming to these external standards will avoid surprises, >> especially when the proposed facilities are even described as being >> the same (except for a noted corner case that is both hard to deal >> with in a portable manner and not actually very interesting, so I'm >> not complaining about that) in the review request email. >> >> Additionally, I think quietly providing a potentially incorrect answer >> in the case of incomplete types renders the offered code quite broken >> for use as a general utility. I think the requirement for complete >> types by the well-known / standard facilities is well justified. >> > > While I disagree it is universally incorrect (as long as it is well defined and tested) to not have the exact same requirement on completeness on the parameters as C++11/boost, who also already have different behaviour in this exact regard as pointed out by yourself, I can try to fix it if it bugs you, but it will make the infrastructure even larger which seems to be a concern. > >>> On the contrary this is a well tested feature on all our supported compilers and I did not intend to engineer it to be identical to some external library if it only causes trouble rather than helping. This is our own class and it can do what we want it to do and be called what we want it to be called, and I think that is fine as long as it is well tested, which it is. At least this is my opinion. Anyone else agree? >> >> Problem is, I think it's well tested code to solve the wrong problem. >> >> I suggest that what we're really looking for is not a base/derived >> relationship between a pair of classes, but that we actually want to >> determine whether a platform-specific class exists. >> >> As a result, I think there is more infrastructure in this change set >> than is actually needed. The present implementation of >> SelectBaseClass uses IsBaseOf, but doesn't need a lot of the >> functionality of that metafunction. There is no actual need for a >> Base/Derived relationship between the (possibly not defined) >> platform-specific class and the default platform-generic class, so the >> use of IsBaseOf is an unnecessary restriction that only >> serendipitously tests for defined or not, due to the previously >> discussed defect in this particular implementation. I also think the >> name SelectBaseClass is perhaps overly general. > > The main idea of SelectBaseClass is to inject an optional class into the class hierarchy for specialization of generalized behaviour. It is absolutely critical (and intended) that this class is constrained to be a derived class of the fall-back class - it is not an artifact nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, A must be able to rely on the superclass being at least C-like so that the behaviour of the superclass can be used reliably and transparently regardless of the existence of the optional B. If this contract would not hold, I think it could lead to some really weird and unreliable code which should not be encouraged (at least not according to the Liskov substitution principle etc). > > So what I need in the contract is a check if the optional class is defined and if it fits in the inheritance hierarchy: they are both important IMO. > > Yes it could have been less code. From the beginning I had a single trait class that checked for both the inheritance and whether the class is defined or not but you said you wanted the behaviour to more closely match that of third party libraries. This is why there is now a lot more code to more closely (but still not exactly) match that behaviour. > >> I might be able to use a similar approach in some code I've been >> working on as a background task. And while writing this reply I've >> thought of another place that might benefit from something along those >> lines. Thinking about the code under review in that context, I think >> some changes would make these other possible use-cases easier and >> clearer. >> >> I believe what is actually needed is a metatfunction something like: >> >> /** >> * Metafunction whose nested type member is Default if Specific is >> * incomplete, otherwise Specific. >> * >> * Specific and Default must both be non-cv qualified class types, and >> * must not be the same type. >> * >> * If Specific is incomplete at the point where this metafunction is >> * invoked, it must never be defined elsewhere in the program. >> */ >> template >> struct SelectPlatformBase; >> >> [Note: I'm not wedded to that name.] >> >> Now, it turns out to be hard / impossible to write a generic >> is_complete metafunction, due to ODR. Whether a type is complete >> can vary between translation units, and even within a translation >> unit. Having is_complete::value have different values depending on >> where it is instantiated is an ODR violation. (One can make progress >> by using anonymous namespaces and macro wrappers with __LINE__ >> concatenation, but there is still the fundamental question of whether >> is_complete really even makes semantic sense.) > > Yes and that makes me wonder if we care enough about the completeness semantics being the same as third party libraries (who are already inconsistent) if we do not need them to be, and on the contrary benefit from them not to be. > >> However, we don't need a fully general is_complete utility. We have a >> much more restricted use-case, where we want to select an optionally >> defined platform-specific class if it exists, and otherwise fall back >> to a more generic class. And we're not using this in arbitrary >> contexts, but only to select a platform-specialized implementation if >> such exists. >> >> Here's a lightly tested implementation of SelectPlatformBase: >> >> // --- SelectPlatformBase >> >> template >> struct EnableIf { typedef T type; }; >> >> template >> struct EnableIf { }; >> >> template >> struct If { typedef Then type; }; >> >> template >> struct If { typedef Else type; }; >> >> // Metafunction whose nested value member is true if T is defined >> // (complete), and false if T is incomplete. T must be a non-cv >> // qualified class type. If T is incomplete at the point where this >> // metafunction is invoked, it must never be defined elsewhere in the >> // program. >> template >> class IsClassDefined { >> typedef char yes[1]; >> typedef char no[2]; >> >> template >> static typename EnableIf::type & check(U*); >> static no& check(...); >> >> public: >> static const bool value = sizeof(check((T*)0)) == sizeof(yes); >> }; >> >> template >> struct SelectPlatformBase { >> typedef typename If< >> IsClassDefined::value, Specific, Default>::type type; >> }; >> >> // --- end SelectPlatformBase >> >> That's ~30 lines of code (+ tests and comments TBD) to do precisely >> what we need, replacing ~100 lines (+ tests and comments) that have >> various issues. (If and EnableIf should probably be hoisted out as >> separate utilities.) We don't actually need SelectPlatformBase, and >> could instead just directly use the If and IsClassDefined >> metafunctions, but giving this functionality a name might be clearer. > > As stated, I think it?s dangerous to allow conditional inheritance where there is no inheritance constraint on the optional class. Therefore I do not think this is better, although I agree it is smaller. > > I agree that our use case here is smaller. If the amount of code is the problem and less code (and hence fewer reusable components) is preferred, then the original implementation with a simple IsBaseOfAndDerived does the trick (without checking types/cv qualifiers) and could be done in just a few lines, and this is what I originally did before you told me to expand it to more closely resemble external libraries: > > template > class IsBaseOfAndDerived { > typedef char yes[1]; > typedef char no[2]; > > template > static yes &check(Derived*, T); > static no &check(Base*, int); > > template > struct IsBaseOfHost { > operator B*() const; > operator D*(); > }; > public: > static const bool value = sizeof(check(IsBaseOfHost(), int())) == sizeof(yes); > }; > > It checks the inheritance and existence of the Derived class which is exactly the constraints I need. > > If you do not want to expose it publicly and remove the reusability because it does not have identical semantics as third party libraries regarding the requirements of the Derived type to be complete, it could be an internal class of SelectBaseClass. > > I?m not going to value the reusability vs LoC, I leave that decision to you. > >> However, while I think there are other use cases for this specific >> utility, I still think the proposed change set as a whole is adding a >> lot of code just to avoid a a few lines of macro usage. That seems >> like a poor tradeoff to me. > > > Okay. > > In the end, we have to make a choice - what is more important, that the traits resemble some external library or code size. Because last time I proposed a small change and you wanted it to be closer to C++11 behaviour and I made it closer to that at the expense of more code. Now you are saying I have to make it even tighter to C++11 (or boost?), but also don?t want more lines of code which I believe is impossible given that I want to constrain both the class hierarchy to be reliable and check the existence of the optional class. This brings me to a few architectural questions which I may have (strong) opinions about but are not up to me to decide. > > 1. Do we want to replace the macros with template metaprogramming? 2 reviewers (and me) liked the metaprogramming approach and it was implemented because they (like me) did not want the macros. But you have a different opinion. I can?t judge who is right and wrong here. > > 2. Do we want to minimize the code size at the cost of reusability by making contracts of dependent components weaker and putting them as private classes in the SelectBaseClass? > > 3. If we do want template metaprogramming and want reusable components, the question is, is it important that the specification of completeness (which in general is almost impossible to implement correctly in a portable way as you pointed out) of the template arguments in IsBaseOf is identical as C++11/boost even though no code needs it to be and on the contrary some code benefits from it not to be? In that case which one of them do we, because they are already different anyway? There is a tradeoff between code size and complexity vs copying either one of the external libraries regarding requirements of completeness of template parameters. > > Thanks, > /Erik > From goetz.lindenmaier at sap.com Mon Jan 5 09:24:41 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 5 Jan 2015 09:24:41 +0000 Subject: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. In-Reply-To: <54A6C6C3.1050806@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF264E2@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CF27CD3@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CF29936@DEWDFEMB12A.global.corp.sap> <547E4F3B.2090501@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF35E66@DEWDFEMB12A.global.corp.sap> <547F4914.1030904@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF36139@DEWDFEMB12A.global.corp.sap> <54809196.7030306@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF39BA5@DEWDFEMB12A.global.corp.sap> <5480AB58.80604@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF3D6BB@DEWDFEMB12A.global.corp.sap> <54889D61.30905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF3E1C5@DEWDFEMB12A.global.corp.sap> <54989B9F.2070903@oracle.com> <549B48A3.1070303@oracle.com> <54A2089D.1020105@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF53F70@DEWDFEMB12A.global.corp.sap> <54A2C966.5010306@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF67EB6@DEWDFEMB12A.global.corp.sap> <54A6C6C3.1050806@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF69210@DEWDFEMB12A.global.corp.sap> Hi Coleen, It would be great if you could sponsor this change. > The metaspace case probably can be fixed to use the try_reserve_heap() code maybe. We wondered why the compressed class space is always above the heap. If the heap is too big for unscaled, the compressed class space could be placed in the lower 4G. The aarch port brings some changes here, and we would like to investigate this on top of the aarch changes, if that's fine with you. Thanks a lot for going through this complex review! Best regards, Goetz. -----Original Message----- From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] Sent: Freitag, 2. Januar 2015 17:27 To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. On 1/2/15, 10:09 AM, Lindenmaier, Goetz wrote: > Hi Coleen, > > I looked at the cleanups and found both not straight forward to implement. > > 1) Merging ReservedHeap::initialize and ReservedHeapSpace::try_reserve_heap. > > The new code calls initialize() with requested_address = NULL, but metaspace.cpp > passes a real requested_address. Therefore I can not clean up the calls to > failed_to_reserve_as_requested() in initialize(). Without that cleanup, I need to pass flags > into initialize() to configure all the different behaviours I would need in a > merged case. > I think with 'large' and 'requested_address' there is already enough cases > in that method, so that I don't want to add more. Okay, this is fine. More flags to special case functions aren't good either. Can you file an RFE to look into refactoring so that the duplicate code can be removed though? The metaspace case probably can be fixed to use the try_reserve_heap() code maybe. > > 2) Moving _noaccess_prefix to ReservedHeapSpace > > This does not work as ReservedHeapSpace is casted to > ReservedSpace and returned by value. So ReservedHeapSpace can not have > fields or virtual methods. > It would be probably better to change the code to return ReservedHeapSpace > by pointer, or, at least under that type, but I think that exceeds the scope of this change. Too bad. > So is it ok if I leave the change as is wrt. to these cleanups? Yes, this is fine. > > I extended test/runtime/CompressedOops/UseCompressedOops.java > to combine the testcases of the different mode with the GCs and UseLargePages. > I added the changes in the test to > http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ Is .03 with the additional changes? If so, you should have made an .04. I reviewed this again and it looks good. I think Vladimir has reviewed an earlier version. I can sponsor this assuming there are no more comments. Coleen > Best regards, > Goetz. > > > > > > -----Original Message----- > From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] > Sent: Dienstag, 30. Dezember 2014 16:49 > To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. > > > On 12/30/14, 8:42 AM, Lindenmaier, Goetz wrote: >> Hi Coleen, >> >>> disjoint >> It tries to say that the bits used for the base (36-63) are different from >> those used for the oops / used as offset into the heap (0-35). >> I chose the name according to this translation >> http://dict.leo.org/#/search=disjoint&searchLoc=0&resultOrder=basic&multiwordShowSingle=on >> Both the german adjective and verb fit to describe the property of the base. >> I can add the following to the docu of the mode in universe.hpp:358 >> // Disjoint: Bits used in base are disjoint from bits used >> // for oops ==> oop = (cOop << 3) | base. One can disjoint >> // the bits of an oop into base and compressed oop. > Yes, that helps. It's a good distinct word so its good that we can make > sense in the documentation. >>> (I filed a bug to move these to the memory directory) >> Good idea! >> >>> 272 void ReservedSpace::establish_noaccess_prefix() { >>> This function should be in ReservedHeapSpace >> Yes, I moved it. I also would like to move the field _noaccess_prefix, >> but I would have to change a lot of code, like ReservedSpace::release(). >> The problem is that _base is not the real base in ReservedHeapSpace. >> Probably ReservedHeapSpace should have heap_size() and heap_base() >> giving the correcte values. >> I'll try to come up with a patch on top of this, maybe we should do it >> in a separate bug. > I just moved _noaccess_prefix and added a virtual > ReservedHeapSpace::release() that adjusted the base and size with > noaccess_prefix. I don't think the footprint of a vtable or virtual > call makes any difference. >>> Can you have arguments.cpp verify_range of HeapSearchSteps so this can't >>> happen? >> Done. >> >>> Rather move this to the end of ReservedHeapSpace constructor. >>> assert(_base == NULL || ... >> Yes, you are right, it should not be there twice. I moved it into the >> constructor. It is only relevant for heaps, right? > Yes. >>> Code in try_reserve_heap() is mostly a copy of ReservedSpace::initialize() >> You are right, requested_address in initialize is dead now. I'll remove that >> and try to merge the two functions. >> >> I removed the verbose printing. Anyways, it's mostly useful when developing this. > Thanks. I was trying to find some way to move the printing so it would > make sense to me but ended up giving up on that. >>> aligned_HBMA >> Changed to aligned_heap_base_min_address. >> >>> Is "size" const in initialize_compressed_heap? >> Yes. Added 'const'. >> >> I'll run tests with +UseLargePages. >> >> I had verified that the heap is at the exact same position for the code in >> my first RFR, and with HeapSearchSteps=1 to make sure I don't break anything. >> Vladimir asked me to increase that to 3. So far I still didn't find a case where >> this changed anything. >> (But I know from SAP JVM, which is tested on all the OS variants we support, >> that there are cases where this can lead to a better - but then different - heap >> placement.) >> The only case where there is always a difference is Solaris 5.11 which >> now can allocate zerobased memory. >> The code passes the CDS jtreg tests. >> >> I updated the webrev.03 with the minor changes. >> I'll come up with a new one with the cleanups of Reserved(Heap)Space >> (_noaccess_prefix and redundant functions). >> >> Thanks for this thorough review!! > Thank you for your willingness to make these changes. > > Happy New Year! > Coleen > >> Happy new year, >> Goetz. >> >> >> >> >> -----Original Message----- >> From: hotspot-runtime-dev [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore >> Sent: Dienstag, 30. Dezember 2014 03:06 >> To: hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. >> >> >> Hi, I've reviewed more code. >> >> I'm sorry about the following comment that should have been asked >> before. The name of the mode is very confusing. I can't figure out why >> it's called "disjoint". I keep looking for a second heap base or some >> code that makes the heap discontinuous. The heap base is really >> specially aligned, right? So I went to the thesaurus.com: >> >> http://www.thesaurus.com/browse/aligned >> >> and "disjoin" is an antonym. Why was "disjoint" chosen? From the >> code, it looks like "aligned" or "scaled" are more descriptive but >> they're overused words in the code. But ScaledBasedNarrowOop seems more >> descriptive than Disjoint. >> >> In virtualspace.cpp (I filed a bug to move these to the memory directory) >> >> 272 void ReservedSpace::establish_noaccess_prefix() { >> >> >> This function should be in ReservedHeapSpace since ReservedSpace never >> does this (and we never want it to). The function it replaced was in >> ReservedSpace and that was wrong. Now that ReservedHeapSpace has >> compressed oops logic in it, I think this function should be moved to >> this class. Actually, I've modified your patch to move the >> _noaccess_prefix field to ReservedHeapSpace. Maybe that's too much >> change to your patch but since the handling of noaccess_prefix is >> changed, it seemed like the thing to do. >> >> Can you have arguments.cpp verify_range of HeapSearchSteps so this can't >> happen? >> >> guarantee(HeapSearchSteps > 0, "Don't set HeapSearchSteps to 0"); >> >> Can this be a function like check_encoding_for_mark_sweep() , or >> something like that? It appears twice. Rather move this to the end of >> ReservedHeapSpace constructor. >> >> assert(_base == NULL || >> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == _base, >> "area must be distinguishable from marks for mark-sweep"); >> assert(_base == NULL || >> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() == >> &_base[size], >> "area must be distinguishable from marks for mark-sweep"); >> >> Code in try_reserve_heap() is a mostly a copy of >> ReservedSpace::initialize(), except the first version has a call to >> failed_to_reserve_as_requested which looks like it's handling the case >> for not being able to reserve the requested address for compressed oops >> which seems like you don't want this. This is very confusing. There >> shouldn't be two of these. >> >> It looks like in the compressed oops case, you call >> ReservedSpace::initialize() only with NULL requested address so >> failed_to_reserve_as_requested is dead code with this change. So it >> seems like try_reserve_heap() could be ReservedSpace::initialize() with >> the failed_to_reserve_as_requested eliminated and a special case for >> handling unaligned addresses. >> >> The logic in initialize_compressed_heap() still doesn't make sense to >> me. If the first try_reserve_range() succeeds at getting an unscaled >> compressed oop base, then does PrintCompressedOopsMode && Verbose still >> print: >> >> tty->print(" == U N S C A L E D ==\n"); >> tty->print(" == Z E R O B A S E D ==\n"); >> tty->print(" == D I S J O I N T B A S E ==\n"); >> tty->print(" == H E A P B A S E D ==\n"); >> >> Why would you print all 4? It's the printing that makes this >> confusing! Now I see how the next case checks that the base address is >> a better allocation. If you changed the printing to a comment like: >> >> // UnscaledMode after testing previous base address is not better (or >> something). >> >> Actually, you have these comments. I think the printing should be >> removed because it's the opposite of helpful. >> >> I had trouble reading the parameter aligned_HBMA, can you change to >> aligned_heap_base_min (I know it's longer but there's too many >> calculations here that words will make easier to read). Or just >> heap_base_min_address. >> >> zerobased_max = (char *)OopEncodingHeapMax - class_space; >> >> Odd that the solaris compiler can compile this and not the code below. >> (minor point) I think this should be: >> >> zerobased_max = zerobased_max - class_space; >> >> Is "size" const in initialize_compressed_heap? (answer appears to be >> yes) It's useful for reading the calculations that it never changes so >> I think it should be const (it's a minor point too). >> >> I don't have any other comments, I've tried to do the calculations a bit >> but I know that you've done this yourself. >> >> Also, can you run tests with -XX:+UseLargePages? I assume you've run >> tests with the 4 garbage collectors. This is a tricky piece of code >> that replaced something else difficult. We've had to change it often to >> support other features like CDS and compressed class pointers so making >> it as understandable as possible is worth the extra effort. I think >> your new design looks cleaner. >> >> Thanks, >> Coleen >> >> >> On 12/24/14, 6:13 PM, Coleen Phillimore wrote: >>> One note for my first reviewed file. >>> >>> On 12/22/14, 5:30 PM, Coleen Phillimore wrote: >>>> Hi Goetz, >>>> >>>> I'm looking at this version of the code >>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >>>> >>>> This passes our JPRT tests. Yeah! >>>> >>>> It's a significant change, so I'm actually reading the code with the >>>> patches applied. >>>> >>>> In src/share/vm/memory/universe.cpp >>>> >>>> Can you put all this code from 742-776 in a new function above >>>> initialize_heap(), like set_narrow_oop_values()? >>>> The comment at 689-695 refer to code that's moved to >>>> virtualspace.cpp. I think it needs to be reworded, updated for >>>> disjoint oops mode or removed. The comment at 744-749 also refers to >>>> code that's moved so should be removed also. The last two lines of >>>> this comment refer to something whose implementation has changed, so >>>> just remove them. >>>> >>>> On line >>>> >>>> 750 if ((uint64_t)Universe::heap()->reserved_region().end() > >>>> UnscaledOopHeapMax) { >>>> >>>> Can you make this expression a variable because the same expression >>>> appears on line 754. >>>> >>>> Why can't this code at 742-776 be with the compressed oops mode code >>>> at line 836-839. If you make this set_narrow_oop_values() function, >>>> can it be moved together? >>> Apparently this doesn't work (it crashes on some platforms). I can't >>> completely follow the control flow that makes this wrong, but ignore >>> this suggestion. >>> >>> Coleen >>>> Can you remove this comment too? I have no idea what it means. There >>>> is no CATCH below (?) >>>> >>>> // We will never reach the CATCH below since Exceptions::_throw >>>> will cause >>>> // the VM to exit if an exception is thrown during initialization >>>> >>>> >>>> ** We should file an RFE to move virtualspace.hpp/cpp to the >>>> src/share/vm/memory directory! >>>> >>>> Sorry, this is partial, I'll continue tomorrow. >>>> >>>> Thanks, >>>> Coleen >>>> >>>> >>>> On 12/11/2014 06:18 AM, Lindenmaier, Goetz wrote: >>>>> Hi Coleen, >>>>> >>>>> thanks for looking at this change! >>>>> >>>>>> get_attach_addresses_for_disjoint_mode function belongs in >>>>>> virtualspace.cpp >>>>> All information about the bounds of the modes comes from universe. >>>>> This is >>>>> related to is_disjoint_heap_base_address() and thus I left it in >>>>> universe. >>>>> But I'll move it, it can also be regarded as an implementation >>>>> detail of >>>>> initialize_compressed_heap(). I can make it static then, so that >>>>> it's removed >>>>> in 32 bit build, which is good. >>>>> >>>>>> Can you reduce some to at least 100? >>>>> Done. I agree that's better . (I have a 2560x1440 screen, causing >>>>> me to have >>>>> big windows :)) >>>>> >>>>>> Why is highest_start not const char* ? >>>>> Removed the const. >>>>> >>>>>> The control flow in initialize_compressed_heap makes no sense to me. >>>>> Vladimir had the same problem, I added comment at line 493 to make >>>>> that clearer. >>>>> One problem of the previous implementation was that, if the os layer >>>>> returned an address with desired properties, but not at the requested >>>>> address, it was discarded. >>>>> To fix this, try_reserve_heap always returns what it allocated, and I >>>>> check afterwards. >>>>> Therefore the checks of the next mode include the check of the >>>>> previous one. So if I try to allocate disjoint, but get unscaled, >>>>> I will detect that and return the unscaled heap. >>>>> If I implement a return, I just have to add tests. I don't get rid of >>>>> other tests. >>>>> >>>>>> Also, our 32 bit platforms don't like: 1 * SIZE_64K * SIZE_32G, >>>>> I changed that to use UCONST64 macro. That should help I guess. >>>>> I also have these in globalDefinitions_xlc.hpp. Should I move them >>>>> to globalDefinitions.hpp:200, after definition of K, M and G? >>>>> >>>>> I uploaded a new webrev: >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >>>>> >>>>> Best regards, >>>>> Goetz. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: hotspot-runtime-dev >>>>> [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of >>>>> Coleen Phillimore >>>>> Sent: Mittwoch, 10. Dezember 2014 20:22 >>>>> To: hotspot-runtime-dev at openjdk.java.net >>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>> "disjoint base" and improve compressed heap handling. >>>>> >>>>> >>>>> Hi Goetz, >>>>> >>>>> I have some initial comments, which are much less detailed than >>>>> Vladimir's comments. I haven't actually processed all the >>>>> implementation details yet. >>>>> >>>>> I think get_attach_addresses_for_disjoint_mode function belongs in >>>>> virtualspace.cpp and not in universe. I like that the compressed oop >>>>> reservation logic is moved to virtualspace.cpp. I think this is an >>>>> improvement over the Universe::preferred_heap_base() logic which was >>>>> also difficult. >>>>> >>>>> The Hotspot coding style doesn't dictate 80 columns anymore (there is >>>>> debate whether it should have or not, which we try not to have this >>>>> debate), but I found the very long lines in this code inconsistent with >>>>> other Hotspot code. I had to expand my window to cover my whole >>>>> screen. Can you reduce some to at least 100? >>>>> >>>>> For example, I aligned these parameters to see better since there >>>>> are so >>>>> many (the indentation isn't right in email). >>>>> >>>>> void ReservedHeapSpace::try_reserve_range(char *const highest_start, >>>>> char *lowest_start, >>>>> size_t >>>>> attach_point_alignment, >>>>> char *aligned_HBMA, // >>>>> HeapBaseMinAddress >>>>> char *upper_bound, >>>>> size_t size, >>>>> size_t alignment, >>>>> bool large) { >>>>> >>>>> Why is highest_start not const char* ? Doesn't char* const >>>>> highest_start just restrict you from changing the pointer and not what >>>>> it points to? This seems odd to do. >>>>> >>>>> The control flow in initialize_compressed_heap makes no sense to me. >>>>> It seems like there should be an exit when the best allocation for >>>>> compression is achieved. But it falls through after >>>>> try_reserve_range(). I can't figure out why it should fall through.... >>>>> >>>>> I was expecting something like: >>>>> >>>>> if (PrintCompressedOopsMode && Verbose) { >>>>> tty->print(" == H E A P B A S E M I N A D D R E S S ==\n"); >>>>> } >>>>> get the heap at aligned HeapBaseMinAddress, return if success... >>>>> >>>>> if (PrintCompressedOopsMode && Verbose) { >>>>> tty->print(" == U N S C A L E D ==\n"); >>>>> } >>>>> try to get heap base for unscaled access, return if successful >>>>> >>>>> etc. >>>>> >>>>> >>>>> You should make this into a little function for each return and for the >>>>> end of the initialize function, or move it to the ReservedHeapSpace >>>>> constructor (caller). >>>>> >>>>> assert(_base == NULL || >>>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == _base, >>>>> "area must be distinguishable from marks for mark-sweep"); >>>>> assert(_base == NULL || >>>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() == >>>>> &_base[size], >>>>> "area must be distinguishable from marks for mark-sweep"); >>>>> } >>>>> >>>>> I ran this through JPRT and this line didn't compile on macos: >>>>> >>>>> const uint64_t num_attempts_to_try = MIN2(HeapSearchSteps, >>>>> num_attempts_possible); >>>>> >>>>> I'm retrying now as: >>>>> >>>>> const uint64_t num_attempts_to_try = >>>>> MIN2((uint64_t)HeapSearchSteps, num_attempts_possible); >>>>> >>>>> Sorry for the delay looking at this. This is a big change that I >>>>> needed >>>>> to make more time to read. I am pleased that it's a performance >>>>> improvement. >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> On 12/8/14, 10:54 AM, Lindenmaier, Goetz wrote: >>>>>> Hi, >>>>>> >>>>>> This is just a ping to gc/rt mailing lists to reach appropriate >>>>>> people. >>>>>> >>>>>> I please need a reviewer from gc or rt, could somebody have a >>>>>> look at this? >>>>>> >>>>>> Short summary: >>>>>> - new cOops mode disjointbase that allows optimizations on PPC >>>>>> improving over heapbased >>>>>> - search for heaps: finds zerobased on sparc Solaris 11 and Aix >>>>>> - concentrate cOops heap allocation code in one function >>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>> >>>>>> Please reply only to the original thread in hotspot-dev to keep this >>>>>> local. >>>>>> >>>>>> Thanks and best regards, >>>>>> Goetz. >>>>>> >>>>>> -----Original Message----- >>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>> Sent: Donnerstag, 4. Dezember 2014 19:44 >>>>>> To: Lindenmaier, Goetz >>>>>> Cc: 'hotspot-dev developers' >>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>> "disjoint base" and improve compressed heap handling. >>>>>> >>>>>> This looks good to me. >>>>>> >>>>>> Now we need second review since changes are significant. Preferable >>>>>> from >>>>>> GC group since you changed ReservedHeapSpace. They will be affected >>>>>> most. Review from Runtime is also welcome. >>>>>> >>>>>> Thanks, >>>>>> Vladimir >>>>>> >>>>>> On 12/4/14 10:27 AM, Lindenmaier, Goetz wrote: >>>>>>> Hi Vladimir. >>>>>>> >>>>>>> Sorry. I updated the webrev once more. Hope it's fine now. >>>>>>> At least I can write comments :) >>>>>>> >>>>>>> Best regards, >>>>>>> Goetz >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>> Sent: Thursday, December 04, 2014 5:54 PM >>>>>>> To: Lindenmaier, Goetz >>>>>>> Cc: 'hotspot-dev developers' >>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>> >>>>>>> I spotted an other bug. >>>>>>> You replaced !_base with _base != NULL when moved code to >>>>>>> try_reserve_range() - it should be _base == NULL. >>>>>>> The same problem in asserts: >>>>>>> >>>>>>> + assert(_base != NULL || >>>>>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == >>>>>>> _base, >>>>>>> + "area must be distinguishable from marks for mark-sweep"); >>>>>>> + assert(_base != NULL || >>>>>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() == >>>>>>> &_base[size], >>>>>>> + "area must be distinguishable from marks for mark-sweep"); >>>>>>> >>>>>>> >>>>>>> Also you did not remove _base && in next place: >>>>>>> >>>>>>> + (_base && _base + size > zerobased_max))) { // Unscaled >>>>>>> delivered an arbitrary address. >>>>>>> >>>>>>> New comment is good. >>>>>>> >>>>>>> Thanks, >>>>>>> Vladimri >>>>>>> >>>>>>> On 12/4/14 1:45 AM, Lindenmaier, Goetz wrote: >>>>>>>> Hi Vladimir, >>>>>>>> >>>>>>>>> Add more extending comment explaining that. >>>>>>>> The comment for try_reserve_heap was meant to explain that. >>>>>>>> I further added a comment in initialize_compressed_heap(). >>>>>>>> >>>>>>>>> You need another parameter to pass UnscaledOopHeapMax or >>>>>>>>> zerobased_max. >>>>>>>> Oh, thanks a lot! That's important. Fixed. >>>>>>>> >>>>>>>>> I mean that you already checked _base == NULL so on other side >>>>>>>>> of || _base != NULL - why you need (_base &&) check? >>>>>>>> Sorry, now I got it. Removed. >>>>>>>> >>>>>>>> I updated the webrev: >>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>> >>>>>>>> Increment on top of the increment :) >>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/incremental_diffs2.patch >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Goetz. >>>>>>>> >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>> Sent: Mittwoch, 3. Dezember 2014 18:32 >>>>>>>> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' >>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>> >>>>>>>> Comments are below. >>>>>>>> >>>>>>>> On 12/3/14 5:49 AM, Lindenmaier, Goetz wrote: >>>>>>>>> Hi Vladimir, >>>>>>>>> >>>>>>>>> thanks for looking at the change! See my comments inline below. >>>>>>>>> >>>>>>>>> I made a new webrev: >>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>>> >>>>>>>>> Incremental changes: >>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/incremental_diffs.patch >>>>>>>>> >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> Goetz. >>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>>> Sent: Mittwoch, 3. Dezember 2014 00:46 >>>>>>>>>> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' >>>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>> >>>>>>>>>> This looks good to me. Someone in runtime/gc have to look on it >>>>>>>>>> too. >>>>>>>>>> >>>>>>>>>> universe.cpp about >>>>>>>>>> SystemProperty("com.sap.vm.test.compressedOopsMode" >>>>>>>>>> we have: >>>>>>>>>> java.vm.info=mixed mode, sharing >>>>>>>>>> so we can have: >>>>>>>>>> java.vm.compressedOopsMode=... >>>>>>>>> Yes, that's good for me. Fixed. >>>>>>>>> >>>>>>>>>> I am not expert in properties names but I don't want to have >>>>>>>>>> 'com.sap' >>>>>>>>>> in VM's property name. >>>>>>>>>> virtualspace.cpp: >>>>>>>>>> Could you fix release() - it does not reset _alignment? >>>>>>>>> Fixed. >>>>>>>>> >>>>>>>>>> In try_reserve_heap(), please, use (base == NULL) instead of >>>>>>>>>> (!base). >>>>>>>>>> And you don't need 'return;' in alignment check at the end of >>>>>>>>>> method. >>>>>>>>> Fixed. >>>>>>>>> >>>>>>>>>> In initialize_compressed_heap() again (!_base). >>>>>>>>> Fixed. >>>>>>>>> >>>>>>>>>> You don't stop (check >>>>>>>>>> (base == NULL)) after successful unscaled, zerobased, disjointbase >>>>>>>>>> allocations. You need to separate them with the check: >>>>>>>>>> >>>>>>>>>> + >>>>>>>>>> + } >>>>>>>>>> + } >>>>>>>>>> + if (_base == NULL) { >>>>>>>>>> + >>>>>>>>>> + if (PrintCompressedOopsMode && Verbose) { >>>>>>>>>> + tty->print(" == Z E R O B A S E D ==\n"); >>>>>>>>>> + } >>>>>>>>>> and so on. >>>>>>>>> No, I can't and don't want to check for _base != NULL. >>>>>>>>> I always keep the result of the last try, also if it didn't >>>>>>>>> fulfil the required properties. >>>>>>>>> So I take that result and go into the next check. That check >>>>>>>>> might succeed >>>>>>>>> with the heap allocated before. >>>>>>>>> This allows me to separate allocation and placement criteria, >>>>>>>>> and to have the >>>>>>>>> placement criteria checked in only one place (per mode). >>>>>>>>> Only for HeapBaseMinAddress I don't do it that way, I explicitly >>>>>>>>> call release(). >>>>>>>>> This way I can enforce mode heapbased. >>>>>>>> I see what you are saying. It was not clear from comments what is >>>>>>>> going on. >>>>>>>> Add more extending comment explaining that. >>>>>>>> >>>>>>>>>> num_attempts calculation and while() loop are similar in >>>>>>>>>> unscaled and >>>>>>>>>> zerobased cases. Could you move it into a separate method? >>>>>>>>> I can do that, but I don't like it as I have to pass in 7 >>>>>>>>> parameters. >>>>>>>> You need an other parameter to pass UnscaledOopHeapMax or >>>>>>>> zerobased_max. >>>>>>>> >>>>>>>>> That makes the code not much more readable. The function will >>>>>>>>> look like this: >>>>>>>> I think initialize_compressed_heap() is more readable now. >>>>>>>> >>>>>>>>> void ReserveHeapSpace::try_reserve_range(char *const >>>>>>>>> highest_start, char *lowest_start, size_t attach_point_alignment, >>>>>>>>> char >>>>>>>>> *aligned_HBMA, size_t size, size_t alignment, bool large) { >>>>>>>>> guarantee(HeapSearchSteps > 0, "Don't set HeapSearchSteps >>>>>>>>> to 0"); >>>>>>>>> >>>>>>>>> const size_t attach_range = highest_start - lowest_start; >>>>>>>>> // Cap num_attempts at possible number. >>>>>>>>> // At least one is possible even for 0 sized attach range. >>>>>>>>> const uint64_t num_attempts_possible = (attach_range / >>>>>>>>> attach_point_alignment) + 1; >>>>>>>>> const uint64_t num_attempts_to_try = >>>>>>>>> MIN2(HeapSearchSteps, num_attempts_possible); >>>>>>>>> >>>>>>>>> const size_t stepsize = align_size_up(attach_range / >>>>>>>>> num_attempts_to_try, attach_point_alignment); >>>>>>>>> >>>>>>>>> // Try attach points from top to bottom. >>>>>>>>> char* attach_point = highest_start; >>>>>>>>> while (attach_point >= lowest_start && >>>>>>>>> attach_point <= highest_start && // Avoid wrap >>>>>>>>> around. >>>>>>>>> (!_base || _base < aligned_HBMA || _base + size > >>>>>>>>> (char *)UnscaledOopHeapMax)) { >>>>>>>>> try_reserve_heap(size, alignment, large, attach_point); >>>>>>>>> attach_point -= stepsize; >>>>>>>>> } >>>>>>>>> } >>>>>>>>> >>>>>>>>> >>>>>>>>>> In disjointbase while() condition no need for _base second check: >>>>>>>>>> + (_base == NULL || >>>>>>>>>> + ((_base + size > (char *)OopEncodingHeapMax) && >>>>>>>>> I need this for the same reason as above: This is the check for >>>>>>>>> successful allocation. >>>>>>>> I mean that you already checked _base == NULL so on other side of >>>>>>>> || _base != NULL - why you need (_base &&) check? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Vladimir >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Vladimir >>>>>>>>> >>>>>>>>> On 11/21/14 5:31 AM, Lindenmaier, Goetz wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I prepared a new webrev trying to cover all the issues >>>>>>>>>> mentioned below. >>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.01/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I moved functionality from os.cpp and universe.cpp into >>>>>>>>>> ReservedHeapSpace::initialize_compressed_heap(). >>>>>>>>>> This class offers to save _base and _special, which I would >>>>>>>>>> have to reimplement >>>>>>>>>> if I had improved the methods I had added to os.cpp to also >>>>>>>>>> allocate large page >>>>>>>>>> heaps. >>>>>>>>>> Anyways, I think this class is the right place to gather code >>>>>>>>>> trying to reserve >>>>>>>>>> the heap. >>>>>>>>>> Also, I get along without setting the shift, base, >>>>>>>>>> implicit_null_check etc. fields >>>>>>>>>> of Universe, so there is no unnecessary calling back and forth >>>>>>>>>> between the two >>>>>>>>>> classes. >>>>>>>>>> Universe gets the heap back, and then sets the properties it >>>>>>>>>> needs to configure >>>>>>>>>> the compressed oops. >>>>>>>>>> All code handling the noaccess prefix is in a single method, too. >>>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> Goetz. >>>>>>>>>> >>>>>>>>>> Btw, I had to workaround a SS12u1 problem: it wouldn't compile >>>>>>>>>> char * x = (char*)UnscaledOopHeapMax - size in 32-bit mode. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] >>>>>>>>>> On Behalf Of Lindenmaier, Goetz >>>>>>>>>> Sent: Montag, 17. November 2014 09:33 >>>>>>>>>> To: 'Vladimir Kozlov'; 'hotspot-dev at openjdk.java.net' >>>>>>>>>> Subject: RE: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>> >>>>>>>>>> Hi Vladimir, >>>>>>>>>> >>>>>>>>>>> It is very significant rewriting and it takes time to evaluate >>>>>>>>>>> it. >>>>>>>>>> Yes, I know ... and I don't want to push, but nevertheless a ping >>>>>>>>>> can be useful sometimes. Thanks a lot for looking at it. >>>>>>>>>> >>>>>>>>>>> And I would not say it is simpler then before :) >>>>>>>>>> If I fix what you propose it's gonna get even more simple ;) >>>>>>>>>>> These is what I found so far. >>>>>>>>>>> The idea to try to allocate in a range instead of just below >>>>>>>>>>> UnscaledOopHeapMax or OopEncodingHeapMax is good. So I would >>>>>>>>>>> ask to do >>>>>>>>>>> several attempts (3?) on non_PPC64 platforms too. >>>>>>>>>> Set to 3. >>>>>>>>>> >>>>>>>>>>> It is matter of preference but I am not comfortable with >>>>>>>>>>> switch in loop. >>>>>>>>>>> For me sequential 'if (addr == 0)' checks is simpler. >>>>>>>>>> I'll fix this. >>>>>>>>>> >>>>>>>>>>> One thing worries me that you release found space and try to >>>>>>>>>>> get it >>>>>>>>>>> again with ReservedHeapSpace. Is it possible to add new >>>>>>>>>>> ReservedHeapSpace ctor which simple use already allocated space? >>>>>>>>>> This was to keep diff's small, but I also think a new >>>>>>>>>> constructor is good. >>>>>>>>>> I'll fix this. >>>>>>>>>> >>>>>>>>>>> The next code in ReservedHeapSpace() is hard to understand (): >>>>>>>>>>> (UseCompressedOops && (requested_address == NULL || >>>>>>>>>> requested_address+size > (char*)OopEncodingHeapMax) ? >>>>>>>>>>> may be move all this into noaccess_prefix_size() and add >>>>>>>>>>> comments. >>>>>>>>>> I have to redo this anyways if I make new constructors. >>>>>>>>>> >>>>>>>>>>> Why you need prefix when requested_address == NULL? >>>>>>>>>> If we allocate with NULL, we most probably will get a heap where >>>>>>>>>> base != NULL and thus need a noaccess prefix. >>>>>>>>>> >>>>>>>>>>> Remove next comment in universe.cpp: >>>>>>>>>>> // SAPJVM GL 2014-09-22 >>>>>>>>>> Removed. >>>>>>>>>> >>>>>>>>>>> Again you will release space so why bother to include space >>>>>>>>>>> for classes?: >>>>>>>>>>> + // For small heaps, save some space for compressed >>>>>>>>>>> class pointer >>>>>>>>>>> + // space so it can be decoded with no base. >>>>>>>>>> This was done like this before. We must assure the upper bound >>>>>>>>>> of the >>>>>>>>>> heap is low enough that the compressed class space still fits >>>>>>>>>> in there. >>>>>>>>>> >>>>>>>>>> virtualspace.cpp >>>>>>>>>> >>>>>>>>>>> With new code size+noaccess_prefix could be requested. But >>>>>>>>>>> later it is >>>>>>>>>>> not used if WIN64_ONLY(&& UseLargePages) and you will have empty >>>>>>>>>>> non-protected page below heap. >>>>>>>>>> There's several points to this: >>>>>>>>>> * Also if not protectable, the heap base has to be below >>>>>>>>>> the real start of the >>>>>>>>>> heap. Else the first object in the heap will be >>>>>>>>>> compressed to 'null' >>>>>>>>>> and decompression will fail. >>>>>>>>>> * If we don't reserve the memory other stuff can end up >>>>>>>>>> in this space. On >>>>>>>>>> errors, if would be quite unexpected to find memory >>>>>>>>>> there. >>>>>>>>>> * To get a heap for the new disjoint mode I must control >>>>>>>>>> the size of this. >>>>>>>>>> Requesting a heap starting at (aligned base + >>>>>>>>>> prefix) is more likely to fail. >>>>>>>>>> * The size for the prefix must anyways be considered >>>>>>>>>> when deciding whether the >>>>>>>>>> heap is small enough to run with compressed oops. >>>>>>>>>> So distinguishing the case where we really can omit this would >>>>>>>>>> require >>>>>>>>>> quite some additional checks everywhere, and I thought it's not >>>>>>>>>> worth it. >>>>>>>>>> >>>>>>>>>> matcher.hpp >>>>>>>>>> >>>>>>>>>>> Universe::narrow_oop_use_implicit_null_checks() should be true >>>>>>>>>>> for such >>>>>>>>>>> case too. So you can add new condition with || to existing >>>>>>>>>>> ones. The >>>>>>>>>>> only condition you relax is base != NULL. Right? >>>>>>>>>> Yes, that's how it's intended. >>>>>>>>>> >>>>>>>>>> arguments.* files >>>>>>>>>> >>>>>>>>>>> Why you need PropertyList_add changes. >>>>>>>>>> Oh, the code using it got lost. I commented on this in the >>>>>>>>>> description in the webrev. >>>>>>>>>> "To more efficiently run expensive tests in various compressed >>>>>>>>>> oop modes, we set a property with the mode the VM is running >>>>>>>>>> in. So far it's called "com.sap.vm.test.compressedOopsMode" >>>>>>>>>> better suggestions are welcome (and necessary I guess). Our >>>>>>>>>> long running tests that are supposed to run in a dedicated >>>>>>>>>> compressed oop mode check this property and abort themselves if >>>>>>>>>> it's not the expected mode." >>>>>>>>>> When I know about the heap I do >>>>>>>>>> Arguments::PropertyList_add(new >>>>>>>>>> SystemProperty("com.sap.vm.test.compressedOopsMode", >>>>>>>>>> narrow_oop_mode_to_string(narrow_oop_mode()), >>>>>>>>>> false)); >>>>>>>>>> in universe.cpp. >>>>>>>>>> On some OSes it's deterministic which modes work, there we >>>>>>>>>> don't start such tests. >>>>>>>>>> Others, as you mentioned OSX, are very indeterministic. Here >>>>>>>>>> we save testruntime with this. >>>>>>>>>> But it's not that important. >>>>>>>>>> We can still parse the PrintCompresseOopsMode output after the >>>>>>>>>> test and discard the >>>>>>>>>> run. >>>>>>>>>> >>>>>>>>>>> Do you have platform specific changes? >>>>>>>>>> Yes, for ppc and aix. I'll submit them once this is in. >>>>>>>>>> >>>>>>>>>> From your other mail: >>>>>>>>>>> One more thing. You should allow an allocation in the range >>>>>>>>>>> when returned from OS allocated address does not match >>>>>>>>>>> requested address. We had such cases on OSX, for example, when >>>>>>>>>>> OS allocates at different address but still inside range. >>>>>>>>>> Good point. I'll fix that in os::attempt_reserve_memory_in_range. >>>>>>>>>> >>>>>>>>>> I'll ping again once a new webrev is done! >>>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> Goetz. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 11/10/14 6:57 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I need to improve a row of things around compressed oops heap >>>>>>>>>>> handling >>>>>>>>>>> to achieve good performance on ppc. >>>>>>>>>>> I prepared a first webrev for review: >>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> A detailed technical description of the change is in the >>>>>>>>>>> webrev and according bug. >>>>>>>>>>> >>>>>>>>>>> If requested, I will split the change into parts with more >>>>>>>>>>> respective less impact on >>>>>>>>>>> non-ppc platforms. >>>>>>>>>>> >>>>>>>>>>> The change is derived from well-tested code in our VM. >>>>>>>>>>> Originally it was >>>>>>>>>>> crafted to require the least changes of VM coding, I changed >>>>>>>>>>> it to be better >>>>>>>>>>> streamlined with the VM. >>>>>>>>>>> I tested this change to deliver heaps at about the same >>>>>>>>>>> addresses as before. >>>>>>>>>>> Heap addresses mostly differ in lower bits. In some cases >>>>>>>>>>> (Solaris 5.11) a heap >>>>>>>>>>> in a better compressed oops mode is found, though. >>>>>>>>>>> I ran (and adapted) test/runtime/CompressedOops and >>>>>>>>>>> gc/arguments/TestUseCompressedOops*. >>>>>>>>>>> >>>>>>>>>>> Best regards, >>>>>>>>>>> Goetz. >>>>>>>>>>> >>>>>>>>>>> From goetz.lindenmaier at sap.com Mon Jan 5 09:35:40 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 5 Jan 2015 09:35:40 +0000 Subject: RFR(M): 8067941: [TESTBUG] Fix tests for OS with 64K page size. In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF43394@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF43394@DEWDFEMB12A.global.corp.sap> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF69221@DEWDFEMB12A.global.corp.sap> Hi, could someone from runtime please have a look at the last of these fixes? runtime/whitebox/WBStackSize.java http://cr.openjdk.java.net/~goetz/webrevs/8067941-64K/webrev.01/test/runtime/whitebox/WBStackSize.java.udiff.html I get the shadow page size from WB to compute the space wasted on the stack so far. And I please need a sponsor for this! Best regards, Goetz. From: Lindenmaier, Goetz Sent: Freitag, 19. Dezember 2014 17:06 To: 'hotspot-dev at openjdk.java.net' Subject: RFR(M): 8067941: [TESTBUG] Fix tests for OS with 64K page size. Hi, please review this simple change. I please need a sponsor. The change should be downported to 8u60. It applies nicely except for some missing files. http://cr.openjdk.java.net/~goetz/webrevs/8067941-64K/webrev.01/ The issue is that we have linux ppc64 machines that have default page size of 64K. This imposes bigger limits on stack and heap sizes than if running with 4k pages. Ergonomics increase some flags if they are too small. Increasing the sizes used in the test slightly helps in most cases. Details: runtime/whitebox/WBStackSize.java The constant esimating the so far occupied stack is too small. Fix: Consider size of shadow pages compiler/uncommontrap/TestStackBangRbp.java: compiler/uncommontrap/TestStackBangMonitorOwned.java compiler/uncommontrap/StackOverflowGuardPagesOff.java compiler/uncommontrap/8009761/Test8009761.java compiler/runtime/6865265/StackOverflowBug.java compiler/exceptions/TestRecursiveReplacedException.java Increase stack size to 392K, need at least 328K. sanity/ExecuteInternalVMTests.java - Assertion in metaspace.cpp simly must fail with vm_page_size == 64K as it basically asserts vm_page_size < 16K - With 64K pages, heap sizes are computed differently starting from the alignment which is card_size * vm_page_size. gc/arguments/TestMaxHeapSizeTools.java Sizes are alinged up being equal in the end. Choose bigger sizes. gc/g1/TestHumongousAllocInitialMark.java Test computes some size based on the -Xmx value it uses. Heap size is increased slightly with 64K making the computation wrong and the test fail. Choose heap size that needs not be aligned. gc/g1/TestGCLogMessages.java Heap of 10M is increased to 32M because of the alignment with 64K pages. This makes the evacuation succeed. Choose 32M from the beginning and adapt size of huge object. Best regards, Goetz. From goetz.lindenmaier at sap.com Mon Jan 5 09:38:25 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 5 Jan 2015 09:38:25 +0000 Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java Message-ID: <4295855A5C1DE049A61835A1887419CC2CF6922E@DEWDFEMB12A.global.corp.sap> Just a little reminder ... this is really small! From: Lindenmaier, Goetz Sent: Montag, 22. Dezember 2014 14:08 To: 'hotspot-dev at openjdk.java.net' Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java Hi, I please need a review and a sponsor for this tiny change: http://cr.openjdk.java.net/~goetz/webrevs/8068026-jtregClient/webrev.01/ The test did not check whether -client is supported. The patch should also go to 8u60, please. Best regards, Goetz From aph at redhat.com Mon Jan 5 09:42:39 2015 From: aph at redhat.com (Andrew Haley) Date: Mon, 05 Jan 2015 09:42:39 +0000 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54A84E35.2000403@linaro.org> References: <54A6A933.8000204@redhat.com> <54A6E3DA.40504@oracle.com> <54A6E748.3070009@redhat.com> <54A6EAF6.106@oracle.com> <54A84E35.2000403@linaro.org> Message-ID: <54AA5C8F.8000605@redhat.com> On 03/01/15 20:16, Edward Nevill wrote: > I think it is necessary. See > > http://mail.openjdk.java.net/pipermail/aarch64-port-dev/2014-October/001503.html Okay, please push it to JDK9. Andrew. From aph at redhat.com Mon Jan 5 09:43:27 2015 From: aph at redhat.com (Andrew Haley) Date: Mon, 05 Jan 2015 09:43:27 +0000 Subject: 8068055: AARCH64: os_cpu/ In-Reply-To: <54A9D349.1000504@oracle.com> References: <54A6A977.4080405@redhat.com> <54A9D349.1000504@oracle.com> Message-ID: <54AA5CBF.8000409@redhat.com> On 04/01/15 23:56, David Holmes wrote: > On 3/01/2015 12:21 AM, Andrew Haley wrote: >> http://cr.openjdk.java.net/~aph/aarch64-8068055-1/ > > Just curious - where are things like __sync_lock_test_and_set and > __atomic_thread_fence defined? Are these external library routines that > are part of your toolset? They are part of GCC. Andrew. From jesper.wilhelmsson at oracle.com Mon Jan 5 10:02:57 2015 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Mon, 05 Jan 2015 11:02:57 +0100 Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF6922E@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF6922E@DEWDFEMB12A.global.corp.sap> Message-ID: <54AA6151.3040603@oracle.com> Hi, It seems your change introduces dead code: + if (output.shouldContain("Unrecognized option: -client") == null) { + throw e; + } If the string is found shouldContain will return a reference to 'output' (!= null), and if the string is not found it will throw an exception, so the introduced throw will never happen, right? This means the original exception 'e' is lost and a different one is thrown if -client was accepted by the VM. /Jesper Lindenmaier, Goetz skrev den 5/1/15 10:38: > Just a little reminder ... this is really small! > > From: Lindenmaier, Goetz > Sent: Montag, 22. Dezember 2014 14:08 > To: 'hotspot-dev at openjdk.java.net' > Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java > > Hi, > > I please need a review and a sponsor for this tiny change: > > http://cr.openjdk.java.net/~goetz/webrevs/8068026-jtregClient/webrev.01/ > The test did not check whether -client is supported. > > The patch should also go to 8u60, please. > > Best regards, > Goetz > From goetz.lindenmaier at sap.com Mon Jan 5 10:46:03 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 5 Jan 2015 10:46:03 +0000 Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java In-Reply-To: <54AA6151.3040603@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF6922E@DEWDFEMB12A.global.corp.sap> <54AA6151.3040603@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF69282@DEWDFEMB12A.global.corp.sap> Hi Jesper, You're right, this throws another exception delivering the wrong one in the end. I guess I should have used firstMatch() == null. I'll fix this. Sorry, Goetz. -----Original Message----- From: Jesper Wilhelmsson [mailto:jesper.wilhelmsson at oracle.com] Sent: Montag, 5. Januar 2015 11:03 To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' Subject: Re: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java Hi, It seems your change introduces dead code: + if (output.shouldContain("Unrecognized option: -client") == null) { + throw e; + } If the string is found shouldContain will return a reference to 'output' (!= null), and if the string is not found it will throw an exception, so the introduced throw will never happen, right? This means the original exception 'e' is lost and a different one is thrown if -client was accepted by the VM. /Jesper Lindenmaier, Goetz skrev den 5/1/15 10:38: > Just a little reminder ... this is really small! > > From: Lindenmaier, Goetz > Sent: Montag, 22. Dezember 2014 14:08 > To: 'hotspot-dev at openjdk.java.net' > Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java > > Hi, > > I please need a review and a sponsor for this tiny change: > > http://cr.openjdk.java.net/~goetz/webrevs/8068026-jtregClient/webrev.01/ > The test did not check whether -client is supported. > > The patch should also go to 8u60, please. > > Best regards, > Goetz > From aph at redhat.com Mon Jan 5 11:28:39 2015 From: aph at redhat.com (Andrew Haley) Date: Mon, 05 Jan 2015 11:28:39 +0000 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54A6EAF6.106@oracle.com> References: <54A6A933.8000204@redhat.com> <54A6E3DA.40504@oracle.com> <54A6E748.3070009@redhat.com> <54A6EAF6.106@oracle.com> Message-ID: <54AA7567.2020006@redhat.com> On 02/01/15 19:01, Coleen Phillimore wrote: > > On 1/2/15, 1:45 PM, Andrew Haley wrote: >> Hi, >> >> On 01/02/2015 06:30 PM, Coleen Phillimore wrote: >>> Hi Andrew, I had a short look at this code and spot-checked some >>> functions in the interpreter. In all these files, were any of the >>> functions added in a different place than in the x86 ports or is this >>> consistent with the other ports? >> Gosh. I had no idea that consistency with any other port would be any >> kind of an issue; I never even considered the question, but it's >> similar to x86. > > The reason I asked is that if I have to add something to one platform, > it would be nice to find it in the same place for another, that's all. We used x86-64 as a template for our port so you should find a close layout correspondence, but we did not adhere strictly to it. >>> +// Method entry for java.lang.ref.Reference.get. >>> +address InterpreterGenerator::generate_Reference_get_entry(void) { >>> + return NULL; >>> +} >>> >>> Why isn't this implemented? I think this needs to have code to support >>> the G1 garbage collector. I don't know this architecture, but I think >>> you can use my code review nonetheless. >> Okay. I think G1 works, but I don't know about this issue. I'll have >> a look. > > You may not need this code; it may be an optimization. The hg history > and bug number it points to would say. Okay. It seems that this code was not checked in to the tree I'm using to produce these patches; it will be fixed shortly. Thanks, Andrew. From goetz.lindenmaier at sap.com Mon Jan 5 11:48:47 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 5 Jan 2015 11:48:47 +0000 Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java In-Reply-To: <54AA6151.3040603@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF6922E@DEWDFEMB12A.global.corp.sap> <54AA6151.3040603@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF692D0@DEWDFEMB12A.global.corp.sap> Hi, I fixed it as proposed. http://cr.openjdk.java.net/~goetz/webrevs/8068026-jtregClient/webrev.01/ I also assured the right exception is thrown, if I change the check for the exit value to check for '1' (so that I get a test error), I now get: java.lang.RuntimeException: Expected to get exit value of [1] while before, it would have said the wrong message: java.lang.RuntimeException: 'Unrecognized option: -client' missing from stdout/stderr Best regards, Goetz. -----Original Message----- From: Jesper Wilhelmsson [mailto:jesper.wilhelmsson at oracle.com] Sent: Montag, 5. Januar 2015 11:03 To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' Subject: Re: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java Hi, It seems your change introduces dead code: + if (output.shouldContain("Unrecognized option: -client") == null) { + throw e; + } If the string is found shouldContain will return a reference to 'output' (!= null), and if the string is not found it will throw an exception, so the introduced throw will never happen, right? This means the original exception 'e' is lost and a different one is thrown if -client was accepted by the VM. /Jesper Lindenmaier, Goetz skrev den 5/1/15 10:38: > Just a little reminder ... this is really small! > > From: Lindenmaier, Goetz > Sent: Montag, 22. Dezember 2014 14:08 > To: 'hotspot-dev at openjdk.java.net' > Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java > > Hi, > > I please need a review and a sponsor for this tiny change: > > http://cr.openjdk.java.net/~goetz/webrevs/8068026-jtregClient/webrev.01/ > The test did not check whether -client is supported. > > The patch should also go to 8u60, please. > > Best regards, > Goetz > From jesper.wilhelmsson at oracle.com Mon Jan 5 12:10:03 2015 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Mon, 05 Jan 2015 13:10:03 +0100 Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF692D0@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF6922E@DEWDFEMB12A.global.corp.sap> <54AA6151.3040603@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF692D0@DEWDFEMB12A.global.corp.sap> Message-ID: <54AA7F1B.2090401@oracle.com> Looks good! In the future I would prefer if you upload a new webrev with a new URL instead of updating the existing webrev also for small changes like this one. Thanks! /Jesper Lindenmaier, Goetz skrev den 5/1/15 12:48: > Hi, > > I fixed it as proposed. > http://cr.openjdk.java.net/~goetz/webrevs/8068026-jtregClient/webrev.01/ > > I also assured the right exception is thrown, if I change the check for > the exit value to check for '1' (so that I get a test error), I now get: > java.lang.RuntimeException: Expected to get exit value of [1] > while before, it would have said the wrong message: > java.lang.RuntimeException: 'Unrecognized option: -client' missing from stdout/stderr > > Best regards, > Goetz. > > > -----Original Message----- > From: Jesper Wilhelmsson [mailto:jesper.wilhelmsson at oracle.com] > Sent: Montag, 5. Januar 2015 11:03 > To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' > Subject: Re: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java > > Hi, > > It seems your change introduces dead code: > > + if (output.shouldContain("Unrecognized option: -client") == null) { > + throw e; > + } > > If the string is found shouldContain will return a reference to 'output' (!= > null), and if the string is not found it will throw an exception, so the > introduced throw will never happen, right? > > This means the original exception 'e' is lost and a different one is thrown if > -client was accepted by the VM. > > /Jesper > > > Lindenmaier, Goetz skrev den 5/1/15 10:38: >> Just a little reminder ... this is really small! >> >> From: Lindenmaier, Goetz >> Sent: Montag, 22. Dezember 2014 14:08 >> To: 'hotspot-dev at openjdk.java.net' >> Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java >> >> Hi, >> >> I please need a review and a sponsor for this tiny change: >> >> http://cr.openjdk.java.net/~goetz/webrevs/8068026-jtregClient/webrev.01/ >> The test did not check whether -client is supported. >> >> The patch should also go to 8u60, please. >> >> Best regards, >> Goetz >> From goetz.lindenmaier at sap.com Mon Jan 5 13:07:26 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 5 Jan 2015 13:07:26 +0000 Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java In-Reply-To: <54AA7F1B.2090401@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF6922E@DEWDFEMB12A.global.corp.sap> <54AA6151.3040603@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF692D0@DEWDFEMB12A.global.corp.sap> <54AA7F1B.2090401@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF692F4@DEWDFEMB12A.global.corp.sap> Hi Jesper, ok, I will make more of them next time. Thanks for looking at this, Goetz. -----Original Message----- From: Jesper Wilhelmsson [mailto:jesper.wilhelmsson at oracle.com] Sent: Montag, 5. Januar 2015 13:10 To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' Subject: Re: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java Looks good! In the future I would prefer if you upload a new webrev with a new URL instead of updating the existing webrev also for small changes like this one. Thanks! /Jesper Lindenmaier, Goetz skrev den 5/1/15 12:48: > Hi, > > I fixed it as proposed. > http://cr.openjdk.java.net/~goetz/webrevs/8068026-jtregClient/webrev.01/ > > I also assured the right exception is thrown, if I change the check for > the exit value to check for '1' (so that I get a test error), I now get: > java.lang.RuntimeException: Expected to get exit value of [1] > while before, it would have said the wrong message: > java.lang.RuntimeException: 'Unrecognized option: -client' missing from stdout/stderr > > Best regards, > Goetz. > > > -----Original Message----- > From: Jesper Wilhelmsson [mailto:jesper.wilhelmsson at oracle.com] > Sent: Montag, 5. Januar 2015 11:03 > To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' > Subject: Re: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java > > Hi, > > It seems your change introduces dead code: > > + if (output.shouldContain("Unrecognized option: -client") == null) { > + throw e; > + } > > If the string is found shouldContain will return a reference to 'output' (!= > null), and if the string is not found it will throw an exception, so the > introduced throw will never happen, right? > > This means the original exception 'e' is lost and a different one is thrown if > -client was accepted by the VM. > > /Jesper > > > Lindenmaier, Goetz skrev den 5/1/15 10:38: >> Just a little reminder ... this is really small! >> >> From: Lindenmaier, Goetz >> Sent: Montag, 22. Dezember 2014 14:08 >> To: 'hotspot-dev at openjdk.java.net' >> Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java >> >> Hi, >> >> I please need a review and a sponsor for this tiny change: >> >> http://cr.openjdk.java.net/~goetz/webrevs/8068026-jtregClient/webrev.01/ >> The test did not check whether -client is supported. >> >> The patch should also go to 8u60, please. >> >> Best regards, >> Goetz >> From coleen.phillimore at oracle.com Mon Jan 5 16:49:10 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 05 Jan 2015 11:49:10 -0500 Subject: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF69210@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF264E2@DEWDFEMB12A.global.corp.sap> <547E4F3B.2090501@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF35E66@DEWDFEMB12A.global.corp.sap> <547F4914.1030904@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF36139@DEWDFEMB12A.global.corp.sap> <54809196.7030306@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF39BA5@DEWDFEMB12A.global.corp.sap> <5480AB58.80604@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF3D6BB@DEWDFEMB12A.global.corp.sap> <54889D61.30905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF3E1C5@DEWDFEMB12A.global.corp.sap> <54989B9F.2070903@oracle.com> <549B48A3.1070303@oracle.com> <54A2089D.1020105@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF53F70@DEWDFEMB12A.global.corp.sap> <54A2C966.5010306@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF67EB6@DEWDFEMB12A.global.corp.sap> <54A6C6C3.1050806@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF69210@DEWDFEMB12A.global.corp.sap> Message-ID: <54AAC086.2060600@oracle.com> On 1/5/15, 4:24 AM, Lindenmaier, Goetz wrote: > Hi Coleen, > > It would be great if you could sponsor this change. I see no other comments so I'll push it today. > >> The metaspace case probably can be fixed to use the try_reserve_heap() code maybe. > We wondered why the compressed class space is always above the heap. > If the heap is too big for unscaled, the compressed class space could be placed > in the lower 4G. The reason that we chose the compressed class space on the top of the heap is for a couple reasons. The time spent compressing and decompressing oops is more critical than the time spent doing the same for the _klass pointer, so we wanted to make the system more likely to get unscaled compressed oops, or even zero based compressed oops. On solaris, the space below the heap is limited because by default malloc uses it so we were limited in low address memory that we can use. We want the compressed class space code not be platform dependent. We also need to have the CDS archive at a fixed address near or adjacent to the compressed class space, so were running out at the bottom of the heap. The compressed class space is fixed at startup (just like PermGen was) and the classes are variable sized, so we had to pick a size fairly large so that it's unlikely for applications to run out. If above 32G, we might be able to implement code to allocate a new space which would use the same compression algorithm. We haven't had a request for that but it's something we could do. > The aarch port brings some changes here, and we would like to investigate > this on top of the aarch changes, if that's fine with you. So one thing I prototyped and really wanted to check in was what we called indirect class pointers. Instead of allocating classes in a fixed class space, we allocate a fixed array of Klass pointers and the objects in the heap point to this. It had a 8% performance degradation in specbb2008 compiler.compiler (iirc, or something like that). This solved so many problems though. If you investigated this sort of approach, I'd be very grateful. I haven't had time to go back to figure out why this had this degradation (we thought it was Java vtable accesses). Thanks, Coleen > > Thanks a lot for going through this complex review! > > Best regards, > Goetz. > > -----Original Message----- > From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] > Sent: Freitag, 2. Januar 2015 17:27 > To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. > > > On 1/2/15, 10:09 AM, Lindenmaier, Goetz wrote: >> Hi Coleen, >> >> I looked at the cleanups and found both not straight forward to implement. >> >> 1) Merging ReservedHeap::initialize and ReservedHeapSpace::try_reserve_heap. >> >> The new code calls initialize() with requested_address = NULL, but metaspace.cpp >> passes a real requested_address. Therefore I can not clean up the calls to >> failed_to_reserve_as_requested() in initialize(). Without that cleanup, I need to pass flags >> into initialize() to configure all the different behaviours I would need in a >> merged case. >> I think with 'large' and 'requested_address' there is already enough cases >> in that method, so that I don't want to add more. > Okay, this is fine. More flags to special case functions aren't good > either. Can you file an RFE to look into refactoring so that the > duplicate code can be removed though? The metaspace case probably can > be fixed to use the try_reserve_heap() code maybe. > >> 2) Moving _noaccess_prefix to ReservedHeapSpace >> >> This does not work as ReservedHeapSpace is casted to >> ReservedSpace and returned by value. So ReservedHeapSpace can not have >> fields or virtual methods. >> It would be probably better to change the code to return ReservedHeapSpace >> by pointer, or, at least under that type, but I think that exceeds the scope of this change. > Too bad. > >> So is it ok if I leave the change as is wrt. to these cleanups? > Yes, this is fine. >> I extended test/runtime/CompressedOops/UseCompressedOops.java >> to combine the testcases of the different mode with the GCs and UseLargePages. >> I added the changes in the test to >> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ > Is .03 with the additional changes? If so, you should have made an .04. > > I reviewed this again and it looks good. I think Vladimir has reviewed > an earlier version. I can sponsor this assuming there are no more comments. > > Coleen >> Best regards, >> Goetz. >> >> >> >> >> >> -----Original Message----- >> From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] >> Sent: Dienstag, 30. Dezember 2014 16:49 >> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. >> >> >> On 12/30/14, 8:42 AM, Lindenmaier, Goetz wrote: >>> Hi Coleen, >>> >>>> disjoint >>> It tries to say that the bits used for the base (36-63) are different from >>> those used for the oops / used as offset into the heap (0-35). >>> I chose the name according to this translation >>> http://dict.leo.org/#/search=disjoint&searchLoc=0&resultOrder=basic&multiwordShowSingle=on >>> Both the german adjective and verb fit to describe the property of the base. >>> I can add the following to the docu of the mode in universe.hpp:358 >>> // Disjoint: Bits used in base are disjoint from bits used >>> // for oops ==> oop = (cOop << 3) | base. One can disjoint >>> // the bits of an oop into base and compressed oop. >> Yes, that helps. It's a good distinct word so its good that we can make >> sense in the documentation. >>>> (I filed a bug to move these to the memory directory) >>> Good idea! >>> >>>> 272 void ReservedSpace::establish_noaccess_prefix() { >>>> This function should be in ReservedHeapSpace >>> Yes, I moved it. I also would like to move the field _noaccess_prefix, >>> but I would have to change a lot of code, like ReservedSpace::release(). >>> The problem is that _base is not the real base in ReservedHeapSpace. >>> Probably ReservedHeapSpace should have heap_size() and heap_base() >>> giving the correcte values. >>> I'll try to come up with a patch on top of this, maybe we should do it >>> in a separate bug. >> I just moved _noaccess_prefix and added a virtual >> ReservedHeapSpace::release() that adjusted the base and size with >> noaccess_prefix. I don't think the footprint of a vtable or virtual >> call makes any difference. >>>> Can you have arguments.cpp verify_range of HeapSearchSteps so this can't >>>> happen? >>> Done. >>> >>>> Rather move this to the end of ReservedHeapSpace constructor. >>>> assert(_base == NULL || ... >>> Yes, you are right, it should not be there twice. I moved it into the >>> constructor. It is only relevant for heaps, right? >> Yes. >>>> Code in try_reserve_heap() is mostly a copy of ReservedSpace::initialize() >>> You are right, requested_address in initialize is dead now. I'll remove that >>> and try to merge the two functions. >>> >>> I removed the verbose printing. Anyways, it's mostly useful when developing this. >> Thanks. I was trying to find some way to move the printing so it would >> make sense to me but ended up giving up on that. >>>> aligned_HBMA >>> Changed to aligned_heap_base_min_address. >>> >>>> Is "size" const in initialize_compressed_heap? >>> Yes. Added 'const'. >>> >>> I'll run tests with +UseLargePages. >>> >>> I had verified that the heap is at the exact same position for the code in >>> my first RFR, and with HeapSearchSteps=1 to make sure I don't break anything. >>> Vladimir asked me to increase that to 3. So far I still didn't find a case where >>> this changed anything. >>> (But I know from SAP JVM, which is tested on all the OS variants we support, >>> that there are cases where this can lead to a better - but then different - heap >>> placement.) >>> The only case where there is always a difference is Solaris 5.11 which >>> now can allocate zerobased memory. >>> The code passes the CDS jtreg tests. >>> >>> I updated the webrev.03 with the minor changes. >>> I'll come up with a new one with the cleanups of Reserved(Heap)Space >>> (_noaccess_prefix and redundant functions). >>> >>> Thanks for this thorough review!! >> Thank you for your willingness to make these changes. >> >> Happy New Year! >> Coleen >> >>> Happy new year, >>> Goetz. >>> >>> >>> >>> >>> -----Original Message----- >>> From: hotspot-runtime-dev [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore >>> Sent: Dienstag, 30. Dezember 2014 03:06 >>> To: hotspot-runtime-dev at openjdk.java.net >>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. >>> >>> >>> Hi, I've reviewed more code. >>> >>> I'm sorry about the following comment that should have been asked >>> before. The name of the mode is very confusing. I can't figure out why >>> it's called "disjoint". I keep looking for a second heap base or some >>> code that makes the heap discontinuous. The heap base is really >>> specially aligned, right? So I went to the thesaurus.com: >>> >>> http://www.thesaurus.com/browse/aligned >>> >>> and "disjoin" is an antonym. Why was "disjoint" chosen? From the >>> code, it looks like "aligned" or "scaled" are more descriptive but >>> they're overused words in the code. But ScaledBasedNarrowOop seems more >>> descriptive than Disjoint. >>> >>> In virtualspace.cpp (I filed a bug to move these to the memory directory) >>> >>> 272 void ReservedSpace::establish_noaccess_prefix() { >>> >>> >>> This function should be in ReservedHeapSpace since ReservedSpace never >>> does this (and we never want it to). The function it replaced was in >>> ReservedSpace and that was wrong. Now that ReservedHeapSpace has >>> compressed oops logic in it, I think this function should be moved to >>> this class. Actually, I've modified your patch to move the >>> _noaccess_prefix field to ReservedHeapSpace. Maybe that's too much >>> change to your patch but since the handling of noaccess_prefix is >>> changed, it seemed like the thing to do. >>> >>> Can you have arguments.cpp verify_range of HeapSearchSteps so this can't >>> happen? >>> >>> guarantee(HeapSearchSteps > 0, "Don't set HeapSearchSteps to 0"); >>> >>> Can this be a function like check_encoding_for_mark_sweep() , or >>> something like that? It appears twice. Rather move this to the end of >>> ReservedHeapSpace constructor. >>> >>> assert(_base == NULL || >>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == _base, >>> "area must be distinguishable from marks for mark-sweep"); >>> assert(_base == NULL || >>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() == >>> &_base[size], >>> "area must be distinguishable from marks for mark-sweep"); >>> >>> Code in try_reserve_heap() is a mostly a copy of >>> ReservedSpace::initialize(), except the first version has a call to >>> failed_to_reserve_as_requested which looks like it's handling the case >>> for not being able to reserve the requested address for compressed oops >>> which seems like you don't want this. This is very confusing. There >>> shouldn't be two of these. >>> >>> It looks like in the compressed oops case, you call >>> ReservedSpace::initialize() only with NULL requested address so >>> failed_to_reserve_as_requested is dead code with this change. So it >>> seems like try_reserve_heap() could be ReservedSpace::initialize() with >>> the failed_to_reserve_as_requested eliminated and a special case for >>> handling unaligned addresses. >>> >>> The logic in initialize_compressed_heap() still doesn't make sense to >>> me. If the first try_reserve_range() succeeds at getting an unscaled >>> compressed oop base, then does PrintCompressedOopsMode && Verbose still >>> print: >>> >>> tty->print(" == U N S C A L E D ==\n"); >>> tty->print(" == Z E R O B A S E D ==\n"); >>> tty->print(" == D I S J O I N T B A S E ==\n"); >>> tty->print(" == H E A P B A S E D ==\n"); >>> >>> Why would you print all 4? It's the printing that makes this >>> confusing! Now I see how the next case checks that the base address is >>> a better allocation. If you changed the printing to a comment like: >>> >>> // UnscaledMode after testing previous base address is not better (or >>> something). >>> >>> Actually, you have these comments. I think the printing should be >>> removed because it's the opposite of helpful. >>> >>> I had trouble reading the parameter aligned_HBMA, can you change to >>> aligned_heap_base_min (I know it's longer but there's too many >>> calculations here that words will make easier to read). Or just >>> heap_base_min_address. >>> >>> zerobased_max = (char *)OopEncodingHeapMax - class_space; >>> >>> Odd that the solaris compiler can compile this and not the code below. >>> (minor point) I think this should be: >>> >>> zerobased_max = zerobased_max - class_space; >>> >>> Is "size" const in initialize_compressed_heap? (answer appears to be >>> yes) It's useful for reading the calculations that it never changes so >>> I think it should be const (it's a minor point too). >>> >>> I don't have any other comments, I've tried to do the calculations a bit >>> but I know that you've done this yourself. >>> >>> Also, can you run tests with -XX:+UseLargePages? I assume you've run >>> tests with the 4 garbage collectors. This is a tricky piece of code >>> that replaced something else difficult. We've had to change it often to >>> support other features like CDS and compressed class pointers so making >>> it as understandable as possible is worth the extra effort. I think >>> your new design looks cleaner. >>> >>> Thanks, >>> Coleen >>> >>> >>> On 12/24/14, 6:13 PM, Coleen Phillimore wrote: >>>> One note for my first reviewed file. >>>> >>>> On 12/22/14, 5:30 PM, Coleen Phillimore wrote: >>>>> Hi Goetz, >>>>> >>>>> I'm looking at this version of the code >>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >>>>> >>>>> This passes our JPRT tests. Yeah! >>>>> >>>>> It's a significant change, so I'm actually reading the code with the >>>>> patches applied. >>>>> >>>>> In src/share/vm/memory/universe.cpp >>>>> >>>>> Can you put all this code from 742-776 in a new function above >>>>> initialize_heap(), like set_narrow_oop_values()? >>>>> The comment at 689-695 refer to code that's moved to >>>>> virtualspace.cpp. I think it needs to be reworded, updated for >>>>> disjoint oops mode or removed. The comment at 744-749 also refers to >>>>> code that's moved so should be removed also. The last two lines of >>>>> this comment refer to something whose implementation has changed, so >>>>> just remove them. >>>>> >>>>> On line >>>>> >>>>> 750 if ((uint64_t)Universe::heap()->reserved_region().end() > >>>>> UnscaledOopHeapMax) { >>>>> >>>>> Can you make this expression a variable because the same expression >>>>> appears on line 754. >>>>> >>>>> Why can't this code at 742-776 be with the compressed oops mode code >>>>> at line 836-839. If you make this set_narrow_oop_values() function, >>>>> can it be moved together? >>>> Apparently this doesn't work (it crashes on some platforms). I can't >>>> completely follow the control flow that makes this wrong, but ignore >>>> this suggestion. >>>> >>>> Coleen >>>>> Can you remove this comment too? I have no idea what it means. There >>>>> is no CATCH below (?) >>>>> >>>>> // We will never reach the CATCH below since Exceptions::_throw >>>>> will cause >>>>> // the VM to exit if an exception is thrown during initialization >>>>> >>>>> >>>>> ** We should file an RFE to move virtualspace.hpp/cpp to the >>>>> src/share/vm/memory directory! >>>>> >>>>> Sorry, this is partial, I'll continue tomorrow. >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> >>>>> On 12/11/2014 06:18 AM, Lindenmaier, Goetz wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> thanks for looking at this change! >>>>>> >>>>>>> get_attach_addresses_for_disjoint_mode function belongs in >>>>>>> virtualspace.cpp >>>>>> All information about the bounds of the modes comes from universe. >>>>>> This is >>>>>> related to is_disjoint_heap_base_address() and thus I left it in >>>>>> universe. >>>>>> But I'll move it, it can also be regarded as an implementation >>>>>> detail of >>>>>> initialize_compressed_heap(). I can make it static then, so that >>>>>> it's removed >>>>>> in 32 bit build, which is good. >>>>>> >>>>>>> Can you reduce some to at least 100? >>>>>> Done. I agree that's better . (I have a 2560x1440 screen, causing >>>>>> me to have >>>>>> big windows :)) >>>>>> >>>>>>> Why is highest_start not const char* ? >>>>>> Removed the const. >>>>>> >>>>>>> The control flow in initialize_compressed_heap makes no sense to me. >>>>>> Vladimir had the same problem, I added comment at line 493 to make >>>>>> that clearer. >>>>>> One problem of the previous implementation was that, if the os layer >>>>>> returned an address with desired properties, but not at the requested >>>>>> address, it was discarded. >>>>>> To fix this, try_reserve_heap always returns what it allocated, and I >>>>>> check afterwards. >>>>>> Therefore the checks of the next mode include the check of the >>>>>> previous one. So if I try to allocate disjoint, but get unscaled, >>>>>> I will detect that and return the unscaled heap. >>>>>> If I implement a return, I just have to add tests. I don't get rid of >>>>>> other tests. >>>>>> >>>>>>> Also, our 32 bit platforms don't like: 1 * SIZE_64K * SIZE_32G, >>>>>> I changed that to use UCONST64 macro. That should help I guess. >>>>>> I also have these in globalDefinitions_xlc.hpp. Should I move them >>>>>> to globalDefinitions.hpp:200, after definition of K, M and G? >>>>>> >>>>>> I uploaded a new webrev: >>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >>>>>> >>>>>> Best regards, >>>>>> Goetz. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -----Original Message----- >>>>>> From: hotspot-runtime-dev >>>>>> [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of >>>>>> Coleen Phillimore >>>>>> Sent: Mittwoch, 10. Dezember 2014 20:22 >>>>>> To: hotspot-runtime-dev at openjdk.java.net >>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>> "disjoint base" and improve compressed heap handling. >>>>>> >>>>>> >>>>>> Hi Goetz, >>>>>> >>>>>> I have some initial comments, which are much less detailed than >>>>>> Vladimir's comments. I haven't actually processed all the >>>>>> implementation details yet. >>>>>> >>>>>> I think get_attach_addresses_for_disjoint_mode function belongs in >>>>>> virtualspace.cpp and not in universe. I like that the compressed oop >>>>>> reservation logic is moved to virtualspace.cpp. I think this is an >>>>>> improvement over the Universe::preferred_heap_base() logic which was >>>>>> also difficult. >>>>>> >>>>>> The Hotspot coding style doesn't dictate 80 columns anymore (there is >>>>>> debate whether it should have or not, which we try not to have this >>>>>> debate), but I found the very long lines in this code inconsistent with >>>>>> other Hotspot code. I had to expand my window to cover my whole >>>>>> screen. Can you reduce some to at least 100? >>>>>> >>>>>> For example, I aligned these parameters to see better since there >>>>>> are so >>>>>> many (the indentation isn't right in email). >>>>>> >>>>>> void ReservedHeapSpace::try_reserve_range(char *const highest_start, >>>>>> char *lowest_start, >>>>>> size_t >>>>>> attach_point_alignment, >>>>>> char *aligned_HBMA, // >>>>>> HeapBaseMinAddress >>>>>> char *upper_bound, >>>>>> size_t size, >>>>>> size_t alignment, >>>>>> bool large) { >>>>>> >>>>>> Why is highest_start not const char* ? Doesn't char* const >>>>>> highest_start just restrict you from changing the pointer and not what >>>>>> it points to? This seems odd to do. >>>>>> >>>>>> The control flow in initialize_compressed_heap makes no sense to me. >>>>>> It seems like there should be an exit when the best allocation for >>>>>> compression is achieved. But it falls through after >>>>>> try_reserve_range(). I can't figure out why it should fall through.... >>>>>> >>>>>> I was expecting something like: >>>>>> >>>>>> if (PrintCompressedOopsMode && Verbose) { >>>>>> tty->print(" == H E A P B A S E M I N A D D R E S S ==\n"); >>>>>> } >>>>>> get the heap at aligned HeapBaseMinAddress, return if success... >>>>>> >>>>>> if (PrintCompressedOopsMode && Verbose) { >>>>>> tty->print(" == U N S C A L E D ==\n"); >>>>>> } >>>>>> try to get heap base for unscaled access, return if successful >>>>>> >>>>>> etc. >>>>>> >>>>>> >>>>>> You should make this into a little function for each return and for the >>>>>> end of the initialize function, or move it to the ReservedHeapSpace >>>>>> constructor (caller). >>>>>> >>>>>> assert(_base == NULL || >>>>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == _base, >>>>>> "area must be distinguishable from marks for mark-sweep"); >>>>>> assert(_base == NULL || >>>>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() == >>>>>> &_base[size], >>>>>> "area must be distinguishable from marks for mark-sweep"); >>>>>> } >>>>>> >>>>>> I ran this through JPRT and this line didn't compile on macos: >>>>>> >>>>>> const uint64_t num_attempts_to_try = MIN2(HeapSearchSteps, >>>>>> num_attempts_possible); >>>>>> >>>>>> I'm retrying now as: >>>>>> >>>>>> const uint64_t num_attempts_to_try = >>>>>> MIN2((uint64_t)HeapSearchSteps, num_attempts_possible); >>>>>> >>>>>> Sorry for the delay looking at this. This is a big change that I >>>>>> needed >>>>>> to make more time to read. I am pleased that it's a performance >>>>>> improvement. >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>> On 12/8/14, 10:54 AM, Lindenmaier, Goetz wrote: >>>>>>> Hi, >>>>>>> >>>>>>> This is just a ping to gc/rt mailing lists to reach appropriate >>>>>>> people. >>>>>>> >>>>>>> I please need a reviewer from gc or rt, could somebody have a >>>>>>> look at this? >>>>>>> >>>>>>> Short summary: >>>>>>> - new cOops mode disjointbase that allows optimizations on PPC >>>>>>> improving over heapbased >>>>>>> - search for heaps: finds zerobased on sparc Solaris 11 and Aix >>>>>>> - concentrate cOops heap allocation code in one function >>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>> >>>>>>> Please reply only to the original thread in hotspot-dev to keep this >>>>>>> local. >>>>>>> >>>>>>> Thanks and best regards, >>>>>>> Goetz. >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>> Sent: Donnerstag, 4. Dezember 2014 19:44 >>>>>>> To: Lindenmaier, Goetz >>>>>>> Cc: 'hotspot-dev developers' >>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>> >>>>>>> This looks good to me. >>>>>>> >>>>>>> Now we need second review since changes are significant. Preferable >>>>>>> from >>>>>>> GC group since you changed ReservedHeapSpace. They will be affected >>>>>>> most. Review from Runtime is also welcome. >>>>>>> >>>>>>> Thanks, >>>>>>> Vladimir >>>>>>> >>>>>>> On 12/4/14 10:27 AM, Lindenmaier, Goetz wrote: >>>>>>>> Hi Vladimir. >>>>>>>> >>>>>>>> Sorry. I updated the webrev once more. Hope it's fine now. >>>>>>>> At least I can write comments :) >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Goetz >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>> Sent: Thursday, December 04, 2014 5:54 PM >>>>>>>> To: Lindenmaier, Goetz >>>>>>>> Cc: 'hotspot-dev developers' >>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>> >>>>>>>> I spotted an other bug. >>>>>>>> You replaced !_base with _base != NULL when moved code to >>>>>>>> try_reserve_range() - it should be _base == NULL. >>>>>>>> The same problem in asserts: >>>>>>>> >>>>>>>> + assert(_base != NULL || >>>>>>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == >>>>>>>> _base, >>>>>>>> + "area must be distinguishable from marks for mark-sweep"); >>>>>>>> + assert(_base != NULL || >>>>>>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() == >>>>>>>> &_base[size], >>>>>>>> + "area must be distinguishable from marks for mark-sweep"); >>>>>>>> >>>>>>>> >>>>>>>> Also you did not remove _base && in next place: >>>>>>>> >>>>>>>> + (_base && _base + size > zerobased_max))) { // Unscaled >>>>>>>> delivered an arbitrary address. >>>>>>>> >>>>>>>> New comment is good. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Vladimri >>>>>>>> >>>>>>>> On 12/4/14 1:45 AM, Lindenmaier, Goetz wrote: >>>>>>>>> Hi Vladimir, >>>>>>>>> >>>>>>>>>> Add more extending comment explaining that. >>>>>>>>> The comment for try_reserve_heap was meant to explain that. >>>>>>>>> I further added a comment in initialize_compressed_heap(). >>>>>>>>> >>>>>>>>>> You need another parameter to pass UnscaledOopHeapMax or >>>>>>>>>> zerobased_max. >>>>>>>>> Oh, thanks a lot! That's important. Fixed. >>>>>>>>> >>>>>>>>>> I mean that you already checked _base == NULL so on other side >>>>>>>>>> of || _base != NULL - why you need (_base &&) check? >>>>>>>>> Sorry, now I got it. Removed. >>>>>>>>> >>>>>>>>> I updated the webrev: >>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>>> >>>>>>>>> Increment on top of the increment :) >>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/incremental_diffs2.patch >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Goetz. >>>>>>>>> >>>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>> Sent: Mittwoch, 3. Dezember 2014 18:32 >>>>>>>>> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' >>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>> >>>>>>>>> Comments are below. >>>>>>>>> >>>>>>>>> On 12/3/14 5:49 AM, Lindenmaier, Goetz wrote: >>>>>>>>>> Hi Vladimir, >>>>>>>>>> >>>>>>>>>> thanks for looking at the change! See my comments inline below. >>>>>>>>>> >>>>>>>>>> I made a new webrev: >>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>>>> >>>>>>>>>> Incremental changes: >>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/incremental_diffs.patch >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> Goetz. >>>>>>>>>> >>>>>>>>>>> -----Original Message----- >>>>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>>>> Sent: Mittwoch, 3. Dezember 2014 00:46 >>>>>>>>>>> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' >>>>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>>> >>>>>>>>>>> This looks good to me. Someone in runtime/gc have to look on it >>>>>>>>>>> too. >>>>>>>>>>> >>>>>>>>>>> universe.cpp about >>>>>>>>>>> SystemProperty("com.sap.vm.test.compressedOopsMode" >>>>>>>>>>> we have: >>>>>>>>>>> java.vm.info=mixed mode, sharing >>>>>>>>>>> so we can have: >>>>>>>>>>> java.vm.compressedOopsMode=... >>>>>>>>>> Yes, that's good for me. Fixed. >>>>>>>>>> >>>>>>>>>>> I am not expert in properties names but I don't want to have >>>>>>>>>>> 'com.sap' >>>>>>>>>>> in VM's property name. >>>>>>>>>>> virtualspace.cpp: >>>>>>>>>>> Could you fix release() - it does not reset _alignment? >>>>>>>>>> Fixed. >>>>>>>>>> >>>>>>>>>>> In try_reserve_heap(), please, use (base == NULL) instead of >>>>>>>>>>> (!base). >>>>>>>>>>> And you don't need 'return;' in alignment check at the end of >>>>>>>>>>> method. >>>>>>>>>> Fixed. >>>>>>>>>> >>>>>>>>>>> In initialize_compressed_heap() again (!_base). >>>>>>>>>> Fixed. >>>>>>>>>> >>>>>>>>>>> You don't stop (check >>>>>>>>>>> (base == NULL)) after successful unscaled, zerobased, disjointbase >>>>>>>>>>> allocations. You need to separate them with the check: >>>>>>>>>>> >>>>>>>>>>> + >>>>>>>>>>> + } >>>>>>>>>>> + } >>>>>>>>>>> + if (_base == NULL) { >>>>>>>>>>> + >>>>>>>>>>> + if (PrintCompressedOopsMode && Verbose) { >>>>>>>>>>> + tty->print(" == Z E R O B A S E D ==\n"); >>>>>>>>>>> + } >>>>>>>>>>> and so on. >>>>>>>>>> No, I can't and don't want to check for _base != NULL. >>>>>>>>>> I always keep the result of the last try, also if it didn't >>>>>>>>>> fulfil the required properties. >>>>>>>>>> So I take that result and go into the next check. That check >>>>>>>>>> might succeed >>>>>>>>>> with the heap allocated before. >>>>>>>>>> This allows me to separate allocation and placement criteria, >>>>>>>>>> and to have the >>>>>>>>>> placement criteria checked in only one place (per mode). >>>>>>>>>> Only for HeapBaseMinAddress I don't do it that way, I explicitly >>>>>>>>>> call release(). >>>>>>>>>> This way I can enforce mode heapbased. >>>>>>>>> I see what you are saying. It was not clear from comments what is >>>>>>>>> going on. >>>>>>>>> Add more extending comment explaining that. >>>>>>>>> >>>>>>>>>>> num_attempts calculation and while() loop are similar in >>>>>>>>>>> unscaled and >>>>>>>>>>> zerobased cases. Could you move it into a separate method? >>>>>>>>>> I can do that, but I don't like it as I have to pass in 7 >>>>>>>>>> parameters. >>>>>>>>> You need an other parameter to pass UnscaledOopHeapMax or >>>>>>>>> zerobased_max. >>>>>>>>> >>>>>>>>>> That makes the code not much more readable. The function will >>>>>>>>>> look like this: >>>>>>>>> I think initialize_compressed_heap() is more readable now. >>>>>>>>> >>>>>>>>>> void ReserveHeapSpace::try_reserve_range(char *const >>>>>>>>>> highest_start, char *lowest_start, size_t attach_point_alignment, >>>>>>>>>> char >>>>>>>>>> *aligned_HBMA, size_t size, size_t alignment, bool large) { >>>>>>>>>> guarantee(HeapSearchSteps > 0, "Don't set HeapSearchSteps >>>>>>>>>> to 0"); >>>>>>>>>> >>>>>>>>>> const size_t attach_range = highest_start - lowest_start; >>>>>>>>>> // Cap num_attempts at possible number. >>>>>>>>>> // At least one is possible even for 0 sized attach range. >>>>>>>>>> const uint64_t num_attempts_possible = (attach_range / >>>>>>>>>> attach_point_alignment) + 1; >>>>>>>>>> const uint64_t num_attempts_to_try = >>>>>>>>>> MIN2(HeapSearchSteps, num_attempts_possible); >>>>>>>>>> >>>>>>>>>> const size_t stepsize = align_size_up(attach_range / >>>>>>>>>> num_attempts_to_try, attach_point_alignment); >>>>>>>>>> >>>>>>>>>> // Try attach points from top to bottom. >>>>>>>>>> char* attach_point = highest_start; >>>>>>>>>> while (attach_point >= lowest_start && >>>>>>>>>> attach_point <= highest_start && // Avoid wrap >>>>>>>>>> around. >>>>>>>>>> (!_base || _base < aligned_HBMA || _base + size > >>>>>>>>>> (char *)UnscaledOopHeapMax)) { >>>>>>>>>> try_reserve_heap(size, alignment, large, attach_point); >>>>>>>>>> attach_point -= stepsize; >>>>>>>>>> } >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> In disjointbase while() condition no need for _base second check: >>>>>>>>>>> + (_base == NULL || >>>>>>>>>>> + ((_base + size > (char *)OopEncodingHeapMax) && >>>>>>>>>> I need this for the same reason as above: This is the check for >>>>>>>>>> successful allocation. >>>>>>>>> I mean that you already checked _base == NULL so on other side of >>>>>>>>> || _base != NULL - why you need (_base &&) check? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Vladimir >>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Vladimir >>>>>>>>>> >>>>>>>>>> On 11/21/14 5:31 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I prepared a new webrev trying to cover all the issues >>>>>>>>>>> mentioned below. >>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.01/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I moved functionality from os.cpp and universe.cpp into >>>>>>>>>>> ReservedHeapSpace::initialize_compressed_heap(). >>>>>>>>>>> This class offers to save _base and _special, which I would >>>>>>>>>>> have to reimplement >>>>>>>>>>> if I had improved the methods I had added to os.cpp to also >>>>>>>>>>> allocate large page >>>>>>>>>>> heaps. >>>>>>>>>>> Anyways, I think this class is the right place to gather code >>>>>>>>>>> trying to reserve >>>>>>>>>>> the heap. >>>>>>>>>>> Also, I get along without setting the shift, base, >>>>>>>>>>> implicit_null_check etc. fields >>>>>>>>>>> of Universe, so there is no unnecessary calling back and forth >>>>>>>>>>> between the two >>>>>>>>>>> classes. >>>>>>>>>>> Universe gets the heap back, and then sets the properties it >>>>>>>>>>> needs to configure >>>>>>>>>>> the compressed oops. >>>>>>>>>>> All code handling the noaccess prefix is in a single method, too. >>>>>>>>>>> >>>>>>>>>>> Best regards, >>>>>>>>>>> Goetz. >>>>>>>>>>> >>>>>>>>>>> Btw, I had to workaround a SS12u1 problem: it wouldn't compile >>>>>>>>>>> char * x = (char*)UnscaledOopHeapMax - size in 32-bit mode. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -----Original Message----- >>>>>>>>>>> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] >>>>>>>>>>> On Behalf Of Lindenmaier, Goetz >>>>>>>>>>> Sent: Montag, 17. November 2014 09:33 >>>>>>>>>>> To: 'Vladimir Kozlov'; 'hotspot-dev at openjdk.java.net' >>>>>>>>>>> Subject: RE: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>>> >>>>>>>>>>> Hi Vladimir, >>>>>>>>>>> >>>>>>>>>>>> It is very significant rewriting and it takes time to evaluate >>>>>>>>>>>> it. >>>>>>>>>>> Yes, I know ... and I don't want to push, but nevertheless a ping >>>>>>>>>>> can be useful sometimes. Thanks a lot for looking at it. >>>>>>>>>>> >>>>>>>>>>>> And I would not say it is simpler then before :) >>>>>>>>>>> If I fix what you propose it's gonna get even more simple ;) >>>>>>>>>>>> These is what I found so far. >>>>>>>>>>>> The idea to try to allocate in a range instead of just below >>>>>>>>>>>> UnscaledOopHeapMax or OopEncodingHeapMax is good. So I would >>>>>>>>>>>> ask to do >>>>>>>>>>>> several attempts (3?) on non_PPC64 platforms too. >>>>>>>>>>> Set to 3. >>>>>>>>>>> >>>>>>>>>>>> It is matter of preference but I am not comfortable with >>>>>>>>>>>> switch in loop. >>>>>>>>>>>> For me sequential 'if (addr == 0)' checks is simpler. >>>>>>>>>>> I'll fix this. >>>>>>>>>>> >>>>>>>>>>>> One thing worries me that you release found space and try to >>>>>>>>>>>> get it >>>>>>>>>>>> again with ReservedHeapSpace. Is it possible to add new >>>>>>>>>>>> ReservedHeapSpace ctor which simple use already allocated space? >>>>>>>>>>> This was to keep diff's small, but I also think a new >>>>>>>>>>> constructor is good. >>>>>>>>>>> I'll fix this. >>>>>>>>>>> >>>>>>>>>>>> The next code in ReservedHeapSpace() is hard to understand (): >>>>>>>>>>>> (UseCompressedOops && (requested_address == NULL || >>>>>>>>>>> requested_address+size > (char*)OopEncodingHeapMax) ? >>>>>>>>>>>> may be move all this into noaccess_prefix_size() and add >>>>>>>>>>>> comments. >>>>>>>>>>> I have to redo this anyways if I make new constructors. >>>>>>>>>>> >>>>>>>>>>>> Why you need prefix when requested_address == NULL? >>>>>>>>>>> If we allocate with NULL, we most probably will get a heap where >>>>>>>>>>> base != NULL and thus need a noaccess prefix. >>>>>>>>>>> >>>>>>>>>>>> Remove next comment in universe.cpp: >>>>>>>>>>>> // SAPJVM GL 2014-09-22 >>>>>>>>>>> Removed. >>>>>>>>>>> >>>>>>>>>>>> Again you will release space so why bother to include space >>>>>>>>>>>> for classes?: >>>>>>>>>>>> + // For small heaps, save some space for compressed >>>>>>>>>>>> class pointer >>>>>>>>>>>> + // space so it can be decoded with no base. >>>>>>>>>>> This was done like this before. We must assure the upper bound >>>>>>>>>>> of the >>>>>>>>>>> heap is low enough that the compressed class space still fits >>>>>>>>>>> in there. >>>>>>>>>>> >>>>>>>>>>> virtualspace.cpp >>>>>>>>>>> >>>>>>>>>>>> With new code size+noaccess_prefix could be requested. But >>>>>>>>>>>> later it is >>>>>>>>>>>> not used if WIN64_ONLY(&& UseLargePages) and you will have empty >>>>>>>>>>>> non-protected page below heap. >>>>>>>>>>> There's several points to this: >>>>>>>>>>> * Also if not protectable, the heap base has to be below >>>>>>>>>>> the real start of the >>>>>>>>>>> heap. Else the first object in the heap will be >>>>>>>>>>> compressed to 'null' >>>>>>>>>>> and decompression will fail. >>>>>>>>>>> * If we don't reserve the memory other stuff can end up >>>>>>>>>>> in this space. On >>>>>>>>>>> errors, if would be quite unexpected to find memory >>>>>>>>>>> there. >>>>>>>>>>> * To get a heap for the new disjoint mode I must control >>>>>>>>>>> the size of this. >>>>>>>>>>> Requesting a heap starting at (aligned base + >>>>>>>>>>> prefix) is more likely to fail. >>>>>>>>>>> * The size for the prefix must anyways be considered >>>>>>>>>>> when deciding whether the >>>>>>>>>>> heap is small enough to run with compressed oops. >>>>>>>>>>> So distinguishing the case where we really can omit this would >>>>>>>>>>> require >>>>>>>>>>> quite some additional checks everywhere, and I thought it's not >>>>>>>>>>> worth it. >>>>>>>>>>> >>>>>>>>>>> matcher.hpp >>>>>>>>>>> >>>>>>>>>>>> Universe::narrow_oop_use_implicit_null_checks() should be true >>>>>>>>>>>> for such >>>>>>>>>>>> case too. So you can add new condition with || to existing >>>>>>>>>>>> ones. The >>>>>>>>>>>> only condition you relax is base != NULL. Right? >>>>>>>>>>> Yes, that's how it's intended. >>>>>>>>>>> >>>>>>>>>>> arguments.* files >>>>>>>>>>> >>>>>>>>>>>> Why you need PropertyList_add changes. >>>>>>>>>>> Oh, the code using it got lost. I commented on this in the >>>>>>>>>>> description in the webrev. >>>>>>>>>>> "To more efficiently run expensive tests in various compressed >>>>>>>>>>> oop modes, we set a property with the mode the VM is running >>>>>>>>>>> in. So far it's called "com.sap.vm.test.compressedOopsMode" >>>>>>>>>>> better suggestions are welcome (and necessary I guess). Our >>>>>>>>>>> long running tests that are supposed to run in a dedicated >>>>>>>>>>> compressed oop mode check this property and abort themselves if >>>>>>>>>>> it's not the expected mode." >>>>>>>>>>> When I know about the heap I do >>>>>>>>>>> Arguments::PropertyList_add(new >>>>>>>>>>> SystemProperty("com.sap.vm.test.compressedOopsMode", >>>>>>>>>>> narrow_oop_mode_to_string(narrow_oop_mode()), >>>>>>>>>>> false)); >>>>>>>>>>> in universe.cpp. >>>>>>>>>>> On some OSes it's deterministic which modes work, there we >>>>>>>>>>> don't start such tests. >>>>>>>>>>> Others, as you mentioned OSX, are very indeterministic. Here >>>>>>>>>>> we save testruntime with this. >>>>>>>>>>> But it's not that important. >>>>>>>>>>> We can still parse the PrintCompresseOopsMode output after the >>>>>>>>>>> test and discard the >>>>>>>>>>> run. >>>>>>>>>>> >>>>>>>>>>>> Do you have platform specific changes? >>>>>>>>>>> Yes, for ppc and aix. I'll submit them once this is in. >>>>>>>>>>> >>>>>>>>>>> From your other mail: >>>>>>>>>>>> One more thing. You should allow an allocation in the range >>>>>>>>>>>> when returned from OS allocated address does not match >>>>>>>>>>>> requested address. We had such cases on OSX, for example, when >>>>>>>>>>>> OS allocates at different address but still inside range. >>>>>>>>>>> Good point. I'll fix that in os::attempt_reserve_memory_in_range. >>>>>>>>>>> >>>>>>>>>>> I'll ping again once a new webrev is done! >>>>>>>>>>> >>>>>>>>>>> Best regards, >>>>>>>>>>> Goetz. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 11/10/14 6:57 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> I need to improve a row of things around compressed oops heap >>>>>>>>>>>> handling >>>>>>>>>>>> to achieve good performance on ppc. >>>>>>>>>>>> I prepared a first webrev for review: >>>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> A detailed technical description of the change is in the >>>>>>>>>>>> webrev and according bug. >>>>>>>>>>>> >>>>>>>>>>>> If requested, I will split the change into parts with more >>>>>>>>>>>> respective less impact on >>>>>>>>>>>> non-ppc platforms. >>>>>>>>>>>> >>>>>>>>>>>> The change is derived from well-tested code in our VM. >>>>>>>>>>>> Originally it was >>>>>>>>>>>> crafted to require the least changes of VM coding, I changed >>>>>>>>>>>> it to be better >>>>>>>>>>>> streamlined with the VM. >>>>>>>>>>>> I tested this change to deliver heaps at about the same >>>>>>>>>>>> addresses as before. >>>>>>>>>>>> Heap addresses mostly differ in lower bits. In some cases >>>>>>>>>>>> (Solaris 5.11) a heap >>>>>>>>>>>> in a better compressed oops mode is found, though. >>>>>>>>>>>> I ran (and adapted) test/runtime/CompressedOops and >>>>>>>>>>>> gc/arguments/TestUseCompressedOops*. >>>>>>>>>>>> >>>>>>>>>>>> Best regards, >>>>>>>>>>>> Goetz. >>>>>>>>>>>> >>>>>>>>>>>> From coleen.phillimore at oracle.com Mon Jan 5 17:01:03 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 05 Jan 2015 12:01:03 -0500 Subject: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. In-Reply-To: <54AAC086.2060600@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF264E2@DEWDFEMB12A.global.corp.sap> <547E4F3B.2090501@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF35E66@DEWDFEMB12A.global.corp.sap> <547F4914.1030904@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF36139@DEWDFEMB12A.global.corp.sap> <54809196.7030306@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF39BA5@DEWDFEMB12A.global.corp.sap> <5480AB58.80604@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF3D6BB@DEWDFEMB12A.global.corp.sap> <54889D61.30905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF3E1C5@DEWDFEMB12A.global.corp.sap> <54989B9F.2070903@oracle.com> <549B48A3.1070303@oracle.com> <54A2089D.1020105@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF53F70@DEWDFEMB12A.global.corp.sap> <54A2C966.5010306@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF67EB6@DEWDFEMB12A.global.corp.sap> <54A6C6C3.1050806@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF69210@DEWDFEMB12A.global.corp.sap> <54AAC086.2060600@oracle.com> Message-ID: <54AAC34F.2050807@oracle.com> On 1/5/15, 11:49 AM, Coleen Phillimore wrote: > > On 1/5/15, 4:24 AM, Lindenmaier, Goetz wrote: >> Hi Coleen, >> >> It would be great if you could sponsor this change. > > I see no other comments so I'll push it today. I am planning to push this to hs-rt, btw. Coleen >> >>> The metaspace case probably can be fixed to use the >>> try_reserve_heap() code maybe. >> We wondered why the compressed class space is always above the heap. >> If the heap is too big for unscaled, the compressed class space could >> be placed >> in the lower 4G. > > The reason that we chose the compressed class space on the top of the > heap is for a couple reasons. The time spent compressing and > decompressing oops is more critical than the time spent doing the same > for the _klass pointer, so we wanted to make the system more likely to > get unscaled compressed oops, or even zero based compressed oops. > > On solaris, the space below the heap is limited because by default > malloc uses it so we were limited in low address memory that we can > use. We want the compressed class space code not be platform > dependent. We also need to have the CDS archive at a fixed address > near or adjacent to the compressed class space, so were running out at > the bottom of the heap. > > The compressed class space is fixed at startup (just like PermGen was) > and the classes are variable sized, so we had to pick a size fairly > large so that it's unlikely for applications to run out. If above 32G, > we might be able to implement code to allocate a new space which would > use the same compression algorithm. We haven't had a request for that > but it's something we could do. > > >> The aarch port brings some changes here, and we would like to >> investigate >> this on top of the aarch changes, if that's fine with you. > > So one thing I prototyped and really wanted to check in was what we > called indirect class pointers. Instead of allocating classes in a > fixed class space, we allocate a fixed array of Klass pointers and the > objects in the heap point to this. It had a 8% performance > degradation in specbb2008 compiler.compiler (iirc, or something like > that). > > This solved so many problems though. If you investigated this sort of > approach, I'd be very grateful. I haven't had time to go back to > figure out why this had this degradation (we thought it was Java > vtable accesses). > > Thanks, > Coleen >> >> Thanks a lot for going through this complex review! >> >> Best regards, >> Goetz. >> >> -----Original Message----- >> From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] >> Sent: Freitag, 2. Januar 2015 17:27 >> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >> "disjoint base" and improve compressed heap handling. >> >> >> On 1/2/15, 10:09 AM, Lindenmaier, Goetz wrote: >>> Hi Coleen, >>> >>> I looked at the cleanups and found both not straight forward to >>> implement. >>> >>> 1) Merging ReservedHeap::initialize and >>> ReservedHeapSpace::try_reserve_heap. >>> The new code calls initialize() with requested_address = NULL, >>> but metaspace.cpp >>> passes a real requested_address. Therefore I can not clean up the >>> calls to >>> failed_to_reserve_as_requested() in initialize(). Without that >>> cleanup, I need to pass flags >>> into initialize() to configure all the different behaviours I would >>> need in a >>> merged case. >>> I think with 'large' and 'requested_address' there is already enough >>> cases >>> in that method, so that I don't want to add more. >> Okay, this is fine. More flags to special case functions aren't good >> either. Can you file an RFE to look into refactoring so that the >> duplicate code can be removed though? The metaspace case probably can >> be fixed to use the try_reserve_heap() code maybe. >> >>> 2) Moving _noaccess_prefix to ReservedHeapSpace >>> >>> This does not work as ReservedHeapSpace is casted to >>> ReservedSpace and returned by value. So ReservedHeapSpace can not have >>> fields or virtual methods. >>> It would be probably better to change the code to return >>> ReservedHeapSpace >>> by pointer, or, at least under that type, but I think that exceeds >>> the scope of this change. >> Too bad. >> >>> So is it ok if I leave the change as is wrt. to these cleanups? >> Yes, this is fine. >>> I extended test/runtime/CompressedOops/UseCompressedOops.java >>> to combine the testcases of the different mode with the GCs and >>> UseLargePages. >>> I added the changes in the test to >>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >> Is .03 with the additional changes? If so, you should have made an .04. >> >> I reviewed this again and it looks good. I think Vladimir has reviewed >> an earlier version. I can sponsor this assuming there are no more >> comments. >> >> Coleen >>> Best regards, >>> Goetz. >>> >>> >>> >>> >>> >>> -----Original Message----- >>> From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] >>> Sent: Dienstag, 30. Dezember 2014 16:49 >>> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net >>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>> "disjoint base" and improve compressed heap handling. >>> >>> >>> On 12/30/14, 8:42 AM, Lindenmaier, Goetz wrote: >>>> Hi Coleen, >>>> >>>>> disjoint >>>> It tries to say that the bits used for the base (36-63) are >>>> different from >>>> those used for the oops / used as offset into the heap (0-35). >>>> I chose the name according to this translation >>>> http://dict.leo.org/#/search=disjoint&searchLoc=0&resultOrder=basic&multiwordShowSingle=on >>>> >>>> Both the german adjective and verb fit to describe the property of >>>> the base. >>>> I can add the following to the docu of the mode in universe.hpp:358 >>>> // Disjoint: Bits used in base are disjoint from bits used >>>> // for oops ==> oop = (cOop << 3) | base. One can disjoint >>>> // the bits of an oop into base and compressed oop. >>> Yes, that helps. It's a good distinct word so its good that we can >>> make >>> sense in the documentation. >>>>> (I filed a bug to move these to the memory directory) >>>> Good idea! >>>> >>>>> 272 void ReservedSpace::establish_noaccess_prefix() { >>>>> This function should be in ReservedHeapSpace >>>> Yes, I moved it. I also would like to move the field >>>> _noaccess_prefix, >>>> but I would have to change a lot of code, like >>>> ReservedSpace::release(). >>>> The problem is that _base is not the real base in ReservedHeapSpace. >>>> Probably ReservedHeapSpace should have heap_size() and heap_base() >>>> giving the correcte values. >>>> I'll try to come up with a patch on top of this, maybe we should do it >>>> in a separate bug. >>> I just moved _noaccess_prefix and added a virtual >>> ReservedHeapSpace::release() that adjusted the base and size with >>> noaccess_prefix. I don't think the footprint of a vtable or virtual >>> call makes any difference. >>>>> Can you have arguments.cpp verify_range of HeapSearchSteps so this >>>>> can't >>>>> happen? >>>> Done. >>>> >>>>> Rather move this to the end of ReservedHeapSpace constructor. >>>>> assert(_base == NULL || ... >>>> Yes, you are right, it should not be there twice. I moved it into the >>>> constructor. It is only relevant for heaps, right? >>> Yes. >>>>> Code in try_reserve_heap() is mostly a copy of >>>>> ReservedSpace::initialize() >>>> You are right, requested_address in initialize is dead now. I'll >>>> remove that >>>> and try to merge the two functions. >>>> >>>> I removed the verbose printing. Anyways, it's mostly useful when >>>> developing this. >>> Thanks. I was trying to find some way to move the printing so it would >>> make sense to me but ended up giving up on that. >>>>> aligned_HBMA >>>> Changed to aligned_heap_base_min_address. >>>> >>>>> Is "size" const in initialize_compressed_heap? >>>> Yes. Added 'const'. >>>> >>>> I'll run tests with +UseLargePages. >>>> >>>> I had verified that the heap is at the exact same position for the >>>> code in >>>> my first RFR, and with HeapSearchSteps=1 to make sure I don't break >>>> anything. >>>> Vladimir asked me to increase that to 3. So far I still didn't >>>> find a case where >>>> this changed anything. >>>> (But I know from SAP JVM, which is tested on all the OS variants we >>>> support, >>>> that there are cases where this can lead to a better - but then >>>> different - heap >>>> placement.) >>>> The only case where there is always a difference is Solaris 5.11 which >>>> now can allocate zerobased memory. >>>> The code passes the CDS jtreg tests. >>>> >>>> I updated the webrev.03 with the minor changes. >>>> I'll come up with a new one with the cleanups of Reserved(Heap)Space >>>> (_noaccess_prefix and redundant functions). >>>> >>>> Thanks for this thorough review!! >>> Thank you for your willingness to make these changes. >>> >>> Happy New Year! >>> Coleen >>> >>>> Happy new year, >>>> Goetz. >>>> >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: hotspot-runtime-dev >>>> [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of >>>> Coleen Phillimore >>>> Sent: Dienstag, 30. Dezember 2014 03:06 >>>> To: hotspot-runtime-dev at openjdk.java.net >>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>> "disjoint base" and improve compressed heap handling. >>>> >>>> >>>> Hi, I've reviewed more code. >>>> >>>> I'm sorry about the following comment that should have been asked >>>> before. The name of the mode is very confusing. I can't figure out >>>> why >>>> it's called "disjoint". I keep looking for a second heap base or some >>>> code that makes the heap discontinuous. The heap base is really >>>> specially aligned, right? So I went to the thesaurus.com: >>>> >>>> http://www.thesaurus.com/browse/aligned >>>> >>>> and "disjoin" is an antonym. Why was "disjoint" chosen? From the >>>> code, it looks like "aligned" or "scaled" are more descriptive but >>>> they're overused words in the code. But ScaledBasedNarrowOop seems >>>> more >>>> descriptive than Disjoint. >>>> >>>> In virtualspace.cpp (I filed a bug to move these to the memory >>>> directory) >>>> >>>> 272 void ReservedSpace::establish_noaccess_prefix() { >>>> >>>> >>>> This function should be in ReservedHeapSpace since ReservedSpace never >>>> does this (and we never want it to). The function it replaced was in >>>> ReservedSpace and that was wrong. Now that ReservedHeapSpace has >>>> compressed oops logic in it, I think this function should be moved to >>>> this class. Actually, I've modified your patch to move the >>>> _noaccess_prefix field to ReservedHeapSpace. Maybe that's too much >>>> change to your patch but since the handling of noaccess_prefix is >>>> changed, it seemed like the thing to do. >>>> >>>> Can you have arguments.cpp verify_range of HeapSearchSteps so this >>>> can't >>>> happen? >>>> >>>> guarantee(HeapSearchSteps > 0, "Don't set HeapSearchSteps to >>>> 0"); >>>> >>>> Can this be a function like check_encoding_for_mark_sweep() , or >>>> something like that? It appears twice. Rather move this to the >>>> end of >>>> ReservedHeapSpace constructor. >>>> >>>> assert(_base == NULL || >>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == _base, >>>> "area must be distinguishable from marks for >>>> mark-sweep"); >>>> assert(_base == NULL || >>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() == >>>> &_base[size], >>>> "area must be distinguishable from marks for >>>> mark-sweep"); >>>> >>>> Code in try_reserve_heap() is a mostly a copy of >>>> ReservedSpace::initialize(), except the first version has a call to >>>> failed_to_reserve_as_requested which looks like it's handling the case >>>> for not being able to reserve the requested address for compressed >>>> oops >>>> which seems like you don't want this. This is very confusing. There >>>> shouldn't be two of these. >>>> >>>> It looks like in the compressed oops case, you call >>>> ReservedSpace::initialize() only with NULL requested address so >>>> failed_to_reserve_as_requested is dead code with this change. So it >>>> seems like try_reserve_heap() could be ReservedSpace::initialize() >>>> with >>>> the failed_to_reserve_as_requested eliminated and a special case for >>>> handling unaligned addresses. >>>> >>>> The logic in initialize_compressed_heap() still doesn't make sense to >>>> me. If the first try_reserve_range() succeeds at getting an unscaled >>>> compressed oop base, then does PrintCompressedOopsMode && Verbose >>>> still >>>> print: >>>> >>>> tty->print(" == U N S C A L E D ==\n"); >>>> tty->print(" == Z E R O B A S E D ==\n"); >>>> tty->print(" == D I S J O I N T B A S E ==\n"); >>>> tty->print(" == H E A P B A S E D ==\n"); >>>> >>>> Why would you print all 4? It's the printing that makes this >>>> confusing! Now I see how the next case checks that the base >>>> address is >>>> a better allocation. If you changed the printing to a comment like: >>>> >>>> // UnscaledMode after testing previous base address is not better (or >>>> something). >>>> >>>> Actually, you have these comments. I think the printing should be >>>> removed because it's the opposite of helpful. >>>> >>>> I had trouble reading the parameter aligned_HBMA, can you change to >>>> aligned_heap_base_min (I know it's longer but there's too many >>>> calculations here that words will make easier to read). Or just >>>> heap_base_min_address. >>>> >>>> zerobased_max = (char *)OopEncodingHeapMax - class_space; >>>> >>>> Odd that the solaris compiler can compile this and not the code below. >>>> (minor point) I think this should be: >>>> >>>> zerobased_max = zerobased_max - class_space; >>>> >>>> Is "size" const in initialize_compressed_heap? (answer appears to be >>>> yes) It's useful for reading the calculations that it never >>>> changes so >>>> I think it should be const (it's a minor point too). >>>> >>>> I don't have any other comments, I've tried to do the calculations >>>> a bit >>>> but I know that you've done this yourself. >>>> >>>> Also, can you run tests with -XX:+UseLargePages? I assume you've run >>>> tests with the 4 garbage collectors. This is a tricky piece of code >>>> that replaced something else difficult. We've had to change it >>>> often to >>>> support other features like CDS and compressed class pointers so >>>> making >>>> it as understandable as possible is worth the extra effort. I think >>>> your new design looks cleaner. >>>> >>>> Thanks, >>>> Coleen >>>> >>>> >>>> On 12/24/14, 6:13 PM, Coleen Phillimore wrote: >>>>> One note for my first reviewed file. >>>>> >>>>> On 12/22/14, 5:30 PM, Coleen Phillimore wrote: >>>>>> Hi Goetz, >>>>>> >>>>>> I'm looking at this version of the code >>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >>>>>> >>>>>> >>>>>> This passes our JPRT tests. Yeah! >>>>>> >>>>>> It's a significant change, so I'm actually reading the code with the >>>>>> patches applied. >>>>>> >>>>>> In src/share/vm/memory/universe.cpp >>>>>> >>>>>> Can you put all this code from 742-776 in a new function above >>>>>> initialize_heap(), like set_narrow_oop_values()? >>>>>> The comment at 689-695 refer to code that's moved to >>>>>> virtualspace.cpp. I think it needs to be reworded, updated for >>>>>> disjoint oops mode or removed. The comment at 744-749 also refers to >>>>>> code that's moved so should be removed also. The last two lines of >>>>>> this comment refer to something whose implementation has changed, so >>>>>> just remove them. >>>>>> >>>>>> On line >>>>>> >>>>>> 750 if ((uint64_t)Universe::heap()->reserved_region().end() > >>>>>> UnscaledOopHeapMax) { >>>>>> >>>>>> Can you make this expression a variable because the same expression >>>>>> appears on line 754. >>>>>> >>>>>> Why can't this code at 742-776 be with the compressed oops mode code >>>>>> at line 836-839. If you make this set_narrow_oop_values() function, >>>>>> can it be moved together? >>>>> Apparently this doesn't work (it crashes on some platforms). I can't >>>>> completely follow the control flow that makes this wrong, but ignore >>>>> this suggestion. >>>>> >>>>> Coleen >>>>>> Can you remove this comment too? I have no idea what it means. There >>>>>> is no CATCH below (?) >>>>>> >>>>>> // We will never reach the CATCH below since Exceptions::_throw >>>>>> will cause >>>>>> // the VM to exit if an exception is thrown during >>>>>> initialization >>>>>> >>>>>> >>>>>> ** We should file an RFE to move virtualspace.hpp/cpp to the >>>>>> src/share/vm/memory directory! >>>>>> >>>>>> Sorry, this is partial, I'll continue tomorrow. >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>> >>>>>> On 12/11/2014 06:18 AM, Lindenmaier, Goetz wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> thanks for looking at this change! >>>>>>> >>>>>>>> get_attach_addresses_for_disjoint_mode function belongs in >>>>>>>> virtualspace.cpp >>>>>>> All information about the bounds of the modes comes from universe. >>>>>>> This is >>>>>>> related to is_disjoint_heap_base_address() and thus I left it in >>>>>>> universe. >>>>>>> But I'll move it, it can also be regarded as an implementation >>>>>>> detail of >>>>>>> initialize_compressed_heap(). I can make it static then, so that >>>>>>> it's removed >>>>>>> in 32 bit build, which is good. >>>>>>> >>>>>>>> Can you reduce some to at least 100? >>>>>>> Done. I agree that's better . (I have a 2560x1440 screen, causing >>>>>>> me to have >>>>>>> big windows :)) >>>>>>> >>>>>>>> Why is highest_start not const char* ? >>>>>>> Removed the const. >>>>>>> >>>>>>>> The control flow in initialize_compressed_heap makes no sense >>>>>>>> to me. >>>>>>> Vladimir had the same problem, I added comment at line 493 to make >>>>>>> that clearer. >>>>>>> One problem of the previous implementation was that, if the os >>>>>>> layer >>>>>>> returned an address with desired properties, but not at the >>>>>>> requested >>>>>>> address, it was discarded. >>>>>>> To fix this, try_reserve_heap always returns what it allocated, >>>>>>> and I >>>>>>> check afterwards. >>>>>>> Therefore the checks of the next mode include the check of the >>>>>>> previous one. So if I try to allocate disjoint, but get unscaled, >>>>>>> I will detect that and return the unscaled heap. >>>>>>> If I implement a return, I just have to add tests. I don't get >>>>>>> rid of >>>>>>> other tests. >>>>>>> >>>>>>>> Also, our 32 bit platforms don't like: 1 * SIZE_64K * SIZE_32G, >>>>>>> I changed that to use UCONST64 macro. That should help I guess. >>>>>>> I also have these in globalDefinitions_xlc.hpp. Should I move them >>>>>>> to globalDefinitions.hpp:200, after definition of K, M and G? >>>>>>> >>>>>>> I uploaded a new webrev: >>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >>>>>>> >>>>>>> >>>>>>> Best regards, >>>>>>> Goetz. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: hotspot-runtime-dev >>>>>>> [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of >>>>>>> Coleen Phillimore >>>>>>> Sent: Mittwoch, 10. Dezember 2014 20:22 >>>>>>> To: hotspot-runtime-dev at openjdk.java.net >>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>> >>>>>>> >>>>>>> Hi Goetz, >>>>>>> >>>>>>> I have some initial comments, which are much less detailed than >>>>>>> Vladimir's comments. I haven't actually processed all the >>>>>>> implementation details yet. >>>>>>> >>>>>>> I think get_attach_addresses_for_disjoint_mode function belongs in >>>>>>> virtualspace.cpp and not in universe. I like that the >>>>>>> compressed oop >>>>>>> reservation logic is moved to virtualspace.cpp. I think this is an >>>>>>> improvement over the Universe::preferred_heap_base() logic which >>>>>>> was >>>>>>> also difficult. >>>>>>> >>>>>>> The Hotspot coding style doesn't dictate 80 columns anymore >>>>>>> (there is >>>>>>> debate whether it should have or not, which we try not to have this >>>>>>> debate), but I found the very long lines in this code >>>>>>> inconsistent with >>>>>>> other Hotspot code. I had to expand my window to cover my whole >>>>>>> screen. Can you reduce some to at least 100? >>>>>>> >>>>>>> For example, I aligned these parameters to see better since there >>>>>>> are so >>>>>>> many (the indentation isn't right in email). >>>>>>> >>>>>>> void ReservedHeapSpace::try_reserve_range(char *const >>>>>>> highest_start, >>>>>>> char *lowest_start, >>>>>>> size_t >>>>>>> attach_point_alignment, >>>>>>> char >>>>>>> *aligned_HBMA, // >>>>>>> HeapBaseMinAddress >>>>>>> char *upper_bound, >>>>>>> size_t size, >>>>>>> size_t alignment, >>>>>>> bool large) { >>>>>>> >>>>>>> Why is highest_start not const char* ? Doesn't char* const >>>>>>> highest_start just restrict you from changing the pointer and >>>>>>> not what >>>>>>> it points to? This seems odd to do. >>>>>>> >>>>>>> The control flow in initialize_compressed_heap makes no sense to >>>>>>> me. >>>>>>> It seems like there should be an exit when the best allocation for >>>>>>> compression is achieved. But it falls through after >>>>>>> try_reserve_range(). I can't figure out why it should fall >>>>>>> through.... >>>>>>> >>>>>>> I was expecting something like: >>>>>>> >>>>>>> if (PrintCompressedOopsMode && Verbose) { >>>>>>> tty->print(" == H E A P B A S E M I N A D D R E S S >>>>>>> ==\n"); >>>>>>> } >>>>>>> get the heap at aligned HeapBaseMinAddress, return if >>>>>>> success... >>>>>>> >>>>>>> if (PrintCompressedOopsMode && Verbose) { >>>>>>> tty->print(" == U N S C A L E D ==\n"); >>>>>>> } >>>>>>> try to get heap base for unscaled access, return if >>>>>>> successful >>>>>>> >>>>>>> etc. >>>>>>> >>>>>>> >>>>>>> You should make this into a little function for each return and >>>>>>> for the >>>>>>> end of the initialize function, or move it to the ReservedHeapSpace >>>>>>> constructor (caller). >>>>>>> >>>>>>> assert(_base == NULL || >>>>>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == >>>>>>> _base, >>>>>>> "area must be distinguishable from marks for >>>>>>> mark-sweep"); >>>>>>> assert(_base == NULL || >>>>>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() >>>>>>> == >>>>>>> &_base[size], >>>>>>> "area must be distinguishable from marks for >>>>>>> mark-sweep"); >>>>>>> } >>>>>>> >>>>>>> I ran this through JPRT and this line didn't compile on macos: >>>>>>> >>>>>>> const uint64_t num_attempts_to_try = MIN2(HeapSearchSteps, >>>>>>> num_attempts_possible); >>>>>>> >>>>>>> I'm retrying now as: >>>>>>> >>>>>>> const uint64_t num_attempts_to_try = >>>>>>> MIN2((uint64_t)HeapSearchSteps, num_attempts_possible); >>>>>>> >>>>>>> Sorry for the delay looking at this. This is a big change that I >>>>>>> needed >>>>>>> to make more time to read. I am pleased that it's a performance >>>>>>> improvement. >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> >>>>>>> On 12/8/14, 10:54 AM, Lindenmaier, Goetz wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> This is just a ping to gc/rt mailing lists to reach appropriate >>>>>>>> people. >>>>>>>> >>>>>>>> I please need a reviewer from gc or rt, could somebody have a >>>>>>>> look at this? >>>>>>>> >>>>>>>> Short summary: >>>>>>>> - new cOops mode disjointbase that allows optimizations on PPC >>>>>>>> improving over heapbased >>>>>>>> - search for heaps: finds zerobased on sparc Solaris 11 and Aix >>>>>>>> - concentrate cOops heap allocation code in one function >>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>> >>>>>>>> >>>>>>>> Please reply only to the original thread in hotspot-dev to keep >>>>>>>> this >>>>>>>> local. >>>>>>>> >>>>>>>> Thanks and best regards, >>>>>>>> Goetz. >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>> Sent: Donnerstag, 4. Dezember 2014 19:44 >>>>>>>> To: Lindenmaier, Goetz >>>>>>>> Cc: 'hotspot-dev developers' >>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>> >>>>>>>> This looks good to me. >>>>>>>> >>>>>>>> Now we need second review since changes are significant. >>>>>>>> Preferable >>>>>>>> from >>>>>>>> GC group since you changed ReservedHeapSpace. They will be >>>>>>>> affected >>>>>>>> most. Review from Runtime is also welcome. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Vladimir >>>>>>>> >>>>>>>> On 12/4/14 10:27 AM, Lindenmaier, Goetz wrote: >>>>>>>>> Hi Vladimir. >>>>>>>>> >>>>>>>>> Sorry. I updated the webrev once more. Hope it's fine now. >>>>>>>>> At least I can write comments :) >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> Goetz >>>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>> Sent: Thursday, December 04, 2014 5:54 PM >>>>>>>>> To: Lindenmaier, Goetz >>>>>>>>> Cc: 'hotspot-dev developers' >>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>> >>>>>>>>> I spotted an other bug. >>>>>>>>> You replaced !_base with _base != NULL when moved code to >>>>>>>>> try_reserve_range() - it should be _base == NULL. >>>>>>>>> The same problem in asserts: >>>>>>>>> >>>>>>>>> + assert(_base != NULL || >>>>>>>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == >>>>>>>>> _base, >>>>>>>>> + "area must be distinguishable from marks for >>>>>>>>> mark-sweep"); >>>>>>>>> + assert(_base != NULL || >>>>>>>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() >>>>>>>>> == >>>>>>>>> &_base[size], >>>>>>>>> + "area must be distinguishable from marks for >>>>>>>>> mark-sweep"); >>>>>>>>> >>>>>>>>> >>>>>>>>> Also you did not remove _base && in next place: >>>>>>>>> >>>>>>>>> + (_base && _base + size > zerobased_max))) { // >>>>>>>>> Unscaled >>>>>>>>> delivered an arbitrary address. >>>>>>>>> >>>>>>>>> New comment is good. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Vladimri >>>>>>>>> >>>>>>>>> On 12/4/14 1:45 AM, Lindenmaier, Goetz wrote: >>>>>>>>>> Hi Vladimir, >>>>>>>>>> >>>>>>>>>>> Add more extending comment explaining that. >>>>>>>>>> The comment for try_reserve_heap was meant to explain that. >>>>>>>>>> I further added a comment in initialize_compressed_heap(). >>>>>>>>>> >>>>>>>>>>> You need another parameter to pass UnscaledOopHeapMax or >>>>>>>>>>> zerobased_max. >>>>>>>>>> Oh, thanks a lot! That's important. Fixed. >>>>>>>>>> >>>>>>>>>>> I mean that you already checked _base == NULL so on other side >>>>>>>>>>> of || _base != NULL - why you need (_base &&) check? >>>>>>>>>> Sorry, now I got it. Removed. >>>>>>>>>> >>>>>>>>>> I updated the webrev: >>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Increment on top of the increment :) >>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/incremental_diffs2.patch >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Goetz. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>>> Sent: Mittwoch, 3. Dezember 2014 18:32 >>>>>>>>>> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' >>>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>> >>>>>>>>>> Comments are below. >>>>>>>>>> >>>>>>>>>> On 12/3/14 5:49 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>> Hi Vladimir, >>>>>>>>>>> >>>>>>>>>>> thanks for looking at the change! See my comments inline >>>>>>>>>>> below. >>>>>>>>>>> >>>>>>>>>>> I made a new webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Incremental changes: >>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/incremental_diffs.patch >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Best regards, >>>>>>>>>>> Goetz. >>>>>>>>>>> >>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>>>>> Sent: Mittwoch, 3. Dezember 2014 00:46 >>>>>>>>>>>> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' >>>>>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>>>> >>>>>>>>>>>> This looks good to me. Someone in runtime/gc have to look >>>>>>>>>>>> on it >>>>>>>>>>>> too. >>>>>>>>>>>> >>>>>>>>>>>> universe.cpp about >>>>>>>>>>>> SystemProperty("com.sap.vm.test.compressedOopsMode" >>>>>>>>>>>> we have: >>>>>>>>>>>> java.vm.info=mixed mode, sharing >>>>>>>>>>>> so we can have: >>>>>>>>>>>> java.vm.compressedOopsMode=... >>>>>>>>>>> Yes, that's good for me. Fixed. >>>>>>>>>>> >>>>>>>>>>>> I am not expert in properties names but I don't want to have >>>>>>>>>>>> 'com.sap' >>>>>>>>>>>> in VM's property name. >>>>>>>>>>>> virtualspace.cpp: >>>>>>>>>>>> Could you fix release() - it does not reset _alignment? >>>>>>>>>>> Fixed. >>>>>>>>>>> >>>>>>>>>>>> In try_reserve_heap(), please, use (base == NULL) instead of >>>>>>>>>>>> (!base). >>>>>>>>>>>> And you don't need 'return;' in alignment check at the end of >>>>>>>>>>>> method. >>>>>>>>>>> Fixed. >>>>>>>>>>> >>>>>>>>>>>> In initialize_compressed_heap() again (!_base). >>>>>>>>>>> Fixed. >>>>>>>>>>> >>>>>>>>>>>> You don't stop (check >>>>>>>>>>>> (base == NULL)) after successful unscaled, zerobased, >>>>>>>>>>>> disjointbase >>>>>>>>>>>> allocations. You need to separate them with the check: >>>>>>>>>>>> >>>>>>>>>>>> + >>>>>>>>>>>> + } >>>>>>>>>>>> + } >>>>>>>>>>>> + if (_base == NULL) { >>>>>>>>>>>> + >>>>>>>>>>>> + if (PrintCompressedOopsMode && Verbose) { >>>>>>>>>>>> + tty->print(" == Z E R O B A S E D ==\n"); >>>>>>>>>>>> + } >>>>>>>>>>>> and so on. >>>>>>>>>>> No, I can't and don't want to check for _base != NULL. >>>>>>>>>>> I always keep the result of the last try, also if it didn't >>>>>>>>>>> fulfil the required properties. >>>>>>>>>>> So I take that result and go into the next check. That check >>>>>>>>>>> might succeed >>>>>>>>>>> with the heap allocated before. >>>>>>>>>>> This allows me to separate allocation and placement criteria, >>>>>>>>>>> and to have the >>>>>>>>>>> placement criteria checked in only one place (per mode). >>>>>>>>>>> Only for HeapBaseMinAddress I don't do it that way, I >>>>>>>>>>> explicitly >>>>>>>>>>> call release(). >>>>>>>>>>> This way I can enforce mode heapbased. >>>>>>>>>> I see what you are saying. It was not clear from comments >>>>>>>>>> what is >>>>>>>>>> going on. >>>>>>>>>> Add more extending comment explaining that. >>>>>>>>>> >>>>>>>>>>>> num_attempts calculation and while() loop are similar in >>>>>>>>>>>> unscaled and >>>>>>>>>>>> zerobased cases. Could you move it into a separate method? >>>>>>>>>>> I can do that, but I don't like it as I have to pass in 7 >>>>>>>>>>> parameters. >>>>>>>>>> You need an other parameter to pass UnscaledOopHeapMax or >>>>>>>>>> zerobased_max. >>>>>>>>>> >>>>>>>>>>> That makes the code not much more readable. The function will >>>>>>>>>>> look like this: >>>>>>>>>> I think initialize_compressed_heap() is more readable now. >>>>>>>>>> >>>>>>>>>>> void ReserveHeapSpace::try_reserve_range(char *const >>>>>>>>>>> highest_start, char *lowest_start, size_t >>>>>>>>>>> attach_point_alignment, >>>>>>>>>>> char >>>>>>>>>>> *aligned_HBMA, size_t size, size_t alignment, bool large) { >>>>>>>>>>> guarantee(HeapSearchSteps > 0, "Don't set >>>>>>>>>>> HeapSearchSteps >>>>>>>>>>> to 0"); >>>>>>>>>>> >>>>>>>>>>> const size_t attach_range = highest_start - >>>>>>>>>>> lowest_start; >>>>>>>>>>> // Cap num_attempts at possible number. >>>>>>>>>>> // At least one is possible even for 0 sized >>>>>>>>>>> attach range. >>>>>>>>>>> const uint64_t num_attempts_possible = >>>>>>>>>>> (attach_range / >>>>>>>>>>> attach_point_alignment) + 1; >>>>>>>>>>> const uint64_t num_attempts_to_try = >>>>>>>>>>> MIN2(HeapSearchSteps, num_attempts_possible); >>>>>>>>>>> >>>>>>>>>>> const size_t stepsize = align_size_up(attach_range / >>>>>>>>>>> num_attempts_to_try, attach_point_alignment); >>>>>>>>>>> >>>>>>>>>>> // Try attach points from top to bottom. >>>>>>>>>>> char* attach_point = highest_start; >>>>>>>>>>> while (attach_point >= lowest_start && >>>>>>>>>>> attach_point <= highest_start && // Avoid >>>>>>>>>>> wrap >>>>>>>>>>> around. >>>>>>>>>>> (!_base || _base < aligned_HBMA || _base + >>>>>>>>>>> size > >>>>>>>>>>> (char *)UnscaledOopHeapMax)) { >>>>>>>>>>> try_reserve_heap(size, alignment, large, >>>>>>>>>>> attach_point); >>>>>>>>>>> attach_point -= stepsize; >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> In disjointbase while() condition no need for _base second >>>>>>>>>>>> check: >>>>>>>>>>>> + (_base == NULL || >>>>>>>>>>>> + ((_base + size > (char *)OopEncodingHeapMax) && >>>>>>>>>>> I need this for the same reason as above: This is the check for >>>>>>>>>>> successful allocation. >>>>>>>>>> I mean that you already checked _base == NULL so on other >>>>>>>>>> side of >>>>>>>>>> || _base != NULL - why you need (_base &&) check? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Vladimir >>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Vladimir >>>>>>>>>>> >>>>>>>>>>> On 11/21/14 5:31 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> I prepared a new webrev trying to cover all the issues >>>>>>>>>>>> mentioned below. >>>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.01/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I moved functionality from os.cpp and universe.cpp into >>>>>>>>>>>> ReservedHeapSpace::initialize_compressed_heap(). >>>>>>>>>>>> This class offers to save _base and _special, which I would >>>>>>>>>>>> have to reimplement >>>>>>>>>>>> if I had improved the methods I had added to os.cpp to also >>>>>>>>>>>> allocate large page >>>>>>>>>>>> heaps. >>>>>>>>>>>> Anyways, I think this class is the right place to gather code >>>>>>>>>>>> trying to reserve >>>>>>>>>>>> the heap. >>>>>>>>>>>> Also, I get along without setting the shift, base, >>>>>>>>>>>> implicit_null_check etc. fields >>>>>>>>>>>> of Universe, so there is no unnecessary calling back and forth >>>>>>>>>>>> between the two >>>>>>>>>>>> classes. >>>>>>>>>>>> Universe gets the heap back, and then sets the properties it >>>>>>>>>>>> needs to configure >>>>>>>>>>>> the compressed oops. >>>>>>>>>>>> All code handling the noaccess prefix is in a single >>>>>>>>>>>> method, too. >>>>>>>>>>>> >>>>>>>>>>>> Best regards, >>>>>>>>>>>> Goetz. >>>>>>>>>>>> >>>>>>>>>>>> Btw, I had to workaround a SS12u1 problem: it wouldn't compile >>>>>>>>>>>> char * x = (char*)UnscaledOopHeapMax - size in 32-bit mode. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>> From: hotspot-dev >>>>>>>>>>>> [mailto:hotspot-dev-bounces at openjdk.java.net] >>>>>>>>>>>> On Behalf Of Lindenmaier, Goetz >>>>>>>>>>>> Sent: Montag, 17. November 2014 09:33 >>>>>>>>>>>> To: 'Vladimir Kozlov'; 'hotspot-dev at openjdk.java.net' >>>>>>>>>>>> Subject: RE: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>>>> >>>>>>>>>>>> Hi Vladimir, >>>>>>>>>>>> >>>>>>>>>>>>> It is very significant rewriting and it takes time to >>>>>>>>>>>>> evaluate >>>>>>>>>>>>> it. >>>>>>>>>>>> Yes, I know ... and I don't want to push, but nevertheless >>>>>>>>>>>> a ping >>>>>>>>>>>> can be useful sometimes. Thanks a lot for looking at it. >>>>>>>>>>>> >>>>>>>>>>>>> And I would not say it is simpler then before :) >>>>>>>>>>>> If I fix what you propose it's gonna get even more simple ;) >>>>>>>>>>>>> These is what I found so far. >>>>>>>>>>>>> The idea to try to allocate in a range instead of just below >>>>>>>>>>>>> UnscaledOopHeapMax or OopEncodingHeapMax is good. So I would >>>>>>>>>>>>> ask to do >>>>>>>>>>>>> several attempts (3?) on non_PPC64 platforms too. >>>>>>>>>>>> Set to 3. >>>>>>>>>>>> >>>>>>>>>>>>> It is matter of preference but I am not comfortable with >>>>>>>>>>>>> switch in loop. >>>>>>>>>>>>> For me sequential 'if (addr == 0)' checks is simpler. >>>>>>>>>>>> I'll fix this. >>>>>>>>>>>> >>>>>>>>>>>>> One thing worries me that you release found space and try to >>>>>>>>>>>>> get it >>>>>>>>>>>>> again with ReservedHeapSpace. Is it possible to add new >>>>>>>>>>>>> ReservedHeapSpace ctor which simple use already allocated >>>>>>>>>>>>> space? >>>>>>>>>>>> This was to keep diff's small, but I also think a new >>>>>>>>>>>> constructor is good. >>>>>>>>>>>> I'll fix this. >>>>>>>>>>>> >>>>>>>>>>>>> The next code in ReservedHeapSpace() is hard to understand >>>>>>>>>>>>> (): >>>>>>>>>>>>> (UseCompressedOops && (requested_address == NULL || >>>>>>>>>>>> requested_address+size > (char*)OopEncodingHeapMax) ? >>>>>>>>>>>>> may be move all this into noaccess_prefix_size() and add >>>>>>>>>>>>> comments. >>>>>>>>>>>> I have to redo this anyways if I make new constructors. >>>>>>>>>>>> >>>>>>>>>>>>> Why you need prefix when requested_address == NULL? >>>>>>>>>>>> If we allocate with NULL, we most probably will get a heap >>>>>>>>>>>> where >>>>>>>>>>>> base != NULL and thus need a noaccess prefix. >>>>>>>>>>>> >>>>>>>>>>>>> Remove next comment in universe.cpp: >>>>>>>>>>>>> // SAPJVM GL 2014-09-22 >>>>>>>>>>>> Removed. >>>>>>>>>>>> >>>>>>>>>>>>> Again you will release space so why bother to include space >>>>>>>>>>>>> for classes?: >>>>>>>>>>>>> + // For small heaps, save some space for compressed >>>>>>>>>>>>> class pointer >>>>>>>>>>>>> + // space so it can be decoded with no base. >>>>>>>>>>>> This was done like this before. We must assure the upper >>>>>>>>>>>> bound >>>>>>>>>>>> of the >>>>>>>>>>>> heap is low enough that the compressed class space still fits >>>>>>>>>>>> in there. >>>>>>>>>>>> >>>>>>>>>>>> virtualspace.cpp >>>>>>>>>>>> >>>>>>>>>>>>> With new code size+noaccess_prefix could be requested. But >>>>>>>>>>>>> later it is >>>>>>>>>>>>> not used if WIN64_ONLY(&& UseLargePages) and you will have >>>>>>>>>>>>> empty >>>>>>>>>>>>> non-protected page below heap. >>>>>>>>>>>> There's several points to this: >>>>>>>>>>>> * Also if not protectable, the heap base has to >>>>>>>>>>>> be below >>>>>>>>>>>> the real start of the >>>>>>>>>>>> heap. Else the first object in the heap will be >>>>>>>>>>>> compressed to 'null' >>>>>>>>>>>> and decompression will fail. >>>>>>>>>>>> * If we don't reserve the memory other stuff can >>>>>>>>>>>> end up >>>>>>>>>>>> in this space. On >>>>>>>>>>>> errors, if would be quite unexpected to find >>>>>>>>>>>> memory >>>>>>>>>>>> there. >>>>>>>>>>>> * To get a heap for the new disjoint mode I must >>>>>>>>>>>> control >>>>>>>>>>>> the size of this. >>>>>>>>>>>> Requesting a heap starting at (aligned base + >>>>>>>>>>>> prefix) is more likely to fail. >>>>>>>>>>>> * The size for the prefix must anyways be considered >>>>>>>>>>>> when deciding whether the >>>>>>>>>>>> heap is small enough to run with compressed >>>>>>>>>>>> oops. >>>>>>>>>>>> So distinguishing the case where we really can omit this would >>>>>>>>>>>> require >>>>>>>>>>>> quite some additional checks everywhere, and I thought it's >>>>>>>>>>>> not >>>>>>>>>>>> worth it. >>>>>>>>>>>> >>>>>>>>>>>> matcher.hpp >>>>>>>>>>>> >>>>>>>>>>>>> Universe::narrow_oop_use_implicit_null_checks() should be >>>>>>>>>>>>> true >>>>>>>>>>>>> for such >>>>>>>>>>>>> case too. So you can add new condition with || to existing >>>>>>>>>>>>> ones. The >>>>>>>>>>>>> only condition you relax is base != NULL. Right? >>>>>>>>>>>> Yes, that's how it's intended. >>>>>>>>>>>> >>>>>>>>>>>> arguments.* files >>>>>>>>>>>> >>>>>>>>>>>>> Why you need PropertyList_add changes. >>>>>>>>>>>> Oh, the code using it got lost. I commented on this in the >>>>>>>>>>>> description in the webrev. >>>>>>>>>>>> "To more efficiently run expensive tests in various compressed >>>>>>>>>>>> oop modes, we set a property with the mode the VM is running >>>>>>>>>>>> in. So far it's called "com.sap.vm.test.compressedOopsMode" >>>>>>>>>>>> better suggestions are welcome (and necessary I guess). Our >>>>>>>>>>>> long running tests that are supposed to run in a dedicated >>>>>>>>>>>> compressed oop mode check this property and abort >>>>>>>>>>>> themselves if >>>>>>>>>>>> it's not the expected mode." >>>>>>>>>>>> When I know about the heap I do >>>>>>>>>>>> Arguments::PropertyList_add(new >>>>>>>>>>>> SystemProperty("com.sap.vm.test.compressedOopsMode", >>>>>>>>>>>> narrow_oop_mode_to_string(narrow_oop_mode()), >>>>>>>>>>>> false)); >>>>>>>>>>>> in universe.cpp. >>>>>>>>>>>> On some OSes it's deterministic which modes work, there we >>>>>>>>>>>> don't start such tests. >>>>>>>>>>>> Others, as you mentioned OSX, are very indeterministic. Here >>>>>>>>>>>> we save testruntime with this. >>>>>>>>>>>> But it's not that important. >>>>>>>>>>>> We can still parse the PrintCompresseOopsMode output after the >>>>>>>>>>>> test and discard the >>>>>>>>>>>> run. >>>>>>>>>>>> >>>>>>>>>>>>> Do you have platform specific changes? >>>>>>>>>>>> Yes, for ppc and aix. I'll submit them once this is in. >>>>>>>>>>>> >>>>>>>>>>>> From your other mail: >>>>>>>>>>>>> One more thing. You should allow an allocation in the range >>>>>>>>>>>>> when returned from OS allocated address does not match >>>>>>>>>>>>> requested address. We had such cases on OSX, for example, >>>>>>>>>>>>> when >>>>>>>>>>>>> OS allocates at different address but still inside range. >>>>>>>>>>>> Good point. I'll fix that in >>>>>>>>>>>> os::attempt_reserve_memory_in_range. >>>>>>>>>>>> >>>>>>>>>>>> I'll ping again once a new webrev is done! >>>>>>>>>>>> >>>>>>>>>>>> Best regards, >>>>>>>>>>>> Goetz. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 11/10/14 6:57 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> I need to improve a row of things around compressed oops heap >>>>>>>>>>>>> handling >>>>>>>>>>>>> to achieve good performance on ppc. >>>>>>>>>>>>> I prepared a first webrev for review: >>>>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> A detailed technical description of the change is in the >>>>>>>>>>>>> webrev and according bug. >>>>>>>>>>>>> >>>>>>>>>>>>> If requested, I will split the change into parts with more >>>>>>>>>>>>> respective less impact on >>>>>>>>>>>>> non-ppc platforms. >>>>>>>>>>>>> >>>>>>>>>>>>> The change is derived from well-tested code in our VM. >>>>>>>>>>>>> Originally it was >>>>>>>>>>>>> crafted to require the least changes of VM coding, I changed >>>>>>>>>>>>> it to be better >>>>>>>>>>>>> streamlined with the VM. >>>>>>>>>>>>> I tested this change to deliver heaps at about the same >>>>>>>>>>>>> addresses as before. >>>>>>>>>>>>> Heap addresses mostly differ in lower bits. In some cases >>>>>>>>>>>>> (Solaris 5.11) a heap >>>>>>>>>>>>> in a better compressed oops mode is found, though. >>>>>>>>>>>>> I ran (and adapted) test/runtime/CompressedOops and >>>>>>>>>>>>> gc/arguments/TestUseCompressedOops*. >>>>>>>>>>>>> >>>>>>>>>>>>> Best regards, >>>>>>>>>>>>> Goetz. >>>>>>>>>>>>> >>>>>>>>>>>>> > From erik.osterlund at lnu.se Mon Jan 5 17:41:39 2015 From: erik.osterlund at lnu.se (=?Windows-1252?Q?Erik_=D6sterlund?=) Date: Mon, 5 Jan 2015 17:41:39 +0000 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <54AA143B.2080900@oracle.com> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <,<8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> <54AA143B.2080900@oracle.com> Message-ID: <775F23B6-41A1-4904-8969-1C2F7407D899@lnu.se> Hi David and Kim, Thank you David for putting this into context. If we want this to be done with the inheritance/template way (specifically the SelectBaseClass template, oh and other names are welcome, InjectOptionalBase?), then I see two options, so I implemented both so you can compare and pick a favourite. They both have exactly identical semantics: they check for the existence of an optional class to inject to the class hierarchy, and check that the inheritance isn?t fishy (the optional class for specialization must be a subclass of the required class for generalization). I think it all comes down to reusability vs code size. Approach 1. ?Reusability Maximum?: Build SelectBaseClass in terms of reusable components resembling boost/C++11 as much as possible. (Note: Since Kim wanted IsBaseOf to require Derived to be complete (defined) to mimic other libraries, I now force a compiler error if parameters are incomplete as per request and instead use her proposal for checking if classes are defined, thanks for that contribution) Pros: 1) Many reusable metafunctions that can be reused in other contexts (for instance automatic closure specialization), and I believe we will have to include them sooner or later anyway. 2) Since they comply very closely with the C++11 standard, it will be easy to make a transition once all compilers support this some time in the distant past. Cons: 380 LoC template infrastructure, including testing, for a perceivably small application (if not reused in the future). Approach 2. ?Code Minimum?: Get the same SelectBaseClass, but with minimal dependencies. Only 20 LoC for the necessary metafunction, some more for IsSame used only for testing purposes, and 156 LoC in total for the template infrastructure, making it much lighter. Pros: Minimal amount of code for the change (less than half). Cons: No reusable metafunctions for the future which we quite likely will need later. I have attached the two solutions in this email so you can have a look and tell me which direction you want me to go before making a final webrev to make everyone happy. Personally I like approach 1 better but if it is concerning to commit too much code and there are strong opinions about this, I?m happy with approach 2 as well. Appendix with code for minimal implementation with no dependencies: template class SelectBaseClass { typedef char yes[1]; typedef char no[2]; template static yes &check(Optional*, T); static no &check(Required*, int); struct OptionalCheckHost { operator Required*() const; operator Optional*(); }; template struct OptionalCheck { typedef O type; }; template struct OptionalCheck { typedef R type; }; public: typedef typename OptionalCheck::type type; }; Thanks, /Erik > On 05 Jan 2015, at 04:34, David Holmes wrote: > > Sorry for the top-post but just wanted to reset the context a little here. Originally [1] I made the comment: > > "It's fine to have generic shared approaches but there really needs to be a way to allow platform specific "overrides"." > > For the actual original RFR I said [2]: > > "Can we pause and give some more thought to a clean mechanism for > allowing a shared implementation if desired with the ability to override if desired. I really do not like to see CPU specific ifdefs being added to shared code. (And I would also not like to see all platforms being forced to reimplement this natively). > > I'm not saying we will find a simple solution, but it would be nice if > we could get a few folk to think about it before proceeding with the > ifdefs :)" > > Erik then proposed three alternative approaches [3] and the simple variant was chosen [4] and presented for review. However Paul Hohensee also made comments about an inheritance-based approach and Erik floated the first template-based variant [5] and there was some discussion with Paul and Kim. But we then ended up with the simple solution, leaving an inheritance-based solution for future work [6] (which is what is what is being discussed now). > > This template-based meta-programming is not something I have any familiarity with. My initial thought is this seems overkill for the situation we have with the Atomic class - but as I said I'm ignorant of the technique being used here. > > David > ----- > > [1] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015292.html > [2] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015303.html > [3] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015362.html > [4] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015401.html > [5] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-October/015586.html > [6] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-November/015889.html > > On 2/01/2015 12:07 AM, Erik ?sterlund wrote: >> Hi Kim, >> >>> On 01 Jan 2015, at 07:42, Kim Barrett wrote: >>> >>> On Dec 31, 2014, at 6:12 AM, Erik ?sterlund wrote: >>>> >>>>> The rest of these comments only matter if, contrary to my suggestion >>>>> above, some additional infrastructure is thought necessary. >>>>> >>>> >>>> This is what I originally thought too and hence the current solution. But voices said that using macros for this is, I quote, "that bad". >>>> >>>>> [?] >>>>> >>>>> This scales with the number of AtomicPlatform definitions, rather than >>>>> the number of specialized functions as happens with the existing code. >>>> >>>> So your proposal is to still use ugly macros but move it to the class level instead, even though at the moment there is really only one member function that needs it. That feels a bit meh to be honest. IMO if we want to do this right, why go half the way and still rely on ugly macros? >>> >>> Please lay off with the perjorative "ugly macros". The preprocessor >>> is part of C++, and isn't going away. It's a tool that can be used, >>> or abused, and I think this is a fine use. To me, "ugly" is adding >>> 800+ lines of code / tests / comments to eliminate a dozen lines of >>> flag macro definitions and associated #ifdef?s. >> >> Well people are entitled to have different opinions of course. As I already mentioned, I did this because other reviewers (as well as me) found it ugly to use macros for conditionally specializing, which is the whole motivation of doing this, and was well understood from the beginning. But I?m fine with leaving it as macros if this is in general preferred and opinions have suddenly changed in this matter. >> >> Personally I don?t see how the number of rows of code define uglyness. Reusability, testing, documentation and well definedness in contracts/behaviours all lead to more lines of code, but I don?t think that equals uglyness. So while I disagree more code is uglier because it?s more, it?s all up to opinion and it?s quite pointless to discuss; you are entitled to think that is ugly. >> >>> If that additional infrastructure were amortized across more use-cases >>> then it might be more appealing. I might even have some additional >>> use-cases for something along that line, though not in its present >>> form. (More on that below.) >> >> Yeah I specifically had in mind the wish to make a split between commercial extensions to open source code actually. This pattern allows easy optional class inheritance injection as a general design pattern to achieve this. And for my own purposes the ability to define asymmetric dekker synchronization to elide storeload fences when the platform (e.g. TSO) supports it but otherwise revert to a generalized implementation (using storeload fences). This is why I think a clean reusable strategy is almost always better because it is widely applicable, even if there would not be obvious other examples where it is useful. >> >>> >>>>> I think the name "IsBaseOfAndDerived" would be better as >>>>> "IsBaseAndDerived?. [?] >>>>> >>>>> The current implementation of IsBaseOfAndDerived (and therefore >>>>> IsBaseOf) may (silently) provide incorrect answers when called with >>>>> incomplete types. [?] >>>>> >>>>> The actual usage in this proposed change-set even relies on this >>>>> flawed behavior of these metafunctions, since AtomicPlatform isn't >>>>> always defined. [?] >>>>> >>>> >>>> So what you are saying here is that I should rename IsBaseOfAndDerived to IsBaseAndDerived to more closely resemble other external libraries we do not use. But if I do that my behaviour is "incorrect" because it is not identical to that of the external library. And therefore my change would not work if it was made "correctly?. >>> >>> The proposed IsBaseOfAndDerived and IsBaseOf are being described and >>> placed as general utilities. Their names are strongly reminiscent of >>> existing facilities in other libraries, including the (C++11) standard >>> library. Conforming to these external standards will avoid surprises, >>> especially when the proposed facilities are even described as being >>> the same (except for a noted corner case that is both hard to deal >>> with in a portable manner and not actually very interesting, so I'm >>> not complaining about that) in the review request email. >>> >>> Additionally, I think quietly providing a potentially incorrect answer >>> in the case of incomplete types renders the offered code quite broken >>> for use as a general utility. I think the requirement for complete >>> types by the well-known / standard facilities is well justified. >>> >> >> While I disagree it is universally incorrect (as long as it is well defined and tested) to not have the exact same requirement on completeness on the parameters as C++11/boost, who also already have different behaviour in this exact regard as pointed out by yourself, I can try to fix it if it bugs you, but it will make the infrastructure even larger which seems to be a concern. >> >>>> On the contrary this is a well tested feature on all our supported compilers and I did not intend to engineer it to be identical to some external library if it only causes trouble rather than helping. This is our own class and it can do what we want it to do and be called what we want it to be called, and I think that is fine as long as it is well tested, which it is. At least this is my opinion. Anyone else agree? >>> >>> Problem is, I think it's well tested code to solve the wrong problem. >>> >>> I suggest that what we're really looking for is not a base/derived >>> relationship between a pair of classes, but that we actually want to >>> determine whether a platform-specific class exists. >>> >>> As a result, I think there is more infrastructure in this change set >>> than is actually needed. The present implementation of >>> SelectBaseClass uses IsBaseOf, but doesn't need a lot of the >>> functionality of that metafunction. There is no actual need for a >>> Base/Derived relationship between the (possibly not defined) >>> platform-specific class and the default platform-generic class, so the >>> use of IsBaseOf is an unnecessary restriction that only >>> serendipitously tests for defined or not, due to the previously >>> discussed defect in this particular implementation. I also think the >>> name SelectBaseClass is perhaps overly general. >> >> The main idea of SelectBaseClass is to inject an optional class into the class hierarchy for specialization of generalized behaviour. It is absolutely critical (and intended) that this class is constrained to be a derived class of the fall-back class - it is not an artifact nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, A must be able to rely on the superclass being at least C-like so that the behaviour of the superclass can be used reliably and transparently regardless of the existence of the optional B. If this contract would not hold, I think it could lead to some really weird and unreliable code which should not be encouraged (at least not according to the Liskov substitution principle etc). >> >> So what I need in the contract is a check if the optional class is defined and if it fits in the inheritance hierarchy: they are both important IMO. >> >> Yes it could have been less code. From the beginning I had a single trait class that checked for both the inheritance and whether the class is defined or not but you said you wanted the behaviour to more closely match that of third party libraries. This is why there is now a lot more code to more closely (but still not exactly) match that behaviour. >> >>> I might be able to use a similar approach in some code I've been >>> working on as a background task. And while writing this reply I've >>> thought of another place that might benefit from something along those >>> lines. Thinking about the code under review in that context, I think >>> some changes would make these other possible use-cases easier and >>> clearer. >>> >>> I believe what is actually needed is a metatfunction something like: >>> >>> /** >>> * Metafunction whose nested type member is Default if Specific is >>> * incomplete, otherwise Specific. >>> * >>> * Specific and Default must both be non-cv qualified class types, and >>> * must not be the same type. >>> * >>> * If Specific is incomplete at the point where this metafunction is >>> * invoked, it must never be defined elsewhere in the program. >>> */ >>> template >>> struct SelectPlatformBase; >>> >>> [Note: I'm not wedded to that name.] >>> >>> Now, it turns out to be hard / impossible to write a generic >>> is_complete metafunction, due to ODR. Whether a type is complete >>> can vary between translation units, and even within a translation >>> unit. Having is_complete::value have different values depending on >>> where it is instantiated is an ODR violation. (One can make progress >>> by using anonymous namespaces and macro wrappers with __LINE__ >>> concatenation, but there is still the fundamental question of whether >>> is_complete really even makes semantic sense.) >> >> Yes and that makes me wonder if we care enough about the completeness semantics being the same as third party libraries (who are already inconsistent) if we do not need them to be, and on the contrary benefit from them not to be. >> >>> However, we don't need a fully general is_complete utility. We have a >>> much more restricted use-case, where we want to select an optionally >>> defined platform-specific class if it exists, and otherwise fall back >>> to a more generic class. And we're not using this in arbitrary >>> contexts, but only to select a platform-specialized implementation if >>> such exists. >>> >>> Here's a lightly tested implementation of SelectPlatformBase: >>> >>> // --- SelectPlatformBase >>> >>> template >>> struct EnableIf { typedef T type; }; >>> >>> template >>> struct EnableIf { }; >>> >>> template >>> struct If { typedef Then type; }; >>> >>> template >>> struct If { typedef Else type; }; >>> >>> // Metafunction whose nested value member is true if T is defined >>> // (complete), and false if T is incomplete. T must be a non-cv >>> // qualified class type. If T is incomplete at the point where this >>> // metafunction is invoked, it must never be defined elsewhere in the >>> // program. >>> template >>> class IsClassDefined { >>> typedef char yes[1]; >>> typedef char no[2]; >>> >>> template >>> static typename EnableIf::type & check(U*); >>> static no& check(...); >>> >>> public: >>> static const bool value = sizeof(check((T*)0)) == sizeof(yes); >>> }; >>> >>> template >>> struct SelectPlatformBase { >>> typedef typename If< >>> IsClassDefined::value, Specific, Default>::type type; >>> }; >>> >>> // --- end SelectPlatformBase >>> >>> That's ~30 lines of code (+ tests and comments TBD) to do precisely >>> what we need, replacing ~100 lines (+ tests and comments) that have >>> various issues. (If and EnableIf should probably be hoisted out as >>> separate utilities.) We don't actually need SelectPlatformBase, and >>> could instead just directly use the If and IsClassDefined >>> metafunctions, but giving this functionality a name might be clearer. >> >> As stated, I think it?s dangerous to allow conditional inheritance where there is no inheritance constraint on the optional class. Therefore I do not think this is better, although I agree it is smaller. >> >> I agree that our use case here is smaller. If the amount of code is the problem and less code (and hence fewer reusable components) is preferred, then the original implementation with a simple IsBaseOfAndDerived does the trick (without checking types/cv qualifiers) and could be done in just a few lines, and this is what I originally did before you told me to expand it to more closely resemble external libraries: >> >> template >> class IsBaseOfAndDerived { >> typedef char yes[1]; >> typedef char no[2]; >> >> template >> static yes &check(Derived*, T); >> static no &check(Base*, int); >> >> template >> struct IsBaseOfHost { >> operator B*() const; >> operator D*(); >> }; >> public: >> static const bool value = sizeof(check(IsBaseOfHost(), int())) == sizeof(yes); >> }; >> >> It checks the inheritance and existence of the Derived class which is exactly the constraints I need. >> >> If you do not want to expose it publicly and remove the reusability because it does not have identical semantics as third party libraries regarding the requirements of the Derived type to be complete, it could be an internal class of SelectBaseClass. >> >> I?m not going to value the reusability vs LoC, I leave that decision to you. >> >>> However, while I think there are other use cases for this specific >>> utility, I still think the proposed change set as a whole is adding a >>> lot of code just to avoid a a few lines of macro usage. That seems >>> like a poor tradeoff to me. >> >> >> Okay. >> >> In the end, we have to make a choice - what is more important, that the traits resemble some external library or code size. Because last time I proposed a small change and you wanted it to be closer to C++11 behaviour and I made it closer to that at the expense of more code. Now you are saying I have to make it even tighter to C++11 (or boost?), but also don?t want more lines of code which I believe is impossible given that I want to constrain both the class hierarchy to be reliable and check the existence of the optional class. This brings me to a few architectural questions which I may have (strong) opinions about but are not up to me to decide. >> >> 1. Do we want to replace the macros with template metaprogramming? 2 reviewers (and me) liked the metaprogramming approach and it was implemented because they (like me) did not want the macros. But you have a different opinion. I can?t judge who is right and wrong here. >> >> 2. Do we want to minimize the code size at the cost of reusability by making contracts of dependent components weaker and putting them as private classes in the SelectBaseClass? >> >> 3. If we do want template metaprogramming and want reusable components, the question is, is it important that the specification of completeness (which in general is almost impossible to implement correctly in a portable way as you pointed out) of the template arguments in IsBaseOf is identical as C++11/boost even though no code needs it to be and on the contrary some code benefits from it not to be? In that case which one of them do we, because they are already different anyway? There is a tradeoff between code size and complexity vs copying either one of the external libraries regarding requirements of completeness of template parameters. >> >> Thanks, >> /Erik >> From roland.westrelin at oracle.com Mon Jan 5 17:50:51 2015 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Mon, 5 Jan 2015 18:50:51 +0100 Subject: 8068053: AARCH64: C1 and C2 compilers In-Reply-To: <54A6A8F5.4040907@redhat.com> References: <54A6A8F5.4040907@redhat.com> Message-ID: <928DC166-7256-4BCA-A9CA-C3ADB05834E5@oracle.com> Hi Andrew, > http://cr.openjdk.java.net/~aph/aarch64-8068053-2/ Shouldn?t c1_FpuStackSim_aarch64.[ch]pp be empty like it is on sparc? What's ad_encode.m4 for? Typo: 831 // advance declaratuons for helper functions to convert register in aarch64.ad That looks good to me. Roland. From aph at redhat.com Mon Jan 5 18:12:12 2015 From: aph at redhat.com (Andrew Haley) Date: Mon, 05 Jan 2015 18:12:12 +0000 Subject: 8068053: AARCH64: C1 and C2 compilers In-Reply-To: <928DC166-7256-4BCA-A9CA-C3ADB05834E5@oracle.com> References: <54A6A8F5.4040907@redhat.com> <928DC166-7256-4BCA-A9CA-C3ADB05834E5@oracle.com> Message-ID: <54AAD3FC.8000003@redhat.com> On 01/05/2015 05:50 PM, Roland Westrelin wrote: > Hi Andrew, > >> http://cr.openjdk.java.net/~aph/aarch64-8068053-2/ > > Shouldn?t c1_FpuStackSim_aarch64.[ch]pp be empty like it is on sparc? Oh yes, that's a hangover from x86. Will remove. How embarrassing. :-) > What's ad_encode.m4 for? The encode patterns in aarch64.md are very repetitive, so I used m4 to generate them. > Typo: > 831 // advance declaratuons for helper functions to convert register OK. > in aarch64.ad > > That looks good to me. Thanks, Andrew. From vladimir.kozlov at oracle.com Mon Jan 5 18:52:16 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 05 Jan 2015 10:52:16 -0800 Subject: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. In-Reply-To: <54AAC34F.2050807@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF264E2@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CF35E66@DEWDFEMB12A.global.corp.sap> <547F4914.1030904@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF36139@DEWDFEMB12A.global.corp.sap> <54809196.7030306@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF39BA5@DEWDFEMB12A.global.corp.sap> <5480AB58.80604@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF3D6BB@DEWDFEMB12A.global.corp.sap> <54889D61.30905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF3E1C5@DEWDFEMB12A.global.corp.sap> <54989B9F.2070903@oracle.com> <549B48A3.1070303@oracle.com> <54A2089D.1020105@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF53F70@DEWDFEMB12A.global.corp.sap> <54A2C966.5010306@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF67EB6@DEWDFEMB12A.global.corp.sap> <54A6C6C3.1050806@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF69210@DEWDFEMB12A.global.corp.sap> <54AAC086.2060600@oracle.com> <54AAC34F.2050807@oracle.com> Message-ID: <54AADD60.6060501@oracle.com> I looked through latest changes (webrev.03) and they look reasonable. Thanks, Vladimir On 1/5/15 9:01 AM, Coleen Phillimore wrote: > > On 1/5/15, 11:49 AM, Coleen Phillimore wrote: >> >> On 1/5/15, 4:24 AM, Lindenmaier, Goetz wrote: >>> Hi Coleen, >>> >>> It would be great if you could sponsor this change. >> >> I see no other comments so I'll push it today. > > I am planning to push this to hs-rt, btw. > Coleen > >>> >>>> The metaspace case probably can be fixed to use the try_reserve_heap() code maybe. >>> We wondered why the compressed class space is always above the heap. >>> If the heap is too big for unscaled, the compressed class space could be placed >>> in the lower 4G. >> >> The reason that we chose the compressed class space on the top of the heap is for a couple reasons. The time spent >> compressing and decompressing oops is more critical than the time spent doing the same for the _klass pointer, so we >> wanted to make the system more likely to get unscaled compressed oops, or even zero based compressed oops. >> >> On solaris, the space below the heap is limited because by default malloc uses it so we were limited in low address >> memory that we can use. We want the compressed class space code not be platform dependent. We also need to have the >> CDS archive at a fixed address near or adjacent to the compressed class space, so were running out at the bottom of >> the heap. >> >> The compressed class space is fixed at startup (just like PermGen was) and the classes are variable sized, so we had >> to pick a size fairly large so that it's unlikely for applications to run out. If above 32G, we might be able to >> implement code to allocate a new space which would use the same compression algorithm. We haven't had a request for >> that but it's something we could do. >> >> >>> The aarch port brings some changes here, and we would like to investigate >>> this on top of the aarch changes, if that's fine with you. >> >> So one thing I prototyped and really wanted to check in was what we called indirect class pointers. Instead of >> allocating classes in a fixed class space, we allocate a fixed array of Klass pointers and the objects in the heap >> point to this. It had a 8% performance degradation in specbb2008 compiler.compiler (iirc, or something like that). >> >> This solved so many problems though. If you investigated this sort of approach, I'd be very grateful. I haven't had >> time to go back to figure out why this had this degradation (we thought it was Java vtable accesses). >> >> Thanks, >> Coleen >>> >>> Thanks a lot for going through this complex review! >>> >>> Best regards, >>> Goetz. >>> >>> -----Original Message----- >>> From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] >>> Sent: Freitag, 2. Januar 2015 17:27 >>> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net >>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. >>> >>> >>> On 1/2/15, 10:09 AM, Lindenmaier, Goetz wrote: >>>> Hi Coleen, >>>> >>>> I looked at the cleanups and found both not straight forward to implement. >>>> >>>> 1) Merging ReservedHeap::initialize and ReservedHeapSpace::try_reserve_heap. >>>> The new code calls initialize() with requested_address = NULL, but metaspace.cpp >>>> passes a real requested_address. Therefore I can not clean up the calls to >>>> failed_to_reserve_as_requested() in initialize(). Without that cleanup, I need to pass flags >>>> into initialize() to configure all the different behaviours I would need in a >>>> merged case. >>>> I think with 'large' and 'requested_address' there is already enough cases >>>> in that method, so that I don't want to add more. >>> Okay, this is fine. More flags to special case functions aren't good >>> either. Can you file an RFE to look into refactoring so that the >>> duplicate code can be removed though? The metaspace case probably can >>> be fixed to use the try_reserve_heap() code maybe. >>> >>>> 2) Moving _noaccess_prefix to ReservedHeapSpace >>>> >>>> This does not work as ReservedHeapSpace is casted to >>>> ReservedSpace and returned by value. So ReservedHeapSpace can not have >>>> fields or virtual methods. >>>> It would be probably better to change the code to return ReservedHeapSpace >>>> by pointer, or, at least under that type, but I think that exceeds the scope of this change. >>> Too bad. >>> >>>> So is it ok if I leave the change as is wrt. to these cleanups? >>> Yes, this is fine. >>>> I extended test/runtime/CompressedOops/UseCompressedOops.java >>>> to combine the testcases of the different mode with the GCs and UseLargePages. >>>> I added the changes in the test to >>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >>> Is .03 with the additional changes? If so, you should have made an .04. >>> >>> I reviewed this again and it looks good. I think Vladimir has reviewed >>> an earlier version. I can sponsor this assuming there are no more comments. >>> >>> Coleen >>>> Best regards, >>>> Goetz. >>>> >>>> >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] >>>> Sent: Dienstag, 30. Dezember 2014 16:49 >>>> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net >>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. >>>> >>>> >>>> On 12/30/14, 8:42 AM, Lindenmaier, Goetz wrote: >>>>> Hi Coleen, >>>>> >>>>>> disjoint >>>>> It tries to say that the bits used for the base (36-63) are different from >>>>> those used for the oops / used as offset into the heap (0-35). >>>>> I chose the name according to this translation >>>>> http://dict.leo.org/#/search=disjoint&searchLoc=0&resultOrder=basic&multiwordShowSingle=on >>>>> Both the german adjective and verb fit to describe the property of the base. >>>>> I can add the following to the docu of the mode in universe.hpp:358 >>>>> // Disjoint: Bits used in base are disjoint from bits used >>>>> // for oops ==> oop = (cOop << 3) | base. One can disjoint >>>>> // the bits of an oop into base and compressed oop. >>>> Yes, that helps. It's a good distinct word so its good that we can make >>>> sense in the documentation. >>>>>> (I filed a bug to move these to the memory directory) >>>>> Good idea! >>>>> >>>>>> 272 void ReservedSpace::establish_noaccess_prefix() { >>>>>> This function should be in ReservedHeapSpace >>>>> Yes, I moved it. I also would like to move the field _noaccess_prefix, >>>>> but I would have to change a lot of code, like ReservedSpace::release(). >>>>> The problem is that _base is not the real base in ReservedHeapSpace. >>>>> Probably ReservedHeapSpace should have heap_size() and heap_base() >>>>> giving the correcte values. >>>>> I'll try to come up with a patch on top of this, maybe we should do it >>>>> in a separate bug. >>>> I just moved _noaccess_prefix and added a virtual >>>> ReservedHeapSpace::release() that adjusted the base and size with >>>> noaccess_prefix. I don't think the footprint of a vtable or virtual >>>> call makes any difference. >>>>>> Can you have arguments.cpp verify_range of HeapSearchSteps so this can't >>>>>> happen? >>>>> Done. >>>>> >>>>>> Rather move this to the end of ReservedHeapSpace constructor. >>>>>> assert(_base == NULL || ... >>>>> Yes, you are right, it should not be there twice. I moved it into the >>>>> constructor. It is only relevant for heaps, right? >>>> Yes. >>>>>> Code in try_reserve_heap() is mostly a copy of ReservedSpace::initialize() >>>>> You are right, requested_address in initialize is dead now. I'll remove that >>>>> and try to merge the two functions. >>>>> >>>>> I removed the verbose printing. Anyways, it's mostly useful when developing this. >>>> Thanks. I was trying to find some way to move the printing so it would >>>> make sense to me but ended up giving up on that. >>>>>> aligned_HBMA >>>>> Changed to aligned_heap_base_min_address. >>>>> >>>>>> Is "size" const in initialize_compressed_heap? >>>>> Yes. Added 'const'. >>>>> >>>>> I'll run tests with +UseLargePages. >>>>> >>>>> I had verified that the heap is at the exact same position for the code in >>>>> my first RFR, and with HeapSearchSteps=1 to make sure I don't break anything. >>>>> Vladimir asked me to increase that to 3. So far I still didn't find a case where >>>>> this changed anything. >>>>> (But I know from SAP JVM, which is tested on all the OS variants we support, >>>>> that there are cases where this can lead to a better - but then different - heap >>>>> placement.) >>>>> The only case where there is always a difference is Solaris 5.11 which >>>>> now can allocate zerobased memory. >>>>> The code passes the CDS jtreg tests. >>>>> >>>>> I updated the webrev.03 with the minor changes. >>>>> I'll come up with a new one with the cleanups of Reserved(Heap)Space >>>>> (_noaccess_prefix and redundant functions). >>>>> >>>>> Thanks for this thorough review!! >>>> Thank you for your willingness to make these changes. >>>> >>>> Happy New Year! >>>> Coleen >>>> >>>>> Happy new year, >>>>> Goetz. >>>>> >>>>> >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: hotspot-runtime-dev [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore >>>>> Sent: Dienstag, 30. Dezember 2014 03:06 >>>>> To: hotspot-runtime-dev at openjdk.java.net >>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. >>>>> >>>>> >>>>> Hi, I've reviewed more code. >>>>> >>>>> I'm sorry about the following comment that should have been asked >>>>> before. The name of the mode is very confusing. I can't figure out why >>>>> it's called "disjoint". I keep looking for a second heap base or some >>>>> code that makes the heap discontinuous. The heap base is really >>>>> specially aligned, right? So I went to the thesaurus.com: >>>>> >>>>> http://www.thesaurus.com/browse/aligned >>>>> >>>>> and "disjoin" is an antonym. Why was "disjoint" chosen? From the >>>>> code, it looks like "aligned" or "scaled" are more descriptive but >>>>> they're overused words in the code. But ScaledBasedNarrowOop seems more >>>>> descriptive than Disjoint. >>>>> >>>>> In virtualspace.cpp (I filed a bug to move these to the memory directory) >>>>> >>>>> 272 void ReservedSpace::establish_noaccess_prefix() { >>>>> >>>>> >>>>> This function should be in ReservedHeapSpace since ReservedSpace never >>>>> does this (and we never want it to). The function it replaced was in >>>>> ReservedSpace and that was wrong. Now that ReservedHeapSpace has >>>>> compressed oops logic in it, I think this function should be moved to >>>>> this class. Actually, I've modified your patch to move the >>>>> _noaccess_prefix field to ReservedHeapSpace. Maybe that's too much >>>>> change to your patch but since the handling of noaccess_prefix is >>>>> changed, it seemed like the thing to do. >>>>> >>>>> Can you have arguments.cpp verify_range of HeapSearchSteps so this can't >>>>> happen? >>>>> >>>>> guarantee(HeapSearchSteps > 0, "Don't set HeapSearchSteps to 0"); >>>>> >>>>> Can this be a function like check_encoding_for_mark_sweep() , or >>>>> something like that? It appears twice. Rather move this to the end of >>>>> ReservedHeapSpace constructor. >>>>> >>>>> assert(_base == NULL || >>>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == _base, >>>>> "area must be distinguishable from marks for mark-sweep"); >>>>> assert(_base == NULL || >>>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() == >>>>> &_base[size], >>>>> "area must be distinguishable from marks for mark-sweep"); >>>>> >>>>> Code in try_reserve_heap() is a mostly a copy of >>>>> ReservedSpace::initialize(), except the first version has a call to >>>>> failed_to_reserve_as_requested which looks like it's handling the case >>>>> for not being able to reserve the requested address for compressed oops >>>>> which seems like you don't want this. This is very confusing. There >>>>> shouldn't be two of these. >>>>> >>>>> It looks like in the compressed oops case, you call >>>>> ReservedSpace::initialize() only with NULL requested address so >>>>> failed_to_reserve_as_requested is dead code with this change. So it >>>>> seems like try_reserve_heap() could be ReservedSpace::initialize() with >>>>> the failed_to_reserve_as_requested eliminated and a special case for >>>>> handling unaligned addresses. >>>>> >>>>> The logic in initialize_compressed_heap() still doesn't make sense to >>>>> me. If the first try_reserve_range() succeeds at getting an unscaled >>>>> compressed oop base, then does PrintCompressedOopsMode && Verbose still >>>>> print: >>>>> >>>>> tty->print(" == U N S C A L E D ==\n"); >>>>> tty->print(" == Z E R O B A S E D ==\n"); >>>>> tty->print(" == D I S J O I N T B A S E ==\n"); >>>>> tty->print(" == H E A P B A S E D ==\n"); >>>>> >>>>> Why would you print all 4? It's the printing that makes this >>>>> confusing! Now I see how the next case checks that the base address is >>>>> a better allocation. If you changed the printing to a comment like: >>>>> >>>>> // UnscaledMode after testing previous base address is not better (or >>>>> something). >>>>> >>>>> Actually, you have these comments. I think the printing should be >>>>> removed because it's the opposite of helpful. >>>>> >>>>> I had trouble reading the parameter aligned_HBMA, can you change to >>>>> aligned_heap_base_min (I know it's longer but there's too many >>>>> calculations here that words will make easier to read). Or just >>>>> heap_base_min_address. >>>>> >>>>> zerobased_max = (char *)OopEncodingHeapMax - class_space; >>>>> >>>>> Odd that the solaris compiler can compile this and not the code below. >>>>> (minor point) I think this should be: >>>>> >>>>> zerobased_max = zerobased_max - class_space; >>>>> >>>>> Is "size" const in initialize_compressed_heap? (answer appears to be >>>>> yes) It's useful for reading the calculations that it never changes so >>>>> I think it should be const (it's a minor point too). >>>>> >>>>> I don't have any other comments, I've tried to do the calculations a bit >>>>> but I know that you've done this yourself. >>>>> >>>>> Also, can you run tests with -XX:+UseLargePages? I assume you've run >>>>> tests with the 4 garbage collectors. This is a tricky piece of code >>>>> that replaced something else difficult. We've had to change it often to >>>>> support other features like CDS and compressed class pointers so making >>>>> it as understandable as possible is worth the extra effort. I think >>>>> your new design looks cleaner. >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> >>>>> On 12/24/14, 6:13 PM, Coleen Phillimore wrote: >>>>>> One note for my first reviewed file. >>>>>> >>>>>> On 12/22/14, 5:30 PM, Coleen Phillimore wrote: >>>>>>> Hi Goetz, >>>>>>> >>>>>>> I'm looking at this version of the code >>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >>>>>>> >>>>>>> This passes our JPRT tests. Yeah! >>>>>>> >>>>>>> It's a significant change, so I'm actually reading the code with the >>>>>>> patches applied. >>>>>>> >>>>>>> In src/share/vm/memory/universe.cpp >>>>>>> >>>>>>> Can you put all this code from 742-776 in a new function above >>>>>>> initialize_heap(), like set_narrow_oop_values()? >>>>>>> The comment at 689-695 refer to code that's moved to >>>>>>> virtualspace.cpp. I think it needs to be reworded, updated for >>>>>>> disjoint oops mode or removed. The comment at 744-749 also refers to >>>>>>> code that's moved so should be removed also. The last two lines of >>>>>>> this comment refer to something whose implementation has changed, so >>>>>>> just remove them. >>>>>>> >>>>>>> On line >>>>>>> >>>>>>> 750 if ((uint64_t)Universe::heap()->reserved_region().end() > >>>>>>> UnscaledOopHeapMax) { >>>>>>> >>>>>>> Can you make this expression a variable because the same expression >>>>>>> appears on line 754. >>>>>>> >>>>>>> Why can't this code at 742-776 be with the compressed oops mode code >>>>>>> at line 836-839. If you make this set_narrow_oop_values() function, >>>>>>> can it be moved together? >>>>>> Apparently this doesn't work (it crashes on some platforms). I can't >>>>>> completely follow the control flow that makes this wrong, but ignore >>>>>> this suggestion. >>>>>> >>>>>> Coleen >>>>>>> Can you remove this comment too? I have no idea what it means. There >>>>>>> is no CATCH below (?) >>>>>>> >>>>>>> // We will never reach the CATCH below since Exceptions::_throw >>>>>>> will cause >>>>>>> // the VM to exit if an exception is thrown during initialization >>>>>>> >>>>>>> >>>>>>> ** We should file an RFE to move virtualspace.hpp/cpp to the >>>>>>> src/share/vm/memory directory! >>>>>>> >>>>>>> Sorry, this is partial, I'll continue tomorrow. >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> >>>>>>> >>>>>>> On 12/11/2014 06:18 AM, Lindenmaier, Goetz wrote: >>>>>>>> Hi Coleen, >>>>>>>> >>>>>>>> thanks for looking at this change! >>>>>>>> >>>>>>>>> get_attach_addresses_for_disjoint_mode function belongs in >>>>>>>>> virtualspace.cpp >>>>>>>> All information about the bounds of the modes comes from universe. >>>>>>>> This is >>>>>>>> related to is_disjoint_heap_base_address() and thus I left it in >>>>>>>> universe. >>>>>>>> But I'll move it, it can also be regarded as an implementation >>>>>>>> detail of >>>>>>>> initialize_compressed_heap(). I can make it static then, so that >>>>>>>> it's removed >>>>>>>> in 32 bit build, which is good. >>>>>>>> >>>>>>>>> Can you reduce some to at least 100? >>>>>>>> Done. I agree that's better . (I have a 2560x1440 screen, causing >>>>>>>> me to have >>>>>>>> big windows :)) >>>>>>>> >>>>>>>>> Why is highest_start not const char* ? >>>>>>>> Removed the const. >>>>>>>> >>>>>>>>> The control flow in initialize_compressed_heap makes no sense to me. >>>>>>>> Vladimir had the same problem, I added comment at line 493 to make >>>>>>>> that clearer. >>>>>>>> One problem of the previous implementation was that, if the os layer >>>>>>>> returned an address with desired properties, but not at the requested >>>>>>>> address, it was discarded. >>>>>>>> To fix this, try_reserve_heap always returns what it allocated, and I >>>>>>>> check afterwards. >>>>>>>> Therefore the checks of the next mode include the check of the >>>>>>>> previous one. So if I try to allocate disjoint, but get unscaled, >>>>>>>> I will detect that and return the unscaled heap. >>>>>>>> If I implement a return, I just have to add tests. I don't get rid of >>>>>>>> other tests. >>>>>>>> >>>>>>>>> Also, our 32 bit platforms don't like: 1 * SIZE_64K * SIZE_32G, >>>>>>>> I changed that to use UCONST64 macro. That should help I guess. >>>>>>>> I also have these in globalDefinitions_xlc.hpp. Should I move them >>>>>>>> to globalDefinitions.hpp:200, after definition of K, M and G? >>>>>>>> >>>>>>>> I uploaded a new webrev: >>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Goetz. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: hotspot-runtime-dev >>>>>>>> [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of >>>>>>>> Coleen Phillimore >>>>>>>> Sent: Mittwoch, 10. Dezember 2014 20:22 >>>>>>>> To: hotspot-runtime-dev at openjdk.java.net >>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>> >>>>>>>> >>>>>>>> Hi Goetz, >>>>>>>> >>>>>>>> I have some initial comments, which are much less detailed than >>>>>>>> Vladimir's comments. I haven't actually processed all the >>>>>>>> implementation details yet. >>>>>>>> >>>>>>>> I think get_attach_addresses_for_disjoint_mode function belongs in >>>>>>>> virtualspace.cpp and not in universe. I like that the compressed oop >>>>>>>> reservation logic is moved to virtualspace.cpp. I think this is an >>>>>>>> improvement over the Universe::preferred_heap_base() logic which was >>>>>>>> also difficult. >>>>>>>> >>>>>>>> The Hotspot coding style doesn't dictate 80 columns anymore (there is >>>>>>>> debate whether it should have or not, which we try not to have this >>>>>>>> debate), but I found the very long lines in this code inconsistent with >>>>>>>> other Hotspot code. I had to expand my window to cover my whole >>>>>>>> screen. Can you reduce some to at least 100? >>>>>>>> >>>>>>>> For example, I aligned these parameters to see better since there >>>>>>>> are so >>>>>>>> many (the indentation isn't right in email). >>>>>>>> >>>>>>>> void ReservedHeapSpace::try_reserve_range(char *const highest_start, >>>>>>>> char *lowest_start, >>>>>>>> size_t >>>>>>>> attach_point_alignment, >>>>>>>> char *aligned_HBMA, // >>>>>>>> HeapBaseMinAddress >>>>>>>> char *upper_bound, >>>>>>>> size_t size, >>>>>>>> size_t alignment, >>>>>>>> bool large) { >>>>>>>> >>>>>>>> Why is highest_start not const char* ? Doesn't char* const >>>>>>>> highest_start just restrict you from changing the pointer and not what >>>>>>>> it points to? This seems odd to do. >>>>>>>> >>>>>>>> The control flow in initialize_compressed_heap makes no sense to me. >>>>>>>> It seems like there should be an exit when the best allocation for >>>>>>>> compression is achieved. But it falls through after >>>>>>>> try_reserve_range(). I can't figure out why it should fall through.... >>>>>>>> >>>>>>>> I was expecting something like: >>>>>>>> >>>>>>>> if (PrintCompressedOopsMode && Verbose) { >>>>>>>> tty->print(" == H E A P B A S E M I N A D D R E S S ==\n"); >>>>>>>> } >>>>>>>> get the heap at aligned HeapBaseMinAddress, return if success... >>>>>>>> >>>>>>>> if (PrintCompressedOopsMode && Verbose) { >>>>>>>> tty->print(" == U N S C A L E D ==\n"); >>>>>>>> } >>>>>>>> try to get heap base for unscaled access, return if successful >>>>>>>> >>>>>>>> etc. >>>>>>>> >>>>>>>> >>>>>>>> You should make this into a little function for each return and for the >>>>>>>> end of the initialize function, or move it to the ReservedHeapSpace >>>>>>>> constructor (caller). >>>>>>>> >>>>>>>> assert(_base == NULL || >>>>>>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == _base, >>>>>>>> "area must be distinguishable from marks for mark-sweep"); >>>>>>>> assert(_base == NULL || >>>>>>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() == >>>>>>>> &_base[size], >>>>>>>> "area must be distinguishable from marks for mark-sweep"); >>>>>>>> } >>>>>>>> >>>>>>>> I ran this through JPRT and this line didn't compile on macos: >>>>>>>> >>>>>>>> const uint64_t num_attempts_to_try = MIN2(HeapSearchSteps, >>>>>>>> num_attempts_possible); >>>>>>>> >>>>>>>> I'm retrying now as: >>>>>>>> >>>>>>>> const uint64_t num_attempts_to_try = >>>>>>>> MIN2((uint64_t)HeapSearchSteps, num_attempts_possible); >>>>>>>> >>>>>>>> Sorry for the delay looking at this. This is a big change that I >>>>>>>> needed >>>>>>>> to make more time to read. I am pleased that it's a performance >>>>>>>> improvement. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>>>>> >>>>>>>> On 12/8/14, 10:54 AM, Lindenmaier, Goetz wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> This is just a ping to gc/rt mailing lists to reach appropriate >>>>>>>>> people. >>>>>>>>> >>>>>>>>> I please need a reviewer from gc or rt, could somebody have a >>>>>>>>> look at this? >>>>>>>>> >>>>>>>>> Short summary: >>>>>>>>> - new cOops mode disjointbase that allows optimizations on PPC >>>>>>>>> improving over heapbased >>>>>>>>> - search for heaps: finds zerobased on sparc Solaris 11 and Aix >>>>>>>>> - concentrate cOops heap allocation code in one function >>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>>> >>>>>>>>> Please reply only to the original thread in hotspot-dev to keep this >>>>>>>>> local. >>>>>>>>> >>>>>>>>> Thanks and best regards, >>>>>>>>> Goetz. >>>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>> Sent: Donnerstag, 4. Dezember 2014 19:44 >>>>>>>>> To: Lindenmaier, Goetz >>>>>>>>> Cc: 'hotspot-dev developers' >>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>> >>>>>>>>> This looks good to me. >>>>>>>>> >>>>>>>>> Now we need second review since changes are significant. Preferable >>>>>>>>> from >>>>>>>>> GC group since you changed ReservedHeapSpace. They will be affected >>>>>>>>> most. Review from Runtime is also welcome. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Vladimir >>>>>>>>> >>>>>>>>> On 12/4/14 10:27 AM, Lindenmaier, Goetz wrote: >>>>>>>>>> Hi Vladimir. >>>>>>>>>> >>>>>>>>>> Sorry. I updated the webrev once more. Hope it's fine now. >>>>>>>>>> At least I can write comments :) >>>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> Goetz >>>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>>> Sent: Thursday, December 04, 2014 5:54 PM >>>>>>>>>> To: Lindenmaier, Goetz >>>>>>>>>> Cc: 'hotspot-dev developers' >>>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>> >>>>>>>>>> I spotted an other bug. >>>>>>>>>> You replaced !_base with _base != NULL when moved code to >>>>>>>>>> try_reserve_range() - it should be _base == NULL. >>>>>>>>>> The same problem in asserts: >>>>>>>>>> >>>>>>>>>> + assert(_base != NULL || >>>>>>>>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == >>>>>>>>>> _base, >>>>>>>>>> + "area must be distinguishable from marks for mark-sweep"); >>>>>>>>>> + assert(_base != NULL || >>>>>>>>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() == >>>>>>>>>> &_base[size], >>>>>>>>>> + "area must be distinguishable from marks for mark-sweep"); >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Also you did not remove _base && in next place: >>>>>>>>>> >>>>>>>>>> + (_base && _base + size > zerobased_max))) { // Unscaled >>>>>>>>>> delivered an arbitrary address. >>>>>>>>>> >>>>>>>>>> New comment is good. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Vladimri >>>>>>>>>> >>>>>>>>>> On 12/4/14 1:45 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>> Hi Vladimir, >>>>>>>>>>> >>>>>>>>>>>> Add more extending comment explaining that. >>>>>>>>>>> The comment for try_reserve_heap was meant to explain that. >>>>>>>>>>> I further added a comment in initialize_compressed_heap(). >>>>>>>>>>> >>>>>>>>>>>> You need another parameter to pass UnscaledOopHeapMax or >>>>>>>>>>>> zerobased_max. >>>>>>>>>>> Oh, thanks a lot! That's important. Fixed. >>>>>>>>>>> >>>>>>>>>>>> I mean that you already checked _base == NULL so on other side >>>>>>>>>>>> of || _base != NULL - why you need (_base &&) check? >>>>>>>>>>> Sorry, now I got it. Removed. >>>>>>>>>>> >>>>>>>>>>> I updated the webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>>>>> >>>>>>>>>>> Increment on top of the increment :) >>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/incremental_diffs2.patch >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Goetz. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -----Original Message----- >>>>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>>>> Sent: Mittwoch, 3. Dezember 2014 18:32 >>>>>>>>>>> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' >>>>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>>> >>>>>>>>>>> Comments are below. >>>>>>>>>>> >>>>>>>>>>> On 12/3/14 5:49 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>>> Hi Vladimir, >>>>>>>>>>>> >>>>>>>>>>>> thanks for looking at the change! See my comments inline below. >>>>>>>>>>>> >>>>>>>>>>>> I made a new webrev: >>>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>>>>>> >>>>>>>>>>>> Incremental changes: >>>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/incremental_diffs.patch >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Best regards, >>>>>>>>>>>> Goetz. >>>>>>>>>>>> >>>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>>>>>> Sent: Mittwoch, 3. Dezember 2014 00:46 >>>>>>>>>>>>> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' >>>>>>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>>>>> >>>>>>>>>>>>> This looks good to me. Someone in runtime/gc have to look on it >>>>>>>>>>>>> too. >>>>>>>>>>>>> >>>>>>>>>>>>> universe.cpp about >>>>>>>>>>>>> SystemProperty("com.sap.vm.test.compressedOopsMode" >>>>>>>>>>>>> we have: >>>>>>>>>>>>> java.vm.info=mixed mode, sharing >>>>>>>>>>>>> so we can have: >>>>>>>>>>>>> java.vm.compressedOopsMode=... >>>>>>>>>>>> Yes, that's good for me. Fixed. >>>>>>>>>>>> >>>>>>>>>>>>> I am not expert in properties names but I don't want to have >>>>>>>>>>>>> 'com.sap' >>>>>>>>>>>>> in VM's property name. >>>>>>>>>>>>> virtualspace.cpp: >>>>>>>>>>>>> Could you fix release() - it does not reset _alignment? >>>>>>>>>>>> Fixed. >>>>>>>>>>>> >>>>>>>>>>>>> In try_reserve_heap(), please, use (base == NULL) instead of >>>>>>>>>>>>> (!base). >>>>>>>>>>>>> And you don't need 'return;' in alignment check at the end of >>>>>>>>>>>>> method. >>>>>>>>>>>> Fixed. >>>>>>>>>>>> >>>>>>>>>>>>> In initialize_compressed_heap() again (!_base). >>>>>>>>>>>> Fixed. >>>>>>>>>>>> >>>>>>>>>>>>> You don't stop (check >>>>>>>>>>>>> (base == NULL)) after successful unscaled, zerobased, disjointbase >>>>>>>>>>>>> allocations. You need to separate them with the check: >>>>>>>>>>>>> >>>>>>>>>>>>> + >>>>>>>>>>>>> + } >>>>>>>>>>>>> + } >>>>>>>>>>>>> + if (_base == NULL) { >>>>>>>>>>>>> + >>>>>>>>>>>>> + if (PrintCompressedOopsMode && Verbose) { >>>>>>>>>>>>> + tty->print(" == Z E R O B A S E D ==\n"); >>>>>>>>>>>>> + } >>>>>>>>>>>>> and so on. >>>>>>>>>>>> No, I can't and don't want to check for _base != NULL. >>>>>>>>>>>> I always keep the result of the last try, also if it didn't >>>>>>>>>>>> fulfil the required properties. >>>>>>>>>>>> So I take that result and go into the next check. That check >>>>>>>>>>>> might succeed >>>>>>>>>>>> with the heap allocated before. >>>>>>>>>>>> This allows me to separate allocation and placement criteria, >>>>>>>>>>>> and to have the >>>>>>>>>>>> placement criteria checked in only one place (per mode). >>>>>>>>>>>> Only for HeapBaseMinAddress I don't do it that way, I explicitly >>>>>>>>>>>> call release(). >>>>>>>>>>>> This way I can enforce mode heapbased. >>>>>>>>>>> I see what you are saying. It was not clear from comments what is >>>>>>>>>>> going on. >>>>>>>>>>> Add more extending comment explaining that. >>>>>>>>>>> >>>>>>>>>>>>> num_attempts calculation and while() loop are similar in >>>>>>>>>>>>> unscaled and >>>>>>>>>>>>> zerobased cases. Could you move it into a separate method? >>>>>>>>>>>> I can do that, but I don't like it as I have to pass in 7 >>>>>>>>>>>> parameters. >>>>>>>>>>> You need an other parameter to pass UnscaledOopHeapMax or >>>>>>>>>>> zerobased_max. >>>>>>>>>>> >>>>>>>>>>>> That makes the code not much more readable. The function will >>>>>>>>>>>> look like this: >>>>>>>>>>> I think initialize_compressed_heap() is more readable now. >>>>>>>>>>> >>>>>>>>>>>> void ReserveHeapSpace::try_reserve_range(char *const >>>>>>>>>>>> highest_start, char *lowest_start, size_t attach_point_alignment, >>>>>>>>>>>> char >>>>>>>>>>>> *aligned_HBMA, size_t size, size_t alignment, bool large) { >>>>>>>>>>>> guarantee(HeapSearchSteps > 0, "Don't set HeapSearchSteps >>>>>>>>>>>> to 0"); >>>>>>>>>>>> >>>>>>>>>>>> const size_t attach_range = highest_start - lowest_start; >>>>>>>>>>>> // Cap num_attempts at possible number. >>>>>>>>>>>> // At least one is possible even for 0 sized attach range. >>>>>>>>>>>> const uint64_t num_attempts_possible = (attach_range / >>>>>>>>>>>> attach_point_alignment) + 1; >>>>>>>>>>>> const uint64_t num_attempts_to_try = >>>>>>>>>>>> MIN2(HeapSearchSteps, num_attempts_possible); >>>>>>>>>>>> >>>>>>>>>>>> const size_t stepsize = align_size_up(attach_range / >>>>>>>>>>>> num_attempts_to_try, attach_point_alignment); >>>>>>>>>>>> >>>>>>>>>>>> // Try attach points from top to bottom. >>>>>>>>>>>> char* attach_point = highest_start; >>>>>>>>>>>> while (attach_point >= lowest_start && >>>>>>>>>>>> attach_point <= highest_start && // Avoid wrap >>>>>>>>>>>> around. >>>>>>>>>>>> (!_base || _base < aligned_HBMA || _base + size > >>>>>>>>>>>> (char *)UnscaledOopHeapMax)) { >>>>>>>>>>>> try_reserve_heap(size, alignment, large, attach_point); >>>>>>>>>>>> attach_point -= stepsize; >>>>>>>>>>>> } >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> In disjointbase while() condition no need for _base second check: >>>>>>>>>>>>> + (_base == NULL || >>>>>>>>>>>>> + ((_base + size > (char *)OopEncodingHeapMax) && >>>>>>>>>>>> I need this for the same reason as above: This is the check for >>>>>>>>>>>> successful allocation. >>>>>>>>>>> I mean that you already checked _base == NULL so on other side of >>>>>>>>>>> || _base != NULL - why you need (_base &&) check? >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Vladimir >>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Vladimir >>>>>>>>>>>> >>>>>>>>>>>> On 11/21/14 5:31 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> I prepared a new webrev trying to cover all the issues >>>>>>>>>>>>> mentioned below. >>>>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.01/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I moved functionality from os.cpp and universe.cpp into >>>>>>>>>>>>> ReservedHeapSpace::initialize_compressed_heap(). >>>>>>>>>>>>> This class offers to save _base and _special, which I would >>>>>>>>>>>>> have to reimplement >>>>>>>>>>>>> if I had improved the methods I had added to os.cpp to also >>>>>>>>>>>>> allocate large page >>>>>>>>>>>>> heaps. >>>>>>>>>>>>> Anyways, I think this class is the right place to gather code >>>>>>>>>>>>> trying to reserve >>>>>>>>>>>>> the heap. >>>>>>>>>>>>> Also, I get along without setting the shift, base, >>>>>>>>>>>>> implicit_null_check etc. fields >>>>>>>>>>>>> of Universe, so there is no unnecessary calling back and forth >>>>>>>>>>>>> between the two >>>>>>>>>>>>> classes. >>>>>>>>>>>>> Universe gets the heap back, and then sets the properties it >>>>>>>>>>>>> needs to configure >>>>>>>>>>>>> the compressed oops. >>>>>>>>>>>>> All code handling the noaccess prefix is in a single method, too. >>>>>>>>>>>>> >>>>>>>>>>>>> Best regards, >>>>>>>>>>>>> Goetz. >>>>>>>>>>>>> >>>>>>>>>>>>> Btw, I had to workaround a SS12u1 problem: it wouldn't compile >>>>>>>>>>>>> char * x = (char*)UnscaledOopHeapMax - size in 32-bit mode. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>>> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] >>>>>>>>>>>>> On Behalf Of Lindenmaier, Goetz >>>>>>>>>>>>> Sent: Montag, 17. November 2014 09:33 >>>>>>>>>>>>> To: 'Vladimir Kozlov'; 'hotspot-dev at openjdk.java.net' >>>>>>>>>>>>> Subject: RE: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Vladimir, >>>>>>>>>>>>> >>>>>>>>>>>>>> It is very significant rewriting and it takes time to evaluate >>>>>>>>>>>>>> it. >>>>>>>>>>>>> Yes, I know ... and I don't want to push, but nevertheless a ping >>>>>>>>>>>>> can be useful sometimes. Thanks a lot for looking at it. >>>>>>>>>>>>> >>>>>>>>>>>>>> And I would not say it is simpler then before :) >>>>>>>>>>>>> If I fix what you propose it's gonna get even more simple ;) >>>>>>>>>>>>>> These is what I found so far. >>>>>>>>>>>>>> The idea to try to allocate in a range instead of just below >>>>>>>>>>>>>> UnscaledOopHeapMax or OopEncodingHeapMax is good. So I would >>>>>>>>>>>>>> ask to do >>>>>>>>>>>>>> several attempts (3?) on non_PPC64 platforms too. >>>>>>>>>>>>> Set to 3. >>>>>>>>>>>>> >>>>>>>>>>>>>> It is matter of preference but I am not comfortable with >>>>>>>>>>>>>> switch in loop. >>>>>>>>>>>>>> For me sequential 'if (addr == 0)' checks is simpler. >>>>>>>>>>>>> I'll fix this. >>>>>>>>>>>>> >>>>>>>>>>>>>> One thing worries me that you release found space and try to >>>>>>>>>>>>>> get it >>>>>>>>>>>>>> again with ReservedHeapSpace. Is it possible to add new >>>>>>>>>>>>>> ReservedHeapSpace ctor which simple use already allocated space? >>>>>>>>>>>>> This was to keep diff's small, but I also think a new >>>>>>>>>>>>> constructor is good. >>>>>>>>>>>>> I'll fix this. >>>>>>>>>>>>> >>>>>>>>>>>>>> The next code in ReservedHeapSpace() is hard to understand (): >>>>>>>>>>>>>> (UseCompressedOops && (requested_address == NULL || >>>>>>>>>>>>> requested_address+size > (char*)OopEncodingHeapMax) ? >>>>>>>>>>>>>> may be move all this into noaccess_prefix_size() and add >>>>>>>>>>>>>> comments. >>>>>>>>>>>>> I have to redo this anyways if I make new constructors. >>>>>>>>>>>>> >>>>>>>>>>>>>> Why you need prefix when requested_address == NULL? >>>>>>>>>>>>> If we allocate with NULL, we most probably will get a heap where >>>>>>>>>>>>> base != NULL and thus need a noaccess prefix. >>>>>>>>>>>>> >>>>>>>>>>>>>> Remove next comment in universe.cpp: >>>>>>>>>>>>>> // SAPJVM GL 2014-09-22 >>>>>>>>>>>>> Removed. >>>>>>>>>>>>> >>>>>>>>>>>>>> Again you will release space so why bother to include space >>>>>>>>>>>>>> for classes?: >>>>>>>>>>>>>> + // For small heaps, save some space for compressed >>>>>>>>>>>>>> class pointer >>>>>>>>>>>>>> + // space so it can be decoded with no base. >>>>>>>>>>>>> This was done like this before. We must assure the upper bound >>>>>>>>>>>>> of the >>>>>>>>>>>>> heap is low enough that the compressed class space still fits >>>>>>>>>>>>> in there. >>>>>>>>>>>>> >>>>>>>>>>>>> virtualspace.cpp >>>>>>>>>>>>> >>>>>>>>>>>>>> With new code size+noaccess_prefix could be requested. But >>>>>>>>>>>>>> later it is >>>>>>>>>>>>>> not used if WIN64_ONLY(&& UseLargePages) and you will have empty >>>>>>>>>>>>>> non-protected page below heap. >>>>>>>>>>>>> There's several points to this: >>>>>>>>>>>>> * Also if not protectable, the heap base has to be below >>>>>>>>>>>>> the real start of the >>>>>>>>>>>>> heap. Else the first object in the heap will be >>>>>>>>>>>>> compressed to 'null' >>>>>>>>>>>>> and decompression will fail. >>>>>>>>>>>>> * If we don't reserve the memory other stuff can end up >>>>>>>>>>>>> in this space. On >>>>>>>>>>>>> errors, if would be quite unexpected to find memory >>>>>>>>>>>>> there. >>>>>>>>>>>>> * To get a heap for the new disjoint mode I must control >>>>>>>>>>>>> the size of this. >>>>>>>>>>>>> Requesting a heap starting at (aligned base + >>>>>>>>>>>>> prefix) is more likely to fail. >>>>>>>>>>>>> * The size for the prefix must anyways be considered >>>>>>>>>>>>> when deciding whether the >>>>>>>>>>>>> heap is small enough to run with compressed oops. >>>>>>>>>>>>> So distinguishing the case where we really can omit this would >>>>>>>>>>>>> require >>>>>>>>>>>>> quite some additional checks everywhere, and I thought it's not >>>>>>>>>>>>> worth it. >>>>>>>>>>>>> >>>>>>>>>>>>> matcher.hpp >>>>>>>>>>>>> >>>>>>>>>>>>>> Universe::narrow_oop_use_implicit_null_checks() should be true >>>>>>>>>>>>>> for such >>>>>>>>>>>>>> case too. So you can add new condition with || to existing >>>>>>>>>>>>>> ones. The >>>>>>>>>>>>>> only condition you relax is base != NULL. Right? >>>>>>>>>>>>> Yes, that's how it's intended. >>>>>>>>>>>>> >>>>>>>>>>>>> arguments.* files >>>>>>>>>>>>> >>>>>>>>>>>>>> Why you need PropertyList_add changes. >>>>>>>>>>>>> Oh, the code using it got lost. I commented on this in the >>>>>>>>>>>>> description in the webrev. >>>>>>>>>>>>> "To more efficiently run expensive tests in various compressed >>>>>>>>>>>>> oop modes, we set a property with the mode the VM is running >>>>>>>>>>>>> in. So far it's called "com.sap.vm.test.compressedOopsMode" >>>>>>>>>>>>> better suggestions are welcome (and necessary I guess). Our >>>>>>>>>>>>> long running tests that are supposed to run in a dedicated >>>>>>>>>>>>> compressed oop mode check this property and abort themselves if >>>>>>>>>>>>> it's not the expected mode." >>>>>>>>>>>>> When I know about the heap I do >>>>>>>>>>>>> Arguments::PropertyList_add(new >>>>>>>>>>>>> SystemProperty("com.sap.vm.test.compressedOopsMode", >>>>>>>>>>>>> narrow_oop_mode_to_string(narrow_oop_mode()), >>>>>>>>>>>>> false)); >>>>>>>>>>>>> in universe.cpp. >>>>>>>>>>>>> On some OSes it's deterministic which modes work, there we >>>>>>>>>>>>> don't start such tests. >>>>>>>>>>>>> Others, as you mentioned OSX, are very indeterministic. Here >>>>>>>>>>>>> we save testruntime with this. >>>>>>>>>>>>> But it's not that important. >>>>>>>>>>>>> We can still parse the PrintCompresseOopsMode output after the >>>>>>>>>>>>> test and discard the >>>>>>>>>>>>> run. >>>>>>>>>>>>> >>>>>>>>>>>>>> Do you have platform specific changes? >>>>>>>>>>>>> Yes, for ppc and aix. I'll submit them once this is in. >>>>>>>>>>>>> >>>>>>>>>>>>> From your other mail: >>>>>>>>>>>>>> One more thing. You should allow an allocation in the range >>>>>>>>>>>>>> when returned from OS allocated address does not match >>>>>>>>>>>>>> requested address. We had such cases on OSX, for example, when >>>>>>>>>>>>>> OS allocates at different address but still inside range. >>>>>>>>>>>>> Good point. I'll fix that in os::attempt_reserve_memory_in_range. >>>>>>>>>>>>> >>>>>>>>>>>>> I'll ping again once a new webrev is done! >>>>>>>>>>>>> >>>>>>>>>>>>> Best regards, >>>>>>>>>>>>> Goetz. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 11/10/14 6:57 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I need to improve a row of things around compressed oops heap >>>>>>>>>>>>>> handling >>>>>>>>>>>>>> to achieve good performance on ppc. >>>>>>>>>>>>>> I prepared a first webrev for review: >>>>>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.00/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> A detailed technical description of the change is in the >>>>>>>>>>>>>> webrev and according bug. >>>>>>>>>>>>>> >>>>>>>>>>>>>> If requested, I will split the change into parts with more >>>>>>>>>>>>>> respective less impact on >>>>>>>>>>>>>> non-ppc platforms. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The change is derived from well-tested code in our VM. >>>>>>>>>>>>>> Originally it was >>>>>>>>>>>>>> crafted to require the least changes of VM coding, I changed >>>>>>>>>>>>>> it to be better >>>>>>>>>>>>>> streamlined with the VM. >>>>>>>>>>>>>> I tested this change to deliver heaps at about the same >>>>>>>>>>>>>> addresses as before. >>>>>>>>>>>>>> Heap addresses mostly differ in lower bits. In some cases >>>>>>>>>>>>>> (Solaris 5.11) a heap >>>>>>>>>>>>>> in a better compressed oops mode is found, though. >>>>>>>>>>>>>> I ran (and adapted) test/runtime/CompressedOops and >>>>>>>>>>>>>> gc/arguments/TestUseCompressedOops*. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>> Goetz. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >> > From vladimir.kozlov at oracle.com Mon Jan 5 19:59:27 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 05 Jan 2015 11:59:27 -0800 Subject: 8068053: AARCH64: C1 and C2 compilers In-Reply-To: <54AAD3FC.8000003@redhat.com> References: <54A6A8F5.4040907@redhat.com> <928DC166-7256-4BCA-A9CA-C3ADB05834E5@oracle.com> <54AAD3FC.8000003@redhat.com> Message-ID: <54AAED1F.1000908@oracle.com> On 1/5/15 10:12 AM, Andrew Haley wrote: > On 01/05/2015 05:50 PM, Roland Westrelin wrote: >> Hi Andrew, >> >>> http://cr.openjdk.java.net/~aph/aarch64-8068053-2/ Looks good. >> >> Shouldn?t c1_FpuStackSim_aarch64.[ch]pp be empty like it is on sparc? > > Oh yes, that's a hangover from x86. Will remove. How embarrassing. :-) > >> What's ad_encode.m4 for? > > The encode patterns in aarch64.ad are very repetitive, so I used m4 to generate > them. Could you explain how you process it? I don't see aarch64_enc_strw_immn() in .ad file. And I don't remember changes in makefiles which use it. And it is missing Copyright header. Thanks, Vladimir > >> Typo: >> 831 // advance declaratuons for helper functions to convert register > > OK. > >> in aarch64.ad >> >> That looks good to me. > > Thanks, > Andrew. > > From vladimir.kozlov at oracle.com Mon Jan 5 20:47:59 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 05 Jan 2015 12:47:59 -0800 Subject: 8068055: AARCH64: os_cpu/ In-Reply-To: <54A6A977.4080405@redhat.com> References: <54A6A977.4080405@redhat.com> Message-ID: <54AAF87F.9090706@oracle.com> linux_aarch64.S is missing Copyright header. linux_aarch64.ad: "AMD64 Linux Architecture". os_linux_aarch64.cpp, print registers methods lists x86 registers. And #ifndef AMD64. The rest seems fine. Thanks, Vladimir On 1/2/15 6:21 AM, Andrew Haley wrote: > http://cr.openjdk.java.net/~aph/aarch64-8068055-1/ > > Thanks, > Andrew. > From jesper.wilhelmsson at oracle.com Mon Jan 5 21:05:18 2015 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Mon, 05 Jan 2015 22:05:18 +0100 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <775F23B6-41A1-4904-8969-1C2F7407D899@lnu.se> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> <54AA143B.2080900@oracle.com> <775F23B6-41A1-4904-8969-1C2F7407D899@lnu.se> Message-ID: <54AAFC8E.1080702@oracle.com> Hi Erik, Attachments are stripped when sent to the list. If you send me two patches I can upload webrevs for the two versions. /Jesper Erik ?sterlund skrev den 5/1/15 18:41: > Hi David and Kim, > > Thank you David for putting this into context. > > If we want this to be done with the inheritance/template way (specifically the SelectBaseClass template, oh and other names are welcome, InjectOptionalBase?), then I see two options, so I implemented both so you can compare and pick a favourite. They both have exactly identical semantics: they check for the existence of an optional class to inject to the class hierarchy, and check that the inheritance isn?t fishy (the optional class for specialization must be a subclass of the required class for generalization). > > I think it all comes down to reusability vs code size. > > Approach 1. ?Reusability Maximum?: Build SelectBaseClass in terms of reusable components resembling boost/C++11 as much as possible. > (Note: Since Kim wanted IsBaseOf to require Derived to be complete (defined) to mimic other libraries, I now force a compiler error if parameters are incomplete as per request and instead use her proposal for checking if classes are defined, thanks for that contribution) > > Pros: 1) Many reusable metafunctions that can be reused in other contexts (for instance automatic closure specialization), and I believe we will have to include them sooner or later anyway. 2) Since they comply very closely with the C++11 standard, it will be easy to make a transition once all compilers support this some time in the distant past. > Cons: 380 LoC template infrastructure, including testing, for a perceivably small application (if not reused in the future). > > Approach 2. ?Code Minimum?: Get the same SelectBaseClass, but with minimal dependencies. Only 20 LoC for the necessary metafunction, some more for IsSame used only for testing purposes, and 156 LoC in total for the template infrastructure, making it much lighter. > > Pros: Minimal amount of code for the change (less than half). > Cons: No reusable metafunctions for the future which we quite likely will need later. > > I have attached the two solutions in this email so you can have a look and tell me which direction you want me to go before making a final webrev to make everyone happy. > > Personally I like approach 1 better but if it is concerning to commit too much code and there are strong opinions about this, I?m happy with approach 2 as well. > > Appendix with code for minimal implementation with no dependencies: > > template > class SelectBaseClass { > typedef char yes[1]; > typedef char no[2]; > > template > static yes &check(Optional*, T); > static no &check(Required*, int); > > struct OptionalCheckHost { > operator Required*() const; > operator Optional*(); > }; > > template > struct OptionalCheck { typedef O type; }; > > template > struct OptionalCheck { typedef R type; }; > public: > typedef typename OptionalCheck::type type; > }; > > Thanks, > /Erik > > >> On 05 Jan 2015, at 04:34, David Holmes wrote: >> >> Sorry for the top-post but just wanted to reset the context a little here. Originally [1] I made the comment: >> >> "It's fine to have generic shared approaches but there really needs to be a way to allow platform specific "overrides"." >> >> For the actual original RFR I said [2]: >> >> "Can we pause and give some more thought to a clean mechanism for >> allowing a shared implementation if desired with the ability to override if desired. I really do not like to see CPU specific ifdefs being added to shared code. (And I would also not like to see all platforms being forced to reimplement this natively). >> >> I'm not saying we will find a simple solution, but it would be nice if >> we could get a few folk to think about it before proceeding with the >> ifdefs :)" >> >> Erik then proposed three alternative approaches [3] and the simple variant was chosen [4] and presented for review. However Paul Hohensee also made comments about an inheritance-based approach and Erik floated the first template-based variant [5] and there was some discussion with Paul and Kim. But we then ended up with the simple solution, leaving an inheritance-based solution for future work [6] (which is what is what is being discussed now). >> >> This template-based meta-programming is not something I have any familiarity with. My initial thought is this seems overkill for the situation we have with the Atomic class - but as I said I'm ignorant of the technique being used here. >> >> David >> ----- >> >> [1] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015292.html >> [2] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015303.html >> [3] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015362.html >> [4] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015401.html >> [5] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-October/015586.html >> [6] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-November/015889.html >> >> On 2/01/2015 12:07 AM, Erik ?sterlund wrote: >>> Hi Kim, >>> >>>> On 01 Jan 2015, at 07:42, Kim Barrett wrote: >>>> >>>> On Dec 31, 2014, at 6:12 AM, Erik ?sterlund wrote: >>>>> >>>>>> The rest of these comments only matter if, contrary to my suggestion >>>>>> above, some additional infrastructure is thought necessary. >>>>>> >>>>> >>>>> This is what I originally thought too and hence the current solution. But voices said that using macros for this is, I quote, "that bad". >>>>> >>>>>> [?] >>>>>> >>>>>> This scales with the number of AtomicPlatform definitions, rather than >>>>>> the number of specialized functions as happens with the existing code. >>>>> >>>>> So your proposal is to still use ugly macros but move it to the class level instead, even though at the moment there is really only one member function that needs it. That feels a bit meh to be honest. IMO if we want to do this right, why go half the way and still rely on ugly macros? >>>> >>>> Please lay off with the perjorative "ugly macros". The preprocessor >>>> is part of C++, and isn't going away. It's a tool that can be used, >>>> or abused, and I think this is a fine use. To me, "ugly" is adding >>>> 800+ lines of code / tests / comments to eliminate a dozen lines of >>>> flag macro definitions and associated #ifdef?s. >>> >>> Well people are entitled to have different opinions of course. As I already mentioned, I did this because other reviewers (as well as me) found it ugly to use macros for conditionally specializing, which is the whole motivation of doing this, and was well understood from the beginning. But I?m fine with leaving it as macros if this is in general preferred and opinions have suddenly changed in this matter. >>> >>> Personally I don?t see how the number of rows of code define uglyness. Reusability, testing, documentation and well definedness in contracts/behaviours all lead to more lines of code, but I don?t think that equals uglyness. So while I disagree more code is uglier because it?s more, it?s all up to opinion and it?s quite pointless to discuss; you are entitled to think that is ugly. >>> >>>> If that additional infrastructure were amortized across more use-cases >>>> then it might be more appealing. I might even have some additional >>>> use-cases for something along that line, though not in its present >>>> form. (More on that below.) >>> >>> Yeah I specifically had in mind the wish to make a split between commercial extensions to open source code actually. This pattern allows easy optional class inheritance injection as a general design pattern to achieve this. And for my own purposes the ability to define asymmetric dekker synchronization to elide storeload fences when the platform (e.g. TSO) supports it but otherwise revert to a generalized implementation (using storeload fences). This is why I think a clean reusable strategy is almost always better because it is widely applicable, even if there would not be obvious other examples where it is useful. >>> >>>> >>>>>> I think the name "IsBaseOfAndDerived" would be better as >>>>>> "IsBaseAndDerived?. [?] >>>>>> >>>>>> The current implementation of IsBaseOfAndDerived (and therefore >>>>>> IsBaseOf) may (silently) provide incorrect answers when called with >>>>>> incomplete types. [?] >>>>>> >>>>>> The actual usage in this proposed change-set even relies on this >>>>>> flawed behavior of these metafunctions, since AtomicPlatform isn't >>>>>> always defined. [?] >>>>>> >>>>> >>>>> So what you are saying here is that I should rename IsBaseOfAndDerived to IsBaseAndDerived to more closely resemble other external libraries we do not use. But if I do that my behaviour is "incorrect" because it is not identical to that of the external library. And therefore my change would not work if it was made "correctly?. >>>> >>>> The proposed IsBaseOfAndDerived and IsBaseOf are being described and >>>> placed as general utilities. Their names are strongly reminiscent of >>>> existing facilities in other libraries, including the (C++11) standard >>>> library. Conforming to these external standards will avoid surprises, >>>> especially when the proposed facilities are even described as being >>>> the same (except for a noted corner case that is both hard to deal >>>> with in a portable manner and not actually very interesting, so I'm >>>> not complaining about that) in the review request email. >>>> >>>> Additionally, I think quietly providing a potentially incorrect answer >>>> in the case of incomplete types renders the offered code quite broken >>>> for use as a general utility. I think the requirement for complete >>>> types by the well-known / standard facilities is well justified. >>>> >>> >>> While I disagree it is universally incorrect (as long as it is well defined and tested) to not have the exact same requirement on completeness on the parameters as C++11/boost, who also already have different behaviour in this exact regard as pointed out by yourself, I can try to fix it if it bugs you, but it will make the infrastructure even larger which seems to be a concern. >>> >>>>> On the contrary this is a well tested feature on all our supported compilers and I did not intend to engineer it to be identical to some external library if it only causes trouble rather than helping. This is our own class and it can do what we want it to do and be called what we want it to be called, and I think that is fine as long as it is well tested, which it is. At least this is my opinion. Anyone else agree? >>>> >>>> Problem is, I think it's well tested code to solve the wrong problem. >>>> >>>> I suggest that what we're really looking for is not a base/derived >>>> relationship between a pair of classes, but that we actually want to >>>> determine whether a platform-specific class exists. >>>> >>>> As a result, I think there is more infrastructure in this change set >>>> than is actually needed. The present implementation of >>>> SelectBaseClass uses IsBaseOf, but doesn't need a lot of the >>>> functionality of that metafunction. There is no actual need for a >>>> Base/Derived relationship between the (possibly not defined) >>>> platform-specific class and the default platform-generic class, so the >>>> use of IsBaseOf is an unnecessary restriction that only >>>> serendipitously tests for defined or not, due to the previously >>>> discussed defect in this particular implementation. I also think the >>>> name SelectBaseClass is perhaps overly general. >>> >>> The main idea of SelectBaseClass is to inject an optional class into the class hierarchy for specialization of generalized behaviour. It is absolutely critical (and intended) that this class is constrained to be a derived class of the fall-back class - it is not an artifact nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, A must be able to rely on the superclass being at least C-like so that the behaviour of the superclass can be used reliably and transparently regardless of the existence of the optional B. If this contract would not hold, I think it could lead to some really weird and unreliable code which should not be encouraged (at least not according to the Liskov substitution principle etc). >>> >>> So what I need in the contract is a check if the optional class is defined and if it fits in the inheritance hierarchy: they are both important IMO. >>> >>> Yes it could have been less code. From the beginning I had a single trait class that checked for both the inheritance and whether the class is defined or not but you said you wanted the behaviour to more closely match that of third party libraries. This is why there is now a lot more code to more closely (but still not exactly) match that behaviour. >>> >>>> I might be able to use a similar approach in some code I've been >>>> working on as a background task. And while writing this reply I've >>>> thought of another place that might benefit from something along those >>>> lines. Thinking about the code under review in that context, I think >>>> some changes would make these other possible use-cases easier and >>>> clearer. >>>> >>>> I believe what is actually needed is a metatfunction something like: >>>> >>>> /** >>>> * Metafunction whose nested type member is Default if Specific is >>>> * incomplete, otherwise Specific. >>>> * >>>> * Specific and Default must both be non-cv qualified class types, and >>>> * must not be the same type. >>>> * >>>> * If Specific is incomplete at the point where this metafunction is >>>> * invoked, it must never be defined elsewhere in the program. >>>> */ >>>> template >>>> struct SelectPlatformBase; >>>> >>>> [Note: I'm not wedded to that name.] >>>> >>>> Now, it turns out to be hard / impossible to write a generic >>>> is_complete metafunction, due to ODR. Whether a type is complete >>>> can vary between translation units, and even within a translation >>>> unit. Having is_complete::value have different values depending on >>>> where it is instantiated is an ODR violation. (One can make progress >>>> by using anonymous namespaces and macro wrappers with __LINE__ >>>> concatenation, but there is still the fundamental question of whether >>>> is_complete really even makes semantic sense.) >>> >>> Yes and that makes me wonder if we care enough about the completeness semantics being the same as third party libraries (who are already inconsistent) if we do not need them to be, and on the contrary benefit from them not to be. >>> >>>> However, we don't need a fully general is_complete utility. We have a >>>> much more restricted use-case, where we want to select an optionally >>>> defined platform-specific class if it exists, and otherwise fall back >>>> to a more generic class. And we're not using this in arbitrary >>>> contexts, but only to select a platform-specialized implementation if >>>> such exists. >>>> >>>> Here's a lightly tested implementation of SelectPlatformBase: >>>> >>>> // --- SelectPlatformBase >>>> >>>> template >>>> struct EnableIf { typedef T type; }; >>>> >>>> template >>>> struct EnableIf { }; >>>> >>>> template >>>> struct If { typedef Then type; }; >>>> >>>> template >>>> struct If { typedef Else type; }; >>>> >>>> // Metafunction whose nested value member is true if T is defined >>>> // (complete), and false if T is incomplete. T must be a non-cv >>>> // qualified class type. If T is incomplete at the point where this >>>> // metafunction is invoked, it must never be defined elsewhere in the >>>> // program. >>>> template >>>> class IsClassDefined { >>>> typedef char yes[1]; >>>> typedef char no[2]; >>>> >>>> template >>>> static typename EnableIf::type & check(U*); >>>> static no& check(...); >>>> >>>> public: >>>> static const bool value = sizeof(check((T*)0)) == sizeof(yes); >>>> }; >>>> >>>> template >>>> struct SelectPlatformBase { >>>> typedef typename If< >>>> IsClassDefined::value, Specific, Default>::type type; >>>> }; >>>> >>>> // --- end SelectPlatformBase >>>> >>>> That's ~30 lines of code (+ tests and comments TBD) to do precisely >>>> what we need, replacing ~100 lines (+ tests and comments) that have >>>> various issues. (If and EnableIf should probably be hoisted out as >>>> separate utilities.) We don't actually need SelectPlatformBase, and >>>> could instead just directly use the If and IsClassDefined >>>> metafunctions, but giving this functionality a name might be clearer. >>> >>> As stated, I think it?s dangerous to allow conditional inheritance where there is no inheritance constraint on the optional class. Therefore I do not think this is better, although I agree it is smaller. >>> >>> I agree that our use case here is smaller. If the amount of code is the problem and less code (and hence fewer reusable components) is preferred, then the original implementation with a simple IsBaseOfAndDerived does the trick (without checking types/cv qualifiers) and could be done in just a few lines, and this is what I originally did before you told me to expand it to more closely resemble external libraries: >>> >>> template >>> class IsBaseOfAndDerived { >>> typedef char yes[1]; >>> typedef char no[2]; >>> >>> template >>> static yes &check(Derived*, T); >>> static no &check(Base*, int); >>> >>> template >>> struct IsBaseOfHost { >>> operator B*() const; >>> operator D*(); >>> }; >>> public: >>> static const bool value = sizeof(check(IsBaseOfHost(), int())) == sizeof(yes); >>> }; >>> >>> It checks the inheritance and existence of the Derived class which is exactly the constraints I need. >>> >>> If you do not want to expose it publicly and remove the reusability because it does not have identical semantics as third party libraries regarding the requirements of the Derived type to be complete, it could be an internal class of SelectBaseClass. >>> >>> I?m not going to value the reusability vs LoC, I leave that decision to you. >>> >>>> However, while I think there are other use cases for this specific >>>> utility, I still think the proposed change set as a whole is adding a >>>> lot of code just to avoid a a few lines of macro usage. That seems >>>> like a poor tradeoff to me. >>> >>> >>> Okay. >>> >>> In the end, we have to make a choice - what is more important, that the traits resemble some external library or code size. Because last time I proposed a small change and you wanted it to be closer to C++11 behaviour and I made it closer to that at the expense of more code. Now you are saying I have to make it even tighter to C++11 (or boost?), but also don?t want more lines of code which I believe is impossible given that I want to constrain both the class hierarchy to be reliable and check the existence of the optional class. This brings me to a few architectural questions which I may have (strong) opinions about but are not up to me to decide. >>> >>> 1. Do we want to replace the macros with template metaprogramming? 2 reviewers (and me) liked the metaprogramming approach and it was implemented because they (like me) did not want the macros. But you have a different opinion. I can?t judge who is right and wrong here. >>> >>> 2. Do we want to minimize the code size at the cost of reusability by making contracts of dependent components weaker and putting them as private classes in the SelectBaseClass? >>> >>> 3. If we do want template metaprogramming and want reusable components, the question is, is it important that the specification of completeness (which in general is almost impossible to implement correctly in a portable way as you pointed out) of the template arguments in IsBaseOf is identical as C++11/boost even though no code needs it to be and on the contrary some code benefits from it not to be? In that case which one of them do we, because they are already different anyway? There is a tradeoff between code size and complexity vs copying either one of the external libraries regarding requirements of completeness of template parameters. >>> >>> Thanks, >>> /Erik >>> > From christian.thalinger at oracle.com Mon Jan 5 23:51:55 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Mon, 5 Jan 2015 15:51:55 -0800 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54A6A933.8000204@redhat.com> References: <54A6A933.8000204@redhat.com> Message-ID: > On Jan 2, 2015, at 6:20 AM, Andrew Haley wrote: > > http://cr.openjdk.java.net/~aph/aarch64-8068054-1/ Sorry if this was discussed somewhere else but is BUILTIN_SIM something that?s still used? > > Thanks, > Andrew. > From kim.barrett at oracle.com Tue Jan 6 00:03:39 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 5 Jan 2015 19:03:39 -0500 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> Message-ID: <2ED21417-E8C1-4E11-AC2F-DED35A993714@oracle.com> On Jan 1, 2015, at 9:07 AM, Erik ?sterlund wrote: > > Personally I don?t see how the number of rows of code define uglyness. Reusability, testing, documentation and well definedness in contracts/behaviours all lead to more lines of code, but I don?t think that equals uglyness. So while I disagree more code is uglier because it?s more, it?s all up to opinion and it?s quite pointless to discuss; you are entitled to think that is ugly. Code surface area has an impact on understanding and maintenance. It's not the only factor, but it *is* a factor. That said, I looked more closely and found the webrev new line count substantially overstates the real size of the proposed change. (Nearly half of the new lines are copyright headers and similar boiler plate. A substantial fraction of the remainder are tests, which I wouldn't count as a maintenance negative.) So rather than 800 lines, its probably more like 100 new lines. That still seems to me to be a lot in order to eliminate a few flag macro definitions and the associated #ifdef, but seems much more plausible to amortize across multiple uses. Since I think between us we've come up with several more possible use-cases, I'm willing to withdraw the size complaint. > While I disagree it is universally incorrect (as long as it is well defined and tested) to not have the exact same requirement on completeness on the parameters as C++11/boost, who also already have different behaviour in this exact regard as pointed out by yourself, I can try to fix it if it bugs you, but it will make the infrastructure even larger which seems to be a concern. [The difference between boost and C++11 is_base_of is minor, with boost being more restrictive than necessary. I fully expect someone to eventually file a bug report about that difference and for the boost trait to be brought into conformance with the C++11 version. I think that difference isn't relevant to this discussion.] It is not correct as a general utility trait because without the requirement that "Derived" be complete one can very easily introduce an ODR violation. That's the rationale for the C++11 requirement. And it's better to check for that possibility, if possible, than to silently risk an unchecked ODR violation that could be challenging to track down. (C++11 doesn't require a diagnostic, leaving that as a QoI issue. g++ and boost issue a diagnostic for this; I haven't tested other compilers.) It could make sense to have a bespoke predicate for determining whether a conditionally defined class is derived from some known base, with a usage requirement that if not defined (e.g. complete) in the current translation unit that it will not be defined anywhere else in the program. However, I think such a predicate should have a more specific / informative name and/or shouldn't be classified as a general utility trait. But I think even renaming or moving it wouldn't be right, since I disagree with the need for such a predicate at all. [More below.] > The main idea of SelectBaseClass is to inject an optional class into the class hierarchy for specialization of generalized behaviour. It is absolutely critical (and intended) that this class is constrained to be a derived class of the fall-back class - it is not an artifact nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, A must be able to rely on the superclass being at least C-like so that the behaviour of the superclass can be used reliably and transparently regardless of the existence of the optional B. If this contract would not hold, I think it could lead to some really weird and unreliable code which should not be encouraged (at least not according to the Liskov substitution principle etc). In the proposed change and in most of the other use-cases I've thought about, no class-based inheritance relationship between the optionally defined class and the core "base" is required; duck type compatibility is sufficient, and I think often preferable. In the specific proposed change, I'm pretty sure there is nothing that requires any class-based inheritance relationship at all among Atomic, AtomicBase, or any AtomicPlatform class. Atomic implements most of the API, and delegates one piece to the selected "AtomicSuper" by explicit name qualification. If there were no inheritance relationship at all among those classes, nobody would notice (assuming the selection process were changed to not require such a relationship). If AtomicBase and AtomicPlatform were changed as I suggested in one of my earlier comments, to provide a protected cmpxchg_byte function with an unqualified call from Atomic, then Atomic would need to be directly derived from either AtomicBase or AtomicPlatform, but there would be no need for any inheritance relationship between AtomicBase or AtomicPlatform. The duck type compatibility of providing an appropriate cmpxchg_byte function is sufficient. (With such an approach I would also consider making the selected base for Atomic be private, since the inheritance is for implementation convenience and there's no external client that cares about that relationship.) > So what I need in the contract is a check if the optional class is defined and if it fits in the inheritance hierarchy: they are both important IMO. I think there are use-cases where appropriate inheritance placement of the optionally defined class is important, but as discussed above, I don't think this is one of those. Nor are most of the others that I've thought of. I don't want to have to introduce unnecessary inheritance in order to use the new infrastructure. In those cases where an inheritance relationship really *is* needed, I think the right approach is to use conditional existence for selection and then static assert the needed base/derived relationship(s). For that we would need an IsBaseAndDerived or IsBaseOf, but their use would be appropriately conditionalized on the existence of the optional class (possibly only implicitly, e.g. if Atomic needed to be derived from AtomicBase then we would assert that, and not care whether it was direct derivation or via AtomicPlatform). > Yes and that makes me wonder if we care enough about the completeness semantics being the same as third party libraries (who are already inconsistent) if we do not need them to be, and on the contrary benefit from them not to be. I strongly care about semantics and risk of error for something that is billed as a general purpose utility. Risk of ODR violations is not something I would treat lightly. And minor maintainance changes could easily lead to such, if this proposed IsBaseOf were used as a general utility. > In the end, we have to make a choice - what is more important, that the traits resemble some external library or code size. [?] I think this badly mis-characterizes my position. I think the trait semantics are critial, and I think the proposed traits have a serious defect that render them inappropriate as general utilities. In addition, I don't think they are what is needed for the more specialized technique exemplared by the proposed change and similar use-cases. Code size doesn't enter into these concerns at all. The case where code size matters to me in this discussion is that we're talking about adding some non-trivial infrastructure to eliminate some simple macro usage. If we can't amortize that infrastructure across some reasonable number of use-cases, then I wouldn't want to add it. I think there probably are sufficient use-cases to make some additional infrastructure worth-while, but I think what was proposed is not the infrastructure we need. > 1. Do we want to replace the macros with template metaprogramming? 2 reviewers (and me) liked the metaprogramming approach and it was implemented because they (like me) did not want the macros. But you have a different opinion. I can?t judge who is right and wrong here. I think this is only beneficial if doing so ultimately makes this code follow a common pattern used elsewhere in our code base. (I say "ultimately" because this would be the first occurrence of such a pattern.) From goetz.lindenmaier at sap.com Tue Jan 6 07:50:18 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 6 Jan 2015 07:50:18 +0000 Subject: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. In-Reply-To: <54AADD60.6060501@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF264E2@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CF35E66@DEWDFEMB12A.global.corp.sap> <547F4914.1030904@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF36139@DEWDFEMB12A.global.corp.sap> <54809196.7030306@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF39BA5@DEWDFEMB12A.global.corp.sap> <5480AB58.80604@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF3D6BB@DEWDFEMB12A.global.corp.sap> <54889D61.30905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF3E1C5@DEWDFEMB12A.global.corp.sap> <54989B9F.2070903@oracle.com> <549B48A3.1070303@oracle.com> <54A2089D.1020105@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF53F70@DEWDFEMB12A.global.corp.sap> <54A2C966.5010306@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF67EB6@DEWDFEMB12A.global.corp.sap> <54A6C6C3.1050806@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF69210@DEWDFEMB12A.global.corp.sap> <54AAC086.2060600@oracle.com> <54AAC34F.2050807@oracle.com> <54AADD60.6060501@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF69431@DEWDFEMB12A.global.corp.sap> Thanks Vladimir! Best regards, Goetz. -----Original Message----- From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] Sent: Monday, January 05, 2015 7:52 PM To: Coleen Phillimore; Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; 'hotspot-dev at openjdk.java.net' Subject: Re: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. I looked through latest changes (webrev.03) and they look reasonable. Thanks, Vladimir On 1/5/15 9:01 AM, Coleen Phillimore wrote: > > On 1/5/15, 11:49 AM, Coleen Phillimore wrote: >> >> On 1/5/15, 4:24 AM, Lindenmaier, Goetz wrote: >>> Hi Coleen, >>> >>> It would be great if you could sponsor this change. >> >> I see no other comments so I'll push it today. > > I am planning to push this to hs-rt, btw. > Coleen > >>> >>>> The metaspace case probably can be fixed to use the try_reserve_heap() code maybe. >>> We wondered why the compressed class space is always above the heap. >>> If the heap is too big for unscaled, the compressed class space could be placed >>> in the lower 4G. >> >> The reason that we chose the compressed class space on the top of the heap is for a couple reasons. The time spent >> compressing and decompressing oops is more critical than the time spent doing the same for the _klass pointer, so we >> wanted to make the system more likely to get unscaled compressed oops, or even zero based compressed oops. >> >> On solaris, the space below the heap is limited because by default malloc uses it so we were limited in low address >> memory that we can use. We want the compressed class space code not be platform dependent. We also need to have the >> CDS archive at a fixed address near or adjacent to the compressed class space, so were running out at the bottom of >> the heap. >> >> The compressed class space is fixed at startup (just like PermGen was) and the classes are variable sized, so we had >> to pick a size fairly large so that it's unlikely for applications to run out. If above 32G, we might be able to >> implement code to allocate a new space which would use the same compression algorithm. We haven't had a request for >> that but it's something we could do. >> >> >>> The aarch port brings some changes here, and we would like to investigate >>> this on top of the aarch changes, if that's fine with you. >> >> So one thing I prototyped and really wanted to check in was what we called indirect class pointers. Instead of >> allocating classes in a fixed class space, we allocate a fixed array of Klass pointers and the objects in the heap >> point to this. It had a 8% performance degradation in specbb2008 compiler.compiler (iirc, or something like that). >> >> This solved so many problems though. If you investigated this sort of approach, I'd be very grateful. I haven't had >> time to go back to figure out why this had this degradation (we thought it was Java vtable accesses). >> >> Thanks, >> Coleen >>> >>> Thanks a lot for going through this complex review! >>> >>> Best regards, >>> Goetz. >>> >>> -----Original Message----- >>> From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] >>> Sent: Freitag, 2. Januar 2015 17:27 >>> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net >>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. >>> >>> >>> On 1/2/15, 10:09 AM, Lindenmaier, Goetz wrote: >>>> Hi Coleen, >>>> >>>> I looked at the cleanups and found both not straight forward to implement. >>>> >>>> 1) Merging ReservedHeap::initialize and ReservedHeapSpace::try_reserve_heap. >>>> The new code calls initialize() with requested_address = NULL, but metaspace.cpp >>>> passes a real requested_address. Therefore I can not clean up the calls to >>>> failed_to_reserve_as_requested() in initialize(). Without that cleanup, I need to pass flags >>>> into initialize() to configure all the different behaviours I would need in a >>>> merged case. >>>> I think with 'large' and 'requested_address' there is already enough cases >>>> in that method, so that I don't want to add more. >>> Okay, this is fine. More flags to special case functions aren't good >>> either. Can you file an RFE to look into refactoring so that the >>> duplicate code can be removed though? The metaspace case probably can >>> be fixed to use the try_reserve_heap() code maybe. >>> >>>> 2) Moving _noaccess_prefix to ReservedHeapSpace >>>> >>>> This does not work as ReservedHeapSpace is casted to >>>> ReservedSpace and returned by value. So ReservedHeapSpace can not have >>>> fields or virtual methods. >>>> It would be probably better to change the code to return ReservedHeapSpace >>>> by pointer, or, at least under that type, but I think that exceeds the scope of this change. >>> Too bad. >>> >>>> So is it ok if I leave the change as is wrt. to these cleanups? >>> Yes, this is fine. >>>> I extended test/runtime/CompressedOops/UseCompressedOops.java >>>> to combine the testcases of the different mode with the GCs and UseLargePages. >>>> I added the changes in the test to >>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >>> Is .03 with the additional changes? If so, you should have made an .04. >>> >>> I reviewed this again and it looks good. I think Vladimir has reviewed >>> an earlier version. I can sponsor this assuming there are no more comments. >>> >>> Coleen >>>> Best regards, >>>> Goetz. >>>> >>>> >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] >>>> Sent: Dienstag, 30. Dezember 2014 16:49 >>>> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net >>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. >>>> >>>> >>>> On 12/30/14, 8:42 AM, Lindenmaier, Goetz wrote: >>>>> Hi Coleen, >>>>> >>>>>> disjoint >>>>> It tries to say that the bits used for the base (36-63) are different from >>>>> those used for the oops / used as offset into the heap (0-35). >>>>> I chose the name according to this translation >>>>> http://dict.leo.org/#/search=disjoint&searchLoc=0&resultOrder=basic&multiwordShowSingle=on >>>>> Both the german adjective and verb fit to describe the property of the base. >>>>> I can add the following to the docu of the mode in universe.hpp:358 >>>>> // Disjoint: Bits used in base are disjoint from bits used >>>>> // for oops ==> oop = (cOop << 3) | base. One can disjoint >>>>> // the bits of an oop into base and compressed oop. >>>> Yes, that helps. It's a good distinct word so its good that we can make >>>> sense in the documentation. >>>>>> (I filed a bug to move these to the memory directory) >>>>> Good idea! >>>>> >>>>>> 272 void ReservedSpace::establish_noaccess_prefix() { >>>>>> This function should be in ReservedHeapSpace >>>>> Yes, I moved it. I also would like to move the field _noaccess_prefix, >>>>> but I would have to change a lot of code, like ReservedSpace::release(). >>>>> The problem is that _base is not the real base in ReservedHeapSpace. >>>>> Probably ReservedHeapSpace should have heap_size() and heap_base() >>>>> giving the correcte values. >>>>> I'll try to come up with a patch on top of this, maybe we should do it >>>>> in a separate bug. >>>> I just moved _noaccess_prefix and added a virtual >>>> ReservedHeapSpace::release() that adjusted the base and size with >>>> noaccess_prefix. I don't think the footprint of a vtable or virtual >>>> call makes any difference. >>>>>> Can you have arguments.cpp verify_range of HeapSearchSteps so this can't >>>>>> happen? >>>>> Done. >>>>> >>>>>> Rather move this to the end of ReservedHeapSpace constructor. >>>>>> assert(_base == NULL || ... >>>>> Yes, you are right, it should not be there twice. I moved it into the >>>>> constructor. It is only relevant for heaps, right? >>>> Yes. >>>>>> Code in try_reserve_heap() is mostly a copy of ReservedSpace::initialize() >>>>> You are right, requested_address in initialize is dead now. I'll remove that >>>>> and try to merge the two functions. >>>>> >>>>> I removed the verbose printing. Anyways, it's mostly useful when developing this. >>>> Thanks. I was trying to find some way to move the printing so it would >>>> make sense to me but ended up giving up on that. >>>>>> aligned_HBMA >>>>> Changed to aligned_heap_base_min_address. >>>>> >>>>>> Is "size" const in initialize_compressed_heap? >>>>> Yes. Added 'const'. >>>>> >>>>> I'll run tests with +UseLargePages. >>>>> >>>>> I had verified that the heap is at the exact same position for the code in >>>>> my first RFR, and with HeapSearchSteps=1 to make sure I don't break anything. >>>>> Vladimir asked me to increase that to 3. So far I still didn't find a case where >>>>> this changed anything. >>>>> (But I know from SAP JVM, which is tested on all the OS variants we support, >>>>> that there are cases where this can lead to a better - but then different - heap >>>>> placement.) >>>>> The only case where there is always a difference is Solaris 5.11 which >>>>> now can allocate zerobased memory. >>>>> The code passes the CDS jtreg tests. >>>>> >>>>> I updated the webrev.03 with the minor changes. >>>>> I'll come up with a new one with the cleanups of Reserved(Heap)Space >>>>> (_noaccess_prefix and redundant functions). >>>>> >>>>> Thanks for this thorough review!! >>>> Thank you for your willingness to make these changes. >>>> >>>> Happy New Year! >>>> Coleen >>>> >>>>> Happy new year, >>>>> Goetz. >>>>> >>>>> >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: hotspot-runtime-dev [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore >>>>> Sent: Dienstag, 30. Dezember 2014 03:06 >>>>> To: hotspot-runtime-dev at openjdk.java.net >>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. >>>>> >>>>> >>>>> Hi, I've reviewed more code. >>>>> >>>>> I'm sorry about the following comment that should have been asked >>>>> before. The name of the mode is very confusing. I can't figure out why >>>>> it's called "disjoint". I keep looking for a second heap base or some >>>>> code that makes the heap discontinuous. The heap base is really >>>>> specially aligned, right? So I went to the thesaurus.com: >>>>> >>>>> http://www.thesaurus.com/browse/aligned >>>>> >>>>> and "disjoin" is an antonym. Why was "disjoint" chosen? From the >>>>> code, it looks like "aligned" or "scaled" are more descriptive but >>>>> they're overused words in the code. But ScaledBasedNarrowOop seems more >>>>> descriptive than Disjoint. >>>>> >>>>> In virtualspace.cpp (I filed a bug to move these to the memory directory) >>>>> >>>>> 272 void ReservedSpace::establish_noaccess_prefix() { >>>>> >>>>> >>>>> This function should be in ReservedHeapSpace since ReservedSpace never >>>>> does this (and we never want it to). The function it replaced was in >>>>> ReservedSpace and that was wrong. Now that ReservedHeapSpace has >>>>> compressed oops logic in it, I think this function should be moved to >>>>> this class. Actually, I've modified your patch to move the >>>>> _noaccess_prefix field to ReservedHeapSpace. Maybe that's too much >>>>> change to your patch but since the handling of noaccess_prefix is >>>>> changed, it seemed like the thing to do. >>>>> >>>>> Can you have arguments.cpp verify_range of HeapSearchSteps so this can't >>>>> happen? >>>>> >>>>> guarantee(HeapSearchSteps > 0, "Don't set HeapSearchSteps to 0"); >>>>> >>>>> Can this be a function like check_encoding_for_mark_sweep() , or >>>>> something like that? It appears twice. Rather move this to the end of >>>>> ReservedHeapSpace constructor. >>>>> >>>>> assert(_base == NULL || >>>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == _base, >>>>> "area must be distinguishable from marks for mark-sweep"); >>>>> assert(_base == NULL || >>>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() == >>>>> &_base[size], >>>>> "area must be distinguishable from marks for mark-sweep"); >>>>> >>>>> Code in try_reserve_heap() is a mostly a copy of >>>>> ReservedSpace::initialize(), except the first version has a call to >>>>> failed_to_reserve_as_requested which looks like it's handling the case >>>>> for not being able to reserve the requested address for compressed oops >>>>> which seems like you don't want this. This is very confusing. There >>>>> shouldn't be two of these. >>>>> >>>>> It looks like in the compressed oops case, you call >>>>> ReservedSpace::initialize() only with NULL requested address so >>>>> failed_to_reserve_as_requested is dead code with this change. So it >>>>> seems like try_reserve_heap() could be ReservedSpace::initialize() with >>>>> the failed_to_reserve_as_requested eliminated and a special case for >>>>> handling unaligned addresses. >>>>> >>>>> The logic in initialize_compressed_heap() still doesn't make sense to >>>>> me. If the first try_reserve_range() succeeds at getting an unscaled >>>>> compressed oop base, then does PrintCompressedOopsMode && Verbose still >>>>> print: >>>>> >>>>> tty->print(" == U N S C A L E D ==\n"); >>>>> tty->print(" == Z E R O B A S E D ==\n"); >>>>> tty->print(" == D I S J O I N T B A S E ==\n"); >>>>> tty->print(" == H E A P B A S E D ==\n"); >>>>> >>>>> Why would you print all 4? It's the printing that makes this >>>>> confusing! Now I see how the next case checks that the base address is >>>>> a better allocation. If you changed the printing to a comment like: >>>>> >>>>> // UnscaledMode after testing previous base address is not better (or >>>>> something). >>>>> >>>>> Actually, you have these comments. I think the printing should be >>>>> removed because it's the opposite of helpful. >>>>> >>>>> I had trouble reading the parameter aligned_HBMA, can you change to >>>>> aligned_heap_base_min (I know it's longer but there's too many >>>>> calculations here that words will make easier to read). Or just >>>>> heap_base_min_address. >>>>> >>>>> zerobased_max = (char *)OopEncodingHeapMax - class_space; >>>>> >>>>> Odd that the solaris compiler can compile this and not the code below. >>>>> (minor point) I think this should be: >>>>> >>>>> zerobased_max = zerobased_max - class_space; >>>>> >>>>> Is "size" const in initialize_compressed_heap? (answer appears to be >>>>> yes) It's useful for reading the calculations that it never changes so >>>>> I think it should be const (it's a minor point too). >>>>> >>>>> I don't have any other comments, I've tried to do the calculations a bit >>>>> but I know that you've done this yourself. >>>>> >>>>> Also, can you run tests with -XX:+UseLargePages? I assume you've run >>>>> tests with the 4 garbage collectors. This is a tricky piece of code >>>>> that replaced something else difficult. We've had to change it often to >>>>> support other features like CDS and compressed class pointers so making >>>>> it as understandable as possible is worth the extra effort. I think >>>>> your new design looks cleaner. >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> >>>>> On 12/24/14, 6:13 PM, Coleen Phillimore wrote: >>>>>> One note for my first reviewed file. >>>>>> >>>>>> On 12/22/14, 5:30 PM, Coleen Phillimore wrote: >>>>>>> Hi Goetz, >>>>>>> >>>>>>> I'm looking at this version of the code >>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >>>>>>> >>>>>>> This passes our JPRT tests. Yeah! >>>>>>> >>>>>>> It's a significant change, so I'm actually reading the code with the >>>>>>> patches applied. >>>>>>> >>>>>>> In src/share/vm/memory/universe.cpp >>>>>>> >>>>>>> Can you put all this code from 742-776 in a new function above >>>>>>> initialize_heap(), like set_narrow_oop_values()? >>>>>>> The comment at 689-695 refer to code that's moved to >>>>>>> virtualspace.cpp. I think it needs to be reworded, updated for >>>>>>> disjoint oops mode or removed. The comment at 744-749 also refers to >>>>>>> code that's moved so should be removed also. The last two lines of >>>>>>> this comment refer to something whose implementation has changed, so >>>>>>> just remove them. >>>>>>> >>>>>>> On line >>>>>>> >>>>>>> 750 if ((uint64_t)Universe::heap()->reserved_region().end() > >>>>>>> UnscaledOopHeapMax) { >>>>>>> >>>>>>> Can you make this expression a variable because the same expression >>>>>>> appears on line 754. >>>>>>> >>>>>>> Why can't this code at 742-776 be with the compressed oops mode code >>>>>>> at line 836-839. If you make this set_narrow_oop_values() function, >>>>>>> can it be moved together? >>>>>> Apparently this doesn't work (it crashes on some platforms). I can't >>>>>> completely follow the control flow that makes this wrong, but ignore >>>>>> this suggestion. >>>>>> >>>>>> Coleen >>>>>>> Can you remove this comment too? I have no idea what it means. There >>>>>>> is no CATCH below (?) >>>>>>> >>>>>>> // We will never reach the CATCH below since Exceptions::_throw >>>>>>> will cause >>>>>>> // the VM to exit if an exception is thrown during initialization >>>>>>> >>>>>>> >>>>>>> ** We should file an RFE to move virtualspace.hpp/cpp to the >>>>>>> src/share/vm/memory directory! >>>>>>> >>>>>>> Sorry, this is partial, I'll continue tomorrow. >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> >>>>>>> >>>>>>> On 12/11/2014 06:18 AM, Lindenmaier, Goetz wrote: >>>>>>>> Hi Coleen, >>>>>>>> >>>>>>>> thanks for looking at this change! >>>>>>>> >>>>>>>>> get_attach_addresses_for_disjoint_mode function belongs in >>>>>>>>> virtualspace.cpp >>>>>>>> All information about the bounds of the modes comes from universe. >>>>>>>> This is >>>>>>>> related to is_disjoint_heap_base_address() and thus I left it in >>>>>>>> universe. >>>>>>>> But I'll move it, it can also be regarded as an implementation >>>>>>>> detail of >>>>>>>> initialize_compressed_heap(). I can make it static then, so that >>>>>>>> it's removed >>>>>>>> in 32 bit build, which is good. >>>>>>>> >>>>>>>>> Can you reduce some to at least 100? >>>>>>>> Done. I agree that's better . (I have a 2560x1440 screen, causing >>>>>>>> me to have >>>>>>>> big windows :)) >>>>>>>> >>>>>>>>> Why is highest_start not const char* ? >>>>>>>> Removed the const. >>>>>>>> >>>>>>>>> The control flow in initialize_compressed_heap makes no sense to me. >>>>>>>> Vladimir had the same problem, I added comment at line 493 to make >>>>>>>> that clearer. >>>>>>>> One problem of the previous implementation was that, if the os layer >>>>>>>> returned an address with desired properties, but not at the requested >>>>>>>> address, it was discarded. >>>>>>>> To fix this, try_reserve_heap always returns what it allocated, and I >>>>>>>> check afterwards. >>>>>>>> Therefore the checks of the next mode include the check of the >>>>>>>> previous one. So if I try to allocate disjoint, but get unscaled, >>>>>>>> I will detect that and return the unscaled heap. >>>>>>>> If I implement a return, I just have to add tests. I don't get rid of >>>>>>>> other tests. >>>>>>>> >>>>>>>>> Also, our 32 bit platforms don't like: 1 * SIZE_64K * SIZE_32G, >>>>>>>> I changed that to use UCONST64 macro. That should help I guess. >>>>>>>> I also have these in globalDefinitions_xlc.hpp. Should I move them >>>>>>>> to globalDefinitions.hpp:200, after definition of K, M and G? >>>>>>>> >>>>>>>> I uploaded a new webrev: >>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Goetz. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: hotspot-runtime-dev >>>>>>>> [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of >>>>>>>> Coleen Phillimore >>>>>>>> Sent: Mittwoch, 10. Dezember 2014 20:22 >>>>>>>> To: hotspot-runtime-dev at openjdk.java.net >>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>> >>>>>>>> >>>>>>>> Hi Goetz, >>>>>>>> >>>>>>>> I have some initial comments, which are much less detailed than >>>>>>>> Vladimir's comments. I haven't actually processed all the >>>>>>>> implementation details yet. >>>>>>>> >>>>>>>> I think get_attach_addresses_for_disjoint_mode function belongs in >>>>>>>> virtualspace.cpp and not in universe. I like that the compressed oop >>>>>>>> reservation logic is moved to virtualspace.cpp. I think this is an >>>>>>>> improvement over the Universe::preferred_heap_base() logic which was >>>>>>>> also difficult. >>>>>>>> >>>>>>>> The Hotspot coding style doesn't dictate 80 columns anymore (there is >>>>>>>> debate whether it should have or not, which we try not to have this >>>>>>>> debate), but I found the very long lines in this code inconsistent with >>>>>>>> other Hotspot code. I had to expand my window to cover my whole >>>>>>>> screen. Can you reduce some to at least 100? >>>>>>>> >>>>>>>> For example, I aligned these parameters to see better since there >>>>>>>> are so >>>>>>>> many (the indentation isn't right in email). >>>>>>>> >>>>>>>> void ReservedHeapSpace::try_reserve_range(char *const highest_start, >>>>>>>> char *lowest_start, >>>>>>>> size_t >>>>>>>> attach_point_alignment, >>>>>>>> char *aligned_HBMA, // >>>>>>>> HeapBaseMinAddress >>>>>>>> char *upper_bound, >>>>>>>> size_t size, >>>>>>>> size_t alignment, >>>>>>>> bool large) { >>>>>>>> >>>>>>>> Why is highest_start not const char* ? Doesn't char* const >>>>>>>> highest_start just restrict you from changing the pointer and not what >>>>>>>> it points to? This seems odd to do. >>>>>>>> >>>>>>>> The control flow in initialize_compressed_heap makes no sense to me. >>>>>>>> It seems like there should be an exit when the best allocation for >>>>>>>> compression is achieved. But it falls through after >>>>>>>> try_reserve_range(). I can't figure out why it should fall through.... >>>>>>>> >>>>>>>> I was expecting something like: >>>>>>>> >>>>>>>> if (PrintCompressedOopsMode && Verbose) { >>>>>>>> tty->print(" == H E A P B A S E M I N A D D R E S S ==\n"); >>>>>>>> } >>>>>>>> get the heap at aligned HeapBaseMinAddress, return if success... >>>>>>>> >>>>>>>> if (PrintCompressedOopsMode && Verbose) { >>>>>>>> tty->print(" == U N S C A L E D ==\n"); >>>>>>>> } >>>>>>>> try to get heap base for unscaled access, return if successful >>>>>>>> >>>>>>>> etc. >>>>>>>> >>>>>>>> >>>>>>>> You should make this into a little function for each return and for the >>>>>>>> end of the initialize function, or move it to the ReservedHeapSpace >>>>>>>> constructor (caller). >>>>>>>> >>>>>>>> assert(_base == NULL || >>>>>>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == _base, >>>>>>>> "area must be distinguishable from marks for mark-sweep"); >>>>>>>> assert(_base == NULL || >>>>>>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() == >>>>>>>> &_base[size], >>>>>>>> "area must be distinguishable from marks for mark-sweep"); >>>>>>>> } >>>>>>>> >>>>>>>> I ran this through JPRT and this line didn't compile on macos: >>>>>>>> >>>>>>>> const uint64_t num_attempts_to_try = MIN2(HeapSearchSteps, >>>>>>>> num_attempts_possible); >>>>>>>> >>>>>>>> I'm retrying now as: >>>>>>>> >>>>>>>> const uint64_t num_attempts_to_try = >>>>>>>> MIN2((uint64_t)HeapSearchSteps, num_attempts_possible); >>>>>>>> >>>>>>>> Sorry for the delay looking at this. This is a big change that I >>>>>>>> needed >>>>>>>> to make more time to read. I am pleased that it's a performance >>>>>>>> improvement. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>>>>> >>>>>>>> On 12/8/14, 10:54 AM, Lindenmaier, Goetz wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> This is just a ping to gc/rt mailing lists to reach appropriate >>>>>>>>> people. >>>>>>>>> >>>>>>>>> I please need a reviewer from gc or rt, could somebody have a >>>>>>>>> look at this? >>>>>>>>> >>>>>>>>> Short summary: >>>>>>>>> - new cOops mode disjointbase that allows optimizations on PPC >>>>>>>>> improving over heapbased >>>>>>>>> - search for heaps: finds zerobased on sparc Solaris 11 and Aix >>>>>>>>> - concentrate cOops heap allocation code in one function >>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>>> >>>>>>>>> Please reply only to the original thread in hotspot-dev to keep this >>>>>>>>> local. >>>>>>>>> >>>>>>>>> Thanks and best regards, >>>>>>>>> Goetz. >>>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>> Sent: Donnerstag, 4. Dezember 2014 19:44 >>>>>>>>> To: Lindenmaier, Goetz >>>>>>>>> Cc: 'hotspot-dev developers' >>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>> >>>>>>>>> This looks good to me. >>>>>>>>> >>>>>>>>> Now we need second review since changes are significant. Preferable >>>>>>>>> from >>>>>>>>> GC group since you changed ReservedHeapSpace. They will be affected >>>>>>>>> most. Review from Runtime is also welcome. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Vladimir >>>>>>>>> >>>>>>>>> On 12/4/14 10:27 AM, Lindenmaier, Goetz wrote: >>>>>>>>>> Hi Vladimir. >>>>>>>>>> >>>>>>>>>> Sorry. I updated the webrev once more. Hope it's fine now. >>>>>>>>>> At least I can write comments :) >>>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> Goetz >>>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>>> Sent: Thursday, December 04, 2014 5:54 PM >>>>>>>>>> To: Lindenmaier, Goetz >>>>>>>>>> Cc: 'hotspot-dev developers' >>>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>> >>>>>>>>>> I spotted an other bug. >>>>>>>>>> You replaced !_base with _base != NULL when moved code to >>>>>>>>>> try_reserve_range() - it should be _base == NULL. >>>>>>>>>> The same problem in asserts: >>>>>>>>>> >>>>>>>>>> + assert(_base != NULL || >>>>>>>>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == >>>>>>>>>> _base, >>>>>>>>>> + "area must be distinguishable from marks for mark-sweep"); >>>>>>>>>> + assert(_base != NULL || >>>>>>>>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() == >>>>>>>>>> &_base[size], >>>>>>>>>> + "area must be distinguishable from marks for mark-sweep"); >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Also you did not remove _base && in next place: >>>>>>>>>> >>>>>>>>>> + (_base && _base + size > zerobased_max))) { // Unscaled >>>>>>>>>> delivered an arbitrary address. >>>>>>>>>> >>>>>>>>>> New comment is good. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Vladimri >>>>>>>>>> >>>>>>>>>> On 12/4/14 1:45 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>> Hi Vladimir, >>>>>>>>>>> >>>>>>>>>>>> Add more extending comment explaining that. >>>>>>>>>>> The comment for try_reserve_heap was meant to explain that. >>>>>>>>>>> I further added a comment in initialize_compressed_heap(). >>>>>>>>>>> >>>>>>>>>>>> You need another parameter to pass UnscaledOopHeapMax or >>>>>>>>>>>> zerobased_max. >>>>>>>>>>> Oh, thanks a lot! That's important. Fixed. >>>>>>>>>>> >>>>>>>>>>>> I mean that you already checked _base == NULL so on other side >>>>>>>>>>>> of || _base != NULL - why you need (_base &&) check? >>>>>>>>>>> Sorry, now I got it. Removed. >>>>>>>>>>> >>>>>>>>>>> I updated the webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>>>>> >>>>>>>>>>> Increment on top of the increment :) >>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/incremental_diffs2.patch >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Goetz. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -----Original Message----- >>>>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>>>> Sent: Mittwoch, 3. Dezember 2014 18:32 >>>>>>>>>>> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' >>>>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>>> >>>>>>>>>>> Comments are below. >>>>>>>>>>> >>>>>>>>>>> On 12/3/14 5:49 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>>> Hi Vladimir, >>>>>>>>>>>> >>>>>>>>>>>> thanks for looking at the change! See my comments inline below. >>>>>>>>>>>> >>>>>>>>>>>> I made a new webrev: >>>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>>>>>> >>>>>>>>>>>> Incremental changes: >>>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/incremental_diffs.patch >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Best regards, >>>>>>>>>>>> Goetz. >>>>>>>>>>>> >>>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>>>>>> Sent: Mittwoch, 3. Dezember 2014 00:46 >>>>>>>>>>>>> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' >>>>>>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>>>>> >>>>>>>>>>>>> This looks good to me. Someone in runtime/gc have to look on it >>>>>>>>>>>>> too. >>>>>>>>>>>>> >>>>>>>>>>>>> universe.cpp about >>>>>>>>>>>>> SystemProperty("com.sap.vm.test.compressedOopsMode" >>>>>>>>>>>>> we have: >>>>>>>>>>>>> java.vm.info=mixed mode, sharing >>>>>>>>>>>>> so we can have: >>>>>>>>>>>>> java.vm.compressedOopsMode=... >>>>>>>>>>>> Yes, that's good for me. Fixed. >>>>>>>>>>>> >>>>>>>>>>>>> I am not expert in properties names but I don't want to have >>>>>>>>>>>>> 'com.sap' >>>>>>>>>>>>> in VM's property name. >>>>>>>>>>>>> virtualspace.cpp: >>>>>>>>>>>>> Could you fix release() - it does not reset _alignment? >>>>>>>>>>>> Fixed. >>>>>>>>>>>> >>>>>>>>>>>>> In try_reserve_heap(), please, use (base == NULL) instead of >>>>>>>>>>>>> (!base). >>>>>>>>>>>>> And you don't need 'return;' in alignment check at the end of >>>>>>>>>>>>> method. >>>>>>>>>>>> Fixed. >>>>>>>>>>>> >>>>>>>>>>>>> In initialize_compressed_heap() again (!_base). >>>>>>>>>>>> Fixed. >>>>>>>>>>>> >>>>>>>>>>>>> You don't stop (check >>>>>>>>>>>>> (base == NULL)) after successful unscaled, zerobased, disjointbase >>>>>>>>>>>>> allocations. You need to separate them with the check: >>>>>>>>>>>>> >>>>>>>>>>>>> + >>>>>>>>>>>>> + } >>>>>>>>>>>>> + } >>>>>>>>>>>>> + if (_base == NULL) { >>>>>>>>>>>>> + >>>>>>>>>>>>> + if (PrintCompressedOopsMode && Verbose) { >>>>>>>>>>>>> + tty->print(" == Z E R O B A S E D ==\n"); >>>>>>>>>>>>> + } >>>>>>>>>>>>> and so on. >>>>>>>>>>>> No, I can't and don't want to check for _base != NULL. >>>>>>>>>>>> I always keep the result of the last try, also if it didn't >>>>>>>>>>>> fulfil the required properties. >>>>>>>>>>>> So I take that result and go into the next check. That check >>>>>>>>>>>> might succeed >>>>>>>>>>>> with the heap allocated before. >>>>>>>>>>>> This allows me to separate allocation and placement criteria, >>>>>>>>>>>> and to have the >>>>>>>>>>>> placement criteria checked in only one place (per mode). >>>>>>>>>>>> Only for HeapBaseMinAddress I don't do it that way, I explicitly >>>>>>>>>>>> call release(). >>>>>>>>>>>> This way I can enforce mode heapbased. >>>>>>>>>>> I see what you are saying. It was not clear from comments what is >>>>>>>>>>> going on. >>>>>>>>>>> Add more extending comment explaining that. >>>>>>>>>>> >>>>>>>>>>>>> num_attempts calculation and while() loop are similar in >>>>>>>>>>>>> unscaled and >>>>>>>>>>>>> zerobased cases. Could you move it into a separate method? >>>>>>>>>>>> I can do that, but I don't like it as I have to pass in 7 >>>>>>>>>>>> parameters. >>>>>>>>>>> You need an other parameter to pass UnscaledOopHeapMax or >>>>>>>>>>> zerobased_max. >>>>>>>>>>> >>>>>>>>>>>> That makes the code not much more readable. The function will >>>>>>>>>>>> look like this: >>>>>>>>>>> I think initialize_compressed_heap() is more readable now. >>>>>>>>>>> >>>>>>>>>>>> void ReserveHeapSpace::try_reserve_range(char *const >>>>>>>>>>>> highest_start, char *lowest_start, size_t attach_point_alignment, >>>>>>>>>>>> char >>>>>>>>>>>> *aligned_HBMA, size_t size, size_t alignment, bool large) { >>>>>>>>>>>> guarantee(HeapSearchSteps > 0, "Don't set HeapSearchSteps >>>>>>>>>>>> to 0"); >>>>>>>>>>>> >>>>>>>>>>>> const size_t attach_range = highest_start - lowest_start; >>>>>>>>>>>> // Cap num_attempts at possible number. >>>>>>>>>>>> // At least one is possible even for 0 sized attach range. >>>>>>>>>>>> const uint64_t num_attempts_possible = (attach_range / >>>>>>>>>>>> attach_point_alignment) + 1; >>>>>>>>>>>> const uint64_t num_attempts_to_try = >>>>>>>>>>>> MIN2(HeapSearchSteps, num_attempts_possible); >>>>>>>>>>>> >>>>>>>>>>>> const size_t stepsize = align_size_up(attach_range / >>>>>>>>>>>> num_attempts_to_try, attach_point_alignment); >>>>>>>>>>>> >>>>>>>>>>>> // Try attach points from top to bottom. >>>>>>>>>>>> char* attach_point = highest_start; >>>>>>>>>>>> while (attach_point >= lowest_start && >>>>>>>>>>>> attach_point <= highest_start && // Avoid wrap >>>>>>>>>>>> around. >>>>>>>>>>>> (!_base || _base < aligned_HBMA || _base + size > >>>>>>>>>>>> (char *)UnscaledOopHeapMax)) { >>>>>>>>>>>> try_reserve_heap(size, alignment, large, attach_point); >>>>>>>>>>>> attach_point -= stepsize; >>>>>>>>>>>> } >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> In disjointbase while() condition no need for _base second check: >>>>>>>>>>>>> + (_base == NULL || >>>>>>>>>>>>> + ((_base + size > (char *)OopEncodingHeapMax) && >>>>>>>>>>>> I need this for the same reason as above: This is the check for >>>>>>>>>>>> successful allocation. >>>>>>>>>>> I mean that you already checked _base == NULL so on other side of >>>>>>>>>>> || _base != NULL - why you need (_base &&) check? >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Vladimir >>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Vladimir >>>>>>>>>>>> >>>>>>>>>>>> On 11/21/14 5:31 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> I prepared a new webrev trying to cover all the issues >>>>>>>>>>>>> mentioned below. >>>>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.01/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I moved functionality from os.cpp and universe.cpp into >>>>>>>>>>>>> ReservedHeapSpace::initialize_compressed_heap(). >>>>>>>>>>>>> This class offers to save _base and _special, which I would >>>>>>>>>>>>> have to reimplement >>>>>>>>>>>>> if I had improved the methods I had added to os.cpp to also >>>>>>>>>>>>> allocate large page >>>>>>>>>>>>> heaps. >>>>>>>>>>>>> Anyways, I think this class is the right place to gather code >>>>>>>>>>>>> trying to reserve >>>>>>>>>>>>> the heap. >>>>>>>>>>>>> Also, I get along without setting the shift, base, >>>>>>>>>>>>> implicit_null_check etc. fields >>>>>>>>>>>>> of Universe, so there is no unnecessary calling back and forth >>>>>>>>>>>>> between the two >>>>>>>>>>>>> classes. >>>>>>>>>>>>> Universe gets the heap back, and then sets the properties it >>>>>>>>>>>>> needs to configure >>>>>>>>>>>>> the compressed oops. >>>>>>>>>>>>> All code handling the noaccess prefix is in a single method, too. >>>>>>>>>>>>> >>>>>>>>>>>>> Best regards, >>>>>>>>>>>>> Goetz. >>>>>>>>>>>>> >>>>>>>>>>>>> Btw, I had to workaround a SS12u1 problem: it wouldn't compile >>>>>>>>>>>>> char * x = (char*)UnscaledOopHeapMax - size in 32-bit mode. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>>> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] >>>>>>>>>>>>> On Behalf Of Lindenmaier, Goetz >>>>>>>>>>>>> Sent: Montag, 17. November 2014 09:33 >>>>>>>>>>>>> To: 'Vladimir Kozlov'; 'hotspot-dev at openjdk.java.net' >>>>>>>>>>>>> Subject: RE: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Vladimir, >>>>>>>>>>>>> >>>>>>>>>>>>>> It is very significant rewriting and it takes time to evaluate >>>>>>>>>>>>>> it. >>>>>>>>>>>>> Yes, I know ... and I don't want to push, but nevertheless a ping >>>>>>>>>>>>> can be useful sometimes. Thanks a lot for looking at it. >>>>>>>>>>>>> >>>>>>>>>>>>>> And I would not say it is simpler then before :) >>>>>>>>>>>>> If I fix what you propose it's gonna get even more simple ;) >>>>>>>>>>>>>> These is what I found so far. >>>>>>>>>>>>>> The idea to try to allocate in a range instead of just below >>>>>>>>>>>>>> UnscaledOopHeapMax or OopEncodingHeapMax is good. So I would >>>>>>>>>>>>>> ask to do >>>>>>>>>>>>>> several attempts (3?) on non_PPC64 platforms too. >>>>>>>>>>>>> Set to 3. >>>>>>>>>>>>> >>>>>>>>>>>>>> It is matter of preference but I am not comfortable with >>>>>>>>>>>>>> switch in loop. >>>>>>>>>>>>>> For me sequential 'if (addr == 0)' checks is simpler. >>>>>>>>>>>>> I'll fix this. >>>>>>>>>>>>> >>>>>>>>>>>>>> One thing worries me that you release found space and try to >>>>>>>>>>>>>> get it >>>>>>>>>>>>>> again with ReservedHeapSpace. Is it possible to add new >>>>>>>>>>>>>> ReservedHeapSpace ctor which simple use already allocated space? >>>>>>>>>>>>> This was to keep diff's small, but I also think a new >>>>>>>>>>>>> constructor is good. >>>>>>>>>>>>> I'll fix this. >>>>>>>>>>>>> >>>>>>>>>>>>>> The next code in ReservedHeapSpace() is hard to understand (): >>>>>>>>>>>>>> (UseCompressedOops && (requested_address == NULL || >>>>>>>>>>>>> requested_address+size > (char*)OopEncodingHeapMax) ? >>>>>>>>>>>>>> may be move all this into noaccess_prefix_size() and add >>>>>>>>>>>>>> comments. >>>>>>>>>>>>> I have to redo this anyways if I make new constructors. >>>>>>>>>>>>> >>>>>>>>>>>>>> Why you need prefix when requested_address == NULL? >>>>>>>>>>>>> If we allocate with NULL, we most probably will get a heap where >>>>>>>>>>>>> base != NULL and thus need a noaccess prefix. >>>>>>>>>>>>> >>>>>>>>>>>>>> Remove next comment in universe.cpp: >>>>>>>>>>>>>> // SAPJVM GL 2014-09-22 >>>>>>>>>>>>> Removed. >>>>>>>>>>>>> >>>>>>>>>>>>>> Again you will release space so why bother to include space >>>>>>>>>>>>>> for classes?: >>>>>>>>>>>>>> + // For small heaps, save some space for compressed >>>>>>>>>>>>>> class pointer >>>>>>>>>>>>>> + // space so it can be decoded with no base. >>>>>>>>>>>>> This was done like this before. We must assure the upper bound >>>>>>>>>>>>> of the >>>>>>>>>>>>> heap is low enough that the compressed class space still fits >>>>>>>>>>>>> in there. >>>>>>>>>>>>> >>>>>>>>>>>>> virtualspace.cpp >>>>>>>>>>>>> >>>>>>>>>>>>>> With new code size+noaccess_prefix could be requested. But >>>>>>>>>>>>>> later it is >>>>>>>>>>>>>> not used if WIN64_ONLY(&& UseLargePages) and you will have empty >>>>>>>>>>>>>> non-protected page below heap. >>>>>>>>>>>>> There's several points to this: >>>>>>>>>>>>> * Also if not protectable, the heap base has to be below >>>>>>>>>>>>> the real start of the >>>>>>>>>>>>> heap. Else the first object in the heap will be >>>>>>>>>>>>> compressed to 'null' >>>>>>>>>>>>> and decompression will fail. >>>>>>>>>>>>> * If we don't reserve the memory other stuff can end up >>>>>>>>>>>>> in this space. On >>>>>>>>>>>>> errors, if would be quite unexpected to find memory >>>>>>>>>>>>> there. >>>>>>>>>>>>> * To get a heap for the new disjoint mode I must control >>>>>>>>>>>>> the size of this. >>>>>>>>>>>>> Requesting a heap starting at (aligned base + >>>>>>>>>>>>> prefix) is more likely to fail. >>>>>>>>>>>>> * The size for the prefix must anyways be considered >>>>>>>>>>>>> when deciding whether the >>>>>>>>>>>>> heap is small enough to run with compressed oops. >>>>>>>>>>>>> So distinguishing the case where we really can omit this would >>>>>>>>>>>>> require >>>>>>>>>>>>> quite some additional checks everywhere, and I thought it's not >>>>>>>>>>>>> worth it. >>>>>>>>>>>>> >>>>>>>>>>>>> matcher.hpp >>>>>>>>>>>>> >>>>>>>>>>>>>> Universe::narrow_oop_use_implicit_null_checks() should be true >>>>>>>>>>>>>> for such >>>>>>>>>>>>>> case too. So you can add new condition with || to existing >>>>>>>>>>>>>> ones. The >>>>>>>>>>>>>> only condition you relax is base != NULL. Right? >>>>>>>>>>>>> Yes, that's how it's intended. >>>>>>>>>>>>> >>>>>>>>>>>>> arguments.* files >>>>>>>>>>>>> >>>>>>>>>>>>>> Why you need PropertyList_add changes. >>>>>>>>>>>>> Oh, the code using it got lost. I commented on this in the >>>>>>>>>>>>> description in the webrev. >>>>>>>>>>>>> "To more efficiently run expensive tests in various compressed >>>>>>>>>>>>> oop modes, we set a property with the mode the VM is running >>>>>>>>>>>>> in. So far it's called "com.sap.vm.test.compressedOopsMode" >>>>>>>>>>>>> better suggestions are welcome (and necessary I guess). Our >>>>>>>>>>>>> long running tests that are supposed to run in a dedicated >>>>>>>>>>>>> compressed oop mode check this property and abort themselves if >>>>>>>>>>>>> it's not the expected mode." >>>>>>>>>>>>> When I know about the heap I do >>>>>>>>>>>>> Arguments::PropertyList_add(new >>>>>>>>>>>>> SystemProperty("com.sap.vm.test.compressedOopsMode", >>>>>>>>>>>>> narrow_oop_mode_to_string(narrow_oop_mode()), >>>>>>>>>>>>> false)); >>>>>>>>>>>>> in universe.cpp. >>>>>>>>>>>>> On some OSes it's deterministic which modes work, there we >>>>>>>>>>>>> don't start such tests. >>>>>>>>>>>>> Others, as you mentioned OSX, are very indeterministic. Here >>>>>>>>>>>>> we save testruntime with this. >>>>>>>>>>>>> But it's not that important. >>>>>>>>>>>>> We can still parse the PrintCompresseOopsMode output after the >>>>>>>>>>>>> test and discard the >>>>>>>>>>>>> run. >>>>>>>>>>>>> >>>>>>>>>>>>>> Do you have platform specific changes? >>>>>>>>>>>>> Yes, for ppc and aix. I'll submit them once this is in. >>>>>>>>>>>>> >>>>>>>>>>>>> From your other mail: >>>>>>>>>>>>>> One more thing. You should allow an allocation in the range >>>>>>>>>>>>>> when returned from OS allocated address does not match >>>>>>>>>>>>>> requested address. We had such cases on OSX, for example, when >>>>>>>>>>>>>> OS allocates at different address but still inside range. >>>>>>>>>>>>> Good point. I'll fix that in os::attempt_reserve_memory_in_range. >>>>>>>>>>>>> >>>>>>>>>>>>> I'll ping again once a new webrev is done! >>>>>>>>>>>>> >>>>>>>>>>>>> Best regards, >>>>>>>>>>>>> Goetz. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 11/10/14 6:57 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I need to improve a row of things around compressed oops heap >>>>>>>>>>>>>> handling >>>>>>>>>>>>>> to achieve good performance on ppc. >>>>>>>>>>>>>> I prepared a first webrev for review: >>>>>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.00/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> A detailed technical description of the change is in the >>>>>>>>>>>>>> webrev and according bug. >>>>>>>>>>>>>> >>>>>>>>>>>>>> If requested, I will split the change into parts with more >>>>>>>>>>>>>> respective less impact on >>>>>>>>>>>>>> non-ppc platforms. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The change is derived from well-tested code in our VM. >>>>>>>>>>>>>> Originally it was >>>>>>>>>>>>>> crafted to require the least changes of VM coding, I changed >>>>>>>>>>>>>> it to be better >>>>>>>>>>>>>> streamlined with the VM. >>>>>>>>>>>>>> I tested this change to deliver heaps at about the same >>>>>>>>>>>>>> addresses as before. >>>>>>>>>>>>>> Heap addresses mostly differ in lower bits. In some cases >>>>>>>>>>>>>> (Solaris 5.11) a heap >>>>>>>>>>>>>> in a better compressed oops mode is found, though. >>>>>>>>>>>>>> I ran (and adapted) test/runtime/CompressedOops and >>>>>>>>>>>>>> gc/arguments/TestUseCompressedOops*. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>> Goetz. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >> > From goetz.lindenmaier at sap.com Tue Jan 6 07:56:22 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 6 Jan 2015 07:56:22 +0000 Subject: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. In-Reply-To: <54AAC34F.2050807@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF264E2@DEWDFEMB12A.global.corp.sap> <547E4F3B.2090501@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF35E66@DEWDFEMB12A.global.corp.sap> <547F4914.1030904@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF36139@DEWDFEMB12A.global.corp.sap> <54809196.7030306@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF39BA5@DEWDFEMB12A.global.corp.sap> <5480AB58.80604@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF3D6BB@DEWDFEMB12A.global.corp.sap> <54889D61.30905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF3E1C5@DEWDFEMB12A.global.corp.sap> <54989B9F.2070903@oracle.com> <549B48A3.1070303@oracle.com> <54A2089D.1020105@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF53F70@DEWDFEMB12A.global.corp.sap> <54A2C966.5010306@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF67EB6@DEWDFEMB12A.global.corp.sap> <54A6C6C3.1050806@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF69210@DEWDFEMB12A.global.corp.sap> <54AAC086.2060600@oracle.com> <54AAC34F.2050807@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF694A5@DEWDFEMB12A.global.corp.sap> Hi Coleen, thanks a lot for pushing this change! Best regards, Goetz. -----Original Message----- From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] Sent: Monday, January 05, 2015 6:01 PM To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; 'hotspot-dev at openjdk.java.net' Subject: Re: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. On 1/5/15, 11:49 AM, Coleen Phillimore wrote: > > On 1/5/15, 4:24 AM, Lindenmaier, Goetz wrote: >> Hi Coleen, >> >> It would be great if you could sponsor this change. > > I see no other comments so I'll push it today. I am planning to push this to hs-rt, btw. Coleen >> >>> The metaspace case probably can be fixed to use the >>> try_reserve_heap() code maybe. >> We wondered why the compressed class space is always above the heap. >> If the heap is too big for unscaled, the compressed class space could >> be placed >> in the lower 4G. > > The reason that we chose the compressed class space on the top of the > heap is for a couple reasons. The time spent compressing and > decompressing oops is more critical than the time spent doing the same > for the _klass pointer, so we wanted to make the system more likely to > get unscaled compressed oops, or even zero based compressed oops. > > On solaris, the space below the heap is limited because by default > malloc uses it so we were limited in low address memory that we can > use. We want the compressed class space code not be platform > dependent. We also need to have the CDS archive at a fixed address > near or adjacent to the compressed class space, so were running out at > the bottom of the heap. > > The compressed class space is fixed at startup (just like PermGen was) > and the classes are variable sized, so we had to pick a size fairly > large so that it's unlikely for applications to run out. If above 32G, > we might be able to implement code to allocate a new space which would > use the same compression algorithm. We haven't had a request for that > but it's something we could do. > > >> The aarch port brings some changes here, and we would like to >> investigate >> this on top of the aarch changes, if that's fine with you. > > So one thing I prototyped and really wanted to check in was what we > called indirect class pointers. Instead of allocating classes in a > fixed class space, we allocate a fixed array of Klass pointers and the > objects in the heap point to this. It had a 8% performance > degradation in specbb2008 compiler.compiler (iirc, or something like > that). > > This solved so many problems though. If you investigated this sort of > approach, I'd be very grateful. I haven't had time to go back to > figure out why this had this degradation (we thought it was Java > vtable accesses). > > Thanks, > Coleen >> >> Thanks a lot for going through this complex review! >> >> Best regards, >> Goetz. >> >> -----Original Message----- >> From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] >> Sent: Freitag, 2. Januar 2015 17:27 >> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >> "disjoint base" and improve compressed heap handling. >> >> >> On 1/2/15, 10:09 AM, Lindenmaier, Goetz wrote: >>> Hi Coleen, >>> >>> I looked at the cleanups and found both not straight forward to >>> implement. >>> >>> 1) Merging ReservedHeap::initialize and >>> ReservedHeapSpace::try_reserve_heap. >>> The new code calls initialize() with requested_address = NULL, >>> but metaspace.cpp >>> passes a real requested_address. Therefore I can not clean up the >>> calls to >>> failed_to_reserve_as_requested() in initialize(). Without that >>> cleanup, I need to pass flags >>> into initialize() to configure all the different behaviours I would >>> need in a >>> merged case. >>> I think with 'large' and 'requested_address' there is already enough >>> cases >>> in that method, so that I don't want to add more. >> Okay, this is fine. More flags to special case functions aren't good >> either. Can you file an RFE to look into refactoring so that the >> duplicate code can be removed though? The metaspace case probably can >> be fixed to use the try_reserve_heap() code maybe. >> >>> 2) Moving _noaccess_prefix to ReservedHeapSpace >>> >>> This does not work as ReservedHeapSpace is casted to >>> ReservedSpace and returned by value. So ReservedHeapSpace can not have >>> fields or virtual methods. >>> It would be probably better to change the code to return >>> ReservedHeapSpace >>> by pointer, or, at least under that type, but I think that exceeds >>> the scope of this change. >> Too bad. >> >>> So is it ok if I leave the change as is wrt. to these cleanups? >> Yes, this is fine. >>> I extended test/runtime/CompressedOops/UseCompressedOops.java >>> to combine the testcases of the different mode with the GCs and >>> UseLargePages. >>> I added the changes in the test to >>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >> Is .03 with the additional changes? If so, you should have made an .04. >> >> I reviewed this again and it looks good. I think Vladimir has reviewed >> an earlier version. I can sponsor this assuming there are no more >> comments. >> >> Coleen >>> Best regards, >>> Goetz. >>> >>> >>> >>> >>> >>> -----Original Message----- >>> From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] >>> Sent: Dienstag, 30. Dezember 2014 16:49 >>> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net >>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>> "disjoint base" and improve compressed heap handling. >>> >>> >>> On 12/30/14, 8:42 AM, Lindenmaier, Goetz wrote: >>>> Hi Coleen, >>>> >>>>> disjoint >>>> It tries to say that the bits used for the base (36-63) are >>>> different from >>>> those used for the oops / used as offset into the heap (0-35). >>>> I chose the name according to this translation >>>> http://dict.leo.org/#/search=disjoint&searchLoc=0&resultOrder=basic&multiwordShowSingle=on >>>> >>>> Both the german adjective and verb fit to describe the property of >>>> the base. >>>> I can add the following to the docu of the mode in universe.hpp:358 >>>> // Disjoint: Bits used in base are disjoint from bits used >>>> // for oops ==> oop = (cOop << 3) | base. One can disjoint >>>> // the bits of an oop into base and compressed oop. >>> Yes, that helps. It's a good distinct word so its good that we can >>> make >>> sense in the documentation. >>>>> (I filed a bug to move these to the memory directory) >>>> Good idea! >>>> >>>>> 272 void ReservedSpace::establish_noaccess_prefix() { >>>>> This function should be in ReservedHeapSpace >>>> Yes, I moved it. I also would like to move the field >>>> _noaccess_prefix, >>>> but I would have to change a lot of code, like >>>> ReservedSpace::release(). >>>> The problem is that _base is not the real base in ReservedHeapSpace. >>>> Probably ReservedHeapSpace should have heap_size() and heap_base() >>>> giving the correcte values. >>>> I'll try to come up with a patch on top of this, maybe we should do it >>>> in a separate bug. >>> I just moved _noaccess_prefix and added a virtual >>> ReservedHeapSpace::release() that adjusted the base and size with >>> noaccess_prefix. I don't think the footprint of a vtable or virtual >>> call makes any difference. >>>>> Can you have arguments.cpp verify_range of HeapSearchSteps so this >>>>> can't >>>>> happen? >>>> Done. >>>> >>>>> Rather move this to the end of ReservedHeapSpace constructor. >>>>> assert(_base == NULL || ... >>>> Yes, you are right, it should not be there twice. I moved it into the >>>> constructor. It is only relevant for heaps, right? >>> Yes. >>>>> Code in try_reserve_heap() is mostly a copy of >>>>> ReservedSpace::initialize() >>>> You are right, requested_address in initialize is dead now. I'll >>>> remove that >>>> and try to merge the two functions. >>>> >>>> I removed the verbose printing. Anyways, it's mostly useful when >>>> developing this. >>> Thanks. I was trying to find some way to move the printing so it would >>> make sense to me but ended up giving up on that. >>>>> aligned_HBMA >>>> Changed to aligned_heap_base_min_address. >>>> >>>>> Is "size" const in initialize_compressed_heap? >>>> Yes. Added 'const'. >>>> >>>> I'll run tests with +UseLargePages. >>>> >>>> I had verified that the heap is at the exact same position for the >>>> code in >>>> my first RFR, and with HeapSearchSteps=1 to make sure I don't break >>>> anything. >>>> Vladimir asked me to increase that to 3. So far I still didn't >>>> find a case where >>>> this changed anything. >>>> (But I know from SAP JVM, which is tested on all the OS variants we >>>> support, >>>> that there are cases where this can lead to a better - but then >>>> different - heap >>>> placement.) >>>> The only case where there is always a difference is Solaris 5.11 which >>>> now can allocate zerobased memory. >>>> The code passes the CDS jtreg tests. >>>> >>>> I updated the webrev.03 with the minor changes. >>>> I'll come up with a new one with the cleanups of Reserved(Heap)Space >>>> (_noaccess_prefix and redundant functions). >>>> >>>> Thanks for this thorough review!! >>> Thank you for your willingness to make these changes. >>> >>> Happy New Year! >>> Coleen >>> >>>> Happy new year, >>>> Goetz. >>>> >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: hotspot-runtime-dev >>>> [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of >>>> Coleen Phillimore >>>> Sent: Dienstag, 30. Dezember 2014 03:06 >>>> To: hotspot-runtime-dev at openjdk.java.net >>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>> "disjoint base" and improve compressed heap handling. >>>> >>>> >>>> Hi, I've reviewed more code. >>>> >>>> I'm sorry about the following comment that should have been asked >>>> before. The name of the mode is very confusing. I can't figure out >>>> why >>>> it's called "disjoint". I keep looking for a second heap base or some >>>> code that makes the heap discontinuous. The heap base is really >>>> specially aligned, right? So I went to the thesaurus.com: >>>> >>>> http://www.thesaurus.com/browse/aligned >>>> >>>> and "disjoin" is an antonym. Why was "disjoint" chosen? From the >>>> code, it looks like "aligned" or "scaled" are more descriptive but >>>> they're overused words in the code. But ScaledBasedNarrowOop seems >>>> more >>>> descriptive than Disjoint. >>>> >>>> In virtualspace.cpp (I filed a bug to move these to the memory >>>> directory) >>>> >>>> 272 void ReservedSpace::establish_noaccess_prefix() { >>>> >>>> >>>> This function should be in ReservedHeapSpace since ReservedSpace never >>>> does this (and we never want it to). The function it replaced was in >>>> ReservedSpace and that was wrong. Now that ReservedHeapSpace has >>>> compressed oops logic in it, I think this function should be moved to >>>> this class. Actually, I've modified your patch to move the >>>> _noaccess_prefix field to ReservedHeapSpace. Maybe that's too much >>>> change to your patch but since the handling of noaccess_prefix is >>>> changed, it seemed like the thing to do. >>>> >>>> Can you have arguments.cpp verify_range of HeapSearchSteps so this >>>> can't >>>> happen? >>>> >>>> guarantee(HeapSearchSteps > 0, "Don't set HeapSearchSteps to >>>> 0"); >>>> >>>> Can this be a function like check_encoding_for_mark_sweep() , or >>>> something like that? It appears twice. Rather move this to the >>>> end of >>>> ReservedHeapSpace constructor. >>>> >>>> assert(_base == NULL || >>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == _base, >>>> "area must be distinguishable from marks for >>>> mark-sweep"); >>>> assert(_base == NULL || >>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() == >>>> &_base[size], >>>> "area must be distinguishable from marks for >>>> mark-sweep"); >>>> >>>> Code in try_reserve_heap() is a mostly a copy of >>>> ReservedSpace::initialize(), except the first version has a call to >>>> failed_to_reserve_as_requested which looks like it's handling the case >>>> for not being able to reserve the requested address for compressed >>>> oops >>>> which seems like you don't want this. This is very confusing. There >>>> shouldn't be two of these. >>>> >>>> It looks like in the compressed oops case, you call >>>> ReservedSpace::initialize() only with NULL requested address so >>>> failed_to_reserve_as_requested is dead code with this change. So it >>>> seems like try_reserve_heap() could be ReservedSpace::initialize() >>>> with >>>> the failed_to_reserve_as_requested eliminated and a special case for >>>> handling unaligned addresses. >>>> >>>> The logic in initialize_compressed_heap() still doesn't make sense to >>>> me. If the first try_reserve_range() succeeds at getting an unscaled >>>> compressed oop base, then does PrintCompressedOopsMode && Verbose >>>> still >>>> print: >>>> >>>> tty->print(" == U N S C A L E D ==\n"); >>>> tty->print(" == Z E R O B A S E D ==\n"); >>>> tty->print(" == D I S J O I N T B A S E ==\n"); >>>> tty->print(" == H E A P B A S E D ==\n"); >>>> >>>> Why would you print all 4? It's the printing that makes this >>>> confusing! Now I see how the next case checks that the base >>>> address is >>>> a better allocation. If you changed the printing to a comment like: >>>> >>>> // UnscaledMode after testing previous base address is not better (or >>>> something). >>>> >>>> Actually, you have these comments. I think the printing should be >>>> removed because it's the opposite of helpful. >>>> >>>> I had trouble reading the parameter aligned_HBMA, can you change to >>>> aligned_heap_base_min (I know it's longer but there's too many >>>> calculations here that words will make easier to read). Or just >>>> heap_base_min_address. >>>> >>>> zerobased_max = (char *)OopEncodingHeapMax - class_space; >>>> >>>> Odd that the solaris compiler can compile this and not the code below. >>>> (minor point) I think this should be: >>>> >>>> zerobased_max = zerobased_max - class_space; >>>> >>>> Is "size" const in initialize_compressed_heap? (answer appears to be >>>> yes) It's useful for reading the calculations that it never >>>> changes so >>>> I think it should be const (it's a minor point too). >>>> >>>> I don't have any other comments, I've tried to do the calculations >>>> a bit >>>> but I know that you've done this yourself. >>>> >>>> Also, can you run tests with -XX:+UseLargePages? I assume you've run >>>> tests with the 4 garbage collectors. This is a tricky piece of code >>>> that replaced something else difficult. We've had to change it >>>> often to >>>> support other features like CDS and compressed class pointers so >>>> making >>>> it as understandable as possible is worth the extra effort. I think >>>> your new design looks cleaner. >>>> >>>> Thanks, >>>> Coleen >>>> >>>> >>>> On 12/24/14, 6:13 PM, Coleen Phillimore wrote: >>>>> One note for my first reviewed file. >>>>> >>>>> On 12/22/14, 5:30 PM, Coleen Phillimore wrote: >>>>>> Hi Goetz, >>>>>> >>>>>> I'm looking at this version of the code >>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >>>>>> >>>>>> >>>>>> This passes our JPRT tests. Yeah! >>>>>> >>>>>> It's a significant change, so I'm actually reading the code with the >>>>>> patches applied. >>>>>> >>>>>> In src/share/vm/memory/universe.cpp >>>>>> >>>>>> Can you put all this code from 742-776 in a new function above >>>>>> initialize_heap(), like set_narrow_oop_values()? >>>>>> The comment at 689-695 refer to code that's moved to >>>>>> virtualspace.cpp. I think it needs to be reworded, updated for >>>>>> disjoint oops mode or removed. The comment at 744-749 also refers to >>>>>> code that's moved so should be removed also. The last two lines of >>>>>> this comment refer to something whose implementation has changed, so >>>>>> just remove them. >>>>>> >>>>>> On line >>>>>> >>>>>> 750 if ((uint64_t)Universe::heap()->reserved_region().end() > >>>>>> UnscaledOopHeapMax) { >>>>>> >>>>>> Can you make this expression a variable because the same expression >>>>>> appears on line 754. >>>>>> >>>>>> Why can't this code at 742-776 be with the compressed oops mode code >>>>>> at line 836-839. If you make this set_narrow_oop_values() function, >>>>>> can it be moved together? >>>>> Apparently this doesn't work (it crashes on some platforms). I can't >>>>> completely follow the control flow that makes this wrong, but ignore >>>>> this suggestion. >>>>> >>>>> Coleen >>>>>> Can you remove this comment too? I have no idea what it means. There >>>>>> is no CATCH below (?) >>>>>> >>>>>> // We will never reach the CATCH below since Exceptions::_throw >>>>>> will cause >>>>>> // the VM to exit if an exception is thrown during >>>>>> initialization >>>>>> >>>>>> >>>>>> ** We should file an RFE to move virtualspace.hpp/cpp to the >>>>>> src/share/vm/memory directory! >>>>>> >>>>>> Sorry, this is partial, I'll continue tomorrow. >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>> >>>>>> On 12/11/2014 06:18 AM, Lindenmaier, Goetz wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> thanks for looking at this change! >>>>>>> >>>>>>>> get_attach_addresses_for_disjoint_mode function belongs in >>>>>>>> virtualspace.cpp >>>>>>> All information about the bounds of the modes comes from universe. >>>>>>> This is >>>>>>> related to is_disjoint_heap_base_address() and thus I left it in >>>>>>> universe. >>>>>>> But I'll move it, it can also be regarded as an implementation >>>>>>> detail of >>>>>>> initialize_compressed_heap(). I can make it static then, so that >>>>>>> it's removed >>>>>>> in 32 bit build, which is good. >>>>>>> >>>>>>>> Can you reduce some to at least 100? >>>>>>> Done. I agree that's better . (I have a 2560x1440 screen, causing >>>>>>> me to have >>>>>>> big windows :)) >>>>>>> >>>>>>>> Why is highest_start not const char* ? >>>>>>> Removed the const. >>>>>>> >>>>>>>> The control flow in initialize_compressed_heap makes no sense >>>>>>>> to me. >>>>>>> Vladimir had the same problem, I added comment at line 493 to make >>>>>>> that clearer. >>>>>>> One problem of the previous implementation was that, if the os >>>>>>> layer >>>>>>> returned an address with desired properties, but not at the >>>>>>> requested >>>>>>> address, it was discarded. >>>>>>> To fix this, try_reserve_heap always returns what it allocated, >>>>>>> and I >>>>>>> check afterwards. >>>>>>> Therefore the checks of the next mode include the check of the >>>>>>> previous one. So if I try to allocate disjoint, but get unscaled, >>>>>>> I will detect that and return the unscaled heap. >>>>>>> If I implement a return, I just have to add tests. I don't get >>>>>>> rid of >>>>>>> other tests. >>>>>>> >>>>>>>> Also, our 32 bit platforms don't like: 1 * SIZE_64K * SIZE_32G, >>>>>>> I changed that to use UCONST64 macro. That should help I guess. >>>>>>> I also have these in globalDefinitions_xlc.hpp. Should I move them >>>>>>> to globalDefinitions.hpp:200, after definition of K, M and G? >>>>>>> >>>>>>> I uploaded a new webrev: >>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >>>>>>> >>>>>>> >>>>>>> Best regards, >>>>>>> Goetz. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: hotspot-runtime-dev >>>>>>> [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of >>>>>>> Coleen Phillimore >>>>>>> Sent: Mittwoch, 10. Dezember 2014 20:22 >>>>>>> To: hotspot-runtime-dev at openjdk.java.net >>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>> >>>>>>> >>>>>>> Hi Goetz, >>>>>>> >>>>>>> I have some initial comments, which are much less detailed than >>>>>>> Vladimir's comments. I haven't actually processed all the >>>>>>> implementation details yet. >>>>>>> >>>>>>> I think get_attach_addresses_for_disjoint_mode function belongs in >>>>>>> virtualspace.cpp and not in universe. I like that the >>>>>>> compressed oop >>>>>>> reservation logic is moved to virtualspace.cpp. I think this is an >>>>>>> improvement over the Universe::preferred_heap_base() logic which >>>>>>> was >>>>>>> also difficult. >>>>>>> >>>>>>> The Hotspot coding style doesn't dictate 80 columns anymore >>>>>>> (there is >>>>>>> debate whether it should have or not, which we try not to have this >>>>>>> debate), but I found the very long lines in this code >>>>>>> inconsistent with >>>>>>> other Hotspot code. I had to expand my window to cover my whole >>>>>>> screen. Can you reduce some to at least 100? >>>>>>> >>>>>>> For example, I aligned these parameters to see better since there >>>>>>> are so >>>>>>> many (the indentation isn't right in email). >>>>>>> >>>>>>> void ReservedHeapSpace::try_reserve_range(char *const >>>>>>> highest_start, >>>>>>> char *lowest_start, >>>>>>> size_t >>>>>>> attach_point_alignment, >>>>>>> char >>>>>>> *aligned_HBMA, // >>>>>>> HeapBaseMinAddress >>>>>>> char *upper_bound, >>>>>>> size_t size, >>>>>>> size_t alignment, >>>>>>> bool large) { >>>>>>> >>>>>>> Why is highest_start not const char* ? Doesn't char* const >>>>>>> highest_start just restrict you from changing the pointer and >>>>>>> not what >>>>>>> it points to? This seems odd to do. >>>>>>> >>>>>>> The control flow in initialize_compressed_heap makes no sense to >>>>>>> me. >>>>>>> It seems like there should be an exit when the best allocation for >>>>>>> compression is achieved. But it falls through after >>>>>>> try_reserve_range(). I can't figure out why it should fall >>>>>>> through.... >>>>>>> >>>>>>> I was expecting something like: >>>>>>> >>>>>>> if (PrintCompressedOopsMode && Verbose) { >>>>>>> tty->print(" == H E A P B A S E M I N A D D R E S S >>>>>>> ==\n"); >>>>>>> } >>>>>>> get the heap at aligned HeapBaseMinAddress, return if >>>>>>> success... >>>>>>> >>>>>>> if (PrintCompressedOopsMode && Verbose) { >>>>>>> tty->print(" == U N S C A L E D ==\n"); >>>>>>> } >>>>>>> try to get heap base for unscaled access, return if >>>>>>> successful >>>>>>> >>>>>>> etc. >>>>>>> >>>>>>> >>>>>>> You should make this into a little function for each return and >>>>>>> for the >>>>>>> end of the initialize function, or move it to the ReservedHeapSpace >>>>>>> constructor (caller). >>>>>>> >>>>>>> assert(_base == NULL || >>>>>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == >>>>>>> _base, >>>>>>> "area must be distinguishable from marks for >>>>>>> mark-sweep"); >>>>>>> assert(_base == NULL || >>>>>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() >>>>>>> == >>>>>>> &_base[size], >>>>>>> "area must be distinguishable from marks for >>>>>>> mark-sweep"); >>>>>>> } >>>>>>> >>>>>>> I ran this through JPRT and this line didn't compile on macos: >>>>>>> >>>>>>> const uint64_t num_attempts_to_try = MIN2(HeapSearchSteps, >>>>>>> num_attempts_possible); >>>>>>> >>>>>>> I'm retrying now as: >>>>>>> >>>>>>> const uint64_t num_attempts_to_try = >>>>>>> MIN2((uint64_t)HeapSearchSteps, num_attempts_possible); >>>>>>> >>>>>>> Sorry for the delay looking at this. This is a big change that I >>>>>>> needed >>>>>>> to make more time to read. I am pleased that it's a performance >>>>>>> improvement. >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> >>>>>>> On 12/8/14, 10:54 AM, Lindenmaier, Goetz wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> This is just a ping to gc/rt mailing lists to reach appropriate >>>>>>>> people. >>>>>>>> >>>>>>>> I please need a reviewer from gc or rt, could somebody have a >>>>>>>> look at this? >>>>>>>> >>>>>>>> Short summary: >>>>>>>> - new cOops mode disjointbase that allows optimizations on PPC >>>>>>>> improving over heapbased >>>>>>>> - search for heaps: finds zerobased on sparc Solaris 11 and Aix >>>>>>>> - concentrate cOops heap allocation code in one function >>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>> >>>>>>>> >>>>>>>> Please reply only to the original thread in hotspot-dev to keep >>>>>>>> this >>>>>>>> local. >>>>>>>> >>>>>>>> Thanks and best regards, >>>>>>>> Goetz. >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>> Sent: Donnerstag, 4. Dezember 2014 19:44 >>>>>>>> To: Lindenmaier, Goetz >>>>>>>> Cc: 'hotspot-dev developers' >>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>> >>>>>>>> This looks good to me. >>>>>>>> >>>>>>>> Now we need second review since changes are significant. >>>>>>>> Preferable >>>>>>>> from >>>>>>>> GC group since you changed ReservedHeapSpace. They will be >>>>>>>> affected >>>>>>>> most. Review from Runtime is also welcome. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Vladimir >>>>>>>> >>>>>>>> On 12/4/14 10:27 AM, Lindenmaier, Goetz wrote: >>>>>>>>> Hi Vladimir. >>>>>>>>> >>>>>>>>> Sorry. I updated the webrev once more. Hope it's fine now. >>>>>>>>> At least I can write comments :) >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> Goetz >>>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>> Sent: Thursday, December 04, 2014 5:54 PM >>>>>>>>> To: Lindenmaier, Goetz >>>>>>>>> Cc: 'hotspot-dev developers' >>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>> >>>>>>>>> I spotted an other bug. >>>>>>>>> You replaced !_base with _base != NULL when moved code to >>>>>>>>> try_reserve_range() - it should be _base == NULL. >>>>>>>>> The same problem in asserts: >>>>>>>>> >>>>>>>>> + assert(_base != NULL || >>>>>>>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == >>>>>>>>> _base, >>>>>>>>> + "area must be distinguishable from marks for >>>>>>>>> mark-sweep"); >>>>>>>>> + assert(_base != NULL || >>>>>>>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() >>>>>>>>> == >>>>>>>>> &_base[size], >>>>>>>>> + "area must be distinguishable from marks for >>>>>>>>> mark-sweep"); >>>>>>>>> >>>>>>>>> >>>>>>>>> Also you did not remove _base && in next place: >>>>>>>>> >>>>>>>>> + (_base && _base + size > zerobased_max))) { // >>>>>>>>> Unscaled >>>>>>>>> delivered an arbitrary address. >>>>>>>>> >>>>>>>>> New comment is good. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Vladimri >>>>>>>>> >>>>>>>>> On 12/4/14 1:45 AM, Lindenmaier, Goetz wrote: >>>>>>>>>> Hi Vladimir, >>>>>>>>>> >>>>>>>>>>> Add more extending comment explaining that. >>>>>>>>>> The comment for try_reserve_heap was meant to explain that. >>>>>>>>>> I further added a comment in initialize_compressed_heap(). >>>>>>>>>> >>>>>>>>>>> You need another parameter to pass UnscaledOopHeapMax or >>>>>>>>>>> zerobased_max. >>>>>>>>>> Oh, thanks a lot! That's important. Fixed. >>>>>>>>>> >>>>>>>>>>> I mean that you already checked _base == NULL so on other side >>>>>>>>>>> of || _base != NULL - why you need (_base &&) check? >>>>>>>>>> Sorry, now I got it. Removed. >>>>>>>>>> >>>>>>>>>> I updated the webrev: >>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Increment on top of the increment :) >>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/incremental_diffs2.patch >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Goetz. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>>> Sent: Mittwoch, 3. Dezember 2014 18:32 >>>>>>>>>> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' >>>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>> >>>>>>>>>> Comments are below. >>>>>>>>>> >>>>>>>>>> On 12/3/14 5:49 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>> Hi Vladimir, >>>>>>>>>>> >>>>>>>>>>> thanks for looking at the change! See my comments inline >>>>>>>>>>> below. >>>>>>>>>>> >>>>>>>>>>> I made a new webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Incremental changes: >>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/incremental_diffs.patch >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Best regards, >>>>>>>>>>> Goetz. >>>>>>>>>>> >>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>>>>> Sent: Mittwoch, 3. Dezember 2014 00:46 >>>>>>>>>>>> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' >>>>>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>>>> >>>>>>>>>>>> This looks good to me. Someone in runtime/gc have to look >>>>>>>>>>>> on it >>>>>>>>>>>> too. >>>>>>>>>>>> >>>>>>>>>>>> universe.cpp about >>>>>>>>>>>> SystemProperty("com.sap.vm.test.compressedOopsMode" >>>>>>>>>>>> we have: >>>>>>>>>>>> java.vm.info=mixed mode, sharing >>>>>>>>>>>> so we can have: >>>>>>>>>>>> java.vm.compressedOopsMode=... >>>>>>>>>>> Yes, that's good for me. Fixed. >>>>>>>>>>> >>>>>>>>>>>> I am not expert in properties names but I don't want to have >>>>>>>>>>>> 'com.sap' >>>>>>>>>>>> in VM's property name. >>>>>>>>>>>> virtualspace.cpp: >>>>>>>>>>>> Could you fix release() - it does not reset _alignment? >>>>>>>>>>> Fixed. >>>>>>>>>>> >>>>>>>>>>>> In try_reserve_heap(), please, use (base == NULL) instead of >>>>>>>>>>>> (!base). >>>>>>>>>>>> And you don't need 'return;' in alignment check at the end of >>>>>>>>>>>> method. >>>>>>>>>>> Fixed. >>>>>>>>>>> >>>>>>>>>>>> In initialize_compressed_heap() again (!_base). >>>>>>>>>>> Fixed. >>>>>>>>>>> >>>>>>>>>>>> You don't stop (check >>>>>>>>>>>> (base == NULL)) after successful unscaled, zerobased, >>>>>>>>>>>> disjointbase >>>>>>>>>>>> allocations. You need to separate them with the check: >>>>>>>>>>>> >>>>>>>>>>>> + >>>>>>>>>>>> + } >>>>>>>>>>>> + } >>>>>>>>>>>> + if (_base == NULL) { >>>>>>>>>>>> + >>>>>>>>>>>> + if (PrintCompressedOopsMode && Verbose) { >>>>>>>>>>>> + tty->print(" == Z E R O B A S E D ==\n"); >>>>>>>>>>>> + } >>>>>>>>>>>> and so on. >>>>>>>>>>> No, I can't and don't want to check for _base != NULL. >>>>>>>>>>> I always keep the result of the last try, also if it didn't >>>>>>>>>>> fulfil the required properties. >>>>>>>>>>> So I take that result and go into the next check. That check >>>>>>>>>>> might succeed >>>>>>>>>>> with the heap allocated before. >>>>>>>>>>> This allows me to separate allocation and placement criteria, >>>>>>>>>>> and to have the >>>>>>>>>>> placement criteria checked in only one place (per mode). >>>>>>>>>>> Only for HeapBaseMinAddress I don't do it that way, I >>>>>>>>>>> explicitly >>>>>>>>>>> call release(). >>>>>>>>>>> This way I can enforce mode heapbased. >>>>>>>>>> I see what you are saying. It was not clear from comments >>>>>>>>>> what is >>>>>>>>>> going on. >>>>>>>>>> Add more extending comment explaining that. >>>>>>>>>> >>>>>>>>>>>> num_attempts calculation and while() loop are similar in >>>>>>>>>>>> unscaled and >>>>>>>>>>>> zerobased cases. Could you move it into a separate method? >>>>>>>>>>> I can do that, but I don't like it as I have to pass in 7 >>>>>>>>>>> parameters. >>>>>>>>>> You need an other parameter to pass UnscaledOopHeapMax or >>>>>>>>>> zerobased_max. >>>>>>>>>> >>>>>>>>>>> That makes the code not much more readable. The function will >>>>>>>>>>> look like this: >>>>>>>>>> I think initialize_compressed_heap() is more readable now. >>>>>>>>>> >>>>>>>>>>> void ReserveHeapSpace::try_reserve_range(char *const >>>>>>>>>>> highest_start, char *lowest_start, size_t >>>>>>>>>>> attach_point_alignment, >>>>>>>>>>> char >>>>>>>>>>> *aligned_HBMA, size_t size, size_t alignment, bool large) { >>>>>>>>>>> guarantee(HeapSearchSteps > 0, "Don't set >>>>>>>>>>> HeapSearchSteps >>>>>>>>>>> to 0"); >>>>>>>>>>> >>>>>>>>>>> const size_t attach_range = highest_start - >>>>>>>>>>> lowest_start; >>>>>>>>>>> // Cap num_attempts at possible number. >>>>>>>>>>> // At least one is possible even for 0 sized >>>>>>>>>>> attach range. >>>>>>>>>>> const uint64_t num_attempts_possible = >>>>>>>>>>> (attach_range / >>>>>>>>>>> attach_point_alignment) + 1; >>>>>>>>>>> const uint64_t num_attempts_to_try = >>>>>>>>>>> MIN2(HeapSearchSteps, num_attempts_possible); >>>>>>>>>>> >>>>>>>>>>> const size_t stepsize = align_size_up(attach_range / >>>>>>>>>>> num_attempts_to_try, attach_point_alignment); >>>>>>>>>>> >>>>>>>>>>> // Try attach points from top to bottom. >>>>>>>>>>> char* attach_point = highest_start; >>>>>>>>>>> while (attach_point >= lowest_start && >>>>>>>>>>> attach_point <= highest_start && // Avoid >>>>>>>>>>> wrap >>>>>>>>>>> around. >>>>>>>>>>> (!_base || _base < aligned_HBMA || _base + >>>>>>>>>>> size > >>>>>>>>>>> (char *)UnscaledOopHeapMax)) { >>>>>>>>>>> try_reserve_heap(size, alignment, large, >>>>>>>>>>> attach_point); >>>>>>>>>>> attach_point -= stepsize; >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> In disjointbase while() condition no need for _base second >>>>>>>>>>>> check: >>>>>>>>>>>> + (_base == NULL || >>>>>>>>>>>> + ((_base + size > (char *)OopEncodingHeapMax) && >>>>>>>>>>> I need this for the same reason as above: This is the check for >>>>>>>>>>> successful allocation. >>>>>>>>>> I mean that you already checked _base == NULL so on other >>>>>>>>>> side of >>>>>>>>>> || _base != NULL - why you need (_base &&) check? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Vladimir >>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Vladimir >>>>>>>>>>> >>>>>>>>>>> On 11/21/14 5:31 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> I prepared a new webrev trying to cover all the issues >>>>>>>>>>>> mentioned below. >>>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.01/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I moved functionality from os.cpp and universe.cpp into >>>>>>>>>>>> ReservedHeapSpace::initialize_compressed_heap(). >>>>>>>>>>>> This class offers to save _base and _special, which I would >>>>>>>>>>>> have to reimplement >>>>>>>>>>>> if I had improved the methods I had added to os.cpp to also >>>>>>>>>>>> allocate large page >>>>>>>>>>>> heaps. >>>>>>>>>>>> Anyways, I think this class is the right place to gather code >>>>>>>>>>>> trying to reserve >>>>>>>>>>>> the heap. >>>>>>>>>>>> Also, I get along without setting the shift, base, >>>>>>>>>>>> implicit_null_check etc. fields >>>>>>>>>>>> of Universe, so there is no unnecessary calling back and forth >>>>>>>>>>>> between the two >>>>>>>>>>>> classes. >>>>>>>>>>>> Universe gets the heap back, and then sets the properties it >>>>>>>>>>>> needs to configure >>>>>>>>>>>> the compressed oops. >>>>>>>>>>>> All code handling the noaccess prefix is in a single >>>>>>>>>>>> method, too. >>>>>>>>>>>> >>>>>>>>>>>> Best regards, >>>>>>>>>>>> Goetz. >>>>>>>>>>>> >>>>>>>>>>>> Btw, I had to workaround a SS12u1 problem: it wouldn't compile >>>>>>>>>>>> char * x = (char*)UnscaledOopHeapMax - size in 32-bit mode. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>> From: hotspot-dev >>>>>>>>>>>> [mailto:hotspot-dev-bounces at openjdk.java.net] >>>>>>>>>>>> On Behalf Of Lindenmaier, Goetz >>>>>>>>>>>> Sent: Montag, 17. November 2014 09:33 >>>>>>>>>>>> To: 'Vladimir Kozlov'; 'hotspot-dev at openjdk.java.net' >>>>>>>>>>>> Subject: RE: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>>>> >>>>>>>>>>>> Hi Vladimir, >>>>>>>>>>>> >>>>>>>>>>>>> It is very significant rewriting and it takes time to >>>>>>>>>>>>> evaluate >>>>>>>>>>>>> it. >>>>>>>>>>>> Yes, I know ... and I don't want to push, but nevertheless >>>>>>>>>>>> a ping >>>>>>>>>>>> can be useful sometimes. Thanks a lot for looking at it. >>>>>>>>>>>> >>>>>>>>>>>>> And I would not say it is simpler then before :) >>>>>>>>>>>> If I fix what you propose it's gonna get even more simple ;) >>>>>>>>>>>>> These is what I found so far. >>>>>>>>>>>>> The idea to try to allocate in a range instead of just below >>>>>>>>>>>>> UnscaledOopHeapMax or OopEncodingHeapMax is good. So I would >>>>>>>>>>>>> ask to do >>>>>>>>>>>>> several attempts (3?) on non_PPC64 platforms too. >>>>>>>>>>>> Set to 3. >>>>>>>>>>>> >>>>>>>>>>>>> It is matter of preference but I am not comfortable with >>>>>>>>>>>>> switch in loop. >>>>>>>>>>>>> For me sequential 'if (addr == 0)' checks is simpler. >>>>>>>>>>>> I'll fix this. >>>>>>>>>>>> >>>>>>>>>>>>> One thing worries me that you release found space and try to >>>>>>>>>>>>> get it >>>>>>>>>>>>> again with ReservedHeapSpace. Is it possible to add new >>>>>>>>>>>>> ReservedHeapSpace ctor which simple use already allocated >>>>>>>>>>>>> space? >>>>>>>>>>>> This was to keep diff's small, but I also think a new >>>>>>>>>>>> constructor is good. >>>>>>>>>>>> I'll fix this. >>>>>>>>>>>> >>>>>>>>>>>>> The next code in ReservedHeapSpace() is hard to understand >>>>>>>>>>>>> (): >>>>>>>>>>>>> (UseCompressedOops && (requested_address == NULL || >>>>>>>>>>>> requested_address+size > (char*)OopEncodingHeapMax) ? >>>>>>>>>>>>> may be move all this into noaccess_prefix_size() and add >>>>>>>>>>>>> comments. >>>>>>>>>>>> I have to redo this anyways if I make new constructors. >>>>>>>>>>>> >>>>>>>>>>>>> Why you need prefix when requested_address == NULL? >>>>>>>>>>>> If we allocate with NULL, we most probably will get a heap >>>>>>>>>>>> where >>>>>>>>>>>> base != NULL and thus need a noaccess prefix. >>>>>>>>>>>> >>>>>>>>>>>>> Remove next comment in universe.cpp: >>>>>>>>>>>>> // SAPJVM GL 2014-09-22 >>>>>>>>>>>> Removed. >>>>>>>>>>>> >>>>>>>>>>>>> Again you will release space so why bother to include space >>>>>>>>>>>>> for classes?: >>>>>>>>>>>>> + // For small heaps, save some space for compressed >>>>>>>>>>>>> class pointer >>>>>>>>>>>>> + // space so it can be decoded with no base. >>>>>>>>>>>> This was done like this before. We must assure the upper >>>>>>>>>>>> bound >>>>>>>>>>>> of the >>>>>>>>>>>> heap is low enough that the compressed class space still fits >>>>>>>>>>>> in there. >>>>>>>>>>>> >>>>>>>>>>>> virtualspace.cpp >>>>>>>>>>>> >>>>>>>>>>>>> With new code size+noaccess_prefix could be requested. But >>>>>>>>>>>>> later it is >>>>>>>>>>>>> not used if WIN64_ONLY(&& UseLargePages) and you will have >>>>>>>>>>>>> empty >>>>>>>>>>>>> non-protected page below heap. >>>>>>>>>>>> There's several points to this: >>>>>>>>>>>> * Also if not protectable, the heap base has to >>>>>>>>>>>> be below >>>>>>>>>>>> the real start of the >>>>>>>>>>>> heap. Else the first object in the heap will be >>>>>>>>>>>> compressed to 'null' >>>>>>>>>>>> and decompression will fail. >>>>>>>>>>>> * If we don't reserve the memory other stuff can >>>>>>>>>>>> end up >>>>>>>>>>>> in this space. On >>>>>>>>>>>> errors, if would be quite unexpected to find >>>>>>>>>>>> memory >>>>>>>>>>>> there. >>>>>>>>>>>> * To get a heap for the new disjoint mode I must >>>>>>>>>>>> control >>>>>>>>>>>> the size of this. >>>>>>>>>>>> Requesting a heap starting at (aligned base + >>>>>>>>>>>> prefix) is more likely to fail. >>>>>>>>>>>> * The size for the prefix must anyways be considered >>>>>>>>>>>> when deciding whether the >>>>>>>>>>>> heap is small enough to run with compressed >>>>>>>>>>>> oops. >>>>>>>>>>>> So distinguishing the case where we really can omit this would >>>>>>>>>>>> require >>>>>>>>>>>> quite some additional checks everywhere, and I thought it's >>>>>>>>>>>> not >>>>>>>>>>>> worth it. >>>>>>>>>>>> >>>>>>>>>>>> matcher.hpp >>>>>>>>>>>> >>>>>>>>>>>>> Universe::narrow_oop_use_implicit_null_checks() should be >>>>>>>>>>>>> true >>>>>>>>>>>>> for such >>>>>>>>>>>>> case too. So you can add new condition with || to existing >>>>>>>>>>>>> ones. The >>>>>>>>>>>>> only condition you relax is base != NULL. Right? >>>>>>>>>>>> Yes, that's how it's intended. >>>>>>>>>>>> >>>>>>>>>>>> arguments.* files >>>>>>>>>>>> >>>>>>>>>>>>> Why you need PropertyList_add changes. >>>>>>>>>>>> Oh, the code using it got lost. I commented on this in the >>>>>>>>>>>> description in the webrev. >>>>>>>>>>>> "To more efficiently run expensive tests in various compressed >>>>>>>>>>>> oop modes, we set a property with the mode the VM is running >>>>>>>>>>>> in. So far it's called "com.sap.vm.test.compressedOopsMode" >>>>>>>>>>>> better suggestions are welcome (and necessary I guess). Our >>>>>>>>>>>> long running tests that are supposed to run in a dedicated >>>>>>>>>>>> compressed oop mode check this property and abort >>>>>>>>>>>> themselves if >>>>>>>>>>>> it's not the expected mode." >>>>>>>>>>>> When I know about the heap I do >>>>>>>>>>>> Arguments::PropertyList_add(new >>>>>>>>>>>> SystemProperty("com.sap.vm.test.compressedOopsMode", >>>>>>>>>>>> narrow_oop_mode_to_string(narrow_oop_mode()), >>>>>>>>>>>> false)); >>>>>>>>>>>> in universe.cpp. >>>>>>>>>>>> On some OSes it's deterministic which modes work, there we >>>>>>>>>>>> don't start such tests. >>>>>>>>>>>> Others, as you mentioned OSX, are very indeterministic. Here >>>>>>>>>>>> we save testruntime with this. >>>>>>>>>>>> But it's not that important. >>>>>>>>>>>> We can still parse the PrintCompresseOopsMode output after the >>>>>>>>>>>> test and discard the >>>>>>>>>>>> run. >>>>>>>>>>>> >>>>>>>>>>>>> Do you have platform specific changes? >>>>>>>>>>>> Yes, for ppc and aix. I'll submit them once this is in. >>>>>>>>>>>> >>>>>>>>>>>> From your other mail: >>>>>>>>>>>>> One more thing. You should allow an allocation in the range >>>>>>>>>>>>> when returned from OS allocated address does not match >>>>>>>>>>>>> requested address. We had such cases on OSX, for example, >>>>>>>>>>>>> when >>>>>>>>>>>>> OS allocates at different address but still inside range. >>>>>>>>>>>> Good point. I'll fix that in >>>>>>>>>>>> os::attempt_reserve_memory_in_range. >>>>>>>>>>>> >>>>>>>>>>>> I'll ping again once a new webrev is done! >>>>>>>>>>>> >>>>>>>>>>>> Best regards, >>>>>>>>>>>> Goetz. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 11/10/14 6:57 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> I need to improve a row of things around compressed oops heap >>>>>>>>>>>>> handling >>>>>>>>>>>>> to achieve good performance on ppc. >>>>>>>>>>>>> I prepared a first webrev for review: >>>>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> A detailed technical description of the change is in the >>>>>>>>>>>>> webrev and according bug. >>>>>>>>>>>>> >>>>>>>>>>>>> If requested, I will split the change into parts with more >>>>>>>>>>>>> respective less impact on >>>>>>>>>>>>> non-ppc platforms. >>>>>>>>>>>>> >>>>>>>>>>>>> The change is derived from well-tested code in our VM. >>>>>>>>>>>>> Originally it was >>>>>>>>>>>>> crafted to require the least changes of VM coding, I changed >>>>>>>>>>>>> it to be better >>>>>>>>>>>>> streamlined with the VM. >>>>>>>>>>>>> I tested this change to deliver heaps at about the same >>>>>>>>>>>>> addresses as before. >>>>>>>>>>>>> Heap addresses mostly differ in lower bits. In some cases >>>>>>>>>>>>> (Solaris 5.11) a heap >>>>>>>>>>>>> in a better compressed oops mode is found, though. >>>>>>>>>>>>> I ran (and adapted) test/runtime/CompressedOops and >>>>>>>>>>>>> gc/arguments/TestUseCompressedOops*. >>>>>>>>>>>>> >>>>>>>>>>>>> Best regards, >>>>>>>>>>>>> Goetz. >>>>>>>>>>>>> >>>>>>>>>>>>> > From aph at redhat.com Tue Jan 6 08:55:06 2015 From: aph at redhat.com (Andrew Haley) Date: Tue, 06 Jan 2015 08:55:06 +0000 Subject: 8068053: AARCH64: C1 and C2 compilers In-Reply-To: <54AAED1F.1000908@oracle.com> References: <54A6A8F5.4040907@redhat.com> <928DC166-7256-4BCA-A9CA-C3ADB05834E5@oracle.com> <54AAD3FC.8000003@redhat.com> <54AAED1F.1000908@oracle.com> Message-ID: <54ABA2EA.4070802@redhat.com> On 05/01/15 19:59, Vladimir Kozlov wrote: > Could you explain how you process [ad_encode.m4]? I don't see aarch64_enc_strw_immn() in .ad file. And I don't remember changes in > makefiles which use it. > > And it is missing Copyright header. Okay, will do. Thanks, Andrew. From aph at redhat.com Tue Jan 6 08:56:29 2015 From: aph at redhat.com (Andrew Haley) Date: Tue, 06 Jan 2015 08:56:29 +0000 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: References: <54A6A933.8000204@redhat.com> Message-ID: <54ABA33D.4020907@redhat.com> On 05/01/15 23:51, Christian Thalinger wrote: > >> On Jan 2, 2015, at 6:20 AM, Andrew Haley wrote: >> >> http://cr.openjdk.java.net/~aph/aarch64-8068054-1/ > > Sorry if this was discussed somewhere else but is BUILTIN_SIM something that?s still used? Not in this version. I did consider stripping it all out, but decided on balance that it would be better to keep it. I could easily be persuaded to remove it. :-) Andrew. From aph at redhat.com Tue Jan 6 08:57:56 2015 From: aph at redhat.com (Andrew Haley) Date: Tue, 06 Jan 2015 08:57:56 +0000 Subject: 8068055: AARCH64: os_cpu/ In-Reply-To: <54AAF87F.9090706@oracle.com> References: <54A6A977.4080405@redhat.com> <54AAF87F.9090706@oracle.com> Message-ID: <54ABA394.5070105@redhat.com> On 05/01/15 20:47, Vladimir Kozlov wrote: > linux_aarch64.S is missing Copyright header. > > linux_aarch64.ad: "AMD64 Linux Architecture". > > os_linux_aarch64.cpp, print registers methods lists x86 registers. And #ifndef AMD64. Yes, that's part of the support for the built-in AArch64 simulator. Perhaps I should take all of that support out. Andrew. From adinn at redhat.com Tue Jan 6 09:14:16 2015 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 06 Jan 2015 09:14:16 +0000 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54ABA33D.4020907@redhat.com> References: <54A6A933.8000204@redhat.com> <54ABA33D.4020907@redhat.com> Message-ID: <54ABA768.1030102@redhat.com> On 06/01/15 08:56, Andrew Haley wrote: > On 05/01/15 23:51, Christian Thalinger wrote: >> >>> On Jan 2, 2015, at 6:20 AM, Andrew Haley wrote: >>> >>> http://cr.openjdk.java.net/~aph/aarch64-8068054-1/ >> >> Sorry if this was discussed somewhere else but is BUILTIN_SIM something that?s still used? > > Not in this version. I did consider stripping it all out, but decided > on balance that it would be better to keep it. I could easily be > persuaded to remove it. :-) Well, perhaps. I am not convinced that is a good idea just yet. Debugging with the AArch64 simulator is still far easier than debugging native code. Now, if we had proper symbol table support for C1/C2 generated code that would be different . . . regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in UK and Wales under Company Registration No. 3798903 Directors: Michael Cunningham (USA), Matt Parson (USA), Charlie Peters (USA), Michael O'Neill (Ireland) From erik.osterlund at lnu.se Tue Jan 6 09:24:31 2015 From: erik.osterlund at lnu.se (=?Windows-1252?Q?Erik_=D6sterlund?=) Date: Tue, 6 Jan 2015 09:24:31 +0000 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <54AAFC8E.1080702@oracle.com> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <,<8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> <54AA143B.2080900@oracle.com> <775F23B6-41A1-4904-8969-1C2F7407D899@lnu.se>, <54AAFC8E.1080702@oracle.com> Message-ID: <692CE9E8-8228-4FD8-8F5C-ADC65DA48A83@lnu.se> Thank you Jesper! With Jespers help, I have some webrevs of my two proposals: Code Minimum: http://cr.openjdk.java.net/~jwilhelm/8067790/webrev.01/ Reuse Maximum: http://cr.openjdk.java.net/~jwilhelm/8067790/webrev.02/ Thanks, /Erik On 5 jan 2015, at 21:05, Jesper Wilhelmsson > wrote: Hi Erik, Attachments are stripped when sent to the list. If you send me two patches I can upload webrevs for the two versions. /Jesper Erik ?sterlund skrev den 5/1/15 18:41: Hi David and Kim, Thank you David for putting this into context. If we want this to be done with the inheritance/template way (specifically the SelectBaseClass template, oh and other names are welcome, InjectOptionalBase?), then I see two options, so I implemented both so you can compare and pick a favourite. They both have exactly identical semantics: they check for the existence of an optional class to inject to the class hierarchy, and check that the inheritance isn?t fishy (the optional class for specialization must be a subclass of the required class for generalization). I think it all comes down to reusability vs code size. Approach 1. ?Reusability Maximum?: Build SelectBaseClass in terms of reusable components resembling boost/C++11 as much as possible. (Note: Since Kim wanted IsBaseOf to require Derived to be complete (defined) to mimic other libraries, I now force a compiler error if parameters are incomplete as per request and instead use her proposal for checking if classes are defined, thanks for that contribution) Pros: 1) Many reusable metafunctions that can be reused in other contexts (for instance automatic closure specialization), and I believe we will have to include them sooner or later anyway. 2) Since they comply very closely with the C++11 standard, it will be easy to make a transition once all compilers support this some time in the distant past. Cons: 380 LoC template infrastructure, including testing, for a perceivably small application (if not reused in the future). Approach 2. ?Code Minimum?: Get the same SelectBaseClass, but with minimal dependencies. Only 20 LoC for the necessary metafunction, some more for IsSame used only for testing purposes, and 156 LoC in total for the template infrastructure, making it much lighter. Pros: Minimal amount of code for the change (less than half). Cons: No reusable metafunctions for the future which we quite likely will need later. I have attached the two solutions in this email so you can have a look and tell me which direction you want me to go before making a final webrev to make everyone happy. Personally I like approach 1 better but if it is concerning to commit too much code and there are strong opinions about this, I?m happy with approach 2 as well. Appendix with code for minimal implementation with no dependencies: template class SelectBaseClass { typedef char yes[1]; typedef char no[2]; template static yes &check(Optional*, T); static no &check(Required*, int); struct OptionalCheckHost { operator Required*() const; operator Optional*(); }; template struct OptionalCheck { typedef O type; }; template struct OptionalCheck { typedef R type; }; public: typedef typename OptionalCheck::type type; }; Thanks, /Erik On 05 Jan 2015, at 04:34, David Holmes > wrote: Sorry for the top-post but just wanted to reset the context a little here. Originally [1] I made the comment: "It's fine to have generic shared approaches but there really needs to be a way to allow platform specific "overrides"." For the actual original RFR I said [2]: "Can we pause and give some more thought to a clean mechanism for allowing a shared implementation if desired with the ability to override if desired. I really do not like to see CPU specific ifdefs being added to shared code. (And I would also not like to see all platforms being forced to reimplement this natively). I'm not saying we will find a simple solution, but it would be nice if we could get a few folk to think about it before proceeding with the ifdefs :)" Erik then proposed three alternative approaches [3] and the simple variant was chosen [4] and presented for review. However Paul Hohensee also made comments about an inheritance-based approach and Erik floated the first template-based variant [5] and there was some discussion with Paul and Kim. But we then ended up with the simple solution, leaving an inheritance-based solution for future work [6] (which is what is what is being discussed now). This template-based meta-programming is not something I have any familiarity with. My initial thought is this seems overkill for the situation we have with the Atomic class - but as I said I'm ignorant of the technique being used here. David ----- [1] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015292.html [2] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015303.html [3] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015362.html [4] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015401.html [5] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-October/015586.html [6] http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-November/015889.html On 2/01/2015 12:07 AM, Erik ?sterlund wrote: Hi Kim, On 01 Jan 2015, at 07:42, Kim Barrett > wrote: On Dec 31, 2014, at 6:12 AM, Erik ?sterlund > wrote: The rest of these comments only matter if, contrary to my suggestion above, some additional infrastructure is thought necessary. This is what I originally thought too and hence the current solution. But voices said that using macros for this is, I quote, "that bad". [?] This scales with the number of AtomicPlatform definitions, rather than the number of specialized functions as happens with the existing code. So your proposal is to still use ugly macros but move it to the class level instead, even though at the moment there is really only one member function that needs it. That feels a bit meh to be honest. IMO if we want to do this right, why go half the way and still rely on ugly macros? Please lay off with the perjorative "ugly macros". The preprocessor is part of C++, and isn't going away. It's a tool that can be used, or abused, and I think this is a fine use. To me, "ugly" is adding 800+ lines of code / tests / comments to eliminate a dozen lines of flag macro definitions and associated #ifdef?s. Well people are entitled to have different opinions of course. As I already mentioned, I did this because other reviewers (as well as me) found it ugly to use macros for conditionally specializing, which is the whole motivation of doing this, and was well understood from the beginning. But I?m fine with leaving it as macros if this is in general preferred and opinions have suddenly changed in this matter. Personally I don?t see how the number of rows of code define uglyness. Reusability, testing, documentation and well definedness in contracts/behaviours all lead to more lines of code, but I don?t think that equals uglyness. So while I disagree more code is uglier because it?s more, it?s all up to opinion and it?s quite pointless to discuss; you are entitled to think that is ugly. If that additional infrastructure were amortized across more use-cases then it might be more appealing. I might even have some additional use-cases for something along that line, though not in its present form. (More on that below.) Yeah I specifically had in mind the wish to make a split between commercial extensions to open source code actually. This pattern allows easy optional class inheritance injection as a general design pattern to achieve this. And for my own purposes the ability to define asymmetric dekker synchronization to elide storeload fences when the platform (e.g. TSO) supports it but otherwise revert to a generalized implementation (using storeload fences). This is why I think a clean reusable strategy is almost always better because it is widely applicable, even if there would not be obvious other examples where it is useful. I think the name "IsBaseOfAndDerived" would be better as "IsBaseAndDerived?. [?] The current implementation of IsBaseOfAndDerived (and therefore IsBaseOf) may (silently) provide incorrect answers when called with incomplete types. [?] The actual usage in this proposed change-set even relies on this flawed behavior of these metafunctions, since AtomicPlatform isn't always defined. [?] So what you are saying here is that I should rename IsBaseOfAndDerived to IsBaseAndDerived to more closely resemble other external libraries we do not use. But if I do that my behaviour is "incorrect" because it is not identical to that of the external library. And therefore my change would not work if it was made "correctly?. The proposed IsBaseOfAndDerived and IsBaseOf are being described and placed as general utilities. Their names are strongly reminiscent of existing facilities in other libraries, including the (C++11) standard library. Conforming to these external standards will avoid surprises, especially when the proposed facilities are even described as being the same (except for a noted corner case that is both hard to deal with in a portable manner and not actually very interesting, so I'm not complaining about that) in the review request email. Additionally, I think quietly providing a potentially incorrect answer in the case of incomplete types renders the offered code quite broken for use as a general utility. I think the requirement for complete types by the well-known / standard facilities is well justified. While I disagree it is universally incorrect (as long as it is well defined and tested) to not have the exact same requirement on completeness on the parameters as C++11/boost, who also already have different behaviour in this exact regard as pointed out by yourself, I can try to fix it if it bugs you, but it will make the infrastructure even larger which seems to be a concern. On the contrary this is a well tested feature on all our supported compilers and I did not intend to engineer it to be identical to some external library if it only causes trouble rather than helping. This is our own class and it can do what we want it to do and be called what we want it to be called, and I think that is fine as long as it is well tested, which it is. At least this is my opinion. Anyone else agree? Problem is, I think it's well tested code to solve the wrong problem. I suggest that what we're really looking for is not a base/derived relationship between a pair of classes, but that we actually want to determine whether a platform-specific class exists. As a result, I think there is more infrastructure in this change set than is actually needed. The present implementation of SelectBaseClass uses IsBaseOf, but doesn't need a lot of the functionality of that metafunction. There is no actual need for a Base/Derived relationship between the (possibly not defined) platform-specific class and the default platform-generic class, so the use of IsBaseOf is an unnecessary restriction that only serendipitously tests for defined or not, due to the previously discussed defect in this particular implementation. I also think the name SelectBaseClass is perhaps overly general. The main idea of SelectBaseClass is to inject an optional class into the class hierarchy for specialization of generalized behaviour. It is absolutely critical (and intended) that this class is constrained to be a derived class of the fall-back class - it is not an artifact nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, A must be able to rely on the superclass being at least C-like so that the behaviour of the superclass can be used reliably and transparently regardless of the existence of the optional B. If this contract would not hold, I think it could lead to some really weird and unreliable code which should not be encouraged (at least not according to the Liskov substitution principle etc). So what I need in the contract is a check if the optional class is defined and if it fits in the inheritance hierarchy: they are both important IMO. Yes it could have been less code. From the beginning I had a single trait class that checked for both the inheritance and whether the class is defined or not but you said you wanted the behaviour to more closely match that of third party libraries. This is why there is now a lot more code to more closely (but still not exactly) match that behaviour. I might be able to use a similar approach in some code I've been working on as a background task. And while writing this reply I've thought of another place that might benefit from something along those lines. Thinking about the code under review in that context, I think some changes would make these other possible use-cases easier and clearer. I believe what is actually needed is a metatfunction something like: /** * Metafunction whose nested type member is Default if Specific is * incomplete, otherwise Specific. * * Specific and Default must both be non-cv qualified class types, and * must not be the same type. * * If Specific is incomplete at the point where this metafunction is * invoked, it must never be defined elsewhere in the program. */ template struct SelectPlatformBase; [Note: I'm not wedded to that name.] Now, it turns out to be hard / impossible to write a generic is_complete metafunction, due to ODR. Whether a type is complete can vary between translation units, and even within a translation unit. Having is_complete::value have different values depending on where it is instantiated is an ODR violation. (One can make progress by using anonymous namespaces and macro wrappers with __LINE__ concatenation, but there is still the fundamental question of whether is_complete really even makes semantic sense.) Yes and that makes me wonder if we care enough about the completeness semantics being the same as third party libraries (who are already inconsistent) if we do not need them to be, and on the contrary benefit from them not to be. However, we don't need a fully general is_complete utility. We have a much more restricted use-case, where we want to select an optionally defined platform-specific class if it exists, and otherwise fall back to a more generic class. And we're not using this in arbitrary contexts, but only to select a platform-specialized implementation if such exists. Here's a lightly tested implementation of SelectPlatformBase: // --- SelectPlatformBase template struct EnableIf { typedef T type; }; template struct EnableIf { }; template struct If { typedef Then type; }; template struct If { typedef Else type; }; // Metafunction whose nested value member is true if T is defined // (complete), and false if T is incomplete. T must be a non-cv // qualified class type. If T is incomplete at the point where this // metafunction is invoked, it must never be defined elsewhere in the // program. template class IsClassDefined { typedef char yes[1]; typedef char no[2]; template static typename EnableIf::type & check(U*); static no& check(...); public: static const bool value = sizeof(check((T*)0)) == sizeof(yes); }; template struct SelectPlatformBase { typedef typename If< IsClassDefined::value, Specific, Default>::type type; }; // --- end SelectPlatformBase That's ~30 lines of code (+ tests and comments TBD) to do precisely what we need, replacing ~100 lines (+ tests and comments) that have various issues. (If and EnableIf should probably be hoisted out as separate utilities.) We don't actually need SelectPlatformBase, and could instead just directly use the If and IsClassDefined metafunctions, but giving this functionality a name might be clearer. As stated, I think it?s dangerous to allow conditional inheritance where there is no inheritance constraint on the optional class. Therefore I do not think this is better, although I agree it is smaller. I agree that our use case here is smaller. If the amount of code is the problem and less code (and hence fewer reusable components) is preferred, then the original implementation with a simple IsBaseOfAndDerived does the trick (without checking types/cv qualifiers) and could be done in just a few lines, and this is what I originally did before you told me to expand it to more closely resemble external libraries: template class IsBaseOfAndDerived { typedef char yes[1]; typedef char no[2]; template static yes &check(Derived*, T); static no &check(Base*, int); template struct IsBaseOfHost { operator B*() const; operator D*(); }; public: static const bool value = sizeof(check(IsBaseOfHost(), int())) == sizeof(yes); }; It checks the inheritance and existence of the Derived class which is exactly the constraints I need. If you do not want to expose it publicly and remove the reusability because it does not have identical semantics as third party libraries regarding the requirements of the Derived type to be complete, it could be an internal class of SelectBaseClass. I?m not going to value the reusability vs LoC, I leave that decision to you. However, while I think there are other use cases for this specific utility, I still think the proposed change set as a whole is adding a lot of code just to avoid a a few lines of macro usage. That seems like a poor tradeoff to me. Okay. In the end, we have to make a choice - what is more important, that the traits resemble some external library or code size. Because last time I proposed a small change and you wanted it to be closer to C++11 behaviour and I made it closer to that at the expense of more code. Now you are saying I have to make it even tighter to C++11 (or boost?), but also don?t want more lines of code which I believe is impossible given that I want to constrain both the class hierarchy to be reliable and check the existence of the optional class. This brings me to a few architectural questions which I may have (strong) opinions about but are not up to me to decide. 1. Do we want to replace the macros with template metaprogramming? 2 reviewers (and me) liked the metaprogramming approach and it was implemented because they (like me) did not want the macros. But you have a different opinion. I can?t judge who is right and wrong here. 2. Do we want to minimize the code size at the cost of reusability by making contracts of dependent components weaker and putting them as private classes in the SelectBaseClass? 3. If we do want template metaprogramming and want reusable components, the question is, is it important that the specification of completeness (which in general is almost impossible to implement correctly in a portable way as you pointed out) of the template arguments in IsBaseOf is identical as C++11/boost even though no code needs it to be and on the contrary some code benefits from it not to be? In that case which one of them do we, because they are already different anyway? There is a tradeoff between code size and complexity vs copying either one of the external libraries regarding requirements of completeness of template parameters. Thanks, /Erik From erik.osterlund at lnu.se Tue Jan 6 10:37:30 2015 From: erik.osterlund at lnu.se (=?Windows-1252?Q?Erik_=D6sterlund?=) Date: Tue, 6 Jan 2015 10:37:30 +0000 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <2ED21417-E8C1-4E11-AC2F-DED35A993714@oracle.com> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <,<8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se>, <2ED21417-E8C1-4E11-AC2F-DED35A993714@oracle.com> Message-ID: <8445D264-6A94-4BF0-AF0D-611C9BC4303A@lnu.se> Hi Kim, Thank you for your detailed feedback. On 6 jan 2015, at 00:03, Kim Barrett > wrote: On Jan 1, 2015, at 9:07 AM, Erik ?sterlund > wrote: Personally I don?t see how the number of rows of code define uglyness. Reusability, testing, documentation and well definedness in contracts/behaviours all lead to more lines of code, but I don?t think that equals uglyness. So while I disagree more code is uglier because it?s more, it?s all up to opinion and it?s quite pointless to discuss; you are entitled to think that is ugly. Code surface area has an impact on understanding and maintenance. It's not the only factor, but it *is* a factor. That said, I looked more closely and found the webrev new line count substantially overstates the real size of the proposed change. (Nearly half of the new lines are copyright headers and similar boiler plate. A substantial fraction of the remainder are tests, which I wouldn't count as a maintenance negative.) So rather than 800 lines, its probably more like 100 new lines. That still seems to me to be a lot in order to eliminate a few flag macro definitions and the associated #ifdef, but seems much more plausible to amortize across multiple uses. Since I think between us we've come up with several more possible use-cases, I'm willing to withdraw the size complaint. :) While I disagree it is universally incorrect (as long as it is well defined and tested) to not have the exact same requirement on completeness on the parameters as C++11/boost, who also already have different behaviour in this exact regard as pointed out by yourself, I can try to fix it if it bugs you, but it will make the infrastructure even larger which seems to be a concern. [The difference between boost and C++11 is_base_of is minor, with boost being more restrictive than necessary. I fully expect someone to eventually file a bug report about that difference and for the boost trait to be brought into conformance with the C++11 version. I think that difference isn't relevant to this discussion.] It is not correct as a general utility trait because without the requirement that "Derived" be complete one can very easily introduce an ODR violation. That's the rationale for the C++11 requirement. And it's better to check for that possibility, if possible, than to silently risk an unchecked ODR violation that could be challenging to track down. (C++11 doesn't require a diagnostic, leaving that as a QoI issue. g++ and boost issue a diagnostic for this; I haven't tested other compilers.) Point taken, and in my latest proposal including IsBaseOf, a compiler error is generated if Derived is incomplete to comply with your request. Instead I use your class IsClassDefined first to determine if it exists and then IsBaseOf to enforce a stronger contract. Thanks for that contribution. The result can be read in http://cr.openjdk.java.net/~jwilhelm/8067790/webrev.02/ It could make sense to have a bespoke predicate for determining whether a conditionally defined class is derived from some known base, with a usage requirement that if not defined (e.g. complete) in the current translation unit that it will not be defined anywhere else in the program. However, I think such a predicate should have a more specific / informative name and/or shouldn't be classified as a general utility trait. But I think even renaming or moving it wouldn't be right, since I disagree with the need for such a predicate at all. [More below.] The main idea of SelectBaseClass is to inject an optional class into the class hierarchy for specialization of generalized behaviour. It is absolutely critical (and intended) that this class is constrained to be a derived class of the fall-back class - it is not an artifact nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, A must be able to rely on the superclass being at least C-like so that the behaviour of the superclass can be used reliably and transparently regardless of the existence of the optional B. If this contract would not hold, I think it could lead to some really weird and unreliable code which should not be encouraged (at least not according to the Liskov substitution principle etc). In the proposed change and in most of the other use-cases I've thought about, no class-based inheritance relationship between the optionally defined class and the core "base" is required; duck type compatibility is sufficient, and I think often preferable. In the specific proposed change, I'm pretty sure there is nothing that requires any class-based inheritance relationship at all among Atomic, AtomicBase, or any AtomicPlatform class. Atomic implements most of the API, and delegates one piece to the selected "AtomicSuper" by explicit name qualification. If there were no inheritance relationship at all among those classes, nobody would notice (assuming the selection process were changed to not require such a relationship). If AtomicBase and AtomicPlatform were changed as I suggested in one of my earlier comments, to provide a protected cmpxchg_byte function with an unqualified call from Atomic, then Atomic would need to be directly derived from either AtomicBase or AtomicPlatform, but there would be no need for any inheritance relationship between AtomicBase or AtomicPlatform. The duck type compatibility of providing an appropriate cmpxchg_byte function is sufficient. (With such an approach I would also consider making the selected base for Atomic be private, since the inheritance is for implementation convenience and there's no external client that cares about that relationship.) It is true that in our case nothing would break if the 3 Atomic classes were not explicitly related by inheritance, but implicitly related with duck typing. However I want to give the user of SelectBaseClass a stronger contract that the base class can be used reliably and must behave consistently because I think that's better than loose implicit promises. I was never a big fan of duck typing. So what I need in the contract is a check if the optional class is defined and if it fits in the inheritance hierarchy: they are both important IMO. I think there are use-cases where appropriate inheritance placement of the optionally defined class is important, but as discussed above, I don't think this is one of those. Nor are most of the others that I've thought of. I don't want to have to introduce unnecessary inheritance in order to use the new infrastructure. With all the template infrastructure provided with my max reuse variant, it would be trivial (~5 rows) to make a variant of SelectBaseClass that does not constrain inheritance in case somebody would need it. The metafunctions have pretty good synergies. And just to make it clear, I know for a fact I will need IsBaseOf (and some of the other metafunctions) anyway to implement automatic closure specialization (e.g. to distinguish between ExtendedOopClosure and OopClosure). So if we do not introduce them now I will still have to introduce them later on. In those cases where an inheritance relationship really *is* needed, I think the right approach is to use conditional existence for selection and then static assert the needed base/derived relationship(s). For that we would need an IsBaseAndDerived or IsBaseOf, but their use would be appropriately conditionalized on the existence of the optional class (possibly only implicitly, e.g. if Atomic needed to be derived from AtomicBase then we would assert that, and not care whether it was direct derivation or via AtomicPlatform). I do not think it is better to leave correctness detection that could be done automatically to be done manually by the programmer for every use of the function. This seems to me equivalent to saying it is better to use duck typing and write unit tests to detect type errors to be less constrained by a type system. Some may argue this is true but I like my automatic type checks and stronger contracts. But then again such arguing could go on forever without reaching a conclusion. :) Yes and that makes me wonder if we care enough about the completeness semantics being the same as third party libraries (who are already inconsistent) if we do not need them to be, and on the contrary benefit from them not to be. I strongly care about semantics and risk of error for something that is billed as a general purpose utility. Risk of ODR violations is not something I would treat lightly. And minor maintainance changes could easily lead to such, if this proposed IsBaseOf were used as a general utility. Point taken, and IsBaseOf is fixed now as you wanted it. In the end, we have to make a choice - what is more important, that the traits resemble some external library or code size. [?] I think this badly mis-characterizes my position. I think the trait semantics are critial, and I think the proposed traits have a serious defect that render them inappropriate as general utilities. In addition, I don't think they are what is needed for the more specialized technique exemplared by the proposed change and similar use-cases. Code size doesn't enter into these concerns at all. With the IsBaseOf completeness now being enforced, and code size being out of the window, the last concern is you may want a SelectBaseClass with weaker contracts not enforcing inheritance at some point in the future, when inheritance for some reason must be disjoint and only duck typing can be relied upon, yes? In that case I'm sure with the current template infrastructure, it will be easy to add those 5 lines needed to make such a metafunction then. :) The case where code size matters to me in this discussion is that we're talking about adding some non-trivial infrastructure to eliminate some simple macro usage. If we can't amortize that infrastructure across some reasonable number of use-cases, then I wouldn't want to add it. I think there probably are sufficient use-cases to make some additional infrastructure worth-while, but I think what was proposed is not the infrastructure we need. I know I am going to need IsBaseOf for automatic closure specialization to detect differences between ExtendedOopClosure and OopClosure. I don't like manually specializing closures when it could be done automatically. But for that I will need this template infrastructure, so I can guarantee they will come in handy and be used more. ;) 1. Do we want to replace the macros with template metaprogramming? 2 reviewers (and me) liked the metaprogramming approach and it was implemented because they (like me) did not want the macros. But you have a different opinion. I can?t judge who is right and wrong here. I think this is only beneficial if doing so ultimately makes this code follow a common pattern used elsewhere in our code base. (I say "ultimately" because this would be the first occurrence of such a pattern.) I will provide automatic closure specialization using this template infrastructure (given somebody is willing to run jprt for me hehe). I hope we are converging a bit now with opinions. With the IsBaseOf implementation fixed the way you want it, code size being toned down as an issue and reuse of template infrastructure promised (for automatic closure specialization), I think that leaves us to a final issue: Stronger contract enforcing inheritance in SelectBaseClass or weaker relying on duck typing. I would like to repeat that with the reusability of the "max reuse" proposal, such a relaxed duck typed conditional class selector metafunction could be implemented when/if needed later with ~5 LoC and I personally do not think it is much to argue about, especially considering IsBaseOf will certainly be needed soon anyway. Thanks, /Erik From albert.noll at oracle.com Tue Jan 6 12:00:03 2015 From: albert.noll at oracle.com (Albert Noll) Date: Tue, 06 Jan 2015 13:00:03 +0100 Subject: RFR: 8066875: VirtualSpace does not use large pages In-Reply-To: <20141215143649.GB24542@ehelin.jrpg.bea.com> References: <54874F17.7070303@oracle.com> <548814C1.8010905@oracle.com> <548AC362.8010800@oracle.com> <20141215143649.GB24542@ehelin.jrpg.bea.com> Message-ID: <54ABCE43.10506@oracle.com> Hi Erik, On 12/15/2014 03:36 PM, Erik Helin wrote: > On 2014-12-12, Albert Noll wrote: >> Hi Erik, >> >> thanks a lot for taking care of this. I have one minor comment (not a >> reviewer): >> >> 1418 size_t os::largest_page_size_less_than(size_t sz) { >> 1419 if (UseLargePages) { >> 1420 // The page sizes are sorted descendingly. >> 1421 for (size_t i = 0; _page_sizes[i] != 0; ++i) { >> 1422 if (_page_sizes[i] <= sz) { >> 1423 return _page_sizes[i]; >> 1424 } >> 1425 } >> 1426 } >> 1427 return vm_page_size(); >> 1428 } >> >> The function name suggests that the function returns a page size that is >> *less* than the value in the argument (sz). >> However, in line 1422 you check for '<=' . I think you should either fix the >> name of the function or the check in line 1422. > Yeah, I wasn't too fond of that name either. I discussed some potential > names with the guys in the office and ended up with: > - os::page_size_for_region_aligned > - os::page_size_for_region_unaligned > > os::page_size_for_region_aligned and os::page_size_for_region_unaligned > takes the same number of parameters, a region size and minimum number of > pages. The difference is that page_size_for_region_aligned guarantees > that returned page size divides the given region size, whereas > page_size_for_region_unaligned does not guarantee this. > > These names were chosen because a call to page_size_for_region_unaligned > indicates that the surrounding code must handle any alignment on its > own. > > Webrevs: > - incremental: http://cr.openjdk.java.net/~ehelin/8066875/webrev.00-01/ > - full: http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/ That looks good to me (not a reviewer). Best, Albert > Testing: > - JPRT > - Verified that the code cache now uses large pages even if > ReservedCodeCacheSize is 241 MB (see bug for more details). > - Added new internal vm tests (also run on SPARC machine with large > pages) > - Run hotspot jtreg tests on SPARC machine with large pages > > Thanks, > Erik > >> Best, >> Albert >> >> >> On 12/10/2014 10:39 AM, Tobias Hartmann wrote: >>> Hi Erik, >>> >>> looks good (not a reviewer). >>> >>> Thanks, >>> Tobias >>> >>> On 09.12.2014 20:35, Erik Helin wrote: >>>> Hi all, >>>> >>>> the fix for JDK-8049864 [0] made os::page_size_for_region slightly more strict >>>> since the function now demands that the given region_size is size aligned with >>>> respect to the chosen page_size. The reason for doing this was that >>>> os::page_size_for_region was used in two ways: >>>> 1. Give me a suitable page size for this amount of memory >>>> 2. Give me the largest page size for this amount of memory >>>> The fix for JDK-8049864 fixed os::page_size_for_region for the "suitable page >>>> size" scenario, but is too strict for the "largest page size" scenario. This >>>> caused a regression in VirtualSpace::initialize, which only needs the largest >>>> possible page size, since VirtualSpace::initialize_with_granularity takes care >>>> of any alignment issues. >>>> >>>> This patch adds the function os::largest_page_size_less_than and updates >>>> VirtualSpace::initialize to use this new function instead of >>>> os::page_size_for_region. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~ehelin/8066875/webrev.00/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8066875 >>>> >>>> Testing: >>>> - JPRT >>>> - Verified that the code cache now uses large pages even if >>>> ReservedCodeCacheSize is 241 MB (see bug for more details). >>>> - Added new internal vm tests (also run on SPARC machine with large >>>> pages) >>>> >>>> Thanks, >>>> Erik >>>> >>>> [0]: http://hg.openjdk.java.net/jdk9/hs-gc/hotspot/rev/b326a3e8dcab From david.holmes at oracle.com Tue Jan 6 12:07:23 2015 From: david.holmes at oracle.com (David Holmes) Date: Tue, 06 Jan 2015 22:07:23 +1000 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <8445D264-6A94-4BF0-AF0D-611C9BC4303A@lnu.se> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se>, <2ED21417-E8C1-4E11-AC2F-DED35A993714@oracle.com> <8445D264-6A94-4BF0-AF0D-611C9BC4303A@lnu.se> Message-ID: <54ABCFFB.6050809@oracle.com> Erik, Can you try re-sending this - I can't tell who wrote what. Thanks, David On 6/01/2015 8:37 PM, Erik ?sterlund wrote: > Hi Kim, > > Thank you for your detailed feedback. > > On 6 jan 2015, at 00:03, Kim Barrett > wrote: > > On Jan 1, 2015, at 9:07 AM, Erik ?sterlund > wrote: > > Personally I don?t see how the number of rows of code define uglyness. Reusability, testing, documentation and well definedness in contracts/behaviours all lead to more lines of code, but I don?t think that equals uglyness. So while I disagree more code is uglier because it?s more, it?s all up to opinion and it?s quite pointless to discuss; you are entitled to think that is ugly. > > Code surface area has an impact on understanding and maintenance. > It's not the only factor, but it *is* a factor. > > That said, I looked more closely and found the webrev new line count > substantially overstates the real size of the proposed change. (Nearly > half of the new lines are copyright headers and similar boiler plate. > A substantial fraction of the remainder are tests, which I wouldn't > count as a maintenance negative.) So rather than 800 lines, its > probably more like 100 new lines. That still seems to me to be a lot > in order to eliminate a few flag macro definitions and the > associated #ifdef, but seems much more plausible to amortize across > multiple uses. Since I think between us we've come up with several > more possible use-cases, I'm willing to withdraw the size complaint. > > :) > > > While I disagree it is universally incorrect (as long as it is well defined and tested) to not have the exact same requirement on completeness on the parameters as C++11/boost, who also already have different behaviour in this exact regard as pointed out by yourself, I can try to fix it if it bugs you, but it will make the infrastructure even larger which seems to be a concern. > > [The difference between boost and C++11 is_base_of is minor, with > boost being more restrictive than necessary. I fully expect someone to > eventually file a bug report about that difference and for the boost > trait to be brought into conformance with the C++11 version. I think > that difference isn't relevant to this discussion.] > > It is not correct as a general utility trait because without the > requirement that "Derived" be complete one can very easily introduce > an ODR violation. That's the rationale for the C++11 requirement. And > it's better to check for that possibility, if possible, than to > silently risk an unchecked ODR violation that could be challenging to > track down. (C++11 doesn't require a diagnostic, leaving that as a QoI > issue. g++ and boost issue a diagnostic for this; I haven't tested > other compilers.) > > Point taken, and in my latest proposal including IsBaseOf, a compiler error is generated if Derived is incomplete to comply with your request. Instead I use your class IsClassDefined first to determine if it exists and then IsBaseOf to enforce a stronger contract. Thanks for that contribution. The result can be read in http://cr.openjdk.java.net/~jwilhelm/8067790/webrev.02/ > > > It could make sense to have a bespoke predicate for determining > whether a conditionally defined class is derived from some known base, > with a usage requirement that if not defined (e.g. complete) in the > current translation unit that it will not be defined anywhere else in > the program. However, I think such a predicate should have a more > specific / informative name and/or shouldn't be classified as a > general utility trait. > > But I think even renaming or moving it wouldn't be right, since I > disagree with the need for such a predicate at all. [More below.] > > The main idea of SelectBaseClass is to inject an optional class into the class hierarchy for specialization of generalized behaviour. It is absolutely critical (and intended) that this class is constrained to be a derived class of the fall-back class - it is not an artifact nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, A must be able to rely on the superclass being at least C-like so that the behaviour of the superclass can be used reliably and transparently regardless of the existence of the optional B. If this contract would not hold, I think it could lead to some really weird and unreliable code which should not be encouraged (at least not according to the Liskov substitution principle etc). > > In the proposed change and in most of the other use-cases I've thought > about, no class-based inheritance relationship between the optionally > defined class and the core "base" is required; duck type compatibility > is sufficient, and I think often preferable. > > In the specific proposed change, I'm pretty sure there is nothing that > requires any class-based inheritance relationship at all among Atomic, > AtomicBase, or any AtomicPlatform class. Atomic implements most of the > API, and delegates one piece to the selected "AtomicSuper" by explicit > name qualification. If there were no inheritance relationship at all > among those classes, nobody would notice (assuming the selection > process were changed to not require such a relationship). > > If AtomicBase and AtomicPlatform were changed as I suggested in one of > my earlier comments, to provide a protected cmpxchg_byte function with > an unqualified call from Atomic, then Atomic would need to be directly > derived from either AtomicBase or AtomicPlatform, but there would be > no need for any inheritance relationship between AtomicBase or > AtomicPlatform. The duck type compatibility of providing an > appropriate cmpxchg_byte function is sufficient. (With such an > approach I would also consider making the selected base for Atomic be > private, since the inheritance is for implementation convenience and > there's no external client that cares about that relationship.) > > It is true that in our case nothing would break if the 3 Atomic classes were not explicitly related by inheritance, but implicitly related with duck typing. > > However I want to give the user of SelectBaseClass a stronger contract that the base class can be used reliably and must behave consistently because I think that's better than loose implicit promises. I was never a big fan of duck typing. > > > So what I need in the contract is a check if the optional class is defined and if it fits in the inheritance hierarchy: they are both important IMO. > > I think there are use-cases where appropriate inheritance placement of > the optionally defined class is important, but as discussed above, I > don't think this is one of those. Nor are most of the others that I've > thought of. I don't want to have to introduce unnecessary inheritance > in order to use the new infrastructure. > > With all the template infrastructure provided with my max reuse variant, it would be trivial (~5 rows) to make a variant of SelectBaseClass that does not constrain inheritance in case somebody would need it. > > The metafunctions have pretty good synergies. And just to make it clear, I know for a fact I will need IsBaseOf (and some of the other metafunctions) anyway to implement automatic closure specialization (e.g. to distinguish between ExtendedOopClosure and OopClosure). So if we do not introduce them now I will still have to introduce them later on. > > > In those cases where an inheritance relationship really *is* needed, I > think the right approach is to use conditional existence for selection > and then static assert the needed base/derived relationship(s). For > that we would need an IsBaseAndDerived or IsBaseOf, but their use > would be appropriately conditionalized on the existence of the > optional class (possibly only implicitly, e.g. if Atomic needed to be > derived from AtomicBase then we would assert that, and not care > whether it was direct derivation or via AtomicPlatform). > > I do not think it is better to leave correctness detection that could be done automatically to be done manually by the programmer for every use of the function. > > This seems to me equivalent to saying it is better to use duck typing and write unit tests to detect type errors to be less constrained by a type system. Some may argue this is true but I like my automatic type checks and stronger contracts. But then again such arguing could go on forever without reaching a conclusion. :) > > > Yes and that makes me wonder if we care enough about the completeness semantics being the same as third party libraries (who are already inconsistent) if we do not need them to be, and on the contrary benefit from them not to be. > > I strongly care about semantics and risk of error for something that > is billed as a general purpose utility. Risk of ODR violations is not > something I would treat lightly. And minor maintainance changes could > easily lead to such, if this proposed IsBaseOf were used as a general > utility. > > Point taken, and IsBaseOf is fixed now as you wanted it. > > > In the end, we have to make a choice - what is more important, that the traits resemble some external library or code size. [?] > > I think this badly mis-characterizes my position. I think the trait > semantics are critial, and I think the proposed traits have a serious > defect that render them inappropriate as general utilities. In > addition, I don't think they are what is needed for the more > specialized technique exemplared by the proposed change and similar > use-cases. Code size doesn't enter into these concerns at all. > > With the IsBaseOf completeness now being enforced, and code size being out of the window, the last concern is you may want a SelectBaseClass with weaker contracts not enforcing inheritance at some point in the future, when inheritance for some reason must be disjoint and only duck typing can be relied upon, yes? > > In that case I'm sure with the current template infrastructure, it will be easy to add those 5 lines needed to make such a metafunction then. :) > > > The case where code size matters to me in this discussion is that > we're talking about adding some non-trivial infrastructure to > eliminate some simple macro usage. If we can't amortize that > infrastructure across some reasonable number of use-cases, then I > wouldn't want to add it. I think there probably are sufficient > use-cases to make some additional infrastructure worth-while, but I > think what was proposed is not the infrastructure we need. > > I know I am going to need IsBaseOf for automatic closure specialization to detect differences between ExtendedOopClosure and OopClosure. I don't like manually specializing closures when it could be done automatically. But for that I will need this template infrastructure, so I can guarantee they will come in handy and be used more. ;) > > > 1. Do we want to replace the macros with template metaprogramming? 2 reviewers (and me) liked the metaprogramming approach and it was implemented because they (like me) did not want the macros. But you have a different opinion. I can?t judge who is right and wrong here. > > I think this is only beneficial if doing so ultimately makes this code > follow a common pattern used elsewhere in our code base. (I say > "ultimately" because this would be the first occurrence of such a > pattern.) > > I will provide automatic closure specialization using this template infrastructure (given somebody is willing to run jprt for me hehe). > > I hope we are converging a bit now with opinions. With the IsBaseOf implementation fixed the way you want it, code size being toned down as an issue and reuse of template infrastructure promised (for automatic closure specialization), I think that leaves us to a final issue: Stronger contract enforcing inheritance in SelectBaseClass or weaker relying on duck typing. I would like to repeat that with the reusability of the "max reuse" proposal, such a relaxed duck typed conditional class selector metafunction could be implemented when/if needed later with ~5 LoC and I personally do not think it is much to argue about, especially considering IsBaseOf will certainly be needed soon anyway. > > Thanks, > /Erik > From erik.osterlund at lnu.se Tue Jan 6 12:41:13 2015 From: erik.osterlund at lnu.se (=?Windows-1252?Q?Erik_=D6sterlund?=) Date: Tue, 6 Jan 2015 12:41:13 +0000 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <54ABCFFB.6050809@oracle.com> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <,<8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> <,> <2ED21417-E8C1-4E11-AC2F-DED35A993714@oracle.com> <8445D264-6A94-4BF0-AF0D-611C9BC4303A@lnu.se> <54ABCFFB.6050809@oracle.com> Message-ID: <344DF776-637E-4131-A12D-2C62DF35804D@lnu.se> Hi David, I do not know why all nesting levels were flattened, but I will try to resend the email with a different email client and hope for the best (or was it flattened by the mailing list server?). Sorry about that. Rule of thumb meanwhile: I?m the optimist, Kim is the pessimist haha. ;) Thanks, /Erik > On 06 Jan 2015, at 12:07, David Holmes wrote: > > Erik, > > Can you try re-sending this - I can't tell who wrote what. > > Thanks, > David > > On 6/01/2015 8:37 PM, Erik ?sterlund wrote: >> Hi Kim, >> >> Thank you for your detailed feedback. >> >> On 6 jan 2015, at 00:03, Kim Barrett > wrote: >> >> On Jan 1, 2015, at 9:07 AM, Erik ?sterlund > wrote: >> >> Personally I don?t see how the number of rows of code define uglyness. Reusability, testing, documentation and well definedness in contracts/behaviours all lead to more lines of code, but I don?t think that equals uglyness. So while I disagree more code is uglier because it?s more, it?s all up to opinion and it?s quite pointless to discuss; you are entitled to think that is ugly. >> >> Code surface area has an impact on understanding and maintenance. >> It's not the only factor, but it *is* a factor. >> >> That said, I looked more closely and found the webrev new line count >> substantially overstates the real size of the proposed change. (Nearly >> half of the new lines are copyright headers and similar boiler plate. >> A substantial fraction of the remainder are tests, which I wouldn't >> count as a maintenance negative.) So rather than 800 lines, its >> probably more like 100 new lines. That still seems to me to be a lot >> in order to eliminate a few flag macro definitions and the >> associated #ifdef, but seems much more plausible to amortize across >> multiple uses. Since I think between us we've come up with several >> more possible use-cases, I'm willing to withdraw the size complaint. >> >> :) >> >> >> While I disagree it is universally incorrect (as long as it is well defined and tested) to not have the exact same requirement on completeness on the parameters as C++11/boost, who also already have different behaviour in this exact regard as pointed out by yourself, I can try to fix it if it bugs you, but it will make the infrastructure even larger which seems to be a concern. >> >> [The difference between boost and C++11 is_base_of is minor, with >> boost being more restrictive than necessary. I fully expect someone to >> eventually file a bug report about that difference and for the boost >> trait to be brought into conformance with the C++11 version. I think >> that difference isn't relevant to this discussion.] >> >> It is not correct as a general utility trait because without the >> requirement that "Derived" be complete one can very easily introduce >> an ODR violation. That's the rationale for the C++11 requirement. And >> it's better to check for that possibility, if possible, than to >> silently risk an unchecked ODR violation that could be challenging to >> track down. (C++11 doesn't require a diagnostic, leaving that as a QoI >> issue. g++ and boost issue a diagnostic for this; I haven't tested >> other compilers.) >> >> Point taken, and in my latest proposal including IsBaseOf, a compiler error is generated if Derived is incomplete to comply with your request. Instead I use your class IsClassDefined first to determine if it exists and then IsBaseOf to enforce a stronger contract. Thanks for that contribution. The result can be read in http://cr.openjdk.java.net/~jwilhelm/8067790/webrev.02/ >> >> >> It could make sense to have a bespoke predicate for determining >> whether a conditionally defined class is derived from some known base, >> with a usage requirement that if not defined (e.g. complete) in the >> current translation unit that it will not be defined anywhere else in >> the program. However, I think such a predicate should have a more >> specific / informative name and/or shouldn't be classified as a >> general utility trait. >> >> But I think even renaming or moving it wouldn't be right, since I >> disagree with the need for such a predicate at all. [More below.] >> >> The main idea of SelectBaseClass is to inject an optional class into the class hierarchy for specialization of generalized behaviour. It is absolutely critical (and intended) that this class is constrained to be a derived class of the fall-back class - it is not an artifact nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, A must be able to rely on the superclass being at least C-like so that the behaviour of the superclass can be used reliably and transparently regardless of the existence of the optional B. If this contract would not hold, I think it could lead to some really weird and unreliable code which should not be encouraged (at least not according to the Liskov substitution principle etc). >> >> In the proposed change and in most of the other use-cases I've thought >> about, no class-based inheritance relationship between the optionally >> defined class and the core "base" is required; duck type compatibility >> is sufficient, and I think often preferable. >> >> In the specific proposed change, I'm pretty sure there is nothing that >> requires any class-based inheritance relationship at all among Atomic, >> AtomicBase, or any AtomicPlatform class. Atomic implements most of the >> API, and delegates one piece to the selected "AtomicSuper" by explicit >> name qualification. If there were no inheritance relationship at all >> among those classes, nobody would notice (assuming the selection >> process were changed to not require such a relationship). >> >> If AtomicBase and AtomicPlatform were changed as I suggested in one of >> my earlier comments, to provide a protected cmpxchg_byte function with >> an unqualified call from Atomic, then Atomic would need to be directly >> derived from either AtomicBase or AtomicPlatform, but there would be >> no need for any inheritance relationship between AtomicBase or >> AtomicPlatform. The duck type compatibility of providing an >> appropriate cmpxchg_byte function is sufficient. (With such an >> approach I would also consider making the selected base for Atomic be >> private, since the inheritance is for implementation convenience and >> there's no external client that cares about that relationship.) >> >> It is true that in our case nothing would break if the 3 Atomic classes were not explicitly related by inheritance, but implicitly related with duck typing. >> >> However I want to give the user of SelectBaseClass a stronger contract that the base class can be used reliably and must behave consistently because I think that's better than loose implicit promises. I was never a big fan of duck typing. >> >> >> So what I need in the contract is a check if the optional class is defined and if it fits in the inheritance hierarchy: they are both important IMO. >> >> I think there are use-cases where appropriate inheritance placement of >> the optionally defined class is important, but as discussed above, I >> don't think this is one of those. Nor are most of the others that I've >> thought of. I don't want to have to introduce unnecessary inheritance >> in order to use the new infrastructure. >> >> With all the template infrastructure provided with my max reuse variant, it would be trivial (~5 rows) to make a variant of SelectBaseClass that does not constrain inheritance in case somebody would need it. >> >> The metafunctions have pretty good synergies. And just to make it clear, I know for a fact I will need IsBaseOf (and some of the other metafunctions) anyway to implement automatic closure specialization (e.g. to distinguish between ExtendedOopClosure and OopClosure). So if we do not introduce them now I will still have to introduce them later on. >> >> >> In those cases where an inheritance relationship really *is* needed, I >> think the right approach is to use conditional existence for selection >> and then static assert the needed base/derived relationship(s). For >> that we would need an IsBaseAndDerived or IsBaseOf, but their use >> would be appropriately conditionalized on the existence of the >> optional class (possibly only implicitly, e.g. if Atomic needed to be >> derived from AtomicBase then we would assert that, and not care >> whether it was direct derivation or via AtomicPlatform). >> >> I do not think it is better to leave correctness detection that could be done automatically to be done manually by the programmer for every use of the function. >> >> This seems to me equivalent to saying it is better to use duck typing and write unit tests to detect type errors to be less constrained by a type system. Some may argue this is true but I like my automatic type checks and stronger contracts. But then again such arguing could go on forever without reaching a conclusion. :) >> >> >> Yes and that makes me wonder if we care enough about the completeness semantics being the same as third party libraries (who are already inconsistent) if we do not need them to be, and on the contrary benefit from them not to be. >> >> I strongly care about semantics and risk of error for something that >> is billed as a general purpose utility. Risk of ODR violations is not >> something I would treat lightly. And minor maintainance changes could >> easily lead to such, if this proposed IsBaseOf were used as a general >> utility. >> >> Point taken, and IsBaseOf is fixed now as you wanted it. >> >> >> In the end, we have to make a choice - what is more important, that the traits resemble some external library or code size. [?] >> >> I think this badly mis-characterizes my position. I think the trait >> semantics are critial, and I think the proposed traits have a serious >> defect that render them inappropriate as general utilities. In >> addition, I don't think they are what is needed for the more >> specialized technique exemplared by the proposed change and similar >> use-cases. Code size doesn't enter into these concerns at all. >> >> With the IsBaseOf completeness now being enforced, and code size being out of the window, the last concern is you may want a SelectBaseClass with weaker contracts not enforcing inheritance at some point in the future, when inheritance for some reason must be disjoint and only duck typing can be relied upon, yes? >> >> In that case I'm sure with the current template infrastructure, it will be easy to add those 5 lines needed to make such a metafunction then. :) >> >> >> The case where code size matters to me in this discussion is that >> we're talking about adding some non-trivial infrastructure to >> eliminate some simple macro usage. If we can't amortize that >> infrastructure across some reasonable number of use-cases, then I >> wouldn't want to add it. I think there probably are sufficient >> use-cases to make some additional infrastructure worth-while, but I >> think what was proposed is not the infrastructure we need. >> >> I know I am going to need IsBaseOf for automatic closure specialization to detect differences between ExtendedOopClosure and OopClosure. I don't like manually specializing closures when it could be done automatically. But for that I will need this template infrastructure, so I can guarantee they will come in handy and be used more. ;) >> >> >> 1. Do we want to replace the macros with template metaprogramming? 2 reviewers (and me) liked the metaprogramming approach and it was implemented because they (like me) did not want the macros. But you have a different opinion. I can?t judge who is right and wrong here. >> >> I think this is only beneficial if doing so ultimately makes this code >> follow a common pattern used elsewhere in our code base. (I say >> "ultimately" because this would be the first occurrence of such a >> pattern.) >> >> I will provide automatic closure specialization using this template infrastructure (given somebody is willing to run jprt for me hehe). >> >> I hope we are converging a bit now with opinions. With the IsBaseOf implementation fixed the way you want it, code size being toned down as an issue and reuse of template infrastructure promised (for automatic closure specialization), I think that leaves us to a final issue: Stronger contract enforcing inheritance in SelectBaseClass or weaker relying on duck typing. I would like to repeat that with the reusability of the "max reuse" proposal, such a relaxed duck typed conditional class selector metafunction could be implemented when/if needed later with ~5 LoC and I personally do not think it is much to argue about, especially considering IsBaseOf will certainly be needed soon anyway. >> >> Thanks, >> /Erik >> From erik.osterlund at lnu.se Tue Jan 6 12:47:10 2015 From: erik.osterlund at lnu.se (=?Windows-1252?Q?Erik_=D6sterlund?=) Date: Tue, 6 Jan 2015 12:47:10 +0000 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <8445D264-6A94-4BF0-AF0D-611C9BC4303A@lnu.se> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <,<8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> <,<2ED21417-E8C1-4E11-AC2F-DED35A993714@oracle.com> <>> <8445D264-6A94-4BF0-AF0D-611C9BC4303A@lnu.se> Message-ID: <5C6292ED-BCB1-4FEA-96A6-6E9D0087E6C8@lnu.se> Hi Kim, [This is a second identical email from before, hoping nesting levels of comments do not get flattened this time.] Thank you for your detailed feedback. On 6 jan 2015, at 00:03, Kim Barrett > wrote: Code surface area has an impact on understanding and maintenance. It's not the only factor, but it *is* a factor. That said, I looked more closely and found the webrev new line count substantially overstates the real size of the proposed change. (Nearly half of the new lines are copyright headers and similar boiler plate. A substantial fraction of the remainder are tests, which I wouldn't count as a maintenance negative.) So rather than 800 lines, its probably more like 100 new lines. That still seems to me to be a lot in order to eliminate a few flag macro definitions and the associated #ifdef, but seems much more plausible to amortize across multiple uses. Since I think between us we've come up with several more possible use-cases, I'm willing to withdraw the size complaint. :) [The difference between boost and C++11 is_base_of is minor, with boost being more restrictive than necessary. I fully expect someone to eventually file a bug report about that difference and for the boost trait to be brought into conformance with the C++11 version. I think that difference isn't relevant to this discussion.] It is not correct as a general utility trait because without the requirement that "Derived" be complete one can very easily introduce an ODR violation. That's the rationale for the C++11 requirement. And it's better to check for that possibility, if possible, than to silently risk an unchecked ODR violation that could be challenging to track down. (C++11 doesn't require a diagnostic, leaving that as a QoI issue. g++ and boost issue a diagnostic for this; I haven't tested other compilers.) Point taken, and in my latest proposal including IsBaseOf, a compiler error is generated if Derived is incomplete to comply with your request. Instead I use your class IsClassDefined first to determine if it exists and then IsBaseOf to enforce a stronger contract. Thanks for that contribution. The result can be read in http://cr.openjdk.java.net/~jwilhelm/8067790/webrev.02/ In the proposed change and in most of the other use-cases I've thought about, no class-based inheritance relationship between the optionally defined class and the core "base" is required; duck type compatibility is sufficient, and I think often preferable. In the specific proposed change, I'm pretty sure there is nothing that requires any class-based inheritance relationship at all among Atomic, AtomicBase, or any AtomicPlatform class. Atomic implements most of the API, and delegates one piece to the selected "AtomicSuper" by explicit name qualification. If there were no inheritance relationship at all among those classes, nobody would notice (assuming the selection process were changed to not require such a relationship). If AtomicBase and AtomicPlatform were changed as I suggested in one of my earlier comments, to provide a protected cmpxchg_byte function with an unqualified call from Atomic, then Atomic would need to be directly derived from either AtomicBase or AtomicPlatform, but there would be no need for any inheritance relationship between AtomicBase or AtomicPlatform. The duck type compatibility of providing an appropriate cmpxchg_byte function is sufficient. (With such an approach I would also consider making the selected base for Atomic be private, since the inheritance is for implementation convenience and there's no external client that cares about that relationship.) It is true that in our case nothing would break if the 3 Atomic classes were not explicitly related by inheritance, but implicitly related with duck typing. However I want to give the user of SelectBaseClass a stronger contract that the base class can be used reliably and must behave consistently because I think that's better than loose implicit promises. I was never a big fan of duck typing. I think there are use-cases where appropriate inheritance placement of the optionally defined class is important, but as discussed above, I don't think this is one of those. Nor are most of the others that I've thought of. I don't want to have to introduce unnecessary inheritance in order to use the new infrastructure. With all the template infrastructure provided with my max reuse variant, it would be trivial (~5 rows) to make a variant of SelectBaseClass that does not constrain inheritance in case somebody would need it. The metafunctions have pretty good synergies. And just to make it clear, I know for a fact I will need IsBaseOf (and some of the other metafunctions) anyway to implement automatic closure specialization (e.g. to distinguish between ExtendedOopClosure and OopClosure). So if we do not introduce them now I will still have to introduce them later on. In those cases where an inheritance relationship really *is* needed, I think the right approach is to use conditional existence for selection and then static assert the needed base/derived relationship(s). For that we would need an IsBaseAndDerived or IsBaseOf, but their use would be appropriately conditionalized on the existence of the optional class (possibly only implicitly, e.g. if Atomic needed to be derived from AtomicBase then we would assert that, and not care whether it was direct derivation or via AtomicPlatform). I do not think it is better to leave correctness detection that could be done automatically to be done manually by the programmer for every use of the function. This seems to me equivalent to saying it is better to use duck typing and write unit tests to detect type errors to be less constrained by a type system. Some may argue this is true but I like my automatic type checks and stronger contracts. But then again such arguing could go on forever without reaching a conclusion. :) I strongly care about semantics and risk of error for something that is billed as a general purpose utility. Risk of ODR violations is not something I would treat lightly. And minor maintainance changes could easily lead to such, if this proposed IsBaseOf were used as a general utility. Point taken, and IsBaseOf is fixed now as you wanted it. I think this badly mis-characterizes my position. I think the trait semantics are critial, and I think the proposed traits have a serious defect that render them inappropriate as general utilities. In addition, I don't think they are what is needed for the more specialized technique exemplared by the proposed change and similar use-cases. Code size doesn't enter into these concerns at all. With the IsBaseOf completeness now being enforced, and code size being out of the window, the last concern is you may want a SelectBaseClass with weaker contracts not enforcing inheritance at some point in the future, when inheritance for some reason must be disjoint and only duck typing can be relied upon, yes? In that case I'm sure with the current template infrastructure, it will be easy to add those 5 lines needed to make such a metafunction then. :) The case where code size matters to me in this discussion is that we're talking about adding some non-trivial infrastructure to eliminate some simple macro usage. If we can't amortize that infrastructure across some reasonable number of use-cases, then I wouldn't want to add it. I think there probably are sufficient use-cases to make some additional infrastructure worth-while, but I think what was proposed is not the infrastructure we need. I know I am going to need IsBaseOf for automatic closure specialization to detect differences between ExtendedOopClosure and OopClosure. I don't like manually specializing closures when it could be done automatically. But for that I will need this template infrastructure, so I can guarantee they will come in handy and be used more. ;) I think this is only beneficial if doing so ultimately makes this code follow a common pattern used elsewhere in our code base. (I say "ultimately" because this would be the first occurrence of such a pattern.) I will provide automatic closure specialization using this template infrastructure (given somebody is willing to run jprt for me hehe). I hope we are converging a bit now with opinions. With the IsBaseOf implementation fixed the way you want it, code size being toned down as an issue and reuse of template infrastructure promised (for automatic closure specialization), I think that leaves us to a final issue: Stronger contract enforcing inheritance in SelectBaseClass or weaker relying on duck typing. I would like to repeat that with the reusability of the "max reuse" proposal, such a relaxed duck typed conditional class selector metafunction could be implemented when/if needed later with ~5 LoC and I personally do not think it is much to argue about, especially considering IsBaseOf will certainly be needed soon anyway. Thanks, /Erik From coleen.phillimore at oracle.com Tue Jan 6 18:28:13 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 06 Jan 2015 13:28:13 -0500 Subject: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF694A5@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF264E2@DEWDFEMB12A.global.corp.sap> <547F4914.1030904@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF36139@DEWDFEMB12A.global.corp.sap> <54809196.7030306@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF39BA5@DEWDFEMB12A.global.corp.sap> <5480AB58.80604@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF3D6BB@DEWDFEMB12A.global.corp.sap> <54889D61.30905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF3E1C5@DEWDFEMB12A.global.corp.sap> <54989B9F.2070903@oracle.com> <549B48A3.1070303@oracle.com> <54A2089D.1020105@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF53F70@DEWDFEMB12A.global.corp.sap> <54A2C966.5010306@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF67EB6@DEWDFEMB12A.global.corp.sap> <54A6C6C3.1050806@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF69210@DEWDFEMB12A.global.corp.sap> <54AAC086.2060600@oracle.com> <54AAC34F.2050807@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF694A5@DEWDFEMB12A.global.corp.sap> Message-ID: <54AC293D.9020903@oracle.com> You're welcome. I updated the files to 2015. Happy New Year! Coleen On 1/6/15, 2:56 AM, Lindenmaier, Goetz wrote: > Hi Coleen, > > thanks a lot for pushing this change! > > Best regards, > Goetz. > > -----Original Message----- > From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] > Sent: Monday, January 05, 2015 6:01 PM > To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net; 'hotspot-dev at openjdk.java.net' > Subject: Re: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. > > > On 1/5/15, 11:49 AM, Coleen Phillimore wrote: >> On 1/5/15, 4:24 AM, Lindenmaier, Goetz wrote: >>> Hi Coleen, >>> >>> It would be great if you could sponsor this change. >> I see no other comments so I'll push it today. > I am planning to push this to hs-rt, btw. > Coleen > >>>> The metaspace case probably can be fixed to use the >>>> try_reserve_heap() code maybe. >>> We wondered why the compressed class space is always above the heap. >>> If the heap is too big for unscaled, the compressed class space could >>> be placed >>> in the lower 4G. >> The reason that we chose the compressed class space on the top of the >> heap is for a couple reasons. The time spent compressing and >> decompressing oops is more critical than the time spent doing the same >> for the _klass pointer, so we wanted to make the system more likely to >> get unscaled compressed oops, or even zero based compressed oops. >> >> On solaris, the space below the heap is limited because by default >> malloc uses it so we were limited in low address memory that we can >> use. We want the compressed class space code not be platform >> dependent. We also need to have the CDS archive at a fixed address >> near or adjacent to the compressed class space, so were running out at >> the bottom of the heap. >> >> The compressed class space is fixed at startup (just like PermGen was) >> and the classes are variable sized, so we had to pick a size fairly >> large so that it's unlikely for applications to run out. If above 32G, >> we might be able to implement code to allocate a new space which would >> use the same compression algorithm. We haven't had a request for that >> but it's something we could do. >> >> >>> The aarch port brings some changes here, and we would like to >>> investigate >>> this on top of the aarch changes, if that's fine with you. >> So one thing I prototyped and really wanted to check in was what we >> called indirect class pointers. Instead of allocating classes in a >> fixed class space, we allocate a fixed array of Klass pointers and the >> objects in the heap point to this. It had a 8% performance >> degradation in specbb2008 compiler.compiler (iirc, or something like >> that). >> >> This solved so many problems though. If you investigated this sort of >> approach, I'd be very grateful. I haven't had time to go back to >> figure out why this had this degradation (we thought it was Java >> vtable accesses). >> >> Thanks, >> Coleen >>> Thanks a lot for going through this complex review! >>> >>> Best regards, >>> Goetz. >>> >>> -----Original Message----- >>> From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] >>> Sent: Freitag, 2. Januar 2015 17:27 >>> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net >>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>> "disjoint base" and improve compressed heap handling. >>> >>> >>> On 1/2/15, 10:09 AM, Lindenmaier, Goetz wrote: >>>> Hi Coleen, >>>> >>>> I looked at the cleanups and found both not straight forward to >>>> implement. >>>> >>>> 1) Merging ReservedHeap::initialize and >>>> ReservedHeapSpace::try_reserve_heap. >>>> The new code calls initialize() with requested_address = NULL, >>>> but metaspace.cpp >>>> passes a real requested_address. Therefore I can not clean up the >>>> calls to >>>> failed_to_reserve_as_requested() in initialize(). Without that >>>> cleanup, I need to pass flags >>>> into initialize() to configure all the different behaviours I would >>>> need in a >>>> merged case. >>>> I think with 'large' and 'requested_address' there is already enough >>>> cases >>>> in that method, so that I don't want to add more. >>> Okay, this is fine. More flags to special case functions aren't good >>> either. Can you file an RFE to look into refactoring so that the >>> duplicate code can be removed though? The metaspace case probably can >>> be fixed to use the try_reserve_heap() code maybe. >>> >>>> 2) Moving _noaccess_prefix to ReservedHeapSpace >>>> >>>> This does not work as ReservedHeapSpace is casted to >>>> ReservedSpace and returned by value. So ReservedHeapSpace can not have >>>> fields or virtual methods. >>>> It would be probably better to change the code to return >>>> ReservedHeapSpace >>>> by pointer, or, at least under that type, but I think that exceeds >>>> the scope of this change. >>> Too bad. >>> >>>> So is it ok if I leave the change as is wrt. to these cleanups? >>> Yes, this is fine. >>>> I extended test/runtime/CompressedOops/UseCompressedOops.java >>>> to combine the testcases of the different mode with the GCs and >>>> UseLargePages. >>>> I added the changes in the test to >>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >>> Is .03 with the additional changes? If so, you should have made an .04. >>> >>> I reviewed this again and it looks good. I think Vladimir has reviewed >>> an earlier version. I can sponsor this assuming there are no more >>> comments. >>> >>> Coleen >>>> Best regards, >>>> Goetz. >>>> >>>> >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] >>>> Sent: Dienstag, 30. Dezember 2014 16:49 >>>> To: Lindenmaier, Goetz; hotspot-runtime-dev at openjdk.java.net >>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>> "disjoint base" and improve compressed heap handling. >>>> >>>> >>>> On 12/30/14, 8:42 AM, Lindenmaier, Goetz wrote: >>>>> Hi Coleen, >>>>> >>>>>> disjoint >>>>> It tries to say that the bits used for the base (36-63) are >>>>> different from >>>>> those used for the oops / used as offset into the heap (0-35). >>>>> I chose the name according to this translation >>>>> http://dict.leo.org/#/search=disjoint&searchLoc=0&resultOrder=basic&multiwordShowSingle=on >>>>> >>>>> Both the german adjective and verb fit to describe the property of >>>>> the base. >>>>> I can add the following to the docu of the mode in universe.hpp:358 >>>>> // Disjoint: Bits used in base are disjoint from bits used >>>>> // for oops ==> oop = (cOop << 3) | base. One can disjoint >>>>> // the bits of an oop into base and compressed oop. >>>> Yes, that helps. It's a good distinct word so its good that we can >>>> make >>>> sense in the documentation. >>>>>> (I filed a bug to move these to the memory directory) >>>>> Good idea! >>>>> >>>>>> 272 void ReservedSpace::establish_noaccess_prefix() { >>>>>> This function should be in ReservedHeapSpace >>>>> Yes, I moved it. I also would like to move the field >>>>> _noaccess_prefix, >>>>> but I would have to change a lot of code, like >>>>> ReservedSpace::release(). >>>>> The problem is that _base is not the real base in ReservedHeapSpace. >>>>> Probably ReservedHeapSpace should have heap_size() and heap_base() >>>>> giving the correcte values. >>>>> I'll try to come up with a patch on top of this, maybe we should do it >>>>> in a separate bug. >>>> I just moved _noaccess_prefix and added a virtual >>>> ReservedHeapSpace::release() that adjusted the base and size with >>>> noaccess_prefix. I don't think the footprint of a vtable or virtual >>>> call makes any difference. >>>>>> Can you have arguments.cpp verify_range of HeapSearchSteps so this >>>>>> can't >>>>>> happen? >>>>> Done. >>>>> >>>>>> Rather move this to the end of ReservedHeapSpace constructor. >>>>>> assert(_base == NULL || ... >>>>> Yes, you are right, it should not be there twice. I moved it into the >>>>> constructor. It is only relevant for heaps, right? >>>> Yes. >>>>>> Code in try_reserve_heap() is mostly a copy of >>>>>> ReservedSpace::initialize() >>>>> You are right, requested_address in initialize is dead now. I'll >>>>> remove that >>>>> and try to merge the two functions. >>>>> >>>>> I removed the verbose printing. Anyways, it's mostly useful when >>>>> developing this. >>>> Thanks. I was trying to find some way to move the printing so it would >>>> make sense to me but ended up giving up on that. >>>>>> aligned_HBMA >>>>> Changed to aligned_heap_base_min_address. >>>>> >>>>>> Is "size" const in initialize_compressed_heap? >>>>> Yes. Added 'const'. >>>>> >>>>> I'll run tests with +UseLargePages. >>>>> >>>>> I had verified that the heap is at the exact same position for the >>>>> code in >>>>> my first RFR, and with HeapSearchSteps=1 to make sure I don't break >>>>> anything. >>>>> Vladimir asked me to increase that to 3. So far I still didn't >>>>> find a case where >>>>> this changed anything. >>>>> (But I know from SAP JVM, which is tested on all the OS variants we >>>>> support, >>>>> that there are cases where this can lead to a better - but then >>>>> different - heap >>>>> placement.) >>>>> The only case where there is always a difference is Solaris 5.11 which >>>>> now can allocate zerobased memory. >>>>> The code passes the CDS jtreg tests. >>>>> >>>>> I updated the webrev.03 with the minor changes. >>>>> I'll come up with a new one with the cleanups of Reserved(Heap)Space >>>>> (_noaccess_prefix and redundant functions). >>>>> >>>>> Thanks for this thorough review!! >>>> Thank you for your willingness to make these changes. >>>> >>>> Happy New Year! >>>> Coleen >>>> >>>>> Happy new year, >>>>> Goetz. >>>>> >>>>> >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: hotspot-runtime-dev >>>>> [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of >>>>> Coleen Phillimore >>>>> Sent: Dienstag, 30. Dezember 2014 03:06 >>>>> To: hotspot-runtime-dev at openjdk.java.net >>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>> "disjoint base" and improve compressed heap handling. >>>>> >>>>> >>>>> Hi, I've reviewed more code. >>>>> >>>>> I'm sorry about the following comment that should have been asked >>>>> before. The name of the mode is very confusing. I can't figure out >>>>> why >>>>> it's called "disjoint". I keep looking for a second heap base or some >>>>> code that makes the heap discontinuous. The heap base is really >>>>> specially aligned, right? So I went to the thesaurus.com: >>>>> >>>>> http://www.thesaurus.com/browse/aligned >>>>> >>>>> and "disjoin" is an antonym. Why was "disjoint" chosen? From the >>>>> code, it looks like "aligned" or "scaled" are more descriptive but >>>>> they're overused words in the code. But ScaledBasedNarrowOop seems >>>>> more >>>>> descriptive than Disjoint. >>>>> >>>>> In virtualspace.cpp (I filed a bug to move these to the memory >>>>> directory) >>>>> >>>>> 272 void ReservedSpace::establish_noaccess_prefix() { >>>>> >>>>> >>>>> This function should be in ReservedHeapSpace since ReservedSpace never >>>>> does this (and we never want it to). The function it replaced was in >>>>> ReservedSpace and that was wrong. Now that ReservedHeapSpace has >>>>> compressed oops logic in it, I think this function should be moved to >>>>> this class. Actually, I've modified your patch to move the >>>>> _noaccess_prefix field to ReservedHeapSpace. Maybe that's too much >>>>> change to your patch but since the handling of noaccess_prefix is >>>>> changed, it seemed like the thing to do. >>>>> >>>>> Can you have arguments.cpp verify_range of HeapSearchSteps so this >>>>> can't >>>>> happen? >>>>> >>>>> guarantee(HeapSearchSteps > 0, "Don't set HeapSearchSteps to >>>>> 0"); >>>>> >>>>> Can this be a function like check_encoding_for_mark_sweep() , or >>>>> something like that? It appears twice. Rather move this to the >>>>> end of >>>>> ReservedHeapSpace constructor. >>>>> >>>>> assert(_base == NULL || >>>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == _base, >>>>> "area must be distinguishable from marks for >>>>> mark-sweep"); >>>>> assert(_base == NULL || >>>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() == >>>>> &_base[size], >>>>> "area must be distinguishable from marks for >>>>> mark-sweep"); >>>>> >>>>> Code in try_reserve_heap() is a mostly a copy of >>>>> ReservedSpace::initialize(), except the first version has a call to >>>>> failed_to_reserve_as_requested which looks like it's handling the case >>>>> for not being able to reserve the requested address for compressed >>>>> oops >>>>> which seems like you don't want this. This is very confusing. There >>>>> shouldn't be two of these. >>>>> >>>>> It looks like in the compressed oops case, you call >>>>> ReservedSpace::initialize() only with NULL requested address so >>>>> failed_to_reserve_as_requested is dead code with this change. So it >>>>> seems like try_reserve_heap() could be ReservedSpace::initialize() >>>>> with >>>>> the failed_to_reserve_as_requested eliminated and a special case for >>>>> handling unaligned addresses. >>>>> >>>>> The logic in initialize_compressed_heap() still doesn't make sense to >>>>> me. If the first try_reserve_range() succeeds at getting an unscaled >>>>> compressed oop base, then does PrintCompressedOopsMode && Verbose >>>>> still >>>>> print: >>>>> >>>>> tty->print(" == U N S C A L E D ==\n"); >>>>> tty->print(" == Z E R O B A S E D ==\n"); >>>>> tty->print(" == D I S J O I N T B A S E ==\n"); >>>>> tty->print(" == H E A P B A S E D ==\n"); >>>>> >>>>> Why would you print all 4? It's the printing that makes this >>>>> confusing! Now I see how the next case checks that the base >>>>> address is >>>>> a better allocation. If you changed the printing to a comment like: >>>>> >>>>> // UnscaledMode after testing previous base address is not better (or >>>>> something). >>>>> >>>>> Actually, you have these comments. I think the printing should be >>>>> removed because it's the opposite of helpful. >>>>> >>>>> I had trouble reading the parameter aligned_HBMA, can you change to >>>>> aligned_heap_base_min (I know it's longer but there's too many >>>>> calculations here that words will make easier to read). Or just >>>>> heap_base_min_address. >>>>> >>>>> zerobased_max = (char *)OopEncodingHeapMax - class_space; >>>>> >>>>> Odd that the solaris compiler can compile this and not the code below. >>>>> (minor point) I think this should be: >>>>> >>>>> zerobased_max = zerobased_max - class_space; >>>>> >>>>> Is "size" const in initialize_compressed_heap? (answer appears to be >>>>> yes) It's useful for reading the calculations that it never >>>>> changes so >>>>> I think it should be const (it's a minor point too). >>>>> >>>>> I don't have any other comments, I've tried to do the calculations >>>>> a bit >>>>> but I know that you've done this yourself. >>>>> >>>>> Also, can you run tests with -XX:+UseLargePages? I assume you've run >>>>> tests with the 4 garbage collectors. This is a tricky piece of code >>>>> that replaced something else difficult. We've had to change it >>>>> often to >>>>> support other features like CDS and compressed class pointers so >>>>> making >>>>> it as understandable as possible is worth the extra effort. I think >>>>> your new design looks cleaner. >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> >>>>> On 12/24/14, 6:13 PM, Coleen Phillimore wrote: >>>>>> One note for my first reviewed file. >>>>>> >>>>>> On 12/22/14, 5:30 PM, Coleen Phillimore wrote: >>>>>>> Hi Goetz, >>>>>>> >>>>>>> I'm looking at this version of the code >>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >>>>>>> >>>>>>> >>>>>>> This passes our JPRT tests. Yeah! >>>>>>> >>>>>>> It's a significant change, so I'm actually reading the code with the >>>>>>> patches applied. >>>>>>> >>>>>>> In src/share/vm/memory/universe.cpp >>>>>>> >>>>>>> Can you put all this code from 742-776 in a new function above >>>>>>> initialize_heap(), like set_narrow_oop_values()? >>>>>>> The comment at 689-695 refer to code that's moved to >>>>>>> virtualspace.cpp. I think it needs to be reworded, updated for >>>>>>> disjoint oops mode or removed. The comment at 744-749 also refers to >>>>>>> code that's moved so should be removed also. The last two lines of >>>>>>> this comment refer to something whose implementation has changed, so >>>>>>> just remove them. >>>>>>> >>>>>>> On line >>>>>>> >>>>>>> 750 if ((uint64_t)Universe::heap()->reserved_region().end() > >>>>>>> UnscaledOopHeapMax) { >>>>>>> >>>>>>> Can you make this expression a variable because the same expression >>>>>>> appears on line 754. >>>>>>> >>>>>>> Why can't this code at 742-776 be with the compressed oops mode code >>>>>>> at line 836-839. If you make this set_narrow_oop_values() function, >>>>>>> can it be moved together? >>>>>> Apparently this doesn't work (it crashes on some platforms). I can't >>>>>> completely follow the control flow that makes this wrong, but ignore >>>>>> this suggestion. >>>>>> >>>>>> Coleen >>>>>>> Can you remove this comment too? I have no idea what it means. There >>>>>>> is no CATCH below (?) >>>>>>> >>>>>>> // We will never reach the CATCH below since Exceptions::_throw >>>>>>> will cause >>>>>>> // the VM to exit if an exception is thrown during >>>>>>> initialization >>>>>>> >>>>>>> >>>>>>> ** We should file an RFE to move virtualspace.hpp/cpp to the >>>>>>> src/share/vm/memory directory! >>>>>>> >>>>>>> Sorry, this is partial, I'll continue tomorrow. >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> >>>>>>> >>>>>>> On 12/11/2014 06:18 AM, Lindenmaier, Goetz wrote: >>>>>>>> Hi Coleen, >>>>>>>> >>>>>>>> thanks for looking at this change! >>>>>>>> >>>>>>>>> get_attach_addresses_for_disjoint_mode function belongs in >>>>>>>>> virtualspace.cpp >>>>>>>> All information about the bounds of the modes comes from universe. >>>>>>>> This is >>>>>>>> related to is_disjoint_heap_base_address() and thus I left it in >>>>>>>> universe. >>>>>>>> But I'll move it, it can also be regarded as an implementation >>>>>>>> detail of >>>>>>>> initialize_compressed_heap(). I can make it static then, so that >>>>>>>> it's removed >>>>>>>> in 32 bit build, which is good. >>>>>>>> >>>>>>>>> Can you reduce some to at least 100? >>>>>>>> Done. I agree that's better . (I have a 2560x1440 screen, causing >>>>>>>> me to have >>>>>>>> big windows :)) >>>>>>>> >>>>>>>>> Why is highest_start not const char* ? >>>>>>>> Removed the const. >>>>>>>> >>>>>>>>> The control flow in initialize_compressed_heap makes no sense >>>>>>>>> to me. >>>>>>>> Vladimir had the same problem, I added comment at line 493 to make >>>>>>>> that clearer. >>>>>>>> One problem of the previous implementation was that, if the os >>>>>>>> layer >>>>>>>> returned an address with desired properties, but not at the >>>>>>>> requested >>>>>>>> address, it was discarded. >>>>>>>> To fix this, try_reserve_heap always returns what it allocated, >>>>>>>> and I >>>>>>>> check afterwards. >>>>>>>> Therefore the checks of the next mode include the check of the >>>>>>>> previous one. So if I try to allocate disjoint, but get unscaled, >>>>>>>> I will detect that and return the unscaled heap. >>>>>>>> If I implement a return, I just have to add tests. I don't get >>>>>>>> rid of >>>>>>>> other tests. >>>>>>>> >>>>>>>>> Also, our 32 bit platforms don't like: 1 * SIZE_64K * SIZE_32G, >>>>>>>> I changed that to use UCONST64 macro. That should help I guess. >>>>>>>> I also have these in globalDefinitions_xlc.hpp. Should I move them >>>>>>>> to globalDefinitions.hpp:200, after definition of K, M and G? >>>>>>>> >>>>>>>> I uploaded a new webrev: >>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.03/ >>>>>>>> >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Goetz. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: hotspot-runtime-dev >>>>>>>> [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of >>>>>>>> Coleen Phillimore >>>>>>>> Sent: Mittwoch, 10. Dezember 2014 20:22 >>>>>>>> To: hotspot-runtime-dev at openjdk.java.net >>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>> >>>>>>>> >>>>>>>> Hi Goetz, >>>>>>>> >>>>>>>> I have some initial comments, which are much less detailed than >>>>>>>> Vladimir's comments. I haven't actually processed all the >>>>>>>> implementation details yet. >>>>>>>> >>>>>>>> I think get_attach_addresses_for_disjoint_mode function belongs in >>>>>>>> virtualspace.cpp and not in universe. I like that the >>>>>>>> compressed oop >>>>>>>> reservation logic is moved to virtualspace.cpp. I think this is an >>>>>>>> improvement over the Universe::preferred_heap_base() logic which >>>>>>>> was >>>>>>>> also difficult. >>>>>>>> >>>>>>>> The Hotspot coding style doesn't dictate 80 columns anymore >>>>>>>> (there is >>>>>>>> debate whether it should have or not, which we try not to have this >>>>>>>> debate), but I found the very long lines in this code >>>>>>>> inconsistent with >>>>>>>> other Hotspot code. I had to expand my window to cover my whole >>>>>>>> screen. Can you reduce some to at least 100? >>>>>>>> >>>>>>>> For example, I aligned these parameters to see better since there >>>>>>>> are so >>>>>>>> many (the indentation isn't right in email). >>>>>>>> >>>>>>>> void ReservedHeapSpace::try_reserve_range(char *const >>>>>>>> highest_start, >>>>>>>> char *lowest_start, >>>>>>>> size_t >>>>>>>> attach_point_alignment, >>>>>>>> char >>>>>>>> *aligned_HBMA, // >>>>>>>> HeapBaseMinAddress >>>>>>>> char *upper_bound, >>>>>>>> size_t size, >>>>>>>> size_t alignment, >>>>>>>> bool large) { >>>>>>>> >>>>>>>> Why is highest_start not const char* ? Doesn't char* const >>>>>>>> highest_start just restrict you from changing the pointer and >>>>>>>> not what >>>>>>>> it points to? This seems odd to do. >>>>>>>> >>>>>>>> The control flow in initialize_compressed_heap makes no sense to >>>>>>>> me. >>>>>>>> It seems like there should be an exit when the best allocation for >>>>>>>> compression is achieved. But it falls through after >>>>>>>> try_reserve_range(). I can't figure out why it should fall >>>>>>>> through.... >>>>>>>> >>>>>>>> I was expecting something like: >>>>>>>> >>>>>>>> if (PrintCompressedOopsMode && Verbose) { >>>>>>>> tty->print(" == H E A P B A S E M I N A D D R E S S >>>>>>>> ==\n"); >>>>>>>> } >>>>>>>> get the heap at aligned HeapBaseMinAddress, return if >>>>>>>> success... >>>>>>>> >>>>>>>> if (PrintCompressedOopsMode && Verbose) { >>>>>>>> tty->print(" == U N S C A L E D ==\n"); >>>>>>>> } >>>>>>>> try to get heap base for unscaled access, return if >>>>>>>> successful >>>>>>>> >>>>>>>> etc. >>>>>>>> >>>>>>>> >>>>>>>> You should make this into a little function for each return and >>>>>>>> for the >>>>>>>> end of the initialize function, or move it to the ReservedHeapSpace >>>>>>>> constructor (caller). >>>>>>>> >>>>>>>> assert(_base == NULL || >>>>>>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == >>>>>>>> _base, >>>>>>>> "area must be distinguishable from marks for >>>>>>>> mark-sweep"); >>>>>>>> assert(_base == NULL || >>>>>>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() >>>>>>>> == >>>>>>>> &_base[size], >>>>>>>> "area must be distinguishable from marks for >>>>>>>> mark-sweep"); >>>>>>>> } >>>>>>>> >>>>>>>> I ran this through JPRT and this line didn't compile on macos: >>>>>>>> >>>>>>>> const uint64_t num_attempts_to_try = MIN2(HeapSearchSteps, >>>>>>>> num_attempts_possible); >>>>>>>> >>>>>>>> I'm retrying now as: >>>>>>>> >>>>>>>> const uint64_t num_attempts_to_try = >>>>>>>> MIN2((uint64_t)HeapSearchSteps, num_attempts_possible); >>>>>>>> >>>>>>>> Sorry for the delay looking at this. This is a big change that I >>>>>>>> needed >>>>>>>> to make more time to read. I am pleased that it's a performance >>>>>>>> improvement. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>>>>> >>>>>>>> On 12/8/14, 10:54 AM, Lindenmaier, Goetz wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> This is just a ping to gc/rt mailing lists to reach appropriate >>>>>>>>> people. >>>>>>>>> >>>>>>>>> I please need a reviewer from gc or rt, could somebody have a >>>>>>>>> look at this? >>>>>>>>> >>>>>>>>> Short summary: >>>>>>>>> - new cOops mode disjointbase that allows optimizations on PPC >>>>>>>>> improving over heapbased >>>>>>>>> - search for heaps: finds zerobased on sparc Solaris 11 and Aix >>>>>>>>> - concentrate cOops heap allocation code in one function >>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Please reply only to the original thread in hotspot-dev to keep >>>>>>>>> this >>>>>>>>> local. >>>>>>>>> >>>>>>>>> Thanks and best regards, >>>>>>>>> Goetz. >>>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>> Sent: Donnerstag, 4. Dezember 2014 19:44 >>>>>>>>> To: Lindenmaier, Goetz >>>>>>>>> Cc: 'hotspot-dev developers' >>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>> >>>>>>>>> This looks good to me. >>>>>>>>> >>>>>>>>> Now we need second review since changes are significant. >>>>>>>>> Preferable >>>>>>>>> from >>>>>>>>> GC group since you changed ReservedHeapSpace. They will be >>>>>>>>> affected >>>>>>>>> most. Review from Runtime is also welcome. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Vladimir >>>>>>>>> >>>>>>>>> On 12/4/14 10:27 AM, Lindenmaier, Goetz wrote: >>>>>>>>>> Hi Vladimir. >>>>>>>>>> >>>>>>>>>> Sorry. I updated the webrev once more. Hope it's fine now. >>>>>>>>>> At least I can write comments :) >>>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> Goetz >>>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>>> Sent: Thursday, December 04, 2014 5:54 PM >>>>>>>>>> To: Lindenmaier, Goetz >>>>>>>>>> Cc: 'hotspot-dev developers' >>>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>> >>>>>>>>>> I spotted an other bug. >>>>>>>>>> You replaced !_base with _base != NULL when moved code to >>>>>>>>>> try_reserve_range() - it should be _base == NULL. >>>>>>>>>> The same problem in asserts: >>>>>>>>>> >>>>>>>>>> + assert(_base != NULL || >>>>>>>>>> markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == >>>>>>>>>> _base, >>>>>>>>>> + "area must be distinguishable from marks for >>>>>>>>>> mark-sweep"); >>>>>>>>>> + assert(_base != NULL || >>>>>>>>>> markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() >>>>>>>>>> == >>>>>>>>>> &_base[size], >>>>>>>>>> + "area must be distinguishable from marks for >>>>>>>>>> mark-sweep"); >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Also you did not remove _base && in next place: >>>>>>>>>> >>>>>>>>>> + (_base && _base + size > zerobased_max))) { // >>>>>>>>>> Unscaled >>>>>>>>>> delivered an arbitrary address. >>>>>>>>>> >>>>>>>>>> New comment is good. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Vladimri >>>>>>>>>> >>>>>>>>>> On 12/4/14 1:45 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>> Hi Vladimir, >>>>>>>>>>> >>>>>>>>>>>> Add more extending comment explaining that. >>>>>>>>>>> The comment for try_reserve_heap was meant to explain that. >>>>>>>>>>> I further added a comment in initialize_compressed_heap(). >>>>>>>>>>> >>>>>>>>>>>> You need another parameter to pass UnscaledOopHeapMax or >>>>>>>>>>>> zerobased_max. >>>>>>>>>>> Oh, thanks a lot! That's important. Fixed. >>>>>>>>>>> >>>>>>>>>>>> I mean that you already checked _base == NULL so on other side >>>>>>>>>>>> of || _base != NULL - why you need (_base &&) check? >>>>>>>>>>> Sorry, now I got it. Removed. >>>>>>>>>>> >>>>>>>>>>> I updated the webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Increment on top of the increment :) >>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/incremental_diffs2.patch >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Goetz. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -----Original Message----- >>>>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>>>> Sent: Mittwoch, 3. Dezember 2014 18:32 >>>>>>>>>>> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' >>>>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>>> >>>>>>>>>>> Comments are below. >>>>>>>>>>> >>>>>>>>>>> On 12/3/14 5:49 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>>> Hi Vladimir, >>>>>>>>>>>> >>>>>>>>>>>> thanks for looking at the change! See my comments inline >>>>>>>>>>>> below. >>>>>>>>>>>> >>>>>>>>>>>> I made a new webrev: >>>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Incremental changes: >>>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.02/incremental_diffs.patch >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Best regards, >>>>>>>>>>>> Goetz. >>>>>>>>>>>> >>>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>>> From: Vladimir Kozlov [mailto:vladimir.kozlov at oracle.com] >>>>>>>>>>>>> Sent: Mittwoch, 3. Dezember 2014 00:46 >>>>>>>>>>>>> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' >>>>>>>>>>>>> Subject: Re: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>>>>> >>>>>>>>>>>>> This looks good to me. Someone in runtime/gc have to look >>>>>>>>>>>>> on it >>>>>>>>>>>>> too. >>>>>>>>>>>>> >>>>>>>>>>>>> universe.cpp about >>>>>>>>>>>>> SystemProperty("com.sap.vm.test.compressedOopsMode" >>>>>>>>>>>>> we have: >>>>>>>>>>>>> java.vm.info=mixed mode, sharing >>>>>>>>>>>>> so we can have: >>>>>>>>>>>>> java.vm.compressedOopsMode=... >>>>>>>>>>>> Yes, that's good for me. Fixed. >>>>>>>>>>>> >>>>>>>>>>>>> I am not expert in properties names but I don't want to have >>>>>>>>>>>>> 'com.sap' >>>>>>>>>>>>> in VM's property name. >>>>>>>>>>>>> virtualspace.cpp: >>>>>>>>>>>>> Could you fix release() - it does not reset _alignment? >>>>>>>>>>>> Fixed. >>>>>>>>>>>> >>>>>>>>>>>>> In try_reserve_heap(), please, use (base == NULL) instead of >>>>>>>>>>>>> (!base). >>>>>>>>>>>>> And you don't need 'return;' in alignment check at the end of >>>>>>>>>>>>> method. >>>>>>>>>>>> Fixed. >>>>>>>>>>>> >>>>>>>>>>>>> In initialize_compressed_heap() again (!_base). >>>>>>>>>>>> Fixed. >>>>>>>>>>>> >>>>>>>>>>>>> You don't stop (check >>>>>>>>>>>>> (base == NULL)) after successful unscaled, zerobased, >>>>>>>>>>>>> disjointbase >>>>>>>>>>>>> allocations. You need to separate them with the check: >>>>>>>>>>>>> >>>>>>>>>>>>> + >>>>>>>>>>>>> + } >>>>>>>>>>>>> + } >>>>>>>>>>>>> + if (_base == NULL) { >>>>>>>>>>>>> + >>>>>>>>>>>>> + if (PrintCompressedOopsMode && Verbose) { >>>>>>>>>>>>> + tty->print(" == Z E R O B A S E D ==\n"); >>>>>>>>>>>>> + } >>>>>>>>>>>>> and so on. >>>>>>>>>>>> No, I can't and don't want to check for _base != NULL. >>>>>>>>>>>> I always keep the result of the last try, also if it didn't >>>>>>>>>>>> fulfil the required properties. >>>>>>>>>>>> So I take that result and go into the next check. That check >>>>>>>>>>>> might succeed >>>>>>>>>>>> with the heap allocated before. >>>>>>>>>>>> This allows me to separate allocation and placement criteria, >>>>>>>>>>>> and to have the >>>>>>>>>>>> placement criteria checked in only one place (per mode). >>>>>>>>>>>> Only for HeapBaseMinAddress I don't do it that way, I >>>>>>>>>>>> explicitly >>>>>>>>>>>> call release(). >>>>>>>>>>>> This way I can enforce mode heapbased. >>>>>>>>>>> I see what you are saying. It was not clear from comments >>>>>>>>>>> what is >>>>>>>>>>> going on. >>>>>>>>>>> Add more extending comment explaining that. >>>>>>>>>>> >>>>>>>>>>>>> num_attempts calculation and while() loop are similar in >>>>>>>>>>>>> unscaled and >>>>>>>>>>>>> zerobased cases. Could you move it into a separate method? >>>>>>>>>>>> I can do that, but I don't like it as I have to pass in 7 >>>>>>>>>>>> parameters. >>>>>>>>>>> You need an other parameter to pass UnscaledOopHeapMax or >>>>>>>>>>> zerobased_max. >>>>>>>>>>> >>>>>>>>>>>> That makes the code not much more readable. The function will >>>>>>>>>>>> look like this: >>>>>>>>>>> I think initialize_compressed_heap() is more readable now. >>>>>>>>>>> >>>>>>>>>>>> void ReserveHeapSpace::try_reserve_range(char *const >>>>>>>>>>>> highest_start, char *lowest_start, size_t >>>>>>>>>>>> attach_point_alignment, >>>>>>>>>>>> char >>>>>>>>>>>> *aligned_HBMA, size_t size, size_t alignment, bool large) { >>>>>>>>>>>> guarantee(HeapSearchSteps > 0, "Don't set >>>>>>>>>>>> HeapSearchSteps >>>>>>>>>>>> to 0"); >>>>>>>>>>>> >>>>>>>>>>>> const size_t attach_range = highest_start - >>>>>>>>>>>> lowest_start; >>>>>>>>>>>> // Cap num_attempts at possible number. >>>>>>>>>>>> // At least one is possible even for 0 sized >>>>>>>>>>>> attach range. >>>>>>>>>>>> const uint64_t num_attempts_possible = >>>>>>>>>>>> (attach_range / >>>>>>>>>>>> attach_point_alignment) + 1; >>>>>>>>>>>> const uint64_t num_attempts_to_try = >>>>>>>>>>>> MIN2(HeapSearchSteps, num_attempts_possible); >>>>>>>>>>>> >>>>>>>>>>>> const size_t stepsize = align_size_up(attach_range / >>>>>>>>>>>> num_attempts_to_try, attach_point_alignment); >>>>>>>>>>>> >>>>>>>>>>>> // Try attach points from top to bottom. >>>>>>>>>>>> char* attach_point = highest_start; >>>>>>>>>>>> while (attach_point >= lowest_start && >>>>>>>>>>>> attach_point <= highest_start && // Avoid >>>>>>>>>>>> wrap >>>>>>>>>>>> around. >>>>>>>>>>>> (!_base || _base < aligned_HBMA || _base + >>>>>>>>>>>> size > >>>>>>>>>>>> (char *)UnscaledOopHeapMax)) { >>>>>>>>>>>> try_reserve_heap(size, alignment, large, >>>>>>>>>>>> attach_point); >>>>>>>>>>>> attach_point -= stepsize; >>>>>>>>>>>> } >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> In disjointbase while() condition no need for _base second >>>>>>>>>>>>> check: >>>>>>>>>>>>> + (_base == NULL || >>>>>>>>>>>>> + ((_base + size > (char *)OopEncodingHeapMax) && >>>>>>>>>>>> I need this for the same reason as above: This is the check for >>>>>>>>>>>> successful allocation. >>>>>>>>>>> I mean that you already checked _base == NULL so on other >>>>>>>>>>> side of >>>>>>>>>>> || _base != NULL - why you need (_base &&) check? >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Vladimir >>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Vladimir >>>>>>>>>>>> >>>>>>>>>>>> On 11/21/14 5:31 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> I prepared a new webrev trying to cover all the issues >>>>>>>>>>>>> mentioned below. >>>>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.01/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I moved functionality from os.cpp and universe.cpp into >>>>>>>>>>>>> ReservedHeapSpace::initialize_compressed_heap(). >>>>>>>>>>>>> This class offers to save _base and _special, which I would >>>>>>>>>>>>> have to reimplement >>>>>>>>>>>>> if I had improved the methods I had added to os.cpp to also >>>>>>>>>>>>> allocate large page >>>>>>>>>>>>> heaps. >>>>>>>>>>>>> Anyways, I think this class is the right place to gather code >>>>>>>>>>>>> trying to reserve >>>>>>>>>>>>> the heap. >>>>>>>>>>>>> Also, I get along without setting the shift, base, >>>>>>>>>>>>> implicit_null_check etc. fields >>>>>>>>>>>>> of Universe, so there is no unnecessary calling back and forth >>>>>>>>>>>>> between the two >>>>>>>>>>>>> classes. >>>>>>>>>>>>> Universe gets the heap back, and then sets the properties it >>>>>>>>>>>>> needs to configure >>>>>>>>>>>>> the compressed oops. >>>>>>>>>>>>> All code handling the noaccess prefix is in a single >>>>>>>>>>>>> method, too. >>>>>>>>>>>>> >>>>>>>>>>>>> Best regards, >>>>>>>>>>>>> Goetz. >>>>>>>>>>>>> >>>>>>>>>>>>> Btw, I had to workaround a SS12u1 problem: it wouldn't compile >>>>>>>>>>>>> char * x = (char*)UnscaledOopHeapMax - size in 32-bit mode. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>>> From: hotspot-dev >>>>>>>>>>>>> [mailto:hotspot-dev-bounces at openjdk.java.net] >>>>>>>>>>>>> On Behalf Of Lindenmaier, Goetz >>>>>>>>>>>>> Sent: Montag, 17. November 2014 09:33 >>>>>>>>>>>>> To: 'Vladimir Kozlov'; 'hotspot-dev at openjdk.java.net' >>>>>>>>>>>>> Subject: RE: RFR(L): 8064457: Introduce compressed oops mode >>>>>>>>>>>>> "disjoint base" and improve compressed heap handling. >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Vladimir, >>>>>>>>>>>>> >>>>>>>>>>>>>> It is very significant rewriting and it takes time to >>>>>>>>>>>>>> evaluate >>>>>>>>>>>>>> it. >>>>>>>>>>>>> Yes, I know ... and I don't want to push, but nevertheless >>>>>>>>>>>>> a ping >>>>>>>>>>>>> can be useful sometimes. Thanks a lot for looking at it. >>>>>>>>>>>>> >>>>>>>>>>>>>> And I would not say it is simpler then before :) >>>>>>>>>>>>> If I fix what you propose it's gonna get even more simple ;) >>>>>>>>>>>>>> These is what I found so far. >>>>>>>>>>>>>> The idea to try to allocate in a range instead of just below >>>>>>>>>>>>>> UnscaledOopHeapMax or OopEncodingHeapMax is good. So I would >>>>>>>>>>>>>> ask to do >>>>>>>>>>>>>> several attempts (3?) on non_PPC64 platforms too. >>>>>>>>>>>>> Set to 3. >>>>>>>>>>>>> >>>>>>>>>>>>>> It is matter of preference but I am not comfortable with >>>>>>>>>>>>>> switch in loop. >>>>>>>>>>>>>> For me sequential 'if (addr == 0)' checks is simpler. >>>>>>>>>>>>> I'll fix this. >>>>>>>>>>>>> >>>>>>>>>>>>>> One thing worries me that you release found space and try to >>>>>>>>>>>>>> get it >>>>>>>>>>>>>> again with ReservedHeapSpace. Is it possible to add new >>>>>>>>>>>>>> ReservedHeapSpace ctor which simple use already allocated >>>>>>>>>>>>>> space? >>>>>>>>>>>>> This was to keep diff's small, but I also think a new >>>>>>>>>>>>> constructor is good. >>>>>>>>>>>>> I'll fix this. >>>>>>>>>>>>> >>>>>>>>>>>>>> The next code in ReservedHeapSpace() is hard to understand >>>>>>>>>>>>>> (): >>>>>>>>>>>>>> (UseCompressedOops && (requested_address == NULL || >>>>>>>>>>>>> requested_address+size > (char*)OopEncodingHeapMax) ? >>>>>>>>>>>>>> may be move all this into noaccess_prefix_size() and add >>>>>>>>>>>>>> comments. >>>>>>>>>>>>> I have to redo this anyways if I make new constructors. >>>>>>>>>>>>> >>>>>>>>>>>>>> Why you need prefix when requested_address == NULL? >>>>>>>>>>>>> If we allocate with NULL, we most probably will get a heap >>>>>>>>>>>>> where >>>>>>>>>>>>> base != NULL and thus need a noaccess prefix. >>>>>>>>>>>>> >>>>>>>>>>>>>> Remove next comment in universe.cpp: >>>>>>>>>>>>>> // SAPJVM GL 2014-09-22 >>>>>>>>>>>>> Removed. >>>>>>>>>>>>> >>>>>>>>>>>>>> Again you will release space so why bother to include space >>>>>>>>>>>>>> for classes?: >>>>>>>>>>>>>> + // For small heaps, save some space for compressed >>>>>>>>>>>>>> class pointer >>>>>>>>>>>>>> + // space so it can be decoded with no base. >>>>>>>>>>>>> This was done like this before. We must assure the upper >>>>>>>>>>>>> bound >>>>>>>>>>>>> of the >>>>>>>>>>>>> heap is low enough that the compressed class space still fits >>>>>>>>>>>>> in there. >>>>>>>>>>>>> >>>>>>>>>>>>> virtualspace.cpp >>>>>>>>>>>>> >>>>>>>>>>>>>> With new code size+noaccess_prefix could be requested. But >>>>>>>>>>>>>> later it is >>>>>>>>>>>>>> not used if WIN64_ONLY(&& UseLargePages) and you will have >>>>>>>>>>>>>> empty >>>>>>>>>>>>>> non-protected page below heap. >>>>>>>>>>>>> There's several points to this: >>>>>>>>>>>>> * Also if not protectable, the heap base has to >>>>>>>>>>>>> be below >>>>>>>>>>>>> the real start of the >>>>>>>>>>>>> heap. Else the first object in the heap will be >>>>>>>>>>>>> compressed to 'null' >>>>>>>>>>>>> and decompression will fail. >>>>>>>>>>>>> * If we don't reserve the memory other stuff can >>>>>>>>>>>>> end up >>>>>>>>>>>>> in this space. On >>>>>>>>>>>>> errors, if would be quite unexpected to find >>>>>>>>>>>>> memory >>>>>>>>>>>>> there. >>>>>>>>>>>>> * To get a heap for the new disjoint mode I must >>>>>>>>>>>>> control >>>>>>>>>>>>> the size of this. >>>>>>>>>>>>> Requesting a heap starting at (aligned base + >>>>>>>>>>>>> prefix) is more likely to fail. >>>>>>>>>>>>> * The size for the prefix must anyways be considered >>>>>>>>>>>>> when deciding whether the >>>>>>>>>>>>> heap is small enough to run with compressed >>>>>>>>>>>>> oops. >>>>>>>>>>>>> So distinguishing the case where we really can omit this would >>>>>>>>>>>>> require >>>>>>>>>>>>> quite some additional checks everywhere, and I thought it's >>>>>>>>>>>>> not >>>>>>>>>>>>> worth it. >>>>>>>>>>>>> >>>>>>>>>>>>> matcher.hpp >>>>>>>>>>>>> >>>>>>>>>>>>>> Universe::narrow_oop_use_implicit_null_checks() should be >>>>>>>>>>>>>> true >>>>>>>>>>>>>> for such >>>>>>>>>>>>>> case too. So you can add new condition with || to existing >>>>>>>>>>>>>> ones. The >>>>>>>>>>>>>> only condition you relax is base != NULL. Right? >>>>>>>>>>>>> Yes, that's how it's intended. >>>>>>>>>>>>> >>>>>>>>>>>>> arguments.* files >>>>>>>>>>>>> >>>>>>>>>>>>>> Why you need PropertyList_add changes. >>>>>>>>>>>>> Oh, the code using it got lost. I commented on this in the >>>>>>>>>>>>> description in the webrev. >>>>>>>>>>>>> "To more efficiently run expensive tests in various compressed >>>>>>>>>>>>> oop modes, we set a property with the mode the VM is running >>>>>>>>>>>>> in. So far it's called "com.sap.vm.test.compressedOopsMode" >>>>>>>>>>>>> better suggestions are welcome (and necessary I guess). Our >>>>>>>>>>>>> long running tests that are supposed to run in a dedicated >>>>>>>>>>>>> compressed oop mode check this property and abort >>>>>>>>>>>>> themselves if >>>>>>>>>>>>> it's not the expected mode." >>>>>>>>>>>>> When I know about the heap I do >>>>>>>>>>>>> Arguments::PropertyList_add(new >>>>>>>>>>>>> SystemProperty("com.sap.vm.test.compressedOopsMode", >>>>>>>>>>>>> narrow_oop_mode_to_string(narrow_oop_mode()), >>>>>>>>>>>>> false)); >>>>>>>>>>>>> in universe.cpp. >>>>>>>>>>>>> On some OSes it's deterministic which modes work, there we >>>>>>>>>>>>> don't start such tests. >>>>>>>>>>>>> Others, as you mentioned OSX, are very indeterministic. Here >>>>>>>>>>>>> we save testruntime with this. >>>>>>>>>>>>> But it's not that important. >>>>>>>>>>>>> We can still parse the PrintCompresseOopsMode output after the >>>>>>>>>>>>> test and discard the >>>>>>>>>>>>> run. >>>>>>>>>>>>> >>>>>>>>>>>>>> Do you have platform specific changes? >>>>>>>>>>>>> Yes, for ppc and aix. I'll submit them once this is in. >>>>>>>>>>>>> >>>>>>>>>>>>> From your other mail: >>>>>>>>>>>>>> One more thing. You should allow an allocation in the range >>>>>>>>>>>>>> when returned from OS allocated address does not match >>>>>>>>>>>>>> requested address. We had such cases on OSX, for example, >>>>>>>>>>>>>> when >>>>>>>>>>>>>> OS allocates at different address but still inside range. >>>>>>>>>>>>> Good point. I'll fix that in >>>>>>>>>>>>> os::attempt_reserve_memory_in_range. >>>>>>>>>>>>> >>>>>>>>>>>>> I'll ping again once a new webrev is done! >>>>>>>>>>>>> >>>>>>>>>>>>> Best regards, >>>>>>>>>>>>> Goetz. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 11/10/14 6:57 AM, Lindenmaier, Goetz wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I need to improve a row of things around compressed oops heap >>>>>>>>>>>>>> handling >>>>>>>>>>>>>> to achieve good performance on ppc. >>>>>>>>>>>>>> I prepared a first webrev for review: >>>>>>>>>>>>>> http://cr.openjdk.java.net/~goetz/webrevs/8064457-disjoint/webrev.00/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> A detailed technical description of the change is in the >>>>>>>>>>>>>> webrev and according bug. >>>>>>>>>>>>>> >>>>>>>>>>>>>> If requested, I will split the change into parts with more >>>>>>>>>>>>>> respective less impact on >>>>>>>>>>>>>> non-ppc platforms. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The change is derived from well-tested code in our VM. >>>>>>>>>>>>>> Originally it was >>>>>>>>>>>>>> crafted to require the least changes of VM coding, I changed >>>>>>>>>>>>>> it to be better >>>>>>>>>>>>>> streamlined with the VM. >>>>>>>>>>>>>> I tested this change to deliver heaps at about the same >>>>>>>>>>>>>> addresses as before. >>>>>>>>>>>>>> Heap addresses mostly differ in lower bits. In some cases >>>>>>>>>>>>>> (Solaris 5.11) a heap >>>>>>>>>>>>>> in a better compressed oops mode is found, though. >>>>>>>>>>>>>> I ran (and adapted) test/runtime/CompressedOops and >>>>>>>>>>>>>> gc/arguments/TestUseCompressedOops*. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>> Goetz. >>>>>>>>>>>>>> >>>>>>>>>>>>>> From kim.barrett at oracle.com Tue Jan 6 20:27:23 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 6 Jan 2015 15:27:23 -0500 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <8445D264-6A94-4BF0-AF0D-611C9BC4303A@lnu.se> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se>, <2ED21417-E8C1-4E11-AC2F-DED35A993714@oracle.com> <8445D264-6A94-4BF0-AF0D-611C9BC4303A@lnu.se> Message-ID: <2F1C0741-3F39-42D9-BDA8-EF853D2F936E@oracle.com> On Jan 6, 2015, at 5:37 AM, Erik ?sterlund wrote: > > On 6 jan 2015, at 00:03, Kim Barrett wrote: > It is true that in our case nothing would break if the 3 Atomic classes were not explicitly related by inheritance, but implicitly related with duck typing. > > However I want to give the user of SelectBaseClass a stronger contract that the base class can be used reliably and must behave consistently because I think that's better than loose implicit promises. Strengthening the selection requirements is a mistake for this general technique. When the optionally defined platform-specific class is defined, we want to (attempt to) use it. If there are additional requirements on the class that it fails to meet, that should result in a compilation error, not a selection failure that results in fallback to the generic class. > I was never a big fan of duck typing. Sorry, you lost me there; duck typing is fundamental to template usage. > The metafunctions have pretty good synergies. And just to make it clear, I know for a fact I will need IsBaseOf (and some of the other metafunctions) anyway to implement automatic closure specialization (e.g. to distinguish between ExtendedOopClosure and OopClosure). So if we do not introduce them now I will still have to introduce them later on. Perhaps IsBaseOf will be needed for that other feature. [I've not looked at it recently or with sufficient care to have a well informed opinion on that right now, though I'm a little suspicious; that sounds like using class inheritance as a stand-in for a concept check.] If IsBaseOf is needed for that (or for some other) feature, then it can be added in conjunction with, or as a prelude to, that feature. I claim that the Atomic cmpxchg support doesn't need (and should not use) IsBaseOf and the associated infrastructure, and so that infrastructure shouldn't be piggy-backing into the code base under the cover of the Atomic cmpxchg changes. >> In those cases where an inheritance relationship really *is* needed, I >> think the right approach is to use conditional existence for selection >> and then static assert the needed base/derived relationship(s). For >> that we would need an IsBaseAndDerived or IsBaseOf, but their use >> would be appropriately conditionalized on the existence of the >> optional class (possibly only implicitly, e.g. if Atomic needed to be >> derived from AtomicBase then we would assert that, and not care >> whether it was direct derivation or via AtomicPlatform). > > I do not think it is better to leave correctness detection that could be done automatically to be done manually by the programmer for every use of the function. > > This seems to me equivalent to saying it is better to use duck typing and write unit tests to detect type errors to be less constrained by a type system. Some may argue this is true but I like my automatic type checks and stronger contracts. But then again such arguing could go on forever without reaching a conclusion. :) The inheritance-based selection criteria does not automatically check for correctness. Instead, it masks errors by falling back to the generic behavior. >> I strongly care about semantics and risk of error for something that >> is billed as a general purpose utility. Risk of ODR violations is not >> something I would treat lightly. And minor maintainance changes could >> easily lead to such, if this proposed IsBaseOf were used as a general >> utility. > > Point taken, and IsBaseOf is fixed now as you wanted it. Good. That's what we'll want, when we want IsBaseOf. > I will provide automatic closure specialization using this template infrastructure (given somebody is willing to run jprt for me hehe). I need to get back to that stuff. Stefan Karlsson and I recently talked about some ideas in that area too. There certainly seems like there should be room for improvement; I may not think macros are inherently ugly, but I would be hard pressed to argue for the beauty of the present oop iterator stuff. > I hope we are converging a bit now with opinions. [?] I think that leaves us to a final issue: Stronger contract enforcing inheritance in SelectBaseClass or weaker relying on duck typing [?] I'm not sure we are converging; we disagree on the selection criteria, and that's quite fundamental. Waiting to see if my arguments above convince you to come around to my position. From coleen.phillimore at oracle.com Tue Jan 6 20:37:20 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 06 Jan 2015 15:37:20 -0500 Subject: RFR (S) 8067836: The Universe::flush_foo methods belong in CodeCache. Message-ID: <54AC4780.1000800@oracle.com> Summary: Move this code to CodeCache. Tested by compiling and running runThese. open webrev at http://cr.openjdk.java.net/~coleenp/8067836/ bug link https://bugs.openjdk.java.net/browse/JDK-8067836 Thanks, Coleen From george.triantafillou at oracle.com Tue Jan 6 20:49:20 2015 From: george.triantafillou at oracle.com (George Triantafillou) Date: Tue, 06 Jan 2015 15:49:20 -0500 Subject: RFR (XS): 8068540 - [TESTBUG] Exclude failing nightly tests Message-ID: <54AC4A50.70101@oracle.com> Please review this small fix for JDK-8068540. The tests were modified with the "@ignore" jtreg tag to exclude them from nightly testing. JBS: https://bugs.openjdk.java.net/browse/JDK-8068540 webrev: http://cr.openjdk.java.net/~gtriantafill/8068540/webrev.00/ The fix was tested locally on Linux with jtreg and the JPRT hotspot testset. -George From coleen.phillimore at oracle.com Tue Jan 6 20:52:38 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 06 Jan 2015 15:52:38 -0500 Subject: RFR (XS): 8068540 - [TESTBUG] Exclude failing nightly tests In-Reply-To: <54AC4A50.70101@oracle.com> References: <54AC4A50.70101@oracle.com> Message-ID: <54AC4B16.7010201@oracle.com> Why are these tests failing? The bug is not helpful. Thanks, Coleen On 1/6/15, 3:49 PM, George Triantafillou wrote: > Please review this small fix for JDK-8068540. The tests were modified > with the "@ignore" jtreg tag to exclude them from nightly testing. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8068540 > webrev: http://cr.openjdk.java.net/~gtriantafill/8068540/webrev.00/ > > > The fix was tested locally on Linux with jtreg and the JPRT hotspot > testset. > > -George From kim.barrett at oracle.com Tue Jan 6 21:04:18 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 6 Jan 2015 16:04:18 -0500 Subject: RFR (S) 8067836: The Universe::flush_foo methods belong in CodeCache. In-Reply-To: <54AC4780.1000800@oracle.com> References: <54AC4780.1000800@oracle.com> Message-ID: <7A189C0D-35DA-4FBB-9462-5458E4199506@oracle.com> On Jan 6, 2015, at 3:37 PM, Coleen Phillimore wrote: > > Summary: Move this code to CodeCache. > > Tested by compiling and running runThese. > > open webrev at http://cr.openjdk.java.net/~coleenp/8067836/ > bug link https://bugs.openjdk.java.net/browse/JDK-8067836 > > Thanks, > Coleen Looks good to me. From erik.osterlund at lnu.se Tue Jan 6 21:42:52 2015 From: erik.osterlund at lnu.se (=?Windows-1252?Q?Erik_=D6sterlund?=) Date: Tue, 6 Jan 2015 21:42:52 +0000 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <2F1C0741-3F39-42D9-BDA8-EF853D2F936E@oracle.com> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <,<8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se>, <2ED21417-E8C1-4E11-AC2F-DED35A993714@oracle.com> <8445D264-6A94-4BF0-AF0D-611C9BC4303A@lnu.se>, <2F1C0741-3F39-42D9-BDA8-EF853D2F936E@oracle.com> Message-ID: Hi Kim, Thank you for your comments. At least now there is only one point we disagree about: the inheritance check. I agree it would be nicer if a compiler error was generated if the derived class was not of expected type rather than reverting to the fallback class automatically. That could trivially be fixed with your permission. (they are already split now anyway - first it checks for existence then for inheritance, and generating an error then is trivial) I also agree that this could indeed be done without the inheritance check, but I still think it is better to perform that inheritance check as I think having a stronger contract is better for reasons already stated. So if you let me I could turn the inheritance check into a check that generates an error if an unexpected class is used rather than selecting the fallback class. However, if you really don't want the inheritance check and require a looser contract, I'm willing to give it up and come to your side so we get somewhere even though I disagree about it. Thanks, /Erik > On 6 jan 2015, at 21:27, Kim Barrett wrote: > >> On Jan 6, 2015, at 5:37 AM, Erik ?sterlund wrote: >> >> On 6 jan 2015, at 00:03, Kim Barrett wrote: >> It is true that in our case nothing would break if the 3 Atomic classes were not explicitly related by inheritance, but implicitly related with duck typing. >> >> However I want to give the user of SelectBaseClass a stronger contract that the base class can be used reliably and must behave consistently because I think that's better than loose implicit promises. > > Strengthening the selection requirements is a mistake for this general > technique. When the optionally defined platform-specific class is > defined, we want to (attempt to) use it. If there are additional > requirements on the class that it fails to meet, that should result in > a compilation error, not a selection failure that results in fallback > to the generic class. > >> I was never a big fan of duck typing. > > Sorry, you lost me there; duck typing is fundamental to template > usage. > >> The metafunctions have pretty good synergies. And just to make it clear, I know for a fact I will need IsBaseOf (and some of the other metafunctions) anyway to implement automatic closure specialization (e.g. to distinguish between ExtendedOopClosure and OopClosure). So if we do not introduce them now I will still have to introduce them later on. > > Perhaps IsBaseOf will be needed for that other feature. [I've not > looked at it recently or with sufficient care to have a well informed > opinion on that right now, though I'm a little suspicious; that sounds > like using class inheritance as a stand-in for a concept check.] If > IsBaseOf is needed for that (or for some other) feature, then it can > be added in conjunction with, or as a prelude to, that feature. > > I claim that the Atomic cmpxchg support doesn't need (and should not > use) IsBaseOf and the associated infrastructure, and so that > infrastructure shouldn't be piggy-backing into the code base under the > cover of the Atomic cmpxchg changes. > >>> In those cases where an inheritance relationship really *is* needed, I >>> think the right approach is to use conditional existence for selection >>> and then static assert the needed base/derived relationship(s). For >>> that we would need an IsBaseAndDerived or IsBaseOf, but their use >>> would be appropriately conditionalized on the existence of the >>> optional class (possibly only implicitly, e.g. if Atomic needed to be >>> derived from AtomicBase then we would assert that, and not care >>> whether it was direct derivation or via AtomicPlatform). >> >> I do not think it is better to leave correctness detection that could be done automatically to be done manually by the programmer for every use of the function. >> >> This seems to me equivalent to saying it is better to use duck typing and write unit tests to detect type errors to be less constrained by a type system. Some may argue this is true but I like my automatic type checks and stronger contracts. But then again such arguing could go on forever without reaching a conclusion. :) > > The inheritance-based selection criteria does not automatically check > for correctness. Instead, it masks errors by falling back to the > generic behavior. > >>> I strongly care about semantics and risk of error for something that >>> is billed as a general purpose utility. Risk of ODR violations is not >>> something I would treat lightly. And minor maintainance changes could >>> easily lead to such, if this proposed IsBaseOf were used as a general >>> utility. >> >> Point taken, and IsBaseOf is fixed now as you wanted it. > > Good. That's what we'll want, when we want IsBaseOf. > >> I will provide automatic closure specialization using this template infrastructure (given somebody is willing to run jprt for me hehe). > > I need to get back to that stuff. Stefan Karlsson and I recently > talked about some ideas in that area too. There certainly seems like > there should be room for improvement; I may not think macros are > inherently ugly, but I would be hard pressed to argue for the beauty > of the present oop iterator stuff. > >> I hope we are converging a bit now with opinions. [?] I think that leaves us to a final issue: Stronger contract enforcing inheritance in SelectBaseClass or weaker relying on duck typing [?] > > I'm not sure we are converging; we disagree on the selection criteria, > and that's quite fundamental. Waiting to see if my arguments above > convince you to come around to my position. > > From christian.thalinger at oracle.com Tue Jan 6 22:00:13 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Tue, 6 Jan 2015 14:00:13 -0800 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54ABA768.1030102@redhat.com> References: <54A6A933.8000204@redhat.com> <54ABA33D.4020907@redhat.com> <54ABA768.1030102@redhat.com> Message-ID: <019599FF-8274-4097-A1C6-237A2461CD64@oracle.com> > On Jan 6, 2015, at 1:14 AM, Andrew Dinn wrote: > > On 06/01/15 08:56, Andrew Haley wrote: >> On 05/01/15 23:51, Christian Thalinger wrote: >>> >>>> On Jan 2, 2015, at 6:20 AM, Andrew Haley wrote: >>>> >>>> http://cr.openjdk.java.net/~aph/aarch64-8068054-1/ >>> >>> Sorry if this was discussed somewhere else but is BUILTIN_SIM something that?s still used? >> >> Not in this version. I did consider stripping it all out, but decided >> on balance that it would be better to keep it. I could easily be >> persuaded to remove it. :-) > > Well, perhaps. I am not convinced that is a good idea just yet. > Debugging with the AArch64 simulator is still far easier than debugging > native code. Now, if we had proper symbol table support for C1/C2 > generated code that would be different . . . No strong opinion here. We can keep it. > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in UK and Wales under Company Registration No. 3798903 > Directors: Michael Cunningham (USA), Matt Parson (USA), Charlie Peters > (USA), Michael O'Neill (Ireland) From coleen.phillimore at oracle.com Tue Jan 6 22:34:58 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 06 Jan 2015 17:34:58 -0500 Subject: RFR (S) 8067836: The Universe::flush_foo methods belong in CodeCache. In-Reply-To: <7A189C0D-35DA-4FBB-9462-5458E4199506@oracle.com> References: <54AC4780.1000800@oracle.com> <7A189C0D-35DA-4FBB-9462-5458E4199506@oracle.com> Message-ID: <54AC6312.90105@oracle.com> Thanks, Kim! Coleen On 1/6/15, 4:04 PM, Kim Barrett wrote: > On Jan 6, 2015, at 3:37 PM, Coleen Phillimore wrote: >> Summary: Move this code to CodeCache. >> >> Tested by compiling and running runThese. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8067836/ >> bug link https://bugs.openjdk.java.net/browse/JDK-8067836 >> >> Thanks, >> Coleen > Looks good to me. > From vladimir.kozlov at oracle.com Tue Jan 6 22:39:06 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 06 Jan 2015 14:39:06 -0800 Subject: RFR (S) 8067836: The Universe::flush_foo methods belong in CodeCache. In-Reply-To: <54AC4780.1000800@oracle.com> References: <54AC4780.1000800@oracle.com> Message-ID: <54AC640A.4090406@oracle.com> Looks fine. Thanks, Vladimir On 1/6/15 12:37 PM, Coleen Phillimore wrote: > Summary: Move this code to CodeCache. > > Tested by compiling and running runThese. > > open webrev at http://cr.openjdk.java.net/~coleenp/8067836/ > bug link https://bugs.openjdk.java.net/browse/JDK-8067836 > > Thanks, > Coleen From kim.barrett at oracle.com Tue Jan 6 22:40:22 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 6 Jan 2015 17:40:22 -0500 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se>, <2ED21417-E8C1-4E11-AC2F-DED35A993714@oracle.com> <8445D264-6A94-4BF0-AF0D-611C9BC4303A@lnu.se>, <2F1C0741-3F39-42D9-BDA8-EF853D2F936E@oracle.com> Message-ID: <992FC1E3-AB9D-43CB-B588-5AD428AC67CD@oracle.com> On Jan 6, 2015, at 4:42 PM, Erik ?sterlund wrote: > > However, if you really don't want the inheritance check and require a looser contract, I'm willing to give it up and come to your side so we get somewhere even though I disagree about it. I would prefer to not have the inheritance check. [I would also prefer better than acquiesence, but it seems I haven't formed my arguments sufficiently well to fully convince you.] I'll watch for an updated webrev; hopefully we can bring this to a conclusion soon. From coleen.phillimore at oracle.com Tue Jan 6 22:40:47 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 06 Jan 2015 17:40:47 -0500 Subject: RFR (S) 8067836: The Universe::flush_foo methods belong in CodeCache. In-Reply-To: <54AC640A.4090406@oracle.com> References: <54AC4780.1000800@oracle.com> <54AC640A.4090406@oracle.com> Message-ID: <54AC646F.4030807@oracle.com> Thank you! Coleen On 1/6/15, 5:39 PM, Vladimir Kozlov wrote: > Looks fine. > > Thanks, > Vladimir > > On 1/6/15 12:37 PM, Coleen Phillimore wrote: >> Summary: Move this code to CodeCache. >> >> Tested by compiling and running runThese. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8067836/ >> bug link https://bugs.openjdk.java.net/browse/JDK-8067836 >> >> Thanks, >> Coleen From john.r.rose at oracle.com Tue Jan 6 22:57:28 2015 From: john.r.rose at oracle.com (John Rose) Date: Tue, 6 Jan 2015 14:57:28 -0800 Subject: RFR(L): 8064457: Introduce compressed oops mode "disjoint base" and improve compressed heap handling. In-Reply-To: <54AAC086.2060600@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF264E2@DEWDFEMB12A.global.corp.sap> <547E4F3B.2090501@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF35E66@DEWDFEMB12A.global.corp.sap> <547F4914.1030904@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF36139@DEWDFEMB12A.global.corp.sap> <54809196.7030306@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF39BA5@DEWDFEMB12A.global.corp.sap> <5480AB58.80604@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF3D6BB@DEWDFEMB12A.global.corp.sap> <54889D61.30905@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF3E1C5@DEWDFEMB12A.global.corp.sap> <54989B9F.2070903@oracle.com> <549B48A3.1070303@oracle.com> <54A2089D.1020105@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF53F70@DEWDFEMB12A.global.corp.sap> <54A2C966.5010306@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF67EB6@DEWDFEMB12A.global.corp.sap> <54A6C6C3.1050806@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF69210@DEWDFEMB12A.global.corp.sap> <54AAC086.2060600@oracle.com> Message-ID: <3CC08A26-61B4-4525-BD93-B0A2A75266A4@oracle.com> On Jan 5, 2015, at 8:49 AM, Coleen Phillimore wrote: > > So one thing I prototyped and really wanted to check in was what we called indirect class pointers. Instead of allocating classes in a fixed class space, we allocate a fixed array of Klass pointers and the objects in the heap point to this. It had a 8% performance degradation in specbb2008 compiler.compiler (iirc, or something like that). > > This solved so many problems though. If you investigated this sort of approach, I'd be very grateful. I haven't had time to go back to figure out why this had this degradation (we thought it was Java vtable accesses). I too would be grateful to see this approach explored more. If klasses can be indexed by a small compact range of values, we have new degrees of freedom for layout of object headers, and (with address masking on LP64) even object references. Moreover, if klasses are split in this way, we have the freedom to explore split classes ("species"), by allowing several splits to get their own near-klass blocks while sharing a semantic far-klass and/or java.lang.Class object. As I've mentioned before, I think a good next experiment would be to pick a small parameter N and hoist the first N vtable entries into the near-klass block (fixed sized array entry). More generally, it would be very interesting to survey (using some sort of injected counters) the access frequencies for the various fields of *Klass, and work towards segregating hot from cold in metadata. A systematic study of access frequency and density would be a worth a PhD or two. ? John From christian.tornqvist at oracle.com Tue Jan 6 23:40:04 2015 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Tue, 6 Jan 2015 18:40:04 -0500 Subject: RFR (XS): 8068540 - [TESTBUG] Exclude failing nightly tests In-Reply-To: <54AC4B16.7010201@oracle.com> References: <54AC4A50.70101@oracle.com> <54AC4B16.7010201@oracle.com> Message-ID: <019601d02a0a$191b8d60$4b52a820$@oracle.com> Hi Coleen and George, @Coleen: They fail because one of our internal platforms doesn't support the full feature set of NMT. @George: The change looks good. Thanks, Christian -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore Sent: Tuesday, January 6, 2015 3:53 PM To: hotspot-dev at openjdk.java.net Subject: Re: RFR (XS): 8068540 - [TESTBUG] Exclude failing nightly tests Why are these tests failing? The bug is not helpful. Thanks, Coleen On 1/6/15, 3:49 PM, George Triantafillou wrote: > Please review this small fix for JDK-8068540. The tests were modified > with the "@ignore" jtreg tag to exclude them from nightly testing. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8068540 > webrev: http://cr.openjdk.java.net/~gtriantafill/8068540/webrev.00/ > > > The fix was tested locally on Linux with jtreg and the JPRT hotspot > testset. > > -George From coleen.phillimore at oracle.com Wed Jan 7 00:00:13 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 06 Jan 2015 19:00:13 -0500 Subject: RFR (XS): 8068540 - [TESTBUG] Exclude failing nightly tests In-Reply-To: <019601d02a0a$191b8d60$4b52a820$@oracle.com> References: <54AC4A50.70101@oracle.com> <54AC4B16.7010201@oracle.com> <019601d02a0a$191b8d60$4b52a820$@oracle.com> Message-ID: <54AC770D.6070802@oracle.com> Ok, change looks good. Coleen On 1/6/15, 6:40 PM, Christian Tornqvist wrote: > Hi Coleen and George, > > @Coleen: They fail because one of our internal platforms doesn't support the full feature set of NMT. > > @George: The change looks good. > > Thanks, > Christian > > > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore > Sent: Tuesday, January 6, 2015 3:53 PM > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR (XS): 8068540 - [TESTBUG] Exclude failing nightly tests > > > Why are these tests failing? The bug is not helpful. > Thanks, > Coleen > > On 1/6/15, 3:49 PM, George Triantafillou wrote: >> Please review this small fix for JDK-8068540. The tests were modified >> with the "@ignore" jtreg tag to exclude them from nightly testing. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8068540 >> webrev: http://cr.openjdk.java.net/~gtriantafill/8068540/webrev.00/ >> >> >> The fix was tested locally on Linux with jtreg and the JPRT hotspot >> testset. >> >> -George > From david.holmes at oracle.com Wed Jan 7 01:18:03 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 07 Jan 2015 11:18:03 +1000 Subject: RFR (XS): 8068540 - [TESTBUG] Exclude failing nightly tests In-Reply-To: <019601d02a0a$191b8d60$4b52a820$@oracle.com> References: <54AC4A50.70101@oracle.com> <54AC4B16.7010201@oracle.com> <019601d02a0a$191b8d60$4b52a820$@oracle.com> Message-ID: <54AC894B.30903@oracle.com> On 7/01/2015 9:40 AM, Christian Tornqvist wrote: > Hi Coleen and George, > > @Coleen: They fail because one of our internal platforms doesn't support the full feature set of NMT. Excluding them completely reduces test coverage. We do we need to exclude them? They simply turn up as known failures (of which there are dozens). ?? David > @George: The change looks good. > > Thanks, > Christian > > > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore > Sent: Tuesday, January 6, 2015 3:53 PM > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR (XS): 8068540 - [TESTBUG] Exclude failing nightly tests > > > Why are these tests failing? The bug is not helpful. > Thanks, > Coleen > > On 1/6/15, 3:49 PM, George Triantafillou wrote: >> Please review this small fix for JDK-8068540. The tests were modified >> with the "@ignore" jtreg tag to exclude them from nightly testing. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8068540 >> webrev: http://cr.openjdk.java.net/~gtriantafill/8068540/webrev.00/ >> >> >> The fix was tested locally on Linux with jtreg and the JPRT hotspot >> testset. >> >> -George > > From david.holmes at oracle.com Wed Jan 7 01:47:20 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 07 Jan 2015 11:47:20 +1000 Subject: RFR(S) 8067331: Zero: Atomic::xchg and Atomic::xchg_ptr need full memory barrier In-Reply-To: <54A9F53C.5040507@oracle.com> References: <1418656152.3297.44.camel@localhost.localdomain> <54A9F53C.5040507@oracle.com> Message-ID: <54AC9028.1040709@oracle.com> Can we get a second hotspot review please - this is quite straight forward. Thanks, David On 5/01/2015 12:21 PM, David Holmes wrote: > Hi Severin, > > Sorry this slipped through the cracks before the holidays. > > On 16/12/2014 1:09 AM, Severin Gehwolf wrote: >> Hi, >> >> Could someone please review and sponsor the following change? >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8067331 >> webrev: >> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8067331/webrev.02/ >> >> The issue is that all atomic operations are expected to be full memory >> barriers, but two implementations of the Zero variant JVM - Atomic::xchg >> and Atomic::xchg_ptr - use GCC's built-in __sync_lock_test_and_set which >> is an acquire barrier only. The fix adds full memory barriers manually >> for those. >> >> Thoughts? > > Looks completely consistent with what I just saw in the aarch64 port code. > > I can sponsor this but we still need a second review. > > Thanks, > David > >> Cheers, >> Severin >> From david.holmes at oracle.com Wed Jan 7 02:06:11 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 07 Jan 2015 12:06:11 +1000 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <5C6292ED-BCB1-4FEA-96A6-6E9D0087E6C8@lnu.se> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> <, <2ED21417-E8C1-4E11-AC2F-DED35A993714@oracle.com> <>> <8445D264-6A94-4BF0-AF0D-611C9BC4303A@lnu.se> <5C6292ED-BCB1-4FEA-96A6-6E9D0087E6C8@lnu.se> Message-ID: <54AC9493.5040600@oracle.com> On 6/01/2015 10:47 PM, Erik ?sterlund wrote: > Hi Kim, > > [This is a second identical email from before, hoping nesting levels of comments do not get flattened this time.] Still flat. :( David > Thank you for your detailed feedback. > > On 6 jan 2015, at 00:03, Kim Barrett > wrote: > > Code surface area has an impact on understanding and maintenance. > It's not the only factor, but it *is* a factor. > > That said, I looked more closely and found the webrev new line count > substantially overstates the real size of the proposed change. (Nearly > half of the new lines are copyright headers and similar boiler plate. > A substantial fraction of the remainder are tests, which I wouldn't > count as a maintenance negative.) So rather than 800 lines, its > probably more like 100 new lines. That still seems to me to be a lot > in order to eliminate a few flag macro definitions and the > associated #ifdef, but seems much more plausible to amortize across > multiple uses. Since I think between us we've come up with several > more possible use-cases, I'm willing to withdraw the size complaint. > > :) > > [The difference between boost and C++11 is_base_of is minor, with > boost being more restrictive than necessary. I fully expect someone to > eventually file a bug report about that difference and for the boost > trait to be brought into conformance with the C++11 version. I think > that difference isn't relevant to this discussion.] > > It is not correct as a general utility trait because without the > requirement that "Derived" be complete one can very easily introduce > an ODR violation. That's the rationale for the C++11 requirement. And > it's better to check for that possibility, if possible, than to > silently risk an unchecked ODR violation that could be challenging to > track down. (C++11 doesn't require a diagnostic, leaving that as a QoI > issue. g++ and boost issue a diagnostic for this; I haven't tested > other compilers.) > > Point taken, and in my latest proposal including IsBaseOf, a compiler error is generated if Derived is incomplete to comply with your request. Instead I use your class IsClassDefined first to determine if it exists and then IsBaseOf to enforce a stronger contract. Thanks for that contribution. The result can be read in http://cr.openjdk.java.net/~jwilhelm/8067790/webrev.02/ > > In the proposed change and in most of the other use-cases I've thought > about, no class-based inheritance relationship between the optionally > defined class and the core "base" is required; duck type compatibility > is sufficient, and I think often preferable. > > In the specific proposed change, I'm pretty sure there is nothing that > requires any class-based inheritance relationship at all among Atomic, > AtomicBase, or any AtomicPlatform class. Atomic implements most of the > API, and delegates one piece to the selected "AtomicSuper" by explicit > name qualification. If there were no inheritance relationship at all > among those classes, nobody would notice (assuming the selection > process were changed to not require such a relationship). > > If AtomicBase and AtomicPlatform were changed as I suggested in one of > my earlier comments, to provide a protected cmpxchg_byte function with > an unqualified call from Atomic, then Atomic would need to be directly > derived from either AtomicBase or AtomicPlatform, but there would be > no need for any inheritance relationship between AtomicBase or > AtomicPlatform. The duck type compatibility of providing an > appropriate cmpxchg_byte function is sufficient. (With such an > approach I would also consider making the selected base for Atomic be > private, since the inheritance is for implementation convenience and > there's no external client that cares about that relationship.) > > It is true that in our case nothing would break if the 3 Atomic classes were not explicitly related by inheritance, but implicitly related with duck typing. > > However I want to give the user of SelectBaseClass a stronger contract that the base class can be used reliably and must behave consistently because I think that's better than loose implicit promises. I was never a big fan of duck typing. > > I think there are use-cases where appropriate inheritance placement of > the optionally defined class is important, but as discussed above, I > don't think this is one of those. Nor are most of the others that I've > thought of. I don't want to have to introduce unnecessary inheritance > in order to use the new infrastructure. > > With all the template infrastructure provided with my max reuse variant, it would be trivial (~5 rows) to make a variant of SelectBaseClass that does not constrain inheritance in case somebody would need it. > > The metafunctions have pretty good synergies. And just to make it clear, I know for a fact I will need IsBaseOf (and some of the other metafunctions) anyway to implement automatic closure specialization (e.g. to distinguish between ExtendedOopClosure and OopClosure). So if we do not introduce them now I will still have to introduce them later on. > > In those cases where an inheritance relationship really *is* needed, I > think the right approach is to use conditional existence for selection > and then static assert the needed base/derived relationship(s). For > that we would need an IsBaseAndDerived or IsBaseOf, but their use > would be appropriately conditionalized on the existence of the > optional class (possibly only implicitly, e.g. if Atomic needed to be > derived from AtomicBase then we would assert that, and not care > whether it was direct derivation or via AtomicPlatform). > > I do not think it is better to leave correctness detection that could be done automatically to be done manually by the programmer for every use of the function. > > This seems to me equivalent to saying it is better to use duck typing and write unit tests to detect type errors to be less constrained by a type system. Some may argue this is true but I like my automatic type checks and stronger contracts. But then again such arguing could go on forever without reaching a conclusion. :) > > I strongly care about semantics and risk of error for something that > is billed as a general purpose utility. Risk of ODR violations is not > something I would treat lightly. And minor maintainance changes could > easily lead to such, if this proposed IsBaseOf were used as a general > utility. > > Point taken, and IsBaseOf is fixed now as you wanted it. > > I think this badly mis-characterizes my position. I think the trait > semantics are critial, and I think the proposed traits have a serious > defect that render them inappropriate as general utilities. In > addition, I don't think they are what is needed for the more > specialized technique exemplared by the proposed change and similar > use-cases. Code size doesn't enter into these concerns at all. > > With the IsBaseOf completeness now being enforced, and code size being out of the window, the last concern is you may want a SelectBaseClass with weaker contracts not enforcing inheritance at some point in the future, when inheritance for some reason must be disjoint and only duck typing can be relied upon, yes? > > In that case I'm sure with the current template infrastructure, it will be easy to add those 5 lines needed to make such a metafunction then. :) > > The case where code size matters to me in this discussion is that > we're talking about adding some non-trivial infrastructure to > eliminate some simple macro usage. If we can't amortize that > infrastructure across some reasonable number of use-cases, then I > wouldn't want to add it. I think there probably are sufficient > use-cases to make some additional infrastructure worth-while, but I > think what was proposed is not the infrastructure we need. > > I know I am going to need IsBaseOf for automatic closure specialization to detect differences between ExtendedOopClosure and OopClosure. I don't like manually specializing closures when it could be done automatically. But for that I will need this template infrastructure, so I can guarantee they will come in handy and be used more. ;) > > I think this is only beneficial if doing so ultimately makes this code > follow a common pattern used elsewhere in our code base. (I say > "ultimately" because this would be the first occurrence of such a > pattern.) > > I will provide automatic closure specialization using this template infrastructure (given somebody is willing to run jprt for me hehe). > > I hope we are converging a bit now with opinions. With the IsBaseOf implementation fixed the way you want it, code size being toned down as an issue and reuse of template infrastructure promised (for automatic closure specialization), I think that leaves us to a final issue: Stronger contract enforcing inheritance in SelectBaseClass or weaker relying on duck typing. I would like to repeat that with the reusability of the "max reuse" proposal, such a relaxed duck typed conditional class selector metafunction could be implemented when/if needed later with ~5 LoC and I personally do not think it is much to argue about, especially considering IsBaseOf will certainly be needed soon anyway. > > Thanks, > /Erik > From coleen.phillimore at oracle.com Wed Jan 7 03:33:11 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 06 Jan 2015 22:33:11 -0500 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <8445D264-6A94-4BF0-AF0D-611C9BC4303A@lnu.se> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se>, <2ED21417-E8C1-4E11-AC2F-DED35A993714@oracle.com> <8445D264-6A94-4BF0-AF0D-611C9BC4303A@lnu.se> Message-ID: <54ACA8F7.8090503@oracle.com> src/os_cpu/linux_x86/vm/atomic_linux_x86.inline.hpp -inline void* Atomic::xchg_ptr(void* exchange_value, volatile void* dest) { +inline void* Atomic::xchg_ptr (void* exchange_value, volatile void* dest) { return (void*)xchg_ptr((intptr_t)exchange_value, (volatile intptr_t*)dest); } This formatting is weird in this file. Why is this like this? Are these os_cpu files essentially the same? *src/os_cpu/bsd_x86/vm/atomic_bsd_x86.hpp * Traditionally we have one declaration of the class AtomicPlatform in a common directory and provide the implementations in their platform dependent directories. There shouldn't be duplicate declarations. http://cr.openjdk.java.net/~jwilhelm/8067790/webrev.02/src/share/vm/utilities/traits/isBaseOf.cpp.html Is this an internal VM test? It should be #ifndef PRODUCT if so. Same for the other .cpp files. http://cr.openjdk.java.net/~jwilhelm/8067790/webrev.02/src/os_cpu/windows_x86/vm/atomic_windows_x86.inline.hpp.udiff.html The parameter spacing is messed up in this file too. Please check all the files to correct spacing. The existing code has some spacing issues also but new code should be correctly formatted. So now I see why there are duplicate AtomicPlatforms. The whole purpose is that some platforms have atomic::cmpxchg and some don't. Sorry I'm late to this thread. I have to admit that I'm biased by Kim's opinion and I know far less about these template tricks than he does. I don't want to review or read all this code in order to avoid some platform specific macros. Is there currently a bug that this code is fixing? Does this code actually add an optimized version of atomic::cmpxchg(byte)? Coleen On 1/6/15, 5:37 AM, Erik ?sterlund wrote: > Hi Kim, > > Thank you for your detailed feedback. > > On 6 jan 2015, at 00:03, Kim Barrett > wrote: > > On Jan 1, 2015, at 9:07 AM, Erik ?sterlund > wrote: > > Personally I don?t see how the number of rows of code define uglyness. Reusability, testing, documentation and well definedness in contracts/behaviours all lead to more lines of code, but I don?t think that equals uglyness. So while I disagree more code is uglier because it?s more, it?s all up to opinion and it?s quite pointless to discuss; you are entitled to think that is ugly. > > Code surface area has an impact on understanding and maintenance. > It's not the only factor, but it *is* a factor. > > That said, I looked more closely and found the webrev new line count > substantially overstates the real size of the proposed change. (Nearly > half of the new lines are copyright headers and similar boiler plate. > A substantial fraction of the remainder are tests, which I wouldn't > count as a maintenance negative.) So rather than 800 lines, its > probably more like 100 new lines. That still seems to me to be a lot > in order to eliminate a few flag macro definitions and the > associated #ifdef, but seems much more plausible to amortize across > multiple uses. Since I think between us we've come up with several > more possible use-cases, I'm willing to withdraw the size complaint. > > :) > > > While I disagree it is universally incorrect (as long as it is well defined and tested) to not have the exact same requirement on completeness on the parameters as C++11/boost, who also already have different behaviour in this exact regard as pointed out by yourself, I can try to fix it if it bugs you, but it will make the infrastructure even larger which seems to be a concern. > > [The difference between boost and C++11 is_base_of is minor, with > boost being more restrictive than necessary. I fully expect someone to > eventually file a bug report about that difference and for the boost > trait to be brought into conformance with the C++11 version. I think > that difference isn't relevant to this discussion.] > > It is not correct as a general utility trait because without the > requirement that "Derived" be complete one can very easily introduce > an ODR violation. That's the rationale for the C++11 requirement. And > it's better to check for that possibility, if possible, than to > silently risk an unchecked ODR violation that could be challenging to > track down. (C++11 doesn't require a diagnostic, leaving that as a QoI > issue. g++ and boost issue a diagnostic for this; I haven't tested > other compilers.) > > Point taken, and in my latest proposal including IsBaseOf, a compiler error is generated if Derived is incomplete to comply with your request. Instead I use your class IsClassDefined first to determine if it exists and then IsBaseOf to enforce a stronger contract. Thanks for that contribution. The result can be read in http://cr.openjdk.java.net/~jwilhelm/8067790/webrev.02/ > > > It could make sense to have a bespoke predicate for determining > whether a conditionally defined class is derived from some known base, > with a usage requirement that if not defined (e.g. complete) in the > current translation unit that it will not be defined anywhere else in > the program. However, I think such a predicate should have a more > specific / informative name and/or shouldn't be classified as a > general utility trait. > > But I think even renaming or moving it wouldn't be right, since I > disagree with the need for such a predicate at all. [More below.] > > The main idea of SelectBaseClass is to inject an optional class into the class hierarchy for specialization of generalized behaviour. It is absolutely critical (and intended) that this class is constrained to be a derived class of the fall-back class - it is not an artifact nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, A must be able to rely on the superclass being at least C-like so that the behaviour of the superclass can be used reliably and transparently regardless of the existence of the optional B. If this contract would not hold, I think it could lead to some really weird and unreliable code which should not be encouraged (at least not according to the Liskov substitution principle etc). > > In the proposed change and in most of the other use-cases I've thought > about, no class-based inheritance relationship between the optionally > defined class and the core "base" is required; duck type compatibility > is sufficient, and I think often preferable. > > In the specific proposed change, I'm pretty sure there is nothing that > requires any class-based inheritance relationship at all among Atomic, > AtomicBase, or any AtomicPlatform class. Atomic implements most of the > API, and delegates one piece to the selected "AtomicSuper" by explicit > name qualification. If there were no inheritance relationship at all > among those classes, nobody would notice (assuming the selection > process were changed to not require such a relationship). > > If AtomicBase and AtomicPlatform were changed as I suggested in one of > my earlier comments, to provide a protected cmpxchg_byte function with > an unqualified call from Atomic, then Atomic would need to be directly > derived from either AtomicBase or AtomicPlatform, but there would be > no need for any inheritance relationship between AtomicBase or > AtomicPlatform. The duck type compatibility of providing an > appropriate cmpxchg_byte function is sufficient. (With such an > approach I would also consider making the selected base for Atomic be > private, since the inheritance is for implementation convenience and > there's no external client that cares about that relationship.) > > It is true that in our case nothing would break if the 3 Atomic classes were not explicitly related by inheritance, but implicitly related with duck typing. > > However I want to give the user of SelectBaseClass a stronger contract that the base class can be used reliably and must behave consistently because I think that's better than loose implicit promises. I was never a big fan of duck typing. > > > So what I need in the contract is a check if the optional class is defined and if it fits in the inheritance hierarchy: they are both important IMO. > > I think there are use-cases where appropriate inheritance placement of > the optionally defined class is important, but as discussed above, I > don't think this is one of those. Nor are most of the others that I've > thought of. I don't want to have to introduce unnecessary inheritance > in order to use the new infrastructure. > > With all the template infrastructure provided with my max reuse variant, it would be trivial (~5 rows) to make a variant of SelectBaseClass that does not constrain inheritance in case somebody would need it. > > The metafunctions have pretty good synergies. And just to make it clear, I know for a fact I will need IsBaseOf (and some of the other metafunctions) anyway to implement automatic closure specialization (e.g. to distinguish between ExtendedOopClosure and OopClosure). So if we do not introduce them now I will still have to introduce them later on. > > > In those cases where an inheritance relationship really *is* needed, I > think the right approach is to use conditional existence for selection > and then static assert the needed base/derived relationship(s). For > that we would need an IsBaseAndDerived or IsBaseOf, but their use > would be appropriately conditionalized on the existence of the > optional class (possibly only implicitly, e.g. if Atomic needed to be > derived from AtomicBase then we would assert that, and not care > whether it was direct derivation or via AtomicPlatform). > > I do not think it is better to leave correctness detection that could be done automatically to be done manually by the programmer for every use of the function. > > This seems to me equivalent to saying it is better to use duck typing and write unit tests to detect type errors to be less constrained by a type system. Some may argue this is true but I like my automatic type checks and stronger contracts. But then again such arguing could go on forever without reaching a conclusion. :) > > > Yes and that makes me wonder if we care enough about the completeness semantics being the same as third party libraries (who are already inconsistent) if we do not need them to be, and on the contrary benefit from them not to be. > > I strongly care about semantics and risk of error for something that > is billed as a general purpose utility. Risk of ODR violations is not > something I would treat lightly. And minor maintainance changes could > easily lead to such, if this proposed IsBaseOf were used as a general > utility. > > Point taken, and IsBaseOf is fixed now as you wanted it. > > > In the end, we have to make a choice - what is more important, that the traits resemble some external library or code size. [?] > > I think this badly mis-characterizes my position. I think the trait > semantics are critial, and I think the proposed traits have a serious > defect that render them inappropriate as general utilities. In > addition, I don't think they are what is needed for the more > specialized technique exemplared by the proposed change and similar > use-cases. Code size doesn't enter into these concerns at all. > > With the IsBaseOf completeness now being enforced, and code size being out of the window, the last concern is you may want a SelectBaseClass with weaker contracts not enforcing inheritance at some point in the future, when inheritance for some reason must be disjoint and only duck typing can be relied upon, yes? > > In that case I'm sure with the current template infrastructure, it will be easy to add those 5 lines needed to make such a metafunction then. :) > > > The case where code size matters to me in this discussion is that > we're talking about adding some non-trivial infrastructure to > eliminate some simple macro usage. If we can't amortize that > infrastructure across some reasonable number of use-cases, then I > wouldn't want to add it. I think there probably are sufficient > use-cases to make some additional infrastructure worth-while, but I > think what was proposed is not the infrastructure we need. > > I know I am going to need IsBaseOf for automatic closure specialization to detect differences between ExtendedOopClosure and OopClosure. I don't like manually specializing closures when it could be done automatically. But for that I will need this template infrastructure, so I can guarantee they will come in handy and be used more. ;) > > > 1. Do we want to replace the macros with template metaprogramming? 2 reviewers (and me) liked the metaprogramming approach and it was implemented because they (like me) did not want the macros. But you have a different opinion. I can?t judge who is right and wrong here. > > I think this is only beneficial if doing so ultimately makes this code > follow a common pattern used elsewhere in our code base. (I say > "ultimately" because this would be the first occurrence of such a > pattern.) > > I will provide automatic closure specialization using this template infrastructure (given somebody is willing to run jprt for me hehe). > > I hope we are converging a bit now with opinions. With the IsBaseOf implementation fixed the way you want it, code size being toned down as an issue and reuse of template infrastructure promised (for automatic closure specialization), I think that leaves us to a final issue: Stronger contract enforcing inheritance in SelectBaseClass or weaker relying on duck typing. I would like to repeat that with the reusability of the "max reuse" proposal, such a relaxed duck typed conditional class selector metafunction could be implemented when/if needed later with ~5 LoC and I personally do not think it is much to argue about, especially considering IsBaseOf will certainly be needed soon anyway. > > Thanks, > /Erik From david.holmes at oracle.com Wed Jan 7 04:17:34 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 07 Jan 2015 14:17:34 +1000 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> Message-ID: <54ACB35E.8020800@oracle.com> It wasn't clear from my earlier misguided comment but consider this Reviewed. Though now the copyright dates need updating to 2015 :) Thanks, David On 31/12/2014 10:47 AM, Kim Barrett wrote: > Please review this change to rename the assert macro to vmassert, as a > step toward eliminating the name clash between the VM's assert() macro > and the standard ( / ) assert() macro. > > [Coleen has volunteered sponsorship, since she's been prodding me to > make this change for months. Changes are based on jdk9/hs-rt repo.] > > CR: > https://bugs.openjdk.java.net/browse/JDK-8068396 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8068396/webrev > > Testing: > jprt. Also manually verified asserts occur as expected in fastdebug > build and not in release build. > > Details of the changes are: > > * Renamed the assert macro to vmassert. Also renamed the assert_status > macro to vmassert_status, for name consistency. > > * Added temporary synonym macros of assert => vmassert and > assert_status => vmassert_status, for backward compatibility. This > change does not modify the (18K+) occurrences of the old macro names > to use the new names. The synonyms allow that renaming to occur > incrementally, with removal of the synonyms once renaming is complete. > > * Simplified the definition of vmassert_status to just use vmassert, > rather than duplicating much of the implementation of vmassert. While > I was at it, added back a comma in the message that was (probably > accidentally) removed by 6888954: argument formatting for assert() and > friends (hg:1410 4/22/2010). > > * Moved the assert_if_no_error macro to utilities/xmlstream.cpp, which > is the only place it is used. Also simplified the definition to just > use vmassert, rather than duplicating much of the implementation of > vmassert. > > * Changed shark/llvmHeaders.hpp to just define assert as a synonym of > vmassert, rather than duplicating the definition of [vm]assert. [This > is an ugly hack, but still an improvement on what was there. This is a > place that could quickly benefit from updating uses.] > > * Removed the non-product AssertRepeat option and the associated > support by the [vm]assert macro. Support for this option has been > broken since 6888954: argument formatting for assert() and friends > (hg:1410 4/22/2010), due to a missing line continuation in the > corresponding definition of the assert macro. Since the feature > doesn't look very interesting, has been broken for nearly 5 years, and > can easily be resurrected if needed, I chose to simplify by removal. > From coleen.phillimore at oracle.com Wed Jan 7 04:27:16 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 06 Jan 2015 23:27:16 -0500 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <54AA143B.2080900@oracle.com> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> <54AA143B.2080900@oracle.com> Message-ID: <54ACB5A4.50603@oracle.com> Thanks David, I'm working through this thread in reverse. Thank you for the context. This helps a lot. I am not familiar with template meta-programming either. A long time ago, I was a proponent of using templates but mostly as a type-checked way of avoiding duplicate code. In this case of template meta-programming, I don't think the intellectual load of the idioms offset the ugliness of the code that they are replacing. Learning these idioms is not trivial. They are really hard to read. Maybe if template classes had proper names rather than X, Y, and Z it would be better. In any case, using this to replace code isn't a clear win in clean code. There is still the duplicate declarations of AtomicPlatform. There's also the unfortunate conditional inclusions in shared code: +// Linux +#ifdef TARGET_OS_ARCH_linux_x86 +# include "atomic_linux_x86.hpp" +#endif + +// Solaris +#ifdef TARGET_OS_ARCH_solaris_x86 +# include "atomic_solaris_x86.hpp" +#endif + +// Windows +#ifdef TARGET_OS_ARCH_windows_x86 +# include "atomic_windows_x86.hpp" +#endif + +// BSD +#ifdef TARGET_OS_ARCH_bsd_x86 +# include "atomic_bsd_x86.hpp" +#endif I don't think this aids maintainability or cleanliness of the code for this use case. I don't think this should be added to the code base at this time. Thanks, Coleen On 1/4/15, 11:34 PM, David Holmes wrote: > Sorry for the top-post but just wanted to reset the context a little > here. Originally [1] I made the comment: > > "It's fine to have generic shared approaches but there really needs to > be a way to allow platform specific "overrides"." > > For the actual original RFR I said [2]: > > "Can we pause and give some more thought to a clean mechanism for > allowing a shared implementation if desired with the ability to > override if desired. I really do not like to see CPU specific ifdefs > being added to shared code. (And I would also not like to see all > platforms being forced to reimplement this natively). > > I'm not saying we will find a simple solution, but it would be nice if > we could get a few folk to think about it before proceeding with the > ifdefs :)" > > Erik then proposed three alternative approaches [3] and the simple > variant was chosen [4] and presented for review. However Paul Hohensee > also made comments about an inheritance-based approach and Erik > floated the first template-based variant [5] and there was some > discussion with Paul and Kim. But we then ended up with the simple > solution, leaving an inheritance-based solution for future work [6] > (which is what is what is being discussed now). > > This template-based meta-programming is not something I have any > familiarity with. My initial thought is this seems overkill for the > situation we have with the Atomic class - but as I said I'm ignorant > of the technique being used here. > > David > ----- > > [1] > http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015292.html > [2] > http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015303.html > [3] > http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015362.html > [4] > http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015401.html > [5] > http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-October/015586.html > [6] > http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-November/015889.html > > On 2/01/2015 12:07 AM, Erik ?sterlund wrote: >> Hi Kim, >> >>> On 01 Jan 2015, at 07:42, Kim Barrett wrote: >>> >>> On Dec 31, 2014, at 6:12 AM, Erik ?sterlund >>> wrote: >>>> >>>>> The rest of these comments only matter if, contrary to my suggestion >>>>> above, some additional infrastructure is thought necessary. >>>>> >>>> >>>> This is what I originally thought too and hence the current >>>> solution. But voices said that using macros for this is, I quote, >>>> "that bad". >>>> >>>>> [?] >>>>> >>>>> This scales with the number of AtomicPlatform definitions, rather >>>>> than >>>>> the number of specialized functions as happens with the existing >>>>> code. >>>> >>>> So your proposal is to still use ugly macros but move it to the >>>> class level instead, even though at the moment there is really only >>>> one member function that needs it. That feels a bit meh to be >>>> honest. IMO if we want to do this right, why go half the way and >>>> still rely on ugly macros? >>> >>> Please lay off with the perjorative "ugly macros". The preprocessor >>> is part of C++, and isn't going away. It's a tool that can be used, >>> or abused, and I think this is a fine use. To me, "ugly" is adding >>> 800+ lines of code / tests / comments to eliminate a dozen lines of >>> flag macro definitions and associated #ifdef?s. >> >> Well people are entitled to have different opinions of course. As I >> already mentioned, I did this because other reviewers (as well as me) >> found it ugly to use macros for conditionally specializing, which is >> the whole motivation of doing this, and was well understood from the >> beginning. But I?m fine with leaving it as macros if this is in >> general preferred and opinions have suddenly changed in this matter. >> >> Personally I don?t see how the number of rows of code define >> uglyness. Reusability, testing, documentation and well definedness in >> contracts/behaviours all lead to more lines of code, but I don?t >> think that equals uglyness. So while I disagree more code is uglier >> because it?s more, it?s all up to opinion and it?s quite pointless to >> discuss; you are entitled to think that is ugly. >> >>> If that additional infrastructure were amortized across more use-cases >>> then it might be more appealing. I might even have some additional >>> use-cases for something along that line, though not in its present >>> form. (More on that below.) >> >> Yeah I specifically had in mind the wish to make a split between >> commercial extensions to open source code actually. This pattern >> allows easy optional class inheritance injection as a general design >> pattern to achieve this. And for my own purposes the ability to >> define asymmetric dekker synchronization to elide storeload fences >> when the platform (e.g. TSO) supports it but otherwise revert to a >> generalized implementation (using storeload fences). This is why I >> think a clean reusable strategy is almost always better because it is >> widely applicable, even if there would not be obvious other examples >> where it is useful. >> >>> >>>>> I think the name "IsBaseOfAndDerived" would be better as >>>>> "IsBaseAndDerived?. [?] >>>>> >>>>> The current implementation of IsBaseOfAndDerived (and therefore >>>>> IsBaseOf) may (silently) provide incorrect answers when called with >>>>> incomplete types. [?] >>>>> >>>>> The actual usage in this proposed change-set even relies on this >>>>> flawed behavior of these metafunctions, since AtomicPlatform isn't >>>>> always defined. [?] >>>>> >>>> >>>> So what you are saying here is that I should rename >>>> IsBaseOfAndDerived to IsBaseAndDerived to more closely resemble >>>> other external libraries we do not use. But if I do that my >>>> behaviour is "incorrect" because it is not identical to that of the >>>> external library. And therefore my change would not work if it was >>>> made "correctly?. >>> >>> The proposed IsBaseOfAndDerived and IsBaseOf are being described and >>> placed as general utilities. Their names are strongly reminiscent of >>> existing facilities in other libraries, including the (C++11) standard >>> library. Conforming to these external standards will avoid surprises, >>> especially when the proposed facilities are even described as being >>> the same (except for a noted corner case that is both hard to deal >>> with in a portable manner and not actually very interesting, so I'm >>> not complaining about that) in the review request email. >>> >>> Additionally, I think quietly providing a potentially incorrect answer >>> in the case of incomplete types renders the offered code quite broken >>> for use as a general utility. I think the requirement for complete >>> types by the well-known / standard facilities is well justified. >>> >> >> While I disagree it is universally incorrect (as long as it is well >> defined and tested) to not have the exact same requirement on >> completeness on the parameters as C++11/boost, who also already have >> different behaviour in this exact regard as pointed out by yourself, >> I can try to fix it if it bugs you, but it will make the >> infrastructure even larger which seems to be a concern. >> >>>> On the contrary this is a well tested feature on all our supported >>>> compilers and I did not intend to engineer it to be identical to >>>> some external library if it only causes trouble rather than >>>> helping. This is our own class and it can do what we want it to do >>>> and be called what we want it to be called, and I think that is >>>> fine as long as it is well tested, which it is. At least this is my >>>> opinion. Anyone else agree? >>> >>> Problem is, I think it's well tested code to solve the wrong problem. >>> >>> I suggest that what we're really looking for is not a base/derived >>> relationship between a pair of classes, but that we actually want to >>> determine whether a platform-specific class exists. >>> >>> As a result, I think there is more infrastructure in this change set >>> than is actually needed. The present implementation of >>> SelectBaseClass uses IsBaseOf, but doesn't need a lot of the >>> functionality of that metafunction. There is no actual need for a >>> Base/Derived relationship between the (possibly not defined) >>> platform-specific class and the default platform-generic class, so the >>> use of IsBaseOf is an unnecessary restriction that only >>> serendipitously tests for defined or not, due to the previously >>> discussed defect in this particular implementation. I also think the >>> name SelectBaseClass is perhaps overly general. >> >> The main idea of SelectBaseClass is to inject an optional class into >> the class hierarchy for specialization of generalized behaviour. It >> is absolutely critical (and intended) that this class is constrained >> to be a derived class of the fall-back class - it is not an artifact >> nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, >> A must be able to rely on the superclass being at least C-like so >> that the behaviour of the superclass can be used reliably and >> transparently regardless of the existence of the optional B. If this >> contract would not hold, I think it could lead to some really weird >> and unreliable code which should not be encouraged (at least not >> according to the Liskov substitution principle etc). >> >> So what I need in the contract is a check if the optional class is >> defined and if it fits in the inheritance hierarchy: they are both >> important IMO. >> >> Yes it could have been less code. From the beginning I had a single >> trait class that checked for both the inheritance and whether the >> class is defined or not but you said you wanted the behaviour to more >> closely match that of third party libraries. This is why there is now >> a lot more code to more closely (but still not exactly) match that >> behaviour. >> >>> I might be able to use a similar approach in some code I've been >>> working on as a background task. And while writing this reply I've >>> thought of another place that might benefit from something along those >>> lines. Thinking about the code under review in that context, I think >>> some changes would make these other possible use-cases easier and >>> clearer. >>> >>> I believe what is actually needed is a metatfunction something like: >>> >>> /** >>> * Metafunction whose nested type member is Default if Specific is >>> * incomplete, otherwise Specific. >>> * >>> * Specific and Default must both be non-cv qualified class types, and >>> * must not be the same type. >>> * >>> * If Specific is incomplete at the point where this metafunction is >>> * invoked, it must never be defined elsewhere in the program. >>> */ >>> template >>> struct SelectPlatformBase; >>> >>> [Note: I'm not wedded to that name.] >>> >>> Now, it turns out to be hard / impossible to write a generic >>> is_complete metafunction, due to ODR. Whether a type is complete >>> can vary between translation units, and even within a translation >>> unit. Having is_complete::value have different values depending on >>> where it is instantiated is an ODR violation. (One can make progress >>> by using anonymous namespaces and macro wrappers with __LINE__ >>> concatenation, but there is still the fundamental question of whether >>> is_complete really even makes semantic sense.) >> >> Yes and that makes me wonder if we care enough about the completeness >> semantics being the same as third party libraries (who are already >> inconsistent) if we do not need them to be, and on the contrary >> benefit from them not to be. >> >>> However, we don't need a fully general is_complete utility. We have a >>> much more restricted use-case, where we want to select an optionally >>> defined platform-specific class if it exists, and otherwise fall back >>> to a more generic class. And we're not using this in arbitrary >>> contexts, but only to select a platform-specialized implementation if >>> such exists. >>> >>> Here's a lightly tested implementation of SelectPlatformBase: >>> >>> // --- SelectPlatformBase >>> >>> template >>> struct EnableIf { typedef T type; }; >>> >>> template >>> struct EnableIf { }; >>> >>> template >>> struct If { typedef Then type; }; >>> >>> template >>> struct If { typedef Else type; }; >>> >>> // Metafunction whose nested value member is true if T is defined >>> // (complete), and false if T is incomplete. T must be a non-cv >>> // qualified class type. If T is incomplete at the point where this >>> // metafunction is invoked, it must never be defined elsewhere in the >>> // program. >>> template >>> class IsClassDefined { >>> typedef char yes[1]; >>> typedef char no[2]; >>> >>> template >>> static typename EnableIf::type & >>> check(U*); >>> static no& check(...); >>> >>> public: >>> static const bool value = sizeof(check((T*)0)) == sizeof(yes); >>> }; >>> >>> template >>> struct SelectPlatformBase { >>> typedef typename If< >>> IsClassDefined::value, Specific, Default>::type type; >>> }; >>> >>> // --- end SelectPlatformBase >>> >>> That's ~30 lines of code (+ tests and comments TBD) to do precisely >>> what we need, replacing ~100 lines (+ tests and comments) that have >>> various issues. (If and EnableIf should probably be hoisted out as >>> separate utilities.) We don't actually need SelectPlatformBase, and >>> could instead just directly use the If and IsClassDefined >>> metafunctions, but giving this functionality a name might be clearer. >> >> As stated, I think it?s dangerous to allow conditional inheritance >> where there is no inheritance constraint on the optional class. >> Therefore I do not think this is better, although I agree it is smaller. >> >> I agree that our use case here is smaller. If the amount of code is >> the problem and less code (and hence fewer reusable components) is >> preferred, then the original implementation with a simple >> IsBaseOfAndDerived does the trick (without checking types/cv >> qualifiers) and could be done in just a few lines, and this is what I >> originally did before you told me to expand it to more closely >> resemble external libraries: >> >> template >> class IsBaseOfAndDerived { >> typedef char yes[1]; >> typedef char no[2]; >> >> template >> static yes &check(Derived*, T); >> static no &check(Base*, int); >> >> template >> struct IsBaseOfHost { >> operator B*() const; >> operator D*(); >> }; >> public: >> static const bool value = sizeof(check(IsBaseOfHost> Derived>(), int())) == sizeof(yes); >> }; >> >> It checks the inheritance and existence of the Derived class which is >> exactly the constraints I need. >> >> If you do not want to expose it publicly and remove the reusability >> because it does not have identical semantics as third party libraries >> regarding the requirements of the Derived type to be complete, it >> could be an internal class of SelectBaseClass. >> >> I?m not going to value the reusability vs LoC, I leave that decision >> to you. >> >>> However, while I think there are other use cases for this specific >>> utility, I still think the proposed change set as a whole is adding a >>> lot of code just to avoid a a few lines of macro usage. That seems >>> like a poor tradeoff to me. >> >> >> Okay. >> >> In the end, we have to make a choice - what is more important, that >> the traits resemble some external library or code size. Because last >> time I proposed a small change and you wanted it to be closer to >> C++11 behaviour and I made it closer to that at the expense of more >> code. Now you are saying I have to make it even tighter to C++11 (or >> boost?), but also don?t want more lines of code which I believe is >> impossible given that I want to constrain both the class hierarchy to >> be reliable and check the existence of the optional class. This >> brings me to a few architectural questions which I may have (strong) >> opinions about but are not up to me to decide. >> >> 1. Do we want to replace the macros with template metaprogramming? 2 >> reviewers (and me) liked the metaprogramming approach and it was >> implemented because they (like me) did not want the macros. But you >> have a different opinion. I can?t judge who is right and wrong here. >> >> 2. Do we want to minimize the code size at the cost of reusability by >> making contracts of dependent components weaker and putting them as >> private classes in the SelectBaseClass? >> >> 3. If we do want template metaprogramming and want reusable >> components, the question is, is it important that the specification >> of completeness (which in general is almost impossible to implement >> correctly in a portable way as you pointed out) of the template >> arguments in IsBaseOf is identical as C++11/boost even though no code >> needs it to be and on the contrary some code benefits from it not to >> be? In that case which one of them do we, because they are already >> different anyway? There is a tradeoff between code size and >> complexity vs copying either one of the external libraries regarding >> requirements of completeness of template parameters. >> >> Thanks, >> /Erik >> From coleen.phillimore at oracle.com Wed Jan 7 04:54:08 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 06 Jan 2015 23:54:08 -0500 Subject: RFR(S) 8067331: Zero: Atomic::xchg and Atomic::xchg_ptr need full memory barrier In-Reply-To: <54AC9028.1040709@oracle.com> References: <1418656152.3297.44.camel@localhost.localdomain> <54A9F53C.5040507@oracle.com> <54AC9028.1040709@oracle.com> Message-ID: <54ACBBF0.3040803@oracle.com> This looks good. Sorry for the delay reviewing it as it looks important. Is there a small test case that can be written to test this? (or no?) Coleen On 1/6/15, 8:47 PM, David Holmes wrote: > Can we get a second hotspot review please - this is quite straight > forward. > > Thanks, > David > > On 5/01/2015 12:21 PM, David Holmes wrote: >> Hi Severin, >> >> Sorry this slipped through the cracks before the holidays. >> >> On 16/12/2014 1:09 AM, Severin Gehwolf wrote: >>> Hi, >>> >>> Could someone please review and sponsor the following change? >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8067331 >>> webrev: >>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8067331/webrev.02/ >>> >>> The issue is that all atomic operations are expected to be full memory >>> barriers, but two implementations of the Zero variant JVM - >>> Atomic::xchg >>> and Atomic::xchg_ptr - use GCC's built-in __sync_lock_test_and_set >>> which >>> is an acquire barrier only. The fix adds full memory barriers manually >>> for those. >>> >>> Thoughts? >> >> Looks completely consistent with what I just saw in the aarch64 port >> code. >> >> I can sponsor this but we still need a second review. >> >> Thanks, >> David >> >>> Cheers, >>> Severin >>> From david.holmes at oracle.com Wed Jan 7 04:58:40 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 07 Jan 2015 14:58:40 +1000 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <54ACB5A4.50603@oracle.com> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> <54AA143B.2080900@oracle.com> <54ACB5A4.50603@oracle.com> Message-ID: <54ACBD00.6030501@oracle.com> I'm inclined to agree with Coleen - while I originally lamented the use of platform specific ifdefs and wondered about using simple inheritance, this has gone off in a direction I never envisaged. I don't think the problem being attacked is sufficiently bad to warrant the complexity** of the solution. And it still adds a bunch of platform-specific ifdefs. :( Sorry. David ----- ** that is subjective of course - if you don't understand template meta-programming then of course it seems complex. I don't know if it remains complex if you do understand template meta-programming ? On 7/01/2015 2:27 PM, Coleen Phillimore wrote: > > Thanks David, I'm working through this thread in reverse. Thank you > for the context. This helps a lot. > > I am not familiar with template meta-programming either. A long time > ago, I was a proponent of using templates but mostly as a type-checked > way of avoiding duplicate code. > > In this case of template meta-programming, I don't think the > intellectual load of the idioms offset the ugliness of the code that > they are replacing. Learning these idioms is not trivial. They are > really hard to read. Maybe if template classes had proper names rather > than X, Y, and Z it would be better. In any case, using this to > replace code isn't a clear win in clean code. There is still the > duplicate declarations of AtomicPlatform. There's also the unfortunate > conditional inclusions in shared code: > > +// Linux > +#ifdef TARGET_OS_ARCH_linux_x86 > +# include "atomic_linux_x86.hpp" > +#endif > + > +// Solaris > +#ifdef TARGET_OS_ARCH_solaris_x86 > +# include "atomic_solaris_x86.hpp" > +#endif > + > +// Windows > +#ifdef TARGET_OS_ARCH_windows_x86 > +# include "atomic_windows_x86.hpp" > +#endif > + > +// BSD > +#ifdef TARGET_OS_ARCH_bsd_x86 > +# include "atomic_bsd_x86.hpp" > +#endif > > > I don't think this aids maintainability or cleanliness of the code for > this use case. I don't think this should be added to the code base at > this time. > > Thanks, > Coleen > > On 1/4/15, 11:34 PM, David Holmes wrote: >> Sorry for the top-post but just wanted to reset the context a little >> here. Originally [1] I made the comment: >> >> "It's fine to have generic shared approaches but there really needs to >> be a way to allow platform specific "overrides"." >> >> For the actual original RFR I said [2]: >> >> "Can we pause and give some more thought to a clean mechanism for >> allowing a shared implementation if desired with the ability to >> override if desired. I really do not like to see CPU specific ifdefs >> being added to shared code. (And I would also not like to see all >> platforms being forced to reimplement this natively). >> >> I'm not saying we will find a simple solution, but it would be nice if >> we could get a few folk to think about it before proceeding with the >> ifdefs :)" >> >> Erik then proposed three alternative approaches [3] and the simple >> variant was chosen [4] and presented for review. However Paul Hohensee >> also made comments about an inheritance-based approach and Erik >> floated the first template-based variant [5] and there was some >> discussion with Paul and Kim. But we then ended up with the simple >> solution, leaving an inheritance-based solution for future work [6] >> (which is what is what is being discussed now). >> >> This template-based meta-programming is not something I have any >> familiarity with. My initial thought is this seems overkill for the >> situation we have with the Atomic class - but as I said I'm ignorant >> of the technique being used here. >> >> David >> ----- >> >> [1] >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015292.html >> >> [2] >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015303.html >> >> [3] >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015362.html >> >> [4] >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015401.html >> >> [5] >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-October/015586.html >> >> [6] >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-November/015889.html >> >> >> On 2/01/2015 12:07 AM, Erik ?sterlund wrote: >>> Hi Kim, >>> >>>> On 01 Jan 2015, at 07:42, Kim Barrett wrote: >>>> >>>> On Dec 31, 2014, at 6:12 AM, Erik ?sterlund >>>> wrote: >>>>> >>>>>> The rest of these comments only matter if, contrary to my suggestion >>>>>> above, some additional infrastructure is thought necessary. >>>>>> >>>>> >>>>> This is what I originally thought too and hence the current >>>>> solution. But voices said that using macros for this is, I quote, >>>>> "that bad". >>>>> >>>>>> [?] >>>>>> >>>>>> This scales with the number of AtomicPlatform definitions, rather >>>>>> than >>>>>> the number of specialized functions as happens with the existing >>>>>> code. >>>>> >>>>> So your proposal is to still use ugly macros but move it to the >>>>> class level instead, even though at the moment there is really only >>>>> one member function that needs it. That feels a bit meh to be >>>>> honest. IMO if we want to do this right, why go half the way and >>>>> still rely on ugly macros? >>>> >>>> Please lay off with the perjorative "ugly macros". The preprocessor >>>> is part of C++, and isn't going away. It's a tool that can be used, >>>> or abused, and I think this is a fine use. To me, "ugly" is adding >>>> 800+ lines of code / tests / comments to eliminate a dozen lines of >>>> flag macro definitions and associated #ifdef?s. >>> >>> Well people are entitled to have different opinions of course. As I >>> already mentioned, I did this because other reviewers (as well as me) >>> found it ugly to use macros for conditionally specializing, which is >>> the whole motivation of doing this, and was well understood from the >>> beginning. But I?m fine with leaving it as macros if this is in >>> general preferred and opinions have suddenly changed in this matter. >>> >>> Personally I don?t see how the number of rows of code define >>> uglyness. Reusability, testing, documentation and well definedness in >>> contracts/behaviours all lead to more lines of code, but I don?t >>> think that equals uglyness. So while I disagree more code is uglier >>> because it?s more, it?s all up to opinion and it?s quite pointless to >>> discuss; you are entitled to think that is ugly. >>> >>>> If that additional infrastructure were amortized across more use-cases >>>> then it might be more appealing. I might even have some additional >>>> use-cases for something along that line, though not in its present >>>> form. (More on that below.) >>> >>> Yeah I specifically had in mind the wish to make a split between >>> commercial extensions to open source code actually. This pattern >>> allows easy optional class inheritance injection as a general design >>> pattern to achieve this. And for my own purposes the ability to >>> define asymmetric dekker synchronization to elide storeload fences >>> when the platform (e.g. TSO) supports it but otherwise revert to a >>> generalized implementation (using storeload fences). This is why I >>> think a clean reusable strategy is almost always better because it is >>> widely applicable, even if there would not be obvious other examples >>> where it is useful. >>> >>>> >>>>>> I think the name "IsBaseOfAndDerived" would be better as >>>>>> "IsBaseAndDerived?. [?] >>>>>> >>>>>> The current implementation of IsBaseOfAndDerived (and therefore >>>>>> IsBaseOf) may (silently) provide incorrect answers when called with >>>>>> incomplete types. [?] >>>>>> >>>>>> The actual usage in this proposed change-set even relies on this >>>>>> flawed behavior of these metafunctions, since AtomicPlatform isn't >>>>>> always defined. [?] >>>>>> >>>>> >>>>> So what you are saying here is that I should rename >>>>> IsBaseOfAndDerived to IsBaseAndDerived to more closely resemble >>>>> other external libraries we do not use. But if I do that my >>>>> behaviour is "incorrect" because it is not identical to that of the >>>>> external library. And therefore my change would not work if it was >>>>> made "correctly?. >>>> >>>> The proposed IsBaseOfAndDerived and IsBaseOf are being described and >>>> placed as general utilities. Their names are strongly reminiscent of >>>> existing facilities in other libraries, including the (C++11) standard >>>> library. Conforming to these external standards will avoid surprises, >>>> especially when the proposed facilities are even described as being >>>> the same (except for a noted corner case that is both hard to deal >>>> with in a portable manner and not actually very interesting, so I'm >>>> not complaining about that) in the review request email. >>>> >>>> Additionally, I think quietly providing a potentially incorrect answer >>>> in the case of incomplete types renders the offered code quite broken >>>> for use as a general utility. I think the requirement for complete >>>> types by the well-known / standard facilities is well justified. >>>> >>> >>> While I disagree it is universally incorrect (as long as it is well >>> defined and tested) to not have the exact same requirement on >>> completeness on the parameters as C++11/boost, who also already have >>> different behaviour in this exact regard as pointed out by yourself, >>> I can try to fix it if it bugs you, but it will make the >>> infrastructure even larger which seems to be a concern. >>> >>>>> On the contrary this is a well tested feature on all our supported >>>>> compilers and I did not intend to engineer it to be identical to >>>>> some external library if it only causes trouble rather than >>>>> helping. This is our own class and it can do what we want it to do >>>>> and be called what we want it to be called, and I think that is >>>>> fine as long as it is well tested, which it is. At least this is my >>>>> opinion. Anyone else agree? >>>> >>>> Problem is, I think it's well tested code to solve the wrong problem. >>>> >>>> I suggest that what we're really looking for is not a base/derived >>>> relationship between a pair of classes, but that we actually want to >>>> determine whether a platform-specific class exists. >>>> >>>> As a result, I think there is more infrastructure in this change set >>>> than is actually needed. The present implementation of >>>> SelectBaseClass uses IsBaseOf, but doesn't need a lot of the >>>> functionality of that metafunction. There is no actual need for a >>>> Base/Derived relationship between the (possibly not defined) >>>> platform-specific class and the default platform-generic class, so the >>>> use of IsBaseOf is an unnecessary restriction that only >>>> serendipitously tests for defined or not, due to the previously >>>> discussed defect in this particular implementation. I also think the >>>> name SelectBaseClass is perhaps overly general. >>> >>> The main idea of SelectBaseClass is to inject an optional class into >>> the class hierarchy for specialization of generalized behaviour. It >>> is absolutely critical (and intended) that this class is constrained >>> to be a derived class of the fall-back class - it is not an artifact >>> nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, >>> A must be able to rely on the superclass being at least C-like so >>> that the behaviour of the superclass can be used reliably and >>> transparently regardless of the existence of the optional B. If this >>> contract would not hold, I think it could lead to some really weird >>> and unreliable code which should not be encouraged (at least not >>> according to the Liskov substitution principle etc). >>> >>> So what I need in the contract is a check if the optional class is >>> defined and if it fits in the inheritance hierarchy: they are both >>> important IMO. >>> >>> Yes it could have been less code. From the beginning I had a single >>> trait class that checked for both the inheritance and whether the >>> class is defined or not but you said you wanted the behaviour to more >>> closely match that of third party libraries. This is why there is now >>> a lot more code to more closely (but still not exactly) match that >>> behaviour. >>> >>>> I might be able to use a similar approach in some code I've been >>>> working on as a background task. And while writing this reply I've >>>> thought of another place that might benefit from something along those >>>> lines. Thinking about the code under review in that context, I think >>>> some changes would make these other possible use-cases easier and >>>> clearer. >>>> >>>> I believe what is actually needed is a metatfunction something like: >>>> >>>> /** >>>> * Metafunction whose nested type member is Default if Specific is >>>> * incomplete, otherwise Specific. >>>> * >>>> * Specific and Default must both be non-cv qualified class types, and >>>> * must not be the same type. >>>> * >>>> * If Specific is incomplete at the point where this metafunction is >>>> * invoked, it must never be defined elsewhere in the program. >>>> */ >>>> template >>>> struct SelectPlatformBase; >>>> >>>> [Note: I'm not wedded to that name.] >>>> >>>> Now, it turns out to be hard / impossible to write a generic >>>> is_complete metafunction, due to ODR. Whether a type is complete >>>> can vary between translation units, and even within a translation >>>> unit. Having is_complete::value have different values depending on >>>> where it is instantiated is an ODR violation. (One can make progress >>>> by using anonymous namespaces and macro wrappers with __LINE__ >>>> concatenation, but there is still the fundamental question of whether >>>> is_complete really even makes semantic sense.) >>> >>> Yes and that makes me wonder if we care enough about the completeness >>> semantics being the same as third party libraries (who are already >>> inconsistent) if we do not need them to be, and on the contrary >>> benefit from them not to be. >>> >>>> However, we don't need a fully general is_complete utility. We have a >>>> much more restricted use-case, where we want to select an optionally >>>> defined platform-specific class if it exists, and otherwise fall back >>>> to a more generic class. And we're not using this in arbitrary >>>> contexts, but only to select a platform-specialized implementation if >>>> such exists. >>>> >>>> Here's a lightly tested implementation of SelectPlatformBase: >>>> >>>> // --- SelectPlatformBase >>>> >>>> template >>>> struct EnableIf { typedef T type; }; >>>> >>>> template >>>> struct EnableIf { }; >>>> >>>> template >>>> struct If { typedef Then type; }; >>>> >>>> template >>>> struct If { typedef Else type; }; >>>> >>>> // Metafunction whose nested value member is true if T is defined >>>> // (complete), and false if T is incomplete. T must be a non-cv >>>> // qualified class type. If T is incomplete at the point where this >>>> // metafunction is invoked, it must never be defined elsewhere in the >>>> // program. >>>> template >>>> class IsClassDefined { >>>> typedef char yes[1]; >>>> typedef char no[2]; >>>> >>>> template >>>> static typename EnableIf::type & >>>> check(U*); >>>> static no& check(...); >>>> >>>> public: >>>> static const bool value = sizeof(check((T*)0)) == sizeof(yes); >>>> }; >>>> >>>> template >>>> struct SelectPlatformBase { >>>> typedef typename If< >>>> IsClassDefined::value, Specific, Default>::type type; >>>> }; >>>> >>>> // --- end SelectPlatformBase >>>> >>>> That's ~30 lines of code (+ tests and comments TBD) to do precisely >>>> what we need, replacing ~100 lines (+ tests and comments) that have >>>> various issues. (If and EnableIf should probably be hoisted out as >>>> separate utilities.) We don't actually need SelectPlatformBase, and >>>> could instead just directly use the If and IsClassDefined >>>> metafunctions, but giving this functionality a name might be clearer. >>> >>> As stated, I think it?s dangerous to allow conditional inheritance >>> where there is no inheritance constraint on the optional class. >>> Therefore I do not think this is better, although I agree it is smaller. >>> >>> I agree that our use case here is smaller. If the amount of code is >>> the problem and less code (and hence fewer reusable components) is >>> preferred, then the original implementation with a simple >>> IsBaseOfAndDerived does the trick (without checking types/cv >>> qualifiers) and could be done in just a few lines, and this is what I >>> originally did before you told me to expand it to more closely >>> resemble external libraries: >>> >>> template >>> class IsBaseOfAndDerived { >>> typedef char yes[1]; >>> typedef char no[2]; >>> >>> template >>> static yes &check(Derived*, T); >>> static no &check(Base*, int); >>> >>> template >>> struct IsBaseOfHost { >>> operator B*() const; >>> operator D*(); >>> }; >>> public: >>> static const bool value = sizeof(check(IsBaseOfHost>> Derived>(), int())) == sizeof(yes); >>> }; >>> >>> It checks the inheritance and existence of the Derived class which is >>> exactly the constraints I need. >>> >>> If you do not want to expose it publicly and remove the reusability >>> because it does not have identical semantics as third party libraries >>> regarding the requirements of the Derived type to be complete, it >>> could be an internal class of SelectBaseClass. >>> >>> I?m not going to value the reusability vs LoC, I leave that decision >>> to you. >>> >>>> However, while I think there are other use cases for this specific >>>> utility, I still think the proposed change set as a whole is adding a >>>> lot of code just to avoid a a few lines of macro usage. That seems >>>> like a poor tradeoff to me. >>> >>> >>> Okay. >>> >>> In the end, we have to make a choice - what is more important, that >>> the traits resemble some external library or code size. Because last >>> time I proposed a small change and you wanted it to be closer to >>> C++11 behaviour and I made it closer to that at the expense of more >>> code. Now you are saying I have to make it even tighter to C++11 (or >>> boost?), but also don?t want more lines of code which I believe is >>> impossible given that I want to constrain both the class hierarchy to >>> be reliable and check the existence of the optional class. This >>> brings me to a few architectural questions which I may have (strong) >>> opinions about but are not up to me to decide. >>> >>> 1. Do we want to replace the macros with template metaprogramming? 2 >>> reviewers (and me) liked the metaprogramming approach and it was >>> implemented because they (like me) did not want the macros. But you >>> have a different opinion. I can?t judge who is right and wrong here. >>> >>> 2. Do we want to minimize the code size at the cost of reusability by >>> making contracts of dependent components weaker and putting them as >>> private classes in the SelectBaseClass? >>> >>> 3. If we do want template metaprogramming and want reusable >>> components, the question is, is it important that the specification >>> of completeness (which in general is almost impossible to implement >>> correctly in a portable way as you pointed out) of the template >>> arguments in IsBaseOf is identical as C++11/boost even though no code >>> needs it to be and on the contrary some code benefits from it not to >>> be? In that case which one of them do we, because they are already >>> different anyway? There is a tradeoff between code size and >>> complexity vs copying either one of the external libraries regarding >>> requirements of completeness of template parameters. >>> >>> Thanks, >>> /Erik >>> > From david.holmes at oracle.com Wed Jan 7 04:59:56 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 07 Jan 2015 14:59:56 +1000 Subject: RFR(S) 8067331: Zero: Atomic::xchg and Atomic::xchg_ptr need full memory barrier In-Reply-To: <54ACBBF0.3040803@oracle.com> References: <1418656152.3297.44.camel@localhost.localdomain> <54A9F53C.5040507@oracle.com> <54AC9028.1040709@oracle.com> <54ACBBF0.3040803@oracle.com> Message-ID: <54ACBD4C.5010307@oracle.com> On 7/01/2015 2:54 PM, Coleen Phillimore wrote: > > This looks good. Sorry for the delay reviewing it as it looks > important. Is there a small test case that can be written to test > this? (or no?) No - I'll add noreg-hard. Testing for a missing memory-barrier is not practical. Thanks, David > Coleen > > On 1/6/15, 8:47 PM, David Holmes wrote: >> Can we get a second hotspot review please - this is quite straight >> forward. >> >> Thanks, >> David >> >> On 5/01/2015 12:21 PM, David Holmes wrote: >>> Hi Severin, >>> >>> Sorry this slipped through the cracks before the holidays. >>> >>> On 16/12/2014 1:09 AM, Severin Gehwolf wrote: >>>> Hi, >>>> >>>> Could someone please review and sponsor the following change? >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8067331 >>>> webrev: >>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8067331/webrev.02/ >>>> >>>> The issue is that all atomic operations are expected to be full memory >>>> barriers, but two implementations of the Zero variant JVM - >>>> Atomic::xchg >>>> and Atomic::xchg_ptr - use GCC's built-in __sync_lock_test_and_set >>>> which >>>> is an acquire barrier only. The fix adds full memory barriers manually >>>> for those. >>>> >>>> Thoughts? >>> >>> Looks completely consistent with what I just saw in the aarch64 port >>> code. >>> >>> I can sponsor this but we still need a second review. >>> >>> Thanks, >>> David >>> >>>> Cheers, >>>> Severin >>>> > From david.holmes at oracle.com Wed Jan 7 05:07:04 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 07 Jan 2015 15:07:04 +1000 Subject: RFR(S) 8067331: Zero: Atomic::xchg and Atomic::xchg_ptr need full memory barrier In-Reply-To: <54ACBD4C.5010307@oracle.com> References: <1418656152.3297.44.camel@localhost.localdomain> <54A9F53C.5040507@oracle.com> <54AC9028.1040709@oracle.com> <54ACBBF0.3040803@oracle.com> <54ACBD4C.5010307@oracle.com> Message-ID: <54ACBEF8.6050703@oracle.com> Severin, Can you create a changeset with myself (dholmes) and Coleen (coleenp) as Reviewers, and with the file copyrights suitably updated. I'm not sure what form that will take given the dual copyright attributions but presumably the Red Hat line will need 2015 added. If you email me the changeset directly I will apply it to the jdk9/hs-rt/hotspot/ forest. Thanks, David On 7/01/2015 2:59 PM, David Holmes wrote: > On 7/01/2015 2:54 PM, Coleen Phillimore wrote: >> >> This looks good. Sorry for the delay reviewing it as it looks >> important. Is there a small test case that can be written to test >> this? (or no?) > > No - I'll add noreg-hard. Testing for a missing memory-barrier is not > practical. > > Thanks, > David > >> Coleen >> >> On 1/6/15, 8:47 PM, David Holmes wrote: >>> Can we get a second hotspot review please - this is quite straight >>> forward. >>> >>> Thanks, >>> David >>> >>> On 5/01/2015 12:21 PM, David Holmes wrote: >>>> Hi Severin, >>>> >>>> Sorry this slipped through the cracks before the holidays. >>>> >>>> On 16/12/2014 1:09 AM, Severin Gehwolf wrote: >>>>> Hi, >>>>> >>>>> Could someone please review and sponsor the following change? >>>>> >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8067331 >>>>> webrev: >>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8067331/webrev.02/ >>>>> >>>>> The issue is that all atomic operations are expected to be full memory >>>>> barriers, but two implementations of the Zero variant JVM - >>>>> Atomic::xchg >>>>> and Atomic::xchg_ptr - use GCC's built-in __sync_lock_test_and_set >>>>> which >>>>> is an acquire barrier only. The fix adds full memory barriers manually >>>>> for those. >>>>> >>>>> Thoughts? >>>> >>>> Looks completely consistent with what I just saw in the aarch64 port >>>> code. >>>> >>>> I can sponsor this but we still need a second review. >>>> >>>> Thanks, >>>> David >>>> >>>>> Cheers, >>>>> Severin >>>>> >> From kim.barrett at oracle.com Wed Jan 7 08:02:07 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 7 Jan 2015 03:02:07 -0500 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: <54ACB35E.8020800@oracle.com> References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> <54ACB35E.8020800@oracle.com> Message-ID: <61A2FAE2-E8EE-410C-9501-56B817F0D71B@oracle.com> On Jan 6, 2015, at 11:17 PM, David Holmes wrote: > > It wasn't clear from my earlier misguided comment but consider this Reviewed. > > Though now the copyright dates need updating to 2015 :) Thanks for your review. I?ll fix the copyright dates. From goetz.lindenmaier at sap.com Wed Jan 7 08:26:56 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 7 Jan 2015 08:26:56 +0000 Subject: RFR(M): 8068503: ppc64: Encode/Decode nodes for disjoint cOops mode Message-ID: <4295855A5C1DE049A61835A1887419CC2CF69B22@DEWDFEMB12A.global.corp.sap> Hi, this change contains the implementation (activation) of the encode/decode nodes exploiting the disjoint heap base available since "8064457: Introduce compressed oops mode disjoint base..." Please review this change. http://cr.openjdk.java.net/~goetz/webrevs/8068503-ppcDisjoint/ This is ppc64-only, so I don't need a sponsor ;) Best regards, Goetz. From jesper.wilhelmsson at oracle.com Wed Jan 7 08:57:41 2015 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 07 Jan 2015 09:57:41 +0100 Subject: RFR (XS): 8068540 - [TESTBUG] Exclude failing nightly tests In-Reply-To: <54AC894B.30903@oracle.com> References: <54AC4A50.70101@oracle.com> <54AC4B16.7010201@oracle.com> <019601d02a0a$191b8d60$4b52a820$@oracle.com> <54AC894B.30903@oracle.com> Message-ID: <54ACF505.3070506@oracle.com> We are trying to get rid of all known failures so unless the test can be modified to run only on the platforms that support NMT (which is my preferred option), I think it is better to exclude the tests than to let them fail on some platforms on a regular basis. /Jesper David Holmes skrev den 7/1/15 02:18: > On 7/01/2015 9:40 AM, Christian Tornqvist wrote: >> Hi Coleen and George, >> >> @Coleen: They fail because one of our internal platforms doesn't support the >> full feature set of NMT. > > Excluding them completely reduces test coverage. We do we need to exclude them? > They simply turn up as known failures (of which there are dozens). > > ?? > > David > > >> @George: The change looks good. >> >> Thanks, >> Christian >> >> >> >> -----Original Message----- >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of >> Coleen Phillimore >> Sent: Tuesday, January 6, 2015 3:53 PM >> To: hotspot-dev at openjdk.java.net >> Subject: Re: RFR (XS): 8068540 - [TESTBUG] Exclude failing nightly tests >> >> >> Why are these tests failing? The bug is not helpful. >> Thanks, >> Coleen >> >> On 1/6/15, 3:49 PM, George Triantafillou wrote: >>> Please review this small fix for JDK-8068540. The tests were modified >>> with the "@ignore" jtreg tag to exclude them from nightly testing. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8068540 >>> webrev: http://cr.openjdk.java.net/~gtriantafill/8068540/webrev.00/ >>> >>> >>> The fix was tested locally on Linux with jtreg and the JPRT hotspot >>> testset. >>> >>> -George >> >> From goetz.lindenmaier at sap.com Wed Jan 7 10:14:18 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 7 Jan 2015 10:14:18 +0000 Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Message-ID: <4295855A5C1DE049A61835A1887419CC2CF69B49@DEWDFEMB12A.global.corp.sap> Hi, A row of hotspot jtreg tests fail because support to test for Aix is missing. To pass them, I would like to submit the following fixes: * runtime/6888954/vmerrors.sh: AIX can read from address '0', then throws SIGILL. * serviceability/dcmd/DynLibDcmdTest.java: Just add check for Aix. * test/test_env.sh: Add support for AIX. Sort OSes alphabetical. * test/testlibrary/com/oracle/java/testlibrary/Platform.java: add isAix(). Sort OSes alphabetical. Please review this change, and I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev-01/ Thanks and best regards, Goetz. From fredrik.arvidsson at oracle.com Wed Jan 7 10:17:36 2015 From: fredrik.arvidsson at oracle.com (Fredrik Arvidsson) Date: Wed, 07 Jan 2015 11:17:36 +0100 Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF69B49@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF69B49@DEWDFEMB12A.global.corp.sap> Message-ID: <54AD07C0.5030100@oracle.com> Hi The changes to the DynLibDcmdTest look good to me. I am not a reviewer though, but I wrote that test some time ago. /Fredrik On 2015-01-07 11:14, Lindenmaier, Goetz wrote: > Hi, > > A row of hotspot jtreg tests fail because support to test for Aix is missing. > To pass them, I would like to submit the following fixes: > > * runtime/6888954/vmerrors.sh: AIX can read from address '0', then throws SIGILL. > * serviceability/dcmd/DynLibDcmdTest.java: Just add check for Aix. > * test/test_env.sh: Add support for AIX. Sort OSes alphabetical. > * test/testlibrary/com/oracle/java/testlibrary/Platform.java: add isAix(). Sort OSes alphabetical. > > Please review this change, and I please need a sponsor. > http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev-01/ > > Thanks and best regards, > Goetz. > From goetz.lindenmaier at sap.com Wed Jan 7 10:52:17 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 7 Jan 2015 10:52:17 +0000 Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests In-Reply-To: <54AD07C0.5030100@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF69B49@DEWDFEMB12A.global.corp.sap> <54AD07C0.5030100@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF69B65@DEWDFEMB12A.global.corp.sap> Thanks Fredrik! Best regards, Goetz. -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Fredrik Arvidsson Sent: Mittwoch, 7. Januar 2015 11:18 To: hotspot-dev at openjdk.java.net Subject: Re: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi The changes to the DynLibDcmdTest look good to me. I am not a reviewer though, but I wrote that test some time ago. /Fredrik On 2015-01-07 11:14, Lindenmaier, Goetz wrote: > Hi, > > A row of hotspot jtreg tests fail because support to test for Aix is missing. > To pass them, I would like to submit the following fixes: > > * runtime/6888954/vmerrors.sh: AIX can read from address '0', then throws SIGILL. > * serviceability/dcmd/DynLibDcmdTest.java: Just add check for Aix. > * test/test_env.sh: Add support for AIX. Sort OSes alphabetical. > * test/testlibrary/com/oracle/java/testlibrary/Platform.java: add isAix(). Sort OSes alphabetical. > > Please review this change, and I please need a sponsor. > http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev-01/ > > Thanks and best regards, > Goetz. > From goetz.lindenmaier at sap.com Wed Jan 7 11:02:36 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 7 Jan 2015 11:02:36 +0000 Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Message-ID: <4295855A5C1DE049A61835A1887419CC2CF69B7E@DEWDFEMB12A.global.corp.sap> Hi, my first upload missed the fix in Platform.shouldSAAttach(): + if (isAix()) { + return false; // SA not implemented. I added that to the webrev. Sorry, Goetz From: Lindenmaier, Goetz Sent: Mittwoch, 7. Januar 2015 11:14 To: 'hotspot-dev at openjdk.java.net' Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi, A row of hotspot jtreg tests fail because support to test for Aix is missing. To pass them, I would like to submit the following fixes: * runtime/6888954/vmerrors.sh: AIX can read from address '0', then throws SIGILL. * serviceability/dcmd/DynLibDcmdTest.java: Just add check for Aix. * test/test_env.sh: Add support for AIX. Sort OSes alphabetical. * test/testlibrary/com/oracle/java/testlibrary/Platform.java: add isAix(). Sort OSes alphabetical. Please review this change, and I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev-01/ Thanks and best regards, Goetz. From filipp.zhinkin at gmail.com Wed Jan 7 13:09:36 2015 From: filipp.zhinkin at gmail.com (Filipp ) Date: Wed, 7 Jan 2015 16:09:36 +0300 Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF69B7E@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF69B7E@DEWDFEMB12A.global.corp.sap> Message-ID: <54ad3027.499b700a.481d.1821@mx.google.com> Hi Goetz, you have to update TestMutuallyExclusivePlatformPredicates test [1], because you've added mutually exclusive predicate. Otherwise that test will fail. Also, I'd prefer to see switch instead of if-else chain in DynLibDcmdTest, but I'm fine with leaving it as it is now. Otherwise change looks good. Regards, Filipp. [1] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/acfdd92cedaa/test/testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java#l48 -----???????? ?????????----- ??: "Lindenmaier, Goetz" ??????????: ?07.?01.?2015 14:03 ????: "'hotspot-dev at openjdk.java.net'" ????: RE: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi, my first upload missed the fix in Platform.shouldSAAttach(): + if (isAix()) { + return false; // SA not implemented. I added that to the webrev. Sorry, Goetz From: Lindenmaier, Goetz Sent: Mittwoch, 7. Januar 2015 11:14 To: 'hotspot-dev at openjdk.java.net' Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi, A row of hotspot jtreg tests fail because support to test for Aix is missing. To pass them, I would like to submit the following fixes: * runtime/6888954/vmerrors.sh: AIX can read from address '0', then throws SIGILL. * serviceability/dcmd/DynLibDcmdTest.java: Just add check for Aix. * test/test_env.sh: Add support for AIX. Sort OSes alphabetical. * test/testlibrary/com/oracle/java/testlibrary/Platform.java: add isAix(). Sort OSes alphabetical. Please review this change, and I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev-01/ Thanks and best regards, Goetz. From roland.westrelin at oracle.com Wed Jan 7 13:58:01 2015 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Wed, 7 Jan 2015 14:58:01 +0100 Subject: 8068055: AARCH64: os_cpu/ In-Reply-To: <54A6A977.4080405@redhat.com> References: <54A6A977.4080405@redhat.com> Message-ID: <3ADEB4FB-87F6-42B9-98DD-B519750A3C99@oracle.com> > http://cr.openjdk.java.net/~aph/aarch64-8068055-1/ That looks good to me. Roland. From tobias.hartmann at oracle.com Wed Jan 7 14:02:30 2015 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Wed, 07 Jan 2015 15:02:30 +0100 Subject: [8u60] Backport RFR: 8066763: fatal error "assert(false) failed: unexpected yanked node" in postaloc.cpp:139 Message-ID: <54AD3C76.40601@oracle.com> Hi, please review the following backport request for 8u60. 8066763: fatal error "assert(false) failed: unexpected yanked node" in postaloc.cpp:139 https://bugs.openjdk.java.net/browse/JDK-8066763 http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/f1cb27c85c83 The changes were pushed to JDK 9 on Dec 12 and nightly testing showed no problems. The patch applies cleanly to 8u60. Thanks, Tobias From goetz.lindenmaier at sap.com Wed Jan 7 14:49:05 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 7 Jan 2015 14:49:05 +0000 Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests In-Reply-To: <54ad3027.499b700a.481d.1821@mx.google.com> References: <4295855A5C1DE049A61835A1887419CC2CF69B7E@DEWDFEMB12A.global.corp.sap> <54ad3027.499b700a.481d.1821@mx.google.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF6ACF5@DEWDFEMB12A.global.corp.sap> Hi Filipp I fixed TestMutuallyExclusivePlatformPredicates.java and uploaded a new webrev: http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev-02/ Thank you for pointing me to that. Somehow it?s not run in our nightly tests. I also fixed the copyrights. Best regards, Goetz. From: Filipp [mailto:filipp.zhinkin at gmail.com] Sent: Mittwoch, 7. Januar 2015 14:10 To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' Subject: RE: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi Goetz, you have to update TestMutuallyExclusivePlatformPredicates test [1], because you've added mutually exclusive predicate. Otherwise that test will fail. Also, I'd prefer to see switch instead of if-else chain in DynLibDcmdTest, but I'm fine with leaving it as it is now. Otherwise change looks good. Regards, Filipp. [1] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/acfdd92cedaa/test/testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java#l48 ________________________________ ??: Lindenmaier, Goetz ??????????: ?07.?01.?2015 14:03 ????: 'hotspot-dev at openjdk.java.net' ????: RE: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi, my first upload missed the fix in Platform.shouldSAAttach(): + if (isAix()) { + return false; // SA not implemented. I added that to the webrev. Sorry, Goetz From: Lindenmaier, Goetz Sent: Mittwoch, 7. Januar 2015 11:14 To: 'hotspot-dev at openjdk.java.net' Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi, A row of hotspot jtreg tests fail because support to test for Aix is missing. To pass them, I would like to submit the following fixes: * runtime/6888954/vmerrors.sh: AIX can read from address '0', then throws SIGILL. * serviceability/dcmd/DynLibDcmdTest.java: Just add check for Aix. * test/test_env.sh: Add support for AIX. Sort OSes alphabetical. * test/testlibrary/com/oracle/java/testlibrary/Platform.java: add isAix(). Sort OSes alphabetical. Please review this change, and I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev-01/ Thanks and best regards, Goetz. From coleen.phillimore at oracle.com Wed Jan 7 14:53:57 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 07 Jan 2015 09:53:57 -0500 Subject: RFR(M): 8068503: ppc64: Encode/Decode nodes for disjoint cOops mode In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF69B22@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF69B22@DEWDFEMB12A.global.corp.sap> Message-ID: <54AD4885.9080700@oracle.com> Don't you have to make (or want to make) changes to MacroAssembler::decode_heap_oop_not_null in the macro assembler files? Coleen On 1/7/15, 3:26 AM, Lindenmaier, Goetz wrote: > Hi, > > this change contains the implementation (activation) of the > encode/decode nodes exploiting the disjoint heap base > available since "8064457: Introduce compressed oops mode disjoint base..." > > Please review this change. > http://cr.openjdk.java.net/~goetz/webrevs/8068503-ppcDisjoint/ > > This is ppc64-only, so I don't need a sponsor ;) > > Best regards, > Goetz. > From roland.westrelin at oracle.com Wed Jan 7 14:54:18 2015 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Wed, 7 Jan 2015 15:54:18 +0100 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54A6A933.8000204@redhat.com> References: <54A6A933.8000204@redhat.com> Message-ID: <7C4F2EEA-04D9-4421-8231-23E0A47A6502@oracle.com> > http://cr.openjdk.java.net/~aph/aarch64-8068054-1/ That looks good to me. A few minor things I spotted: assembler_aarch64.cpp Typo: 45 // and decode functiosn I guess you want that removed: 50 // #include "gc_interface/collectedHeap.inline.hpp" 51 // #include "interpreter/interpreter.hpp" 52 // #include "memory/cardTableModRefBS.hpp" 53 // #include "prims/methodHandles.hpp" 54 // #include "runtime/biasedLocking.hpp" 55 // #include "runtime/interfaceSupport.hpp" 56 // #include "runtime/objectMonitor.hpp" 57 // #include "runtime/os.hpp" 58 // #include "runtime/sharedRuntime.hpp" 59 // #include "runtime/stubRoutines.hpp" 60 // #if INCLUDE_ALL_GCS 61 // #include "gc_implementation/g1/g1CollectedHeap.inline.hpp" 62 // #include "gc_implementation/g1/g1SATBCardTableModRefBS.hpp" 63 // #include "gc_implementation/g1/heapRegion.hpp" 64 // #endif Should not be indented: 1408 // An "all-purpose" add/subtract immediate, per ARM documentation: 1409 // A "programmer-friendly" assembler may accept a negative immediate 1410 // between -(2^24 -1) and -1 inclusive, causing it to convert a 1411 // requested ADD operation to a SUB, or vice versa, and then encode 1412 // the absolute value of the immediate as for uimm24. bytecodeInterpreter_aarch64.cpp This can?t be needed, right? 45 #ifdef TARGET_ARCH_MODEL_x86_32 46 # include "interp_masm_x86_32.hpp" 47 #endif 48 #ifdef TARGET_ARCH_MODEL_x86_64 49 # include "interp_masm_x86_64.hpp" 50 #endif stubRoutines_aarch64.cpp How come you need crc_table? templateTable_aarch64.cpp 50 // No amd64 specific initialization Roland. From aph at redhat.com Wed Jan 7 15:02:48 2015 From: aph at redhat.com (Andrew Haley) Date: Wed, 07 Jan 2015 15:02:48 +0000 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <7C4F2EEA-04D9-4421-8231-23E0A47A6502@oracle.com> References: <54A6A933.8000204@redhat.com> <7C4F2EEA-04D9-4421-8231-23E0A47A6502@oracle.com> Message-ID: <54AD4A98.9080501@redhat.com> On 01/07/2015 02:54 PM, Roland Westrelin wrote: > How come you need crc_table? CRC instructions are optional, so are not provided by some implementations. We have provided an implementation of CRC32 which uses SIMD instructions. ACK to the rest. Thanks, Andrew. From erik.osterlund at lnu.se Wed Jan 7 15:58:17 2015 From: erik.osterlund at lnu.se (=?Windows-1252?Q?Erik_=D6sterlund?=) Date: Wed, 7 Jan 2015 15:58:17 +0000 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <54ACBD00.6030501@oracle.com> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> <54AA143B.2080900@oracle.com> <54ACB5A4.50603@oracle.com> <54ACBD00.6030501@oracle.com> Message-ID: Hi Coleen and David, I was under the impression that this change was wanted to remove the error prone conditional #define for supported specialized/generalized atomics at almost any cost. I was fine with the #define myself but implemented this conditional class injection since voices were raised by reviewers to find a better way, and the actual optimization (using #define) was only conditionally accepted, given that I would provide the template inheritance solution to get rid of the #define after. Personally I don?t think the change is too complicated, especially not from the user?s point of view as you don?t really need to understand the internals to use the traits. For this specific change, only one trait is really used by atomics, the rest are dependencies designed to be reusable ?bonuses? if anyone (like myself) needs it later on (for instance for automatic closure specialization). And the trait/metafunction actually used by Atomic is SelectBaseClass which simply selects among two classes the most specific class which is defined (a general, for the AtomicBase class and optionally AtomicPlatform if implemented, otherwise ignored and AtomicBase is used as fallback). But of course, if this fix is perceived as complicated and that this complexity is not motivated, then I understand and we can just close the issue and leave it the way it is I guess, if reviewers are now fine with that. Thanks, /Erik > On 07 Jan 2015, at 05:58, David Holmes wrote: > > I'm inclined to agree with Coleen - while I originally lamented the use of platform specific ifdefs and wondered about using simple inheritance, this has gone off in a direction I never envisaged. I don't think the problem being attacked is sufficiently bad to warrant the complexity** of the solution. And it still adds a bunch of platform-specific ifdefs. :( > > Sorry. > > David > ----- > > ** that is subjective of course - if you don't understand template meta-programming then of course it seems complex. I don't know if it remains complex if you do understand template meta-programming ? > > On 7/01/2015 2:27 PM, Coleen Phillimore wrote: >> >> Thanks David, I'm working through this thread in reverse. Thank you >> for the context. This helps a lot. >> >> I am not familiar with template meta-programming either. A long time >> ago, I was a proponent of using templates but mostly as a type-checked >> way of avoiding duplicate code. >> >> In this case of template meta-programming, I don't think the >> intellectual load of the idioms offset the ugliness of the code that >> they are replacing. Learning these idioms is not trivial. They are >> really hard to read. Maybe if template classes had proper names rather >> than X, Y, and Z it would be better. In any case, using this to >> replace code isn't a clear win in clean code. There is still the >> duplicate declarations of AtomicPlatform. There's also the unfortunate >> conditional inclusions in shared code: >> >> +// Linux >> +#ifdef TARGET_OS_ARCH_linux_x86 >> +# include "atomic_linux_x86.hpp" >> +#endif >> + >> +// Solaris >> +#ifdef TARGET_OS_ARCH_solaris_x86 >> +# include "atomic_solaris_x86.hpp" >> +#endif >> + >> +// Windows >> +#ifdef TARGET_OS_ARCH_windows_x86 >> +# include "atomic_windows_x86.hpp" >> +#endif >> + >> +// BSD >> +#ifdef TARGET_OS_ARCH_bsd_x86 >> +# include "atomic_bsd_x86.hpp" >> +#endif >> >> >> I don't think this aids maintainability or cleanliness of the code for >> this use case. I don't think this should be added to the code base at >> this time. >> >> Thanks, >> Coleen >> >> On 1/4/15, 11:34 PM, David Holmes wrote: >>> Sorry for the top-post but just wanted to reset the context a little >>> here. Originally [1] I made the comment: >>> >>> "It's fine to have generic shared approaches but there really needs to >>> be a way to allow platform specific "overrides"." >>> >>> For the actual original RFR I said [2]: >>> >>> "Can we pause and give some more thought to a clean mechanism for >>> allowing a shared implementation if desired with the ability to >>> override if desired. I really do not like to see CPU specific ifdefs >>> being added to shared code. (And I would also not like to see all >>> platforms being forced to reimplement this natively). >>> >>> I'm not saying we will find a simple solution, but it would be nice if >>> we could get a few folk to think about it before proceeding with the >>> ifdefs :)" >>> >>> Erik then proposed three alternative approaches [3] and the simple >>> variant was chosen [4] and presented for review. However Paul Hohensee >>> also made comments about an inheritance-based approach and Erik >>> floated the first template-based variant [5] and there was some >>> discussion with Paul and Kim. But we then ended up with the simple >>> solution, leaving an inheritance-based solution for future work [6] >>> (which is what is what is being discussed now). >>> >>> This template-based meta-programming is not something I have any >>> familiarity with. My initial thought is this seems overkill for the >>> situation we have with the Atomic class - but as I said I'm ignorant >>> of the technique being used here. >>> >>> David >>> ----- >>> >>> [1] >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015292.html >>> >>> [2] >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015303.html >>> >>> [3] >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015362.html >>> >>> [4] >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015401.html >>> >>> [5] >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-October/015586.html >>> >>> [6] >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-November/015889.html >>> >>> >>> On 2/01/2015 12:07 AM, Erik ?sterlund wrote: >>>> Hi Kim, >>>> >>>>> On 01 Jan 2015, at 07:42, Kim Barrett wrote: >>>>> >>>>> On Dec 31, 2014, at 6:12 AM, Erik ?sterlund >>>>> wrote: >>>>>> >>>>>>> The rest of these comments only matter if, contrary to my suggestion >>>>>>> above, some additional infrastructure is thought necessary. >>>>>>> >>>>>> >>>>>> This is what I originally thought too and hence the current >>>>>> solution. But voices said that using macros for this is, I quote, >>>>>> "that bad". >>>>>> >>>>>>> [?] >>>>>>> >>>>>>> This scales with the number of AtomicPlatform definitions, rather >>>>>>> than >>>>>>> the number of specialized functions as happens with the existing >>>>>>> code. >>>>>> >>>>>> So your proposal is to still use ugly macros but move it to the >>>>>> class level instead, even though at the moment there is really only >>>>>> one member function that needs it. That feels a bit meh to be >>>>>> honest. IMO if we want to do this right, why go half the way and >>>>>> still rely on ugly macros? >>>>> >>>>> Please lay off with the perjorative "ugly macros". The preprocessor >>>>> is part of C++, and isn't going away. It's a tool that can be used, >>>>> or abused, and I think this is a fine use. To me, "ugly" is adding >>>>> 800+ lines of code / tests / comments to eliminate a dozen lines of >>>>> flag macro definitions and associated #ifdef?s. >>>> >>>> Well people are entitled to have different opinions of course. As I >>>> already mentioned, I did this because other reviewers (as well as me) >>>> found it ugly to use macros for conditionally specializing, which is >>>> the whole motivation of doing this, and was well understood from the >>>> beginning. But I?m fine with leaving it as macros if this is in >>>> general preferred and opinions have suddenly changed in this matter. >>>> >>>> Personally I don?t see how the number of rows of code define >>>> uglyness. Reusability, testing, documentation and well definedness in >>>> contracts/behaviours all lead to more lines of code, but I don?t >>>> think that equals uglyness. So while I disagree more code is uglier >>>> because it?s more, it?s all up to opinion and it?s quite pointless to >>>> discuss; you are entitled to think that is ugly. >>>> >>>>> If that additional infrastructure were amortized across more use-cases >>>>> then it might be more appealing. I might even have some additional >>>>> use-cases for something along that line, though not in its present >>>>> form. (More on that below.) >>>> >>>> Yeah I specifically had in mind the wish to make a split between >>>> commercial extensions to open source code actually. This pattern >>>> allows easy optional class inheritance injection as a general design >>>> pattern to achieve this. And for my own purposes the ability to >>>> define asymmetric dekker synchronization to elide storeload fences >>>> when the platform (e.g. TSO) supports it but otherwise revert to a >>>> generalized implementation (using storeload fences). This is why I >>>> think a clean reusable strategy is almost always better because it is >>>> widely applicable, even if there would not be obvious other examples >>>> where it is useful. >>>> >>>>> >>>>>>> I think the name "IsBaseOfAndDerived" would be better as >>>>>>> "IsBaseAndDerived?. [?] >>>>>>> >>>>>>> The current implementation of IsBaseOfAndDerived (and therefore >>>>>>> IsBaseOf) may (silently) provide incorrect answers when called with >>>>>>> incomplete types. [?] >>>>>>> >>>>>>> The actual usage in this proposed change-set even relies on this >>>>>>> flawed behavior of these metafunctions, since AtomicPlatform isn't >>>>>>> always defined. [?] >>>>>>> >>>>>> >>>>>> So what you are saying here is that I should rename >>>>>> IsBaseOfAndDerived to IsBaseAndDerived to more closely resemble >>>>>> other external libraries we do not use. But if I do that my >>>>>> behaviour is "incorrect" because it is not identical to that of the >>>>>> external library. And therefore my change would not work if it was >>>>>> made "correctly?. >>>>> >>>>> The proposed IsBaseOfAndDerived and IsBaseOf are being described and >>>>> placed as general utilities. Their names are strongly reminiscent of >>>>> existing facilities in other libraries, including the (C++11) standard >>>>> library. Conforming to these external standards will avoid surprises, >>>>> especially when the proposed facilities are even described as being >>>>> the same (except for a noted corner case that is both hard to deal >>>>> with in a portable manner and not actually very interesting, so I'm >>>>> not complaining about that) in the review request email. >>>>> >>>>> Additionally, I think quietly providing a potentially incorrect answer >>>>> in the case of incomplete types renders the offered code quite broken >>>>> for use as a general utility. I think the requirement for complete >>>>> types by the well-known / standard facilities is well justified. >>>>> >>>> >>>> While I disagree it is universally incorrect (as long as it is well >>>> defined and tested) to not have the exact same requirement on >>>> completeness on the parameters as C++11/boost, who also already have >>>> different behaviour in this exact regard as pointed out by yourself, >>>> I can try to fix it if it bugs you, but it will make the >>>> infrastructure even larger which seems to be a concern. >>>> >>>>>> On the contrary this is a well tested feature on all our supported >>>>>> compilers and I did not intend to engineer it to be identical to >>>>>> some external library if it only causes trouble rather than >>>>>> helping. This is our own class and it can do what we want it to do >>>>>> and be called what we want it to be called, and I think that is >>>>>> fine as long as it is well tested, which it is. At least this is my >>>>>> opinion. Anyone else agree? >>>>> >>>>> Problem is, I think it's well tested code to solve the wrong problem. >>>>> >>>>> I suggest that what we're really looking for is not a base/derived >>>>> relationship between a pair of classes, but that we actually want to >>>>> determine whether a platform-specific class exists. >>>>> >>>>> As a result, I think there is more infrastructure in this change set >>>>> than is actually needed. The present implementation of >>>>> SelectBaseClass uses IsBaseOf, but doesn't need a lot of the >>>>> functionality of that metafunction. There is no actual need for a >>>>> Base/Derived relationship between the (possibly not defined) >>>>> platform-specific class and the default platform-generic class, so the >>>>> use of IsBaseOf is an unnecessary restriction that only >>>>> serendipitously tests for defined or not, due to the previously >>>>> discussed defect in this particular implementation. I also think the >>>>> name SelectBaseClass is perhaps overly general. >>>> >>>> The main idea of SelectBaseClass is to inject an optional class into >>>> the class hierarchy for specialization of generalized behaviour. It >>>> is absolutely critical (and intended) that this class is constrained >>>> to be a derived class of the fall-back class - it is not an artifact >>>> nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, >>>> A must be able to rely on the superclass being at least C-like so >>>> that the behaviour of the superclass can be used reliably and >>>> transparently regardless of the existence of the optional B. If this >>>> contract would not hold, I think it could lead to some really weird >>>> and unreliable code which should not be encouraged (at least not >>>> according to the Liskov substitution principle etc). >>>> >>>> So what I need in the contract is a check if the optional class is >>>> defined and if it fits in the inheritance hierarchy: they are both >>>> important IMO. >>>> >>>> Yes it could have been less code. From the beginning I had a single >>>> trait class that checked for both the inheritance and whether the >>>> class is defined or not but you said you wanted the behaviour to more >>>> closely match that of third party libraries. This is why there is now >>>> a lot more code to more closely (but still not exactly) match that >>>> behaviour. >>>> >>>>> I might be able to use a similar approach in some code I've been >>>>> working on as a background task. And while writing this reply I've >>>>> thought of another place that might benefit from something along those >>>>> lines. Thinking about the code under review in that context, I think >>>>> some changes would make these other possible use-cases easier and >>>>> clearer. >>>>> >>>>> I believe what is actually needed is a metatfunction something like: >>>>> >>>>> /** >>>>> * Metafunction whose nested type member is Default if Specific is >>>>> * incomplete, otherwise Specific. >>>>> * >>>>> * Specific and Default must both be non-cv qualified class types, and >>>>> * must not be the same type. >>>>> * >>>>> * If Specific is incomplete at the point where this metafunction is >>>>> * invoked, it must never be defined elsewhere in the program. >>>>> */ >>>>> template >>>>> struct SelectPlatformBase; >>>>> >>>>> [Note: I'm not wedded to that name.] >>>>> >>>>> Now, it turns out to be hard / impossible to write a generic >>>>> is_complete metafunction, due to ODR. Whether a type is complete >>>>> can vary between translation units, and even within a translation >>>>> unit. Having is_complete::value have different values depending on >>>>> where it is instantiated is an ODR violation. (One can make progress >>>>> by using anonymous namespaces and macro wrappers with __LINE__ >>>>> concatenation, but there is still the fundamental question of whether >>>>> is_complete really even makes semantic sense.) >>>> >>>> Yes and that makes me wonder if we care enough about the completeness >>>> semantics being the same as third party libraries (who are already >>>> inconsistent) if we do not need them to be, and on the contrary >>>> benefit from them not to be. >>>> >>>>> However, we don't need a fully general is_complete utility. We have a >>>>> much more restricted use-case, where we want to select an optionally >>>>> defined platform-specific class if it exists, and otherwise fall back >>>>> to a more generic class. And we're not using this in arbitrary >>>>> contexts, but only to select a platform-specialized implementation if >>>>> such exists. >>>>> >>>>> Here's a lightly tested implementation of SelectPlatformBase: >>>>> >>>>> // --- SelectPlatformBase >>>>> >>>>> template >>>>> struct EnableIf { typedef T type; }; >>>>> >>>>> template >>>>> struct EnableIf { }; >>>>> >>>>> template >>>>> struct If { typedef Then type; }; >>>>> >>>>> template >>>>> struct If { typedef Else type; }; >>>>> >>>>> // Metafunction whose nested value member is true if T is defined >>>>> // (complete), and false if T is incomplete. T must be a non-cv >>>>> // qualified class type. If T is incomplete at the point where this >>>>> // metafunction is invoked, it must never be defined elsewhere in the >>>>> // program. >>>>> template >>>>> class IsClassDefined { >>>>> typedef char yes[1]; >>>>> typedef char no[2]; >>>>> >>>>> template >>>>> static typename EnableIf::type & >>>>> check(U*); >>>>> static no& check(...); >>>>> >>>>> public: >>>>> static const bool value = sizeof(check((T*)0)) == sizeof(yes); >>>>> }; >>>>> >>>>> template >>>>> struct SelectPlatformBase { >>>>> typedef typename If< >>>>> IsClassDefined::value, Specific, Default>::type type; >>>>> }; >>>>> >>>>> // --- end SelectPlatformBase >>>>> >>>>> That's ~30 lines of code (+ tests and comments TBD) to do precisely >>>>> what we need, replacing ~100 lines (+ tests and comments) that have >>>>> various issues. (If and EnableIf should probably be hoisted out as >>>>> separate utilities.) We don't actually need SelectPlatformBase, and >>>>> could instead just directly use the If and IsClassDefined >>>>> metafunctions, but giving this functionality a name might be clearer. >>>> >>>> As stated, I think it?s dangerous to allow conditional inheritance >>>> where there is no inheritance constraint on the optional class. >>>> Therefore I do not think this is better, although I agree it is smaller. >>>> >>>> I agree that our use case here is smaller. If the amount of code is >>>> the problem and less code (and hence fewer reusable components) is >>>> preferred, then the original implementation with a simple >>>> IsBaseOfAndDerived does the trick (without checking types/cv >>>> qualifiers) and could be done in just a few lines, and this is what I >>>> originally did before you told me to expand it to more closely >>>> resemble external libraries: >>>> >>>> template >>>> class IsBaseOfAndDerived { >>>> typedef char yes[1]; >>>> typedef char no[2]; >>>> >>>> template >>>> static yes &check(Derived*, T); >>>> static no &check(Base*, int); >>>> >>>> template >>>> struct IsBaseOfHost { >>>> operator B*() const; >>>> operator D*(); >>>> }; >>>> public: >>>> static const bool value = sizeof(check(IsBaseOfHost>>> Derived>(), int())) == sizeof(yes); >>>> }; >>>> >>>> It checks the inheritance and existence of the Derived class which is >>>> exactly the constraints I need. >>>> >>>> If you do not want to expose it publicly and remove the reusability >>>> because it does not have identical semantics as third party libraries >>>> regarding the requirements of the Derived type to be complete, it >>>> could be an internal class of SelectBaseClass. >>>> >>>> I?m not going to value the reusability vs LoC, I leave that decision >>>> to you. >>>> >>>>> However, while I think there are other use cases for this specific >>>>> utility, I still think the proposed change set as a whole is adding a >>>>> lot of code just to avoid a a few lines of macro usage. That seems >>>>> like a poor tradeoff to me. >>>> >>>> >>>> Okay. >>>> >>>> In the end, we have to make a choice - what is more important, that >>>> the traits resemble some external library or code size. Because last >>>> time I proposed a small change and you wanted it to be closer to >>>> C++11 behaviour and I made it closer to that at the expense of more >>>> code. Now you are saying I have to make it even tighter to C++11 (or >>>> boost?), but also don?t want more lines of code which I believe is >>>> impossible given that I want to constrain both the class hierarchy to >>>> be reliable and check the existence of the optional class. This >>>> brings me to a few architectural questions which I may have (strong) >>>> opinions about but are not up to me to decide. >>>> >>>> 1. Do we want to replace the macros with template metaprogramming? 2 >>>> reviewers (and me) liked the metaprogramming approach and it was >>>> implemented because they (like me) did not want the macros. But you >>>> have a different opinion. I can?t judge who is right and wrong here. >>>> >>>> 2. Do we want to minimize the code size at the cost of reusability by >>>> making contracts of dependent components weaker and putting them as >>>> private classes in the SelectBaseClass? >>>> >>>> 3. If we do want template metaprogramming and want reusable >>>> components, the question is, is it important that the specification >>>> of completeness (which in general is almost impossible to implement >>>> correctly in a portable way as you pointed out) of the template >>>> arguments in IsBaseOf is identical as C++11/boost even though no code >>>> needs it to be and on the contrary some code benefits from it not to >>>> be? In that case which one of them do we, because they are already >>>> different anyway? There is a tradeoff between code size and >>>> complexity vs copying either one of the external libraries regarding >>>> requirements of completeness of template parameters. >>>> >>>> Thanks, >>>> /Erik >>>> >> From filipp.zhinkin at gmail.com Wed Jan 7 16:26:05 2015 From: filipp.zhinkin at gmail.com (Filipp ) Date: Wed, 7 Jan 2015 19:26:05 +0300 Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF6ACF5@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF69B7E@DEWDFEMB12A.global.corp.sap> <54ad3027.499b700a.481d.1821@mx.google.com> <4295855A5C1DE049A61835A1887419CC2CF6ACF5@DEWDFEMB12A.global.corp.sap> Message-ID: <54ad5e35.a72c980a.723d.1f8c@mx.google.com> Hi Goetz, Looks good to me. Thanks, Filipp. -----???????? ?????????----- ??: "Lindenmaier, Goetz" ??????????: ?07.?01.?2015 17:49 ????: "Filipp " ; "'hotspot-dev at openjdk.java.net'" ????: RE: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi Filipp I fixed TestMutuallyExclusivePlatformPredicates.java and uploaded a new webrev: http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev-02/ Thank you for pointing me to that. Somehow it?s not run in our nightly tests. I also fixed the copyrights. Best regards, Goetz. From: Filipp [mailto:filipp.zhinkin at gmail.com] Sent: Mittwoch, 7. Januar 2015 14:10 To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' Subject: RE: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi Goetz, you have to update TestMutuallyExclusivePlatformPredicates test [1], because you've added mutually exclusive predicate. Otherwise that test will fail. Also, I'd prefer to see switch instead of if-else chain in DynLibDcmdTest, but I'm fine with leaving it as it is now. Otherwise change looks good. Regards, Filipp. [1] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/acfdd92cedaa/test/testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java#l48 ??: Lindenmaier, Goetz ??????????: ?07.?01.?2015 14:03 ????: 'hotspot-dev at openjdk.java.net' ????: RE: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi, my first upload missed the fix in Platform.shouldSAAttach(): + if (isAix()) { + return false; // SA not implemented. I added that to the webrev. Sorry, Goetz From: Lindenmaier, Goetz Sent: Mittwoch, 7. Januar 2015 11:14 To: 'hotspot-dev at openjdk.java.net' Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi, A row of hotspot jtreg tests fail because support to test for Aix is missing. To pass them, I would like to submit the following fixes: * runtime/6888954/vmerrors.sh: AIX can read from address '0', then throws SIGILL. * serviceability/dcmd/DynLibDcmdTest.java: Just add check for Aix. * test/test_env.sh: Add support for AIX. Sort OSes alphabetical. * test/testlibrary/com/oracle/java/testlibrary/Platform.java: add isAix(). Sort OSes alphabetical. Please review this change, and I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev-01/ Thanks and best regards, Goetz. From christian.tornqvist at oracle.com Wed Jan 7 16:32:58 2015 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Wed, 7 Jan 2015 11:32:58 -0500 Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF69B49@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF69B49@DEWDFEMB12A.global.corp.sap> Message-ID: <02ed01d02a97$9b445780$d1cd0680$@oracle.com> Hi Goetz, This looks good, I can sponsor this for you. Thanks, Christian -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz Sent: Wednesday, January 7, 2015 5:14 AM To: 'hotspot-dev at openjdk.java.net' Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi, A row of hotspot jtreg tests fail because support to test for Aix is missing. To pass them, I would like to submit the following fixes: * runtime/6888954/vmerrors.sh: AIX can read from address '0', then throws SIGILL. * serviceability/dcmd/DynLibDcmdTest.java: Just add check for Aix. * test/test_env.sh: Add support for AIX. Sort OSes alphabetical. * test/testlibrary/com/oracle/java/testlibrary/Platform.java: add isAix(). Sort OSes alphabetical. Please review this change, and I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev-01/ Thanks and best regards, Goetz. From volker.simonis at gmail.com Wed Jan 7 17:02:23 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 7 Jan 2015 18:02:23 +0100 Subject: [9] RFR(XS): 8068505: interpreter profiling incorrect on PPC In-Reply-To: <54ABE6EE.8070605@oracle.com> References: <54ABE6EE.8070605@oracle.com> Message-ID: Hi Zoltan, thanks for this change. I think you're right with your observation that the code is wrong. It seems this is an old copy and paste error which has been in the port since the first day. The historical background is that we implemented tiered compilation with the C2 compiler only but haven't contributed that to the OpenJDK. But even with our C2-based tiered compilation, the code region fixed by you was never executed and still isn't executed because we have no C1 in the ppc64 port (and therefore no tiered compilation). So the only test I can currently do is to verify that the new code compiles - which I've just verified. Nevertheless your change is good since we might start working on C1 later this year. I can sponsor this change (and push it because it only touches a ppc-specific file). Regards, Volker PS: just out of curiosity - how did you stumble over this problem without even having ppc64 hardware? Maybe you just like reading ppc-assembly:) On Tue, Jan 6, 2015 at 2:45 PM, Zolt?n Maj? wrote: > Hi, > > > I've recently looked at the PPC port in hotspot and I noticed the following > small problem. > > Problem: The method TemplateInterpreterGenerator::generate_counter_incr() > increments method invocations counters on all supported architectures, > except on PPC (w/ tiered compilation enabled). As a result, methods might be > compiled later and OSR happens sooner than specified by compilation > thresholds. > > Solution: Change generate_counter_incr() in templateInterpreter_ppc.cpp to > use invocation counters instead of backedge counters. > > Webrev: http://cr.openjdk.java.net/~zmajo/8068505/webrev.00/ > > JDK Bug: https://bugs.openjdk.java.net/browse/JDK-8068505 > > I was not able to test the change as I don't have access to appropriate > infrastructure. > > Maybe this is the wrong list to send this RFR to. If yes, could you please > tell me where to send it? > > Thank you and best regards, > > > Zoltan > From kim.barrett at oracle.com Wed Jan 7 17:07:25 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 7 Jan 2015 12:07:25 -0500 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> Message-ID: Erik Helin pointed out to me that the change to the assertion text from "assert(...)" to "vmassert(...)" might break some existing JBS RULE comments for matching known failures. I'll look at how bad this really is, but it seems likely that the thing to do is to revert the message text back to using "assert" rather than "vmassert". From zoltan.majo at oracle.com Wed Jan 7 17:14:13 2015 From: zoltan.majo at oracle.com (=?UTF-8?B?Wm9sdMOhbiBNYWrDsw==?=) Date: Wed, 07 Jan 2015 18:14:13 +0100 Subject: [9] RFR(XS): 8068505: interpreter profiling incorrect on PPC In-Reply-To: References: <54ABE6EE.8070605@oracle.com> Message-ID: <54AD6965.30207@oracle.com> Hi Volker, thank you for the feedback! On 01/07/2015 06:02 PM, Volker Simonis wrote: > Hi Zoltan, > > thanks for this change. I think you're right with your observation > that the code is wrong. It seems this is an old copy and paste error > which has been in the port since the first day. The historical > background is that we implemented tiered compilation with the C2 > compiler only but haven't contributed that to the OpenJDK. But even > with our C2-based tiered compilation, the code region fixed by you was > never executed and still isn't executed because we have no C1 in the > ppc64 port (and therefore no tiered compilation). This makes things much clearer to me. > So the only test I can currently do is to verify that the new code > compiles - which I've just verified. Nevertheless your change is good > since we might start working on C1 later this year. > > I can sponsor this change (and push it because it only touches a > ppc-specific file). Thank you! It would be great if you could push the change. > > Regards, > Volker > > PS: just out of curiosity - how did you stumble over this problem > without even having ppc64 hardware? Maybe you just like reading > ppc-assembly:) I'm currently working on JDK-8059606 (enabling per-method compilation thresholds) and I was looking into how invocation and backedge counters are used by hotspot. So I stumbled upon this small problem... :-) Best regards, Zoltan > > > On Tue, Jan 6, 2015 at 2:45 PM, Zolt?n Maj? wrote: >> Hi, >> >> >> I've recently looked at the PPC port in hotspot and I noticed the following >> small problem. >> >> Problem: The method TemplateInterpreterGenerator::generate_counter_incr() >> increments method invocations counters on all supported architectures, >> except on PPC (w/ tiered compilation enabled). As a result, methods might be >> compiled later and OSR happens sooner than specified by compilation >> thresholds. >> >> Solution: Change generate_counter_incr() in templateInterpreter_ppc.cpp to >> use invocation counters instead of backedge counters. >> >> Webrev: http://cr.openjdk.java.net/~zmajo/8068505/webrev.00/ >> >> JDK Bug: https://bugs.openjdk.java.net/browse/JDK-8068505 >> >> I was not able to test the change as I don't have access to appropriate >> infrastructure. >> >> Maybe this is the wrong list to send this RFR to. If yes, could you please >> tell me where to send it? >> >> Thank you and best regards, >> >> >> Zoltan >> From volker.simonis at gmail.com Wed Jan 7 18:52:04 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 7 Jan 2015 19:52:04 +0100 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> <54AA143B.2080900@oracle.com> <54ACB5A4.50603@oracle.com> <54ACBD00.6030501@oracle.com> Message-ID: Hi Erik, I reviewed this change positively but I can live with either solution (I always feel a little like Dr. Jekyll and Mr. Hyde when it comes to template meta programming:) Nevertheless there's one question I'd like to ask: what's your experience with IDE support when it comes to template meta programming (haven't tried that for a while)? More concrete, I don't specially like C-macros either but IDE support for macros has become quite powerful and good in the last years. So if I correctly configure my IDE, it is easily possible to navigate to the correct Atomic::cmpxchg implementation depending on the current configuration I use. Would this still work with the meta programming trick - i.e. is it possible for the IDE (Netbeans or Eclipse) to select the correct base class? I think the C++ - parser in the IDE would have to evaluate the template arguments for this to work and I don't now if it really does that. Do you have any experience with this? Thanks, Volker On Wed, Jan 7, 2015 at 4:58 PM, Erik ?sterlund wrote: > Hi Coleen and David, > > I was under the impression that this change was wanted to remove the error prone conditional #define for supported specialized/generalized atomics at almost any cost. I was fine with the #define myself but implemented this conditional class injection since voices were raised by reviewers to find a better way, and the actual optimization (using #define) was only conditionally accepted, given that I would provide the template inheritance solution to get rid of the #define after. > > Personally I don?t think the change is too complicated, especially not from the user?s point of view as you don?t really need to understand the internals to use the traits. For this specific change, only one trait is really used by atomics, the rest are dependencies designed to be reusable ?bonuses? if anyone (like myself) needs it later on (for instance for automatic closure specialization). And the trait/metafunction actually used by Atomic is SelectBaseClass which simply selects among two classes the most specific class which is defined (a general, for the AtomicBase class and optionally AtomicPlatform if implemented, otherwise ignored and AtomicBase is used as fallback). > > But of course, if this fix is perceived as complicated and that this complexity is not motivated, then I understand and we can just close the issue and leave it the way it is I guess, if reviewers are now fine with that. > > Thanks, > /Erik > >> On 07 Jan 2015, at 05:58, David Holmes wrote: >> >> I'm inclined to agree with Coleen - while I originally lamented the use of platform specific ifdefs and wondered about using simple inheritance, this has gone off in a direction I never envisaged. I don't think the problem being attacked is sufficiently bad to warrant the complexity** of the solution. And it still adds a bunch of platform-specific ifdefs. :( >> >> Sorry. >> >> David >> ----- >> >> ** that is subjective of course - if you don't understand template meta-programming then of course it seems complex. I don't know if it remains complex if you do understand template meta-programming ? >> >> On 7/01/2015 2:27 PM, Coleen Phillimore wrote: >>> >>> Thanks David, I'm working through this thread in reverse. Thank you >>> for the context. This helps a lot. >>> >>> I am not familiar with template meta-programming either. A long time >>> ago, I was a proponent of using templates but mostly as a type-checked >>> way of avoiding duplicate code. >>> >>> In this case of template meta-programming, I don't think the >>> intellectual load of the idioms offset the ugliness of the code that >>> they are replacing. Learning these idioms is not trivial. They are >>> really hard to read. Maybe if template classes had proper names rather >>> than X, Y, and Z it would be better. In any case, using this to >>> replace code isn't a clear win in clean code. There is still the >>> duplicate declarations of AtomicPlatform. There's also the unfortunate >>> conditional inclusions in shared code: >>> >>> +// Linux >>> +#ifdef TARGET_OS_ARCH_linux_x86 >>> +# include "atomic_linux_x86.hpp" >>> +#endif >>> + >>> +// Solaris >>> +#ifdef TARGET_OS_ARCH_solaris_x86 >>> +# include "atomic_solaris_x86.hpp" >>> +#endif >>> + >>> +// Windows >>> +#ifdef TARGET_OS_ARCH_windows_x86 >>> +# include "atomic_windows_x86.hpp" >>> +#endif >>> + >>> +// BSD >>> +#ifdef TARGET_OS_ARCH_bsd_x86 >>> +# include "atomic_bsd_x86.hpp" >>> +#endif >>> >>> >>> I don't think this aids maintainability or cleanliness of the code for >>> this use case. I don't think this should be added to the code base at >>> this time. >>> >>> Thanks, >>> Coleen >>> >>> On 1/4/15, 11:34 PM, David Holmes wrote: >>>> Sorry for the top-post but just wanted to reset the context a little >>>> here. Originally [1] I made the comment: >>>> >>>> "It's fine to have generic shared approaches but there really needs to >>>> be a way to allow platform specific "overrides"." >>>> >>>> For the actual original RFR I said [2]: >>>> >>>> "Can we pause and give some more thought to a clean mechanism for >>>> allowing a shared implementation if desired with the ability to >>>> override if desired. I really do not like to see CPU specific ifdefs >>>> being added to shared code. (And I would also not like to see all >>>> platforms being forced to reimplement this natively). >>>> >>>> I'm not saying we will find a simple solution, but it would be nice if >>>> we could get a few folk to think about it before proceeding with the >>>> ifdefs :)" >>>> >>>> Erik then proposed three alternative approaches [3] and the simple >>>> variant was chosen [4] and presented for review. However Paul Hohensee >>>> also made comments about an inheritance-based approach and Erik >>>> floated the first template-based variant [5] and there was some >>>> discussion with Paul and Kim. But we then ended up with the simple >>>> solution, leaving an inheritance-based solution for future work [6] >>>> (which is what is what is being discussed now). >>>> >>>> This template-based meta-programming is not something I have any >>>> familiarity with. My initial thought is this seems overkill for the >>>> situation we have with the Atomic class - but as I said I'm ignorant >>>> of the technique being used here. >>>> >>>> David >>>> ----- >>>> >>>> [1] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015292.html >>>> >>>> [2] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015303.html >>>> >>>> [3] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015362.html >>>> >>>> [4] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015401.html >>>> >>>> [5] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-October/015586.html >>>> >>>> [6] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-November/015889.html >>>> >>>> >>>> On 2/01/2015 12:07 AM, Erik ?sterlund wrote: >>>>> Hi Kim, >>>>> >>>>>> On 01 Jan 2015, at 07:42, Kim Barrett wrote: >>>>>> >>>>>> On Dec 31, 2014, at 6:12 AM, Erik ?sterlund >>>>>> wrote: >>>>>>> >>>>>>>> The rest of these comments only matter if, contrary to my suggestion >>>>>>>> above, some additional infrastructure is thought necessary. >>>>>>>> >>>>>>> >>>>>>> This is what I originally thought too and hence the current >>>>>>> solution. But voices said that using macros for this is, I quote, >>>>>>> "that bad". >>>>>>> >>>>>>>> [?] >>>>>>>> >>>>>>>> This scales with the number of AtomicPlatform definitions, rather >>>>>>>> than >>>>>>>> the number of specialized functions as happens with the existing >>>>>>>> code. >>>>>>> >>>>>>> So your proposal is to still use ugly macros but move it to the >>>>>>> class level instead, even though at the moment there is really only >>>>>>> one member function that needs it. That feels a bit meh to be >>>>>>> honest. IMO if we want to do this right, why go half the way and >>>>>>> still rely on ugly macros? >>>>>> >>>>>> Please lay off with the perjorative "ugly macros". The preprocessor >>>>>> is part of C++, and isn't going away. It's a tool that can be used, >>>>>> or abused, and I think this is a fine use. To me, "ugly" is adding >>>>>> 800+ lines of code / tests / comments to eliminate a dozen lines of >>>>>> flag macro definitions and associated #ifdef?s. >>>>> >>>>> Well people are entitled to have different opinions of course. As I >>>>> already mentioned, I did this because other reviewers (as well as me) >>>>> found it ugly to use macros for conditionally specializing, which is >>>>> the whole motivation of doing this, and was well understood from the >>>>> beginning. But I?m fine with leaving it as macros if this is in >>>>> general preferred and opinions have suddenly changed in this matter. >>>>> >>>>> Personally I don?t see how the number of rows of code define >>>>> uglyness. Reusability, testing, documentation and well definedness in >>>>> contracts/behaviours all lead to more lines of code, but I don?t >>>>> think that equals uglyness. So while I disagree more code is uglier >>>>> because it?s more, it?s all up to opinion and it?s quite pointless to >>>>> discuss; you are entitled to think that is ugly. >>>>> >>>>>> If that additional infrastructure were amortized across more use-cases >>>>>> then it might be more appealing. I might even have some additional >>>>>> use-cases for something along that line, though not in its present >>>>>> form. (More on that below.) >>>>> >>>>> Yeah I specifically had in mind the wish to make a split between >>>>> commercial extensions to open source code actually. This pattern >>>>> allows easy optional class inheritance injection as a general design >>>>> pattern to achieve this. And for my own purposes the ability to >>>>> define asymmetric dekker synchronization to elide storeload fences >>>>> when the platform (e.g. TSO) supports it but otherwise revert to a >>>>> generalized implementation (using storeload fences). This is why I >>>>> think a clean reusable strategy is almost always better because it is >>>>> widely applicable, even if there would not be obvious other examples >>>>> where it is useful. >>>>> >>>>>> >>>>>>>> I think the name "IsBaseOfAndDerived" would be better as >>>>>>>> "IsBaseAndDerived?. [?] >>>>>>>> >>>>>>>> The current implementation of IsBaseOfAndDerived (and therefore >>>>>>>> IsBaseOf) may (silently) provide incorrect answers when called with >>>>>>>> incomplete types. [?] >>>>>>>> >>>>>>>> The actual usage in this proposed change-set even relies on this >>>>>>>> flawed behavior of these metafunctions, since AtomicPlatform isn't >>>>>>>> always defined. [?] >>>>>>>> >>>>>>> >>>>>>> So what you are saying here is that I should rename >>>>>>> IsBaseOfAndDerived to IsBaseAndDerived to more closely resemble >>>>>>> other external libraries we do not use. But if I do that my >>>>>>> behaviour is "incorrect" because it is not identical to that of the >>>>>>> external library. And therefore my change would not work if it was >>>>>>> made "correctly?. >>>>>> >>>>>> The proposed IsBaseOfAndDerived and IsBaseOf are being described and >>>>>> placed as general utilities. Their names are strongly reminiscent of >>>>>> existing facilities in other libraries, including the (C++11) standard >>>>>> library. Conforming to these external standards will avoid surprises, >>>>>> especially when the proposed facilities are even described as being >>>>>> the same (except for a noted corner case that is both hard to deal >>>>>> with in a portable manner and not actually very interesting, so I'm >>>>>> not complaining about that) in the review request email. >>>>>> >>>>>> Additionally, I think quietly providing a potentially incorrect answer >>>>>> in the case of incomplete types renders the offered code quite broken >>>>>> for use as a general utility. I think the requirement for complete >>>>>> types by the well-known / standard facilities is well justified. >>>>>> >>>>> >>>>> While I disagree it is universally incorrect (as long as it is well >>>>> defined and tested) to not have the exact same requirement on >>>>> completeness on the parameters as C++11/boost, who also already have >>>>> different behaviour in this exact regard as pointed out by yourself, >>>>> I can try to fix it if it bugs you, but it will make the >>>>> infrastructure even larger which seems to be a concern. >>>>> >>>>>>> On the contrary this is a well tested feature on all our supported >>>>>>> compilers and I did not intend to engineer it to be identical to >>>>>>> some external library if it only causes trouble rather than >>>>>>> helping. This is our own class and it can do what we want it to do >>>>>>> and be called what we want it to be called, and I think that is >>>>>>> fine as long as it is well tested, which it is. At least this is my >>>>>>> opinion. Anyone else agree? >>>>>> >>>>>> Problem is, I think it's well tested code to solve the wrong problem. >>>>>> >>>>>> I suggest that what we're really looking for is not a base/derived >>>>>> relationship between a pair of classes, but that we actually want to >>>>>> determine whether a platform-specific class exists. >>>>>> >>>>>> As a result, I think there is more infrastructure in this change set >>>>>> than is actually needed. The present implementation of >>>>>> SelectBaseClass uses IsBaseOf, but doesn't need a lot of the >>>>>> functionality of that metafunction. There is no actual need for a >>>>>> Base/Derived relationship between the (possibly not defined) >>>>>> platform-specific class and the default platform-generic class, so the >>>>>> use of IsBaseOf is an unnecessary restriction that only >>>>>> serendipitously tests for defined or not, due to the previously >>>>>> discussed defect in this particular implementation. I also think the >>>>>> name SelectBaseClass is perhaps overly general. >>>>> >>>>> The main idea of SelectBaseClass is to inject an optional class into >>>>> the class hierarchy for specialization of generalized behaviour. It >>>>> is absolutely critical (and intended) that this class is constrained >>>>> to be a derived class of the fall-back class - it is not an artifact >>>>> nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, >>>>> A must be able to rely on the superclass being at least C-like so >>>>> that the behaviour of the superclass can be used reliably and >>>>> transparently regardless of the existence of the optional B. If this >>>>> contract would not hold, I think it could lead to some really weird >>>>> and unreliable code which should not be encouraged (at least not >>>>> according to the Liskov substitution principle etc). >>>>> >>>>> So what I need in the contract is a check if the optional class is >>>>> defined and if it fits in the inheritance hierarchy: they are both >>>>> important IMO. >>>>> >>>>> Yes it could have been less code. From the beginning I had a single >>>>> trait class that checked for both the inheritance and whether the >>>>> class is defined or not but you said you wanted the behaviour to more >>>>> closely match that of third party libraries. This is why there is now >>>>> a lot more code to more closely (but still not exactly) match that >>>>> behaviour. >>>>> >>>>>> I might be able to use a similar approach in some code I've been >>>>>> working on as a background task. And while writing this reply I've >>>>>> thought of another place that might benefit from something along those >>>>>> lines. Thinking about the code under review in that context, I think >>>>>> some changes would make these other possible use-cases easier and >>>>>> clearer. >>>>>> >>>>>> I believe what is actually needed is a metatfunction something like: >>>>>> >>>>>> /** >>>>>> * Metafunction whose nested type member is Default if Specific is >>>>>> * incomplete, otherwise Specific. >>>>>> * >>>>>> * Specific and Default must both be non-cv qualified class types, and >>>>>> * must not be the same type. >>>>>> * >>>>>> * If Specific is incomplete at the point where this metafunction is >>>>>> * invoked, it must never be defined elsewhere in the program. >>>>>> */ >>>>>> template >>>>>> struct SelectPlatformBase; >>>>>> >>>>>> [Note: I'm not wedded to that name.] >>>>>> >>>>>> Now, it turns out to be hard / impossible to write a generic >>>>>> is_complete metafunction, due to ODR. Whether a type is complete >>>>>> can vary between translation units, and even within a translation >>>>>> unit. Having is_complete::value have different values depending on >>>>>> where it is instantiated is an ODR violation. (One can make progress >>>>>> by using anonymous namespaces and macro wrappers with __LINE__ >>>>>> concatenation, but there is still the fundamental question of whether >>>>>> is_complete really even makes semantic sense.) >>>>> >>>>> Yes and that makes me wonder if we care enough about the completeness >>>>> semantics being the same as third party libraries (who are already >>>>> inconsistent) if we do not need them to be, and on the contrary >>>>> benefit from them not to be. >>>>> >>>>>> However, we don't need a fully general is_complete utility. We have a >>>>>> much more restricted use-case, where we want to select an optionally >>>>>> defined platform-specific class if it exists, and otherwise fall back >>>>>> to a more generic class. And we're not using this in arbitrary >>>>>> contexts, but only to select a platform-specialized implementation if >>>>>> such exists. >>>>>> >>>>>> Here's a lightly tested implementation of SelectPlatformBase: >>>>>> >>>>>> // --- SelectPlatformBase >>>>>> >>>>>> template >>>>>> struct EnableIf { typedef T type; }; >>>>>> >>>>>> template >>>>>> struct EnableIf { }; >>>>>> >>>>>> template >>>>>> struct If { typedef Then type; }; >>>>>> >>>>>> template >>>>>> struct If { typedef Else type; }; >>>>>> >>>>>> // Metafunction whose nested value member is true if T is defined >>>>>> // (complete), and false if T is incomplete. T must be a non-cv >>>>>> // qualified class type. If T is incomplete at the point where this >>>>>> // metafunction is invoked, it must never be defined elsewhere in the >>>>>> // program. >>>>>> template >>>>>> class IsClassDefined { >>>>>> typedef char yes[1]; >>>>>> typedef char no[2]; >>>>>> >>>>>> template >>>>>> static typename EnableIf::type & >>>>>> check(U*); >>>>>> static no& check(...); >>>>>> >>>>>> public: >>>>>> static const bool value = sizeof(check((T*)0)) == sizeof(yes); >>>>>> }; >>>>>> >>>>>> template >>>>>> struct SelectPlatformBase { >>>>>> typedef typename If< >>>>>> IsClassDefined::value, Specific, Default>::type type; >>>>>> }; >>>>>> >>>>>> // --- end SelectPlatformBase >>>>>> >>>>>> That's ~30 lines of code (+ tests and comments TBD) to do precisely >>>>>> what we need, replacing ~100 lines (+ tests and comments) that have >>>>>> various issues. (If and EnableIf should probably be hoisted out as >>>>>> separate utilities.) We don't actually need SelectPlatformBase, and >>>>>> could instead just directly use the If and IsClassDefined >>>>>> metafunctions, but giving this functionality a name might be clearer. >>>>> >>>>> As stated, I think it?s dangerous to allow conditional inheritance >>>>> where there is no inheritance constraint on the optional class. >>>>> Therefore I do not think this is better, although I agree it is smaller. >>>>> >>>>> I agree that our use case here is smaller. If the amount of code is >>>>> the problem and less code (and hence fewer reusable components) is >>>>> preferred, then the original implementation with a simple >>>>> IsBaseOfAndDerived does the trick (without checking types/cv >>>>> qualifiers) and could be done in just a few lines, and this is what I >>>>> originally did before you told me to expand it to more closely >>>>> resemble external libraries: >>>>> >>>>> template >>>>> class IsBaseOfAndDerived { >>>>> typedef char yes[1]; >>>>> typedef char no[2]; >>>>> >>>>> template >>>>> static yes &check(Derived*, T); >>>>> static no &check(Base*, int); >>>>> >>>>> template >>>>> struct IsBaseOfHost { >>>>> operator B*() const; >>>>> operator D*(); >>>>> }; >>>>> public: >>>>> static const bool value = sizeof(check(IsBaseOfHost>>>> Derived>(), int())) == sizeof(yes); >>>>> }; >>>>> >>>>> It checks the inheritance and existence of the Derived class which is >>>>> exactly the constraints I need. >>>>> >>>>> If you do not want to expose it publicly and remove the reusability >>>>> because it does not have identical semantics as third party libraries >>>>> regarding the requirements of the Derived type to be complete, it >>>>> could be an internal class of SelectBaseClass. >>>>> >>>>> I?m not going to value the reusability vs LoC, I leave that decision >>>>> to you. >>>>> >>>>>> However, while I think there are other use cases for this specific >>>>>> utility, I still think the proposed change set as a whole is adding a >>>>>> lot of code just to avoid a a few lines of macro usage. That seems >>>>>> like a poor tradeoff to me. >>>>> >>>>> >>>>> Okay. >>>>> >>>>> In the end, we have to make a choice - what is more important, that >>>>> the traits resemble some external library or code size. Because last >>>>> time I proposed a small change and you wanted it to be closer to >>>>> C++11 behaviour and I made it closer to that at the expense of more >>>>> code. Now you are saying I have to make it even tighter to C++11 (or >>>>> boost?), but also don?t want more lines of code which I believe is >>>>> impossible given that I want to constrain both the class hierarchy to >>>>> be reliable and check the existence of the optional class. This >>>>> brings me to a few architectural questions which I may have (strong) >>>>> opinions about but are not up to me to decide. >>>>> >>>>> 1. Do we want to replace the macros with template metaprogramming? 2 >>>>> reviewers (and me) liked the metaprogramming approach and it was >>>>> implemented because they (like me) did not want the macros. But you >>>>> have a different opinion. I can?t judge who is right and wrong here. >>>>> >>>>> 2. Do we want to minimize the code size at the cost of reusability by >>>>> making contracts of dependent components weaker and putting them as >>>>> private classes in the SelectBaseClass? >>>>> >>>>> 3. If we do want template metaprogramming and want reusable >>>>> components, the question is, is it important that the specification >>>>> of completeness (which in general is almost impossible to implement >>>>> correctly in a portable way as you pointed out) of the template >>>>> arguments in IsBaseOf is identical as C++11/boost even though no code >>>>> needs it to be and on the contrary some code benefits from it not to >>>>> be? In that case which one of them do we, because they are already >>>>> different anyway? There is a tradeoff between code size and >>>>> complexity vs copying either one of the external libraries regarding >>>>> requirements of completeness of template parameters. >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>> > From sgehwolf at redhat.com Wed Jan 7 18:56:55 2015 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 07 Jan 2015 19:56:55 +0100 Subject: RFR(S) 8067331: Zero: Atomic::xchg and Atomic::xchg_ptr need full memory barrier In-Reply-To: <54ACBEF8.6050703@oracle.com> References: <1418656152.3297.44.camel@localhost.localdomain> <54A9F53C.5040507@oracle.com> <54AC9028.1040709@oracle.com> <54ACBBF0.3040803@oracle.com> <54ACBD4C.5010307@oracle.com> <54ACBEF8.6050703@oracle.com> Message-ID: <1420657015.5372.8.camel@redhat.com> On Wed, 2015-01-07 at 15:07 +1000, David Holmes wrote: > Severin, > > Can you create a changeset with myself (dholmes) and Coleen (coleenp) as > Reviewers, and with the file copyrights suitably updated. I'm not sure > what form that will take given the dual copyright attributions but > presumably the Red Hat line will need 2015 added. > > If you email me the changeset directly I will apply it to the > jdk9/hs-rt/hotspot/ forest. I've sent the exported changeset to you privately, David. Thanks for the reviews, David and Coleen! Cheers, Severin > Thanks, > David > > On 7/01/2015 2:59 PM, David Holmes wrote: > > On 7/01/2015 2:54 PM, Coleen Phillimore wrote: > >> > >> This looks good. Sorry for the delay reviewing it as it looks > >> important. Is there a small test case that can be written to test > >> this? (or no?) > > > > No - I'll add noreg-hard. Testing for a missing memory-barrier is not > > practical. > > > > Thanks, > > David > > > >> Coleen > >> > >> On 1/6/15, 8:47 PM, David Holmes wrote: > >>> Can we get a second hotspot review please - this is quite straight > >>> forward. > >>> > >>> Thanks, > >>> David > >>> > >>> On 5/01/2015 12:21 PM, David Holmes wrote: > >>>> Hi Severin, > >>>> > >>>> Sorry this slipped through the cracks before the holidays. > >>>> > >>>> On 16/12/2014 1:09 AM, Severin Gehwolf wrote: > >>>>> Hi, > >>>>> > >>>>> Could someone please review and sponsor the following change? > >>>>> > >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8067331 > >>>>> webrev: > >>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8067331/webrev.02/ > >>>>> > >>>>> The issue is that all atomic operations are expected to be full memory > >>>>> barriers, but two implementations of the Zero variant JVM - > >>>>> Atomic::xchg > >>>>> and Atomic::xchg_ptr - use GCC's built-in __sync_lock_test_and_set > >>>>> which > >>>>> is an acquire barrier only. The fix adds full memory barriers manually > >>>>> for those. > >>>>> > >>>>> Thoughts? > >>>> > >>>> Looks completely consistent with what I just saw in the aarch64 port > >>>> code. > >>>> > >>>> I can sponsor this but we still need a second review. > >>>> > >>>> Thanks, > >>>> David > >>>> > >>>>> Cheers, > >>>>> Severin > >>>>> > >> From goetz.lindenmaier at sap.com Wed Jan 7 21:31:38 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 7 Jan 2015 21:31:38 +0000 Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests In-Reply-To: <02ed01d02a97$9b445780$d1cd0680$@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF69B49@DEWDFEMB12A.global.corp.sap> <02ed01d02a97$9b445780$d1cd0680$@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF6AE34@DEWDFEMB12A.global.corp.sap> Hi Christian, It would be great if you sponsor this, thanks! Best regards, Goetz. -----Original Message----- From: Christian Tornqvist [mailto:christian.tornqvist at oracle.com] Sent: Wednesday, January 07, 2015 5:33 PM To: Lindenmaier, Goetz; hotspot-dev at openjdk.java.net Subject: RE: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi Goetz, This looks good, I can sponsor this for you. Thanks, Christian -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz Sent: Wednesday, January 7, 2015 5:14 AM To: 'hotspot-dev at openjdk.java.net' Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi, A row of hotspot jtreg tests fail because support to test for Aix is missing. To pass them, I would like to submit the following fixes: * runtime/6888954/vmerrors.sh: AIX can read from address '0', then throws SIGILL. * serviceability/dcmd/DynLibDcmdTest.java: Just add check for Aix. * test/test_env.sh: Add support for AIX. Sort OSes alphabetical. * test/testlibrary/com/oracle/java/testlibrary/Platform.java: add isAix(). Sort OSes alphabetical. Please review this change, and I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev-01/ Thanks and best regards, Goetz. From goetz.lindenmaier at sap.com Wed Jan 7 21:31:55 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 7 Jan 2015 21:31:55 +0000 Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests In-Reply-To: <54ad5e35.a72c980a.723d.1f8c@mx.google.com> References: <4295855A5C1DE049A61835A1887419CC2CF69B7E@DEWDFEMB12A.global.corp.sap> <54ad3027.499b700a.481d.1821@mx.google.com> <4295855A5C1DE049A61835A1887419CC2CF6ACF5@DEWDFEMB12A.global.corp.sap> <54ad5e35.a72c980a.723d.1f8c@mx.google.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF6AE51@DEWDFEMB12A.global.corp.sap> Thanks Filipp! Best regards, Goetz. From: Filipp [mailto:filipp.zhinkin at gmail.com] Sent: Wednesday, January 07, 2015 5:26 PM To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' Subject: RE: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi Goetz, Looks good to me. Thanks, Filipp. ________________________________ ??: Lindenmaier, Goetz ??????????: ?07.?01.?2015 17:49 ????: Filipp ; 'hotspot-dev at openjdk.java.net' ????: RE: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi Filipp I fixed TestMutuallyExclusivePlatformPredicates.java and uploaded a new webrev: http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev-02/ Thank you for pointing me to that. Somehow it?s not run in our nightly tests. I also fixed the copyrights. Best regards, Goetz. From: Filipp [mailto:filipp.zhinkin at gmail.com] Sent: Mittwoch, 7. Januar 2015 14:10 To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' Subject: RE: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi Goetz, you have to update TestMutuallyExclusivePlatformPredicates test [1], because you've added mutually exclusive predicate. Otherwise that test will fail. Also, I'd prefer to see switch instead of if-else chain in DynLibDcmdTest, but I'm fine with leaving it as it is now. Otherwise change looks good. Regards, Filipp. [1] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/acfdd92cedaa/test/testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java#l48 ________________________________ ??: Lindenmaier, Goetz ??????????: ?07.?01.?2015 14:03 ????: 'hotspot-dev at openjdk.java.net' ????: RE: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi, my first upload missed the fix in Platform.shouldSAAttach(): + if (isAix()) { + return false; // SA not implemented. I added that to the webrev. Sorry, Goetz From: Lindenmaier, Goetz Sent: Mittwoch, 7. Januar 2015 11:14 To: 'hotspot-dev at openjdk.java.net' Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi, A row of hotspot jtreg tests fail because support to test for Aix is missing. To pass them, I would like to submit the following fixes: * runtime/6888954/vmerrors.sh: AIX can read from address '0', then throws SIGILL. * serviceability/dcmd/DynLibDcmdTest.java: Just add check for Aix. * test/test_env.sh: Add support for AIX. Sort OSes alphabetical. * test/testlibrary/com/oracle/java/testlibrary/Platform.java: add isAix(). Sort OSes alphabetical. Please review this change, and I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev-01/ Thanks and best regards, Goetz. From erik.osterlund at lnu.se Wed Jan 7 22:42:07 2015 From: erik.osterlund at lnu.se (=?utf-8?B?RXJpayDDlnN0ZXJsdW5k?=) Date: Wed, 7 Jan 2015 22:42:07 +0000 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> <54AA143B.2080900@oracle.com> <54ACB5A4.50603@oracle.com> <54ACBD00.6030501@oracle.com> Message-ID: <85BF2CA5-4B5F-4610-B367-906C24377D60@lnu.se> Hi Volker, Personally I abandoned Eclipse in favour of tmux, vim, and a bunch of vim plugins because I found it difficult to configure Eclipse the way I wanted it to (perhaps due to lack of Eclipse knowledge). To go to declaration/definition I use the YouCompleteMe plugin for vim. It plugs straight into the clang library for code sense and is easy to set up with the compiler flags you want (basically mirroring flags the makefiles produce), and it therefore understands the metaprogramming perfectly and selects the right base class in my previously proposed code. Since the same library is used for compiling and code sensing, it seems like pretty much everything that compiles is also semantically well understood since it uses the compiler?s AST. :) I just tried it in Eclipse and it selected the wrong base class (selected the generic variant when the platform specialization was available). But I don?t know if that?s because I haven?t set everything right in Eclipse or not. But it seems more likely it?s simply not clever enough to understand the SFINAE stuff very well. But perhaps somebody more knowledgeable about Eclipse knows some setting I don?t (for instance how to plug it into libclang? I mean if vim can do it, surely eclipse should be able to as well? right?!) Hope this partially answers your question. Maybe somebody else using IDEs like Eclipse and Netbeans more in their workflow has a better clue! Vim users are safe anyway haha! /Erik > On 07 Jan 2015, at 19:52, Volker Simonis wrote: > > Hi Erik, > > I reviewed this change positively but I can live with either solution > (I always feel a little like Dr. Jekyll and Mr. Hyde when it comes to > template meta programming:) > > Nevertheless there's one question I'd like to ask: what's your > experience with IDE support when it comes to template meta programming > (haven't tried that for a while)? > > More concrete, I don't specially like C-macros either but IDE support > for macros has become quite powerful and good in the last years. So if > I correctly configure my IDE, it is easily possible to navigate to the > correct Atomic::cmpxchg implementation depending on the current > configuration I use. > > Would this still work with the meta programming trick - i.e. is it > possible for the IDE (Netbeans or Eclipse) to select the correct base > class? I think the C++ - parser in the IDE would have to evaluate the > template arguments for this to work and I don't now if it really does > that. Do you have any experience with this? > > Thanks, > Volker > > > On Wed, Jan 7, 2015 at 4:58 PM, Erik ?sterlund wrote: >> Hi Coleen and David, >> >> I was under the impression that this change was wanted to remove the error prone conditional #define for supported specialized/generalized atomics at almost any cost. I was fine with the #define myself but implemented this conditional class injection since voices were raised by reviewers to find a better way, and the actual optimization (using #define) was only conditionally accepted, given that I would provide the template inheritance solution to get rid of the #define after. >> >> Personally I don?t think the change is too complicated, especially not from the user?s point of view as you don?t really need to understand the internals to use the traits. For this specific change, only one trait is really used by atomics, the rest are dependencies designed to be reusable ?bonuses? if anyone (like myself) needs it later on (for instance for automatic closure specialization). And the trait/metafunction actually used by Atomic is SelectBaseClass which simply selects among two classes the most specific class which is defined (a general, for the AtomicBase class and optionally AtomicPlatform if implemented, otherwise ignored and AtomicBase is used as fallback). >> >> But of course, if this fix is perceived as complicated and that this complexity is not motivated, then I understand and we can just close the issue and leave it the way it is I guess, if reviewers are now fine with that. >> >> Thanks, >> /Erik >> >>> On 07 Jan 2015, at 05:58, David Holmes wrote: >>> >>> I'm inclined to agree with Coleen - while I originally lamented the use of platform specific ifdefs and wondered about using simple inheritance, this has gone off in a direction I never envisaged. I don't think the problem being attacked is sufficiently bad to warrant the complexity** of the solution. And it still adds a bunch of platform-specific ifdefs. :( >>> >>> Sorry. >>> >>> David >>> ----- >>> >>> ** that is subjective of course - if you don't understand template meta-programming then of course it seems complex. I don't know if it remains complex if you do understand template meta-programming ? >>> >>> On 7/01/2015 2:27 PM, Coleen Phillimore wrote: >>>> >>>> Thanks David, I'm working through this thread in reverse. Thank you >>>> for the context. This helps a lot. >>>> >>>> I am not familiar with template meta-programming either. A long time >>>> ago, I was a proponent of using templates but mostly as a type-checked >>>> way of avoiding duplicate code. >>>> >>>> In this case of template meta-programming, I don't think the >>>> intellectual load of the idioms offset the ugliness of the code that >>>> they are replacing. Learning these idioms is not trivial. They are >>>> really hard to read. Maybe if template classes had proper names rather >>>> than X, Y, and Z it would be better. In any case, using this to >>>> replace code isn't a clear win in clean code. There is still the >>>> duplicate declarations of AtomicPlatform. There's also the unfortunate >>>> conditional inclusions in shared code: >>>> >>>> +// Linux >>>> +#ifdef TARGET_OS_ARCH_linux_x86 >>>> +# include "atomic_linux_x86.hpp" >>>> +#endif >>>> + >>>> +// Solaris >>>> +#ifdef TARGET_OS_ARCH_solaris_x86 >>>> +# include "atomic_solaris_x86.hpp" >>>> +#endif >>>> + >>>> +// Windows >>>> +#ifdef TARGET_OS_ARCH_windows_x86 >>>> +# include "atomic_windows_x86.hpp" >>>> +#endif >>>> + >>>> +// BSD >>>> +#ifdef TARGET_OS_ARCH_bsd_x86 >>>> +# include "atomic_bsd_x86.hpp" >>>> +#endif >>>> >>>> >>>> I don't think this aids maintainability or cleanliness of the code for >>>> this use case. I don't think this should be added to the code base at >>>> this time. >>>> >>>> Thanks, >>>> Coleen >>>> >>>> On 1/4/15, 11:34 PM, David Holmes wrote: >>>>> Sorry for the top-post but just wanted to reset the context a little >>>>> here. Originally [1] I made the comment: >>>>> >>>>> "It's fine to have generic shared approaches but there really needs to >>>>> be a way to allow platform specific "overrides"." >>>>> >>>>> For the actual original RFR I said [2]: >>>>> >>>>> "Can we pause and give some more thought to a clean mechanism for >>>>> allowing a shared implementation if desired with the ability to >>>>> override if desired. I really do not like to see CPU specific ifdefs >>>>> being added to shared code. (And I would also not like to see all >>>>> platforms being forced to reimplement this natively). >>>>> >>>>> I'm not saying we will find a simple solution, but it would be nice if >>>>> we could get a few folk to think about it before proceeding with the >>>>> ifdefs :)" >>>>> >>>>> Erik then proposed three alternative approaches [3] and the simple >>>>> variant was chosen [4] and presented for review. However Paul Hohensee >>>>> also made comments about an inheritance-based approach and Erik >>>>> floated the first template-based variant [5] and there was some >>>>> discussion with Paul and Kim. But we then ended up with the simple >>>>> solution, leaving an inheritance-based solution for future work [6] >>>>> (which is what is what is being discussed now). >>>>> >>>>> This template-based meta-programming is not something I have any >>>>> familiarity with. My initial thought is this seems overkill for the >>>>> situation we have with the Atomic class - but as I said I'm ignorant >>>>> of the technique being used here. >>>>> >>>>> David >>>>> ----- >>>>> >>>>> [1] >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015292.html >>>>> >>>>> [2] >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015303.html >>>>> >>>>> [3] >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015362.html >>>>> >>>>> [4] >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015401.html >>>>> >>>>> [5] >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-October/015586.html >>>>> >>>>> [6] >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-November/015889.html >>>>> >>>>> >>>>> On 2/01/2015 12:07 AM, Erik ?sterlund wrote: >>>>>> Hi Kim, >>>>>> >>>>>>> On 01 Jan 2015, at 07:42, Kim Barrett wrote: >>>>>>> >>>>>>> On Dec 31, 2014, at 6:12 AM, Erik ?sterlund >>>>>>> wrote: >>>>>>>> >>>>>>>>> The rest of these comments only matter if, contrary to my suggestion >>>>>>>>> above, some additional infrastructure is thought necessary. >>>>>>>>> >>>>>>>> >>>>>>>> This is what I originally thought too and hence the current >>>>>>>> solution. But voices said that using macros for this is, I quote, >>>>>>>> "that bad". >>>>>>>> >>>>>>>>> [?] >>>>>>>>> >>>>>>>>> This scales with the number of AtomicPlatform definitions, rather >>>>>>>>> than >>>>>>>>> the number of specialized functions as happens with the existing >>>>>>>>> code. >>>>>>>> >>>>>>>> So your proposal is to still use ugly macros but move it to the >>>>>>>> class level instead, even though at the moment there is really only >>>>>>>> one member function that needs it. That feels a bit meh to be >>>>>>>> honest. IMO if we want to do this right, why go half the way and >>>>>>>> still rely on ugly macros? >>>>>>> >>>>>>> Please lay off with the perjorative "ugly macros". The preprocessor >>>>>>> is part of C++, and isn't going away. It's a tool that can be used, >>>>>>> or abused, and I think this is a fine use. To me, "ugly" is adding >>>>>>> 800+ lines of code / tests / comments to eliminate a dozen lines of >>>>>>> flag macro definitions and associated #ifdef?s. >>>>>> >>>>>> Well people are entitled to have different opinions of course. As I >>>>>> already mentioned, I did this because other reviewers (as well as me) >>>>>> found it ugly to use macros for conditionally specializing, which is >>>>>> the whole motivation of doing this, and was well understood from the >>>>>> beginning. But I?m fine with leaving it as macros if this is in >>>>>> general preferred and opinions have suddenly changed in this matter. >>>>>> >>>>>> Personally I don?t see how the number of rows of code define >>>>>> uglyness. Reusability, testing, documentation and well definedness in >>>>>> contracts/behaviours all lead to more lines of code, but I don?t >>>>>> think that equals uglyness. So while I disagree more code is uglier >>>>>> because it?s more, it?s all up to opinion and it?s quite pointless to >>>>>> discuss; you are entitled to think that is ugly. >>>>>> >>>>>>> If that additional infrastructure were amortized across more use-cases >>>>>>> then it might be more appealing. I might even have some additional >>>>>>> use-cases for something along that line, though not in its present >>>>>>> form. (More on that below.) >>>>>> >>>>>> Yeah I specifically had in mind the wish to make a split between >>>>>> commercial extensions to open source code actually. This pattern >>>>>> allows easy optional class inheritance injection as a general design >>>>>> pattern to achieve this. And for my own purposes the ability to >>>>>> define asymmetric dekker synchronization to elide storeload fences >>>>>> when the platform (e.g. TSO) supports it but otherwise revert to a >>>>>> generalized implementation (using storeload fences). This is why I >>>>>> think a clean reusable strategy is almost always better because it is >>>>>> widely applicable, even if there would not be obvious other examples >>>>>> where it is useful. >>>>>> >>>>>>> >>>>>>>>> I think the name "IsBaseOfAndDerived" would be better as >>>>>>>>> "IsBaseAndDerived?. [?] >>>>>>>>> >>>>>>>>> The current implementation of IsBaseOfAndDerived (and therefore >>>>>>>>> IsBaseOf) may (silently) provide incorrect answers when called with >>>>>>>>> incomplete types. [?] >>>>>>>>> >>>>>>>>> The actual usage in this proposed change-set even relies on this >>>>>>>>> flawed behavior of these metafunctions, since AtomicPlatform isn't >>>>>>>>> always defined. [?] >>>>>>>>> >>>>>>>> >>>>>>>> So what you are saying here is that I should rename >>>>>>>> IsBaseOfAndDerived to IsBaseAndDerived to more closely resemble >>>>>>>> other external libraries we do not use. But if I do that my >>>>>>>> behaviour is "incorrect" because it is not identical to that of the >>>>>>>> external library. And therefore my change would not work if it was >>>>>>>> made "correctly?. >>>>>>> >>>>>>> The proposed IsBaseOfAndDerived and IsBaseOf are being described and >>>>>>> placed as general utilities. Their names are strongly reminiscent of >>>>>>> existing facilities in other libraries, including the (C++11) standard >>>>>>> library. Conforming to these external standards will avoid surprises, >>>>>>> especially when the proposed facilities are even described as being >>>>>>> the same (except for a noted corner case that is both hard to deal >>>>>>> with in a portable manner and not actually very interesting, so I'm >>>>>>> not complaining about that) in the review request email. >>>>>>> >>>>>>> Additionally, I think quietly providing a potentially incorrect answer >>>>>>> in the case of incomplete types renders the offered code quite broken >>>>>>> for use as a general utility. I think the requirement for complete >>>>>>> types by the well-known / standard facilities is well justified. >>>>>>> >>>>>> >>>>>> While I disagree it is universally incorrect (as long as it is well >>>>>> defined and tested) to not have the exact same requirement on >>>>>> completeness on the parameters as C++11/boost, who also already have >>>>>> different behaviour in this exact regard as pointed out by yourself, >>>>>> I can try to fix it if it bugs you, but it will make the >>>>>> infrastructure even larger which seems to be a concern. >>>>>> >>>>>>>> On the contrary this is a well tested feature on all our supported >>>>>>>> compilers and I did not intend to engineer it to be identical to >>>>>>>> some external library if it only causes trouble rather than >>>>>>>> helping. This is our own class and it can do what we want it to do >>>>>>>> and be called what we want it to be called, and I think that is >>>>>>>> fine as long as it is well tested, which it is. At least this is my >>>>>>>> opinion. Anyone else agree? >>>>>>> >>>>>>> Problem is, I think it's well tested code to solve the wrong problem. >>>>>>> >>>>>>> I suggest that what we're really looking for is not a base/derived >>>>>>> relationship between a pair of classes, but that we actually want to >>>>>>> determine whether a platform-specific class exists. >>>>>>> >>>>>>> As a result, I think there is more infrastructure in this change set >>>>>>> than is actually needed. The present implementation of >>>>>>> SelectBaseClass uses IsBaseOf, but doesn't need a lot of the >>>>>>> functionality of that metafunction. There is no actual need for a >>>>>>> Base/Derived relationship between the (possibly not defined) >>>>>>> platform-specific class and the default platform-generic class, so the >>>>>>> use of IsBaseOf is an unnecessary restriction that only >>>>>>> serendipitously tests for defined or not, due to the previously >>>>>>> discussed defect in this particular implementation. I also think the >>>>>>> name SelectBaseClass is perhaps overly general. >>>>>> >>>>>> The main idea of SelectBaseClass is to inject an optional class into >>>>>> the class hierarchy for specialization of generalized behaviour. It >>>>>> is absolutely critical (and intended) that this class is constrained >>>>>> to be a derived class of the fall-back class - it is not an artifact >>>>>> nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, >>>>>> A must be able to rely on the superclass being at least C-like so >>>>>> that the behaviour of the superclass can be used reliably and >>>>>> transparently regardless of the existence of the optional B. If this >>>>>> contract would not hold, I think it could lead to some really weird >>>>>> and unreliable code which should not be encouraged (at least not >>>>>> according to the Liskov substitution principle etc). >>>>>> >>>>>> So what I need in the contract is a check if the optional class is >>>>>> defined and if it fits in the inheritance hierarchy: they are both >>>>>> important IMO. >>>>>> >>>>>> Yes it could have been less code. From the beginning I had a single >>>>>> trait class that checked for both the inheritance and whether the >>>>>> class is defined or not but you said you wanted the behaviour to more >>>>>> closely match that of third party libraries. This is why there is now >>>>>> a lot more code to more closely (but still not exactly) match that >>>>>> behaviour. >>>>>> >>>>>>> I might be able to use a similar approach in some code I've been >>>>>>> working on as a background task. And while writing this reply I've >>>>>>> thought of another place that might benefit from something along those >>>>>>> lines. Thinking about the code under review in that context, I think >>>>>>> some changes would make these other possible use-cases easier and >>>>>>> clearer. >>>>>>> >>>>>>> I believe what is actually needed is a metatfunction something like: >>>>>>> >>>>>>> /** >>>>>>> * Metafunction whose nested type member is Default if Specific is >>>>>>> * incomplete, otherwise Specific. >>>>>>> * >>>>>>> * Specific and Default must both be non-cv qualified class types, and >>>>>>> * must not be the same type. >>>>>>> * >>>>>>> * If Specific is incomplete at the point where this metafunction is >>>>>>> * invoked, it must never be defined elsewhere in the program. >>>>>>> */ >>>>>>> template >>>>>>> struct SelectPlatformBase; >>>>>>> >>>>>>> [Note: I'm not wedded to that name.] >>>>>>> >>>>>>> Now, it turns out to be hard / impossible to write a generic >>>>>>> is_complete metafunction, due to ODR. Whether a type is complete >>>>>>> can vary between translation units, and even within a translation >>>>>>> unit. Having is_complete::value have different values depending on >>>>>>> where it is instantiated is an ODR violation. (One can make progress >>>>>>> by using anonymous namespaces and macro wrappers with __LINE__ >>>>>>> concatenation, but there is still the fundamental question of whether >>>>>>> is_complete really even makes semantic sense.) >>>>>> >>>>>> Yes and that makes me wonder if we care enough about the completeness >>>>>> semantics being the same as third party libraries (who are already >>>>>> inconsistent) if we do not need them to be, and on the contrary >>>>>> benefit from them not to be. >>>>>> >>>>>>> However, we don't need a fully general is_complete utility. We have a >>>>>>> much more restricted use-case, where we want to select an optionally >>>>>>> defined platform-specific class if it exists, and otherwise fall back >>>>>>> to a more generic class. And we're not using this in arbitrary >>>>>>> contexts, but only to select a platform-specialized implementation if >>>>>>> such exists. >>>>>>> >>>>>>> Here's a lightly tested implementation of SelectPlatformBase: >>>>>>> >>>>>>> // --- SelectPlatformBase >>>>>>> >>>>>>> template >>>>>>> struct EnableIf { typedef T type; }; >>>>>>> >>>>>>> template >>>>>>> struct EnableIf { }; >>>>>>> >>>>>>> template >>>>>>> struct If { typedef Then type; }; >>>>>>> >>>>>>> template >>>>>>> struct If { typedef Else type; }; >>>>>>> >>>>>>> // Metafunction whose nested value member is true if T is defined >>>>>>> // (complete), and false if T is incomplete. T must be a non-cv >>>>>>> // qualified class type. If T is incomplete at the point where this >>>>>>> // metafunction is invoked, it must never be defined elsewhere in the >>>>>>> // program. >>>>>>> template >>>>>>> class IsClassDefined { >>>>>>> typedef char yes[1]; >>>>>>> typedef char no[2]; >>>>>>> >>>>>>> template >>>>>>> static typename EnableIf::type & >>>>>>> check(U*); >>>>>>> static no& check(...); >>>>>>> >>>>>>> public: >>>>>>> static const bool value = sizeof(check((T*)0)) == sizeof(yes); >>>>>>> }; >>>>>>> >>>>>>> template >>>>>>> struct SelectPlatformBase { >>>>>>> typedef typename If< >>>>>>> IsClassDefined::value, Specific, Default>::type type; >>>>>>> }; >>>>>>> >>>>>>> // --- end SelectPlatformBase >>>>>>> >>>>>>> That's ~30 lines of code (+ tests and comments TBD) to do precisely >>>>>>> what we need, replacing ~100 lines (+ tests and comments) that have >>>>>>> various issues. (If and EnableIf should probably be hoisted out as >>>>>>> separate utilities.) We don't actually need SelectPlatformBase, and >>>>>>> could instead just directly use the If and IsClassDefined >>>>>>> metafunctions, but giving this functionality a name might be clearer. >>>>>> >>>>>> As stated, I think it?s dangerous to allow conditional inheritance >>>>>> where there is no inheritance constraint on the optional class. >>>>>> Therefore I do not think this is better, although I agree it is smaller. >>>>>> >>>>>> I agree that our use case here is smaller. If the amount of code is >>>>>> the problem and less code (and hence fewer reusable components) is >>>>>> preferred, then the original implementation with a simple >>>>>> IsBaseOfAndDerived does the trick (without checking types/cv >>>>>> qualifiers) and could be done in just a few lines, and this is what I >>>>>> originally did before you told me to expand it to more closely >>>>>> resemble external libraries: >>>>>> >>>>>> template >>>>>> class IsBaseOfAndDerived { >>>>>> typedef char yes[1]; >>>>>> typedef char no[2]; >>>>>> >>>>>> template >>>>>> static yes &check(Derived*, T); >>>>>> static no &check(Base*, int); >>>>>> >>>>>> template >>>>>> struct IsBaseOfHost { >>>>>> operator B*() const; >>>>>> operator D*(); >>>>>> }; >>>>>> public: >>>>>> static const bool value = sizeof(check(IsBaseOfHost>>>>> Derived>(), int())) == sizeof(yes); >>>>>> }; >>>>>> >>>>>> It checks the inheritance and existence of the Derived class which is >>>>>> exactly the constraints I need. >>>>>> >>>>>> If you do not want to expose it publicly and remove the reusability >>>>>> because it does not have identical semantics as third party libraries >>>>>> regarding the requirements of the Derived type to be complete, it >>>>>> could be an internal class of SelectBaseClass. >>>>>> >>>>>> I?m not going to value the reusability vs LoC, I leave that decision >>>>>> to you. >>>>>> >>>>>>> However, while I think there are other use cases for this specific >>>>>>> utility, I still think the proposed change set as a whole is adding a >>>>>>> lot of code just to avoid a a few lines of macro usage. That seems >>>>>>> like a poor tradeoff to me. >>>>>> >>>>>> >>>>>> Okay. >>>>>> >>>>>> In the end, we have to make a choice - what is more important, that >>>>>> the traits resemble some external library or code size. Because last >>>>>> time I proposed a small change and you wanted it to be closer to >>>>>> C++11 behaviour and I made it closer to that at the expense of more >>>>>> code. Now you are saying I have to make it even tighter to C++11 (or >>>>>> boost?), but also don?t want more lines of code which I believe is >>>>>> impossible given that I want to constrain both the class hierarchy to >>>>>> be reliable and check the existence of the optional class. This >>>>>> brings me to a few architectural questions which I may have (strong) >>>>>> opinions about but are not up to me to decide. >>>>>> >>>>>> 1. Do we want to replace the macros with template metaprogramming? 2 >>>>>> reviewers (and me) liked the metaprogramming approach and it was >>>>>> implemented because they (like me) did not want the macros. But you >>>>>> have a different opinion. I can?t judge who is right and wrong here. >>>>>> >>>>>> 2. Do we want to minimize the code size at the cost of reusability by >>>>>> making contracts of dependent components weaker and putting them as >>>>>> private classes in the SelectBaseClass? >>>>>> >>>>>> 3. If we do want template metaprogramming and want reusable >>>>>> components, the question is, is it important that the specification >>>>>> of completeness (which in general is almost impossible to implement >>>>>> correctly in a portable way as you pointed out) of the template >>>>>> arguments in IsBaseOf is identical as C++11/boost even though no code >>>>>> needs it to be and on the contrary some code benefits from it not to >>>>>> be? In that case which one of them do we, because they are already >>>>>> different anyway? There is a tradeoff between code size and >>>>>> complexity vs copying either one of the external libraries regarding >>>>>> requirements of completeness of template parameters. >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>> >> From david.holmes at oracle.com Thu Jan 8 03:02:25 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 08 Jan 2015 13:02:25 +1000 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> <54AA143B.2080900@oracle.com> <54ACB5A4.50603@oracle.com> <54ACBD00.6030501@oracle.com> Message-ID: <54ADF341.2070802@oracle.com> On 8/01/2015 1:58 AM, Erik ?sterlund wrote: > Hi Coleen and David, > > I was under the impression that this change was wanted to remove the > error prone conditional #define for supported specialized/generalized > atomics at almost any cost. Not at "almost any cost". :) As I said this went off in a different direction to what I had envisaged when considering inheritance. > I was fine with the #define myself but > implemented this conditional class injection since voices were raised by > reviewers to find a better way, and the actual optimization (using > #define) was only conditionally accepted, given that I would provide the > template inheritance solution to get rid of the #define after. I'm sorry you wasted your time on this, the template solution was not something I advocated. David > > Personally I don?t think the change is too complicated, especially not from the user?s point of view as you don?t really need to understand the internals to use the traits. For this specific change, only one trait is really used by atomics, the rest are dependencies designed to be reusable ?bonuses? if anyone (like myself) needs it later on (for instance for automatic closure specialization). And the trait/metafunction actually used by Atomic is SelectBaseClass which simply selects among two classes the most specific class which is defined (a general, for the AtomicBase class and optionally AtomicPlatform if implemented, otherwise ignored and AtomicBase is used as fallback). > > But of course, if this fix is perceived as complicated and that this complexity is not motivated, then I understand and we can just close the issue and leave it the way it is I guess, if reviewers are now fine with that. > > Thanks, > /Erik > >> On 07 Jan 2015, at 05:58, David Holmes wrote: >> >> I'm inclined to agree with Coleen - while I originally lamented the use of platform specific ifdefs and wondered about using simple inheritance, this has gone off in a direction I never envisaged. I don't think the problem being attacked is sufficiently bad to warrant the complexity** of the solution. And it still adds a bunch of platform-specific ifdefs. :( >> >> Sorry. >> >> David >> ----- >> >> ** that is subjective of course - if you don't understand template meta-programming then of course it seems complex. I don't know if it remains complex if you do understand template meta-programming ? >> >> On 7/01/2015 2:27 PM, Coleen Phillimore wrote: >>> >>> Thanks David, I'm working through this thread in reverse. Thank you >>> for the context. This helps a lot. >>> >>> I am not familiar with template meta-programming either. A long time >>> ago, I was a proponent of using templates but mostly as a type-checked >>> way of avoiding duplicate code. >>> >>> In this case of template meta-programming, I don't think the >>> intellectual load of the idioms offset the ugliness of the code that >>> they are replacing. Learning these idioms is not trivial. They are >>> really hard to read. Maybe if template classes had proper names rather >>> than X, Y, and Z it would be better. In any case, using this to >>> replace code isn't a clear win in clean code. There is still the >>> duplicate declarations of AtomicPlatform. There's also the unfortunate >>> conditional inclusions in shared code: >>> >>> +// Linux >>> +#ifdef TARGET_OS_ARCH_linux_x86 >>> +# include "atomic_linux_x86.hpp" >>> +#endif >>> + >>> +// Solaris >>> +#ifdef TARGET_OS_ARCH_solaris_x86 >>> +# include "atomic_solaris_x86.hpp" >>> +#endif >>> + >>> +// Windows >>> +#ifdef TARGET_OS_ARCH_windows_x86 >>> +# include "atomic_windows_x86.hpp" >>> +#endif >>> + >>> +// BSD >>> +#ifdef TARGET_OS_ARCH_bsd_x86 >>> +# include "atomic_bsd_x86.hpp" >>> +#endif >>> >>> >>> I don't think this aids maintainability or cleanliness of the code for >>> this use case. I don't think this should be added to the code base at >>> this time. >>> >>> Thanks, >>> Coleen >>> >>> On 1/4/15, 11:34 PM, David Holmes wrote: >>>> Sorry for the top-post but just wanted to reset the context a little >>>> here. Originally [1] I made the comment: >>>> >>>> "It's fine to have generic shared approaches but there really needs to >>>> be a way to allow platform specific "overrides"." >>>> >>>> For the actual original RFR I said [2]: >>>> >>>> "Can we pause and give some more thought to a clean mechanism for >>>> allowing a shared implementation if desired with the ability to >>>> override if desired. I really do not like to see CPU specific ifdefs >>>> being added to shared code. (And I would also not like to see all >>>> platforms being forced to reimplement this natively). >>>> >>>> I'm not saying we will find a simple solution, but it would be nice if >>>> we could get a few folk to think about it before proceeding with the >>>> ifdefs :)" >>>> >>>> Erik then proposed three alternative approaches [3] and the simple >>>> variant was chosen [4] and presented for review. However Paul Hohensee >>>> also made comments about an inheritance-based approach and Erik >>>> floated the first template-based variant [5] and there was some >>>> discussion with Paul and Kim. But we then ended up with the simple >>>> solution, leaving an inheritance-based solution for future work [6] >>>> (which is what is what is being discussed now). >>>> >>>> This template-based meta-programming is not something I have any >>>> familiarity with. My initial thought is this seems overkill for the >>>> situation we have with the Atomic class - but as I said I'm ignorant >>>> of the technique being used here. >>>> >>>> David >>>> ----- >>>> >>>> [1] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015292.html >>>> >>>> [2] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015303.html >>>> >>>> [3] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015362.html >>>> >>>> [4] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015401.html >>>> >>>> [5] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-October/015586.html >>>> >>>> [6] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-November/015889.html >>>> >>>> >>>> On 2/01/2015 12:07 AM, Erik ?sterlund wrote: >>>>> Hi Kim, >>>>> >>>>>> On 01 Jan 2015, at 07:42, Kim Barrett wrote: >>>>>> >>>>>> On Dec 31, 2014, at 6:12 AM, Erik ?sterlund >>>>>> wrote: >>>>>>> >>>>>>>> The rest of these comments only matter if, contrary to my suggestion >>>>>>>> above, some additional infrastructure is thought necessary. >>>>>>>> >>>>>>> >>>>>>> This is what I originally thought too and hence the current >>>>>>> solution. But voices said that using macros for this is, I quote, >>>>>>> "that bad". >>>>>>> >>>>>>>> [?] >>>>>>>> >>>>>>>> This scales with the number of AtomicPlatform definitions, rather >>>>>>>> than >>>>>>>> the number of specialized functions as happens with the existing >>>>>>>> code. >>>>>>> >>>>>>> So your proposal is to still use ugly macros but move it to the >>>>>>> class level instead, even though at the moment there is really only >>>>>>> one member function that needs it. That feels a bit meh to be >>>>>>> honest. IMO if we want to do this right, why go half the way and >>>>>>> still rely on ugly macros? >>>>>> >>>>>> Please lay off with the perjorative "ugly macros". The preprocessor >>>>>> is part of C++, and isn't going away. It's a tool that can be used, >>>>>> or abused, and I think this is a fine use. To me, "ugly" is adding >>>>>> 800+ lines of code / tests / comments to eliminate a dozen lines of >>>>>> flag macro definitions and associated #ifdef?s. >>>>> >>>>> Well people are entitled to have different opinions of course. As I >>>>> already mentioned, I did this because other reviewers (as well as me) >>>>> found it ugly to use macros for conditionally specializing, which is >>>>> the whole motivation of doing this, and was well understood from the >>>>> beginning. But I?m fine with leaving it as macros if this is in >>>>> general preferred and opinions have suddenly changed in this matter. >>>>> >>>>> Personally I don?t see how the number of rows of code define >>>>> uglyness. Reusability, testing, documentation and well definedness in >>>>> contracts/behaviours all lead to more lines of code, but I don?t >>>>> think that equals uglyness. So while I disagree more code is uglier >>>>> because it?s more, it?s all up to opinion and it?s quite pointless to >>>>> discuss; you are entitled to think that is ugly. >>>>> >>>>>> If that additional infrastructure were amortized across more use-cases >>>>>> then it might be more appealing. I might even have some additional >>>>>> use-cases for something along that line, though not in its present >>>>>> form. (More on that below.) >>>>> >>>>> Yeah I specifically had in mind the wish to make a split between >>>>> commercial extensions to open source code actually. This pattern >>>>> allows easy optional class inheritance injection as a general design >>>>> pattern to achieve this. And for my own purposes the ability to >>>>> define asymmetric dekker synchronization to elide storeload fences >>>>> when the platform (e.g. TSO) supports it but otherwise revert to a >>>>> generalized implementation (using storeload fences). This is why I >>>>> think a clean reusable strategy is almost always better because it is >>>>> widely applicable, even if there would not be obvious other examples >>>>> where it is useful. >>>>> >>>>>> >>>>>>>> I think the name "IsBaseOfAndDerived" would be better as >>>>>>>> "IsBaseAndDerived?. [?] >>>>>>>> >>>>>>>> The current implementation of IsBaseOfAndDerived (and therefore >>>>>>>> IsBaseOf) may (silently) provide incorrect answers when called with >>>>>>>> incomplete types. [?] >>>>>>>> >>>>>>>> The actual usage in this proposed change-set even relies on this >>>>>>>> flawed behavior of these metafunctions, since AtomicPlatform isn't >>>>>>>> always defined. [?] >>>>>>>> >>>>>>> >>>>>>> So what you are saying here is that I should rename >>>>>>> IsBaseOfAndDerived to IsBaseAndDerived to more closely resemble >>>>>>> other external libraries we do not use. But if I do that my >>>>>>> behaviour is "incorrect" because it is not identical to that of the >>>>>>> external library. And therefore my change would not work if it was >>>>>>> made "correctly?. >>>>>> >>>>>> The proposed IsBaseOfAndDerived and IsBaseOf are being described and >>>>>> placed as general utilities. Their names are strongly reminiscent of >>>>>> existing facilities in other libraries, including the (C++11) standard >>>>>> library. Conforming to these external standards will avoid surprises, >>>>>> especially when the proposed facilities are even described as being >>>>>> the same (except for a noted corner case that is both hard to deal >>>>>> with in a portable manner and not actually very interesting, so I'm >>>>>> not complaining about that) in the review request email. >>>>>> >>>>>> Additionally, I think quietly providing a potentially incorrect answer >>>>>> in the case of incomplete types renders the offered code quite broken >>>>>> for use as a general utility. I think the requirement for complete >>>>>> types by the well-known / standard facilities is well justified. >>>>>> >>>>> >>>>> While I disagree it is universally incorrect (as long as it is well >>>>> defined and tested) to not have the exact same requirement on >>>>> completeness on the parameters as C++11/boost, who also already have >>>>> different behaviour in this exact regard as pointed out by yourself, >>>>> I can try to fix it if it bugs you, but it will make the >>>>> infrastructure even larger which seems to be a concern. >>>>> >>>>>>> On the contrary this is a well tested feature on all our supported >>>>>>> compilers and I did not intend to engineer it to be identical to >>>>>>> some external library if it only causes trouble rather than >>>>>>> helping. This is our own class and it can do what we want it to do >>>>>>> and be called what we want it to be called, and I think that is >>>>>>> fine as long as it is well tested, which it is. At least this is my >>>>>>> opinion. Anyone else agree? >>>>>> >>>>>> Problem is, I think it's well tested code to solve the wrong problem. >>>>>> >>>>>> I suggest that what we're really looking for is not a base/derived >>>>>> relationship between a pair of classes, but that we actually want to >>>>>> determine whether a platform-specific class exists. >>>>>> >>>>>> As a result, I think there is more infrastructure in this change set >>>>>> than is actually needed. The present implementation of >>>>>> SelectBaseClass uses IsBaseOf, but doesn't need a lot of the >>>>>> functionality of that metafunction. There is no actual need for a >>>>>> Base/Derived relationship between the (possibly not defined) >>>>>> platform-specific class and the default platform-generic class, so the >>>>>> use of IsBaseOf is an unnecessary restriction that only >>>>>> serendipitously tests for defined or not, due to the previously >>>>>> discussed defect in this particular implementation. I also think the >>>>>> name SelectBaseClass is perhaps overly general. >>>>> >>>>> The main idea of SelectBaseClass is to inject an optional class into >>>>> the class hierarchy for specialization of generalized behaviour. It >>>>> is absolutely critical (and intended) that this class is constrained >>>>> to be a derived class of the fall-back class - it is not an artifact >>>>> nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, >>>>> A must be able to rely on the superclass being at least C-like so >>>>> that the behaviour of the superclass can be used reliably and >>>>> transparently regardless of the existence of the optional B. If this >>>>> contract would not hold, I think it could lead to some really weird >>>>> and unreliable code which should not be encouraged (at least not >>>>> according to the Liskov substitution principle etc). >>>>> >>>>> So what I need in the contract is a check if the optional class is >>>>> defined and if it fits in the inheritance hierarchy: they are both >>>>> important IMO. >>>>> >>>>> Yes it could have been less code. From the beginning I had a single >>>>> trait class that checked for both the inheritance and whether the >>>>> class is defined or not but you said you wanted the behaviour to more >>>>> closely match that of third party libraries. This is why there is now >>>>> a lot more code to more closely (but still not exactly) match that >>>>> behaviour. >>>>> >>>>>> I might be able to use a similar approach in some code I've been >>>>>> working on as a background task. And while writing this reply I've >>>>>> thought of another place that might benefit from something along those >>>>>> lines. Thinking about the code under review in that context, I think >>>>>> some changes would make these other possible use-cases easier and >>>>>> clearer. >>>>>> >>>>>> I believe what is actually needed is a metatfunction something like: >>>>>> >>>>>> /** >>>>>> * Metafunction whose nested type member is Default if Specific is >>>>>> * incomplete, otherwise Specific. >>>>>> * >>>>>> * Specific and Default must both be non-cv qualified class types, and >>>>>> * must not be the same type. >>>>>> * >>>>>> * If Specific is incomplete at the point where this metafunction is >>>>>> * invoked, it must never be defined elsewhere in the program. >>>>>> */ >>>>>> template >>>>>> struct SelectPlatformBase; >>>>>> >>>>>> [Note: I'm not wedded to that name.] >>>>>> >>>>>> Now, it turns out to be hard / impossible to write a generic >>>>>> is_complete metafunction, due to ODR. Whether a type is complete >>>>>> can vary between translation units, and even within a translation >>>>>> unit. Having is_complete::value have different values depending on >>>>>> where it is instantiated is an ODR violation. (One can make progress >>>>>> by using anonymous namespaces and macro wrappers with __LINE__ >>>>>> concatenation, but there is still the fundamental question of whether >>>>>> is_complete really even makes semantic sense.) >>>>> >>>>> Yes and that makes me wonder if we care enough about the completeness >>>>> semantics being the same as third party libraries (who are already >>>>> inconsistent) if we do not need them to be, and on the contrary >>>>> benefit from them not to be. >>>>> >>>>>> However, we don't need a fully general is_complete utility. We have a >>>>>> much more restricted use-case, where we want to select an optionally >>>>>> defined platform-specific class if it exists, and otherwise fall back >>>>>> to a more generic class. And we're not using this in arbitrary >>>>>> contexts, but only to select a platform-specialized implementation if >>>>>> such exists. >>>>>> >>>>>> Here's a lightly tested implementation of SelectPlatformBase: >>>>>> >>>>>> // --- SelectPlatformBase >>>>>> >>>>>> template >>>>>> struct EnableIf { typedef T type; }; >>>>>> >>>>>> template >>>>>> struct EnableIf { }; >>>>>> >>>>>> template >>>>>> struct If { typedef Then type; }; >>>>>> >>>>>> template >>>>>> struct If { typedef Else type; }; >>>>>> >>>>>> // Metafunction whose nested value member is true if T is defined >>>>>> // (complete), and false if T is incomplete. T must be a non-cv >>>>>> // qualified class type. If T is incomplete at the point where this >>>>>> // metafunction is invoked, it must never be defined elsewhere in the >>>>>> // program. >>>>>> template >>>>>> class IsClassDefined { >>>>>> typedef char yes[1]; >>>>>> typedef char no[2]; >>>>>> >>>>>> template >>>>>> static typename EnableIf::type & >>>>>> check(U*); >>>>>> static no& check(...); >>>>>> >>>>>> public: >>>>>> static const bool value = sizeof(check((T*)0)) == sizeof(yes); >>>>>> }; >>>>>> >>>>>> template >>>>>> struct SelectPlatformBase { >>>>>> typedef typename If< >>>>>> IsClassDefined::value, Specific, Default>::type type; >>>>>> }; >>>>>> >>>>>> // --- end SelectPlatformBase >>>>>> >>>>>> That's ~30 lines of code (+ tests and comments TBD) to do precisely >>>>>> what we need, replacing ~100 lines (+ tests and comments) that have >>>>>> various issues. (If and EnableIf should probably be hoisted out as >>>>>> separate utilities.) We don't actually need SelectPlatformBase, and >>>>>> could instead just directly use the If and IsClassDefined >>>>>> metafunctions, but giving this functionality a name might be clearer. >>>>> >>>>> As stated, I think it?s dangerous to allow conditional inheritance >>>>> where there is no inheritance constraint on the optional class. >>>>> Therefore I do not think this is better, although I agree it is smaller. >>>>> >>>>> I agree that our use case here is smaller. If the amount of code is >>>>> the problem and less code (and hence fewer reusable components) is >>>>> preferred, then the original implementation with a simple >>>>> IsBaseOfAndDerived does the trick (without checking types/cv >>>>> qualifiers) and could be done in just a few lines, and this is what I >>>>> originally did before you told me to expand it to more closely >>>>> resemble external libraries: >>>>> >>>>> template >>>>> class IsBaseOfAndDerived { >>>>> typedef char yes[1]; >>>>> typedef char no[2]; >>>>> >>>>> template >>>>> static yes &check(Derived*, T); >>>>> static no &check(Base*, int); >>>>> >>>>> template >>>>> struct IsBaseOfHost { >>>>> operator B*() const; >>>>> operator D*(); >>>>> }; >>>>> public: >>>>> static const bool value = sizeof(check(IsBaseOfHost>>>> Derived>(), int())) == sizeof(yes); >>>>> }; >>>>> >>>>> It checks the inheritance and existence of the Derived class which is >>>>> exactly the constraints I need. >>>>> >>>>> If you do not want to expose it publicly and remove the reusability >>>>> because it does not have identical semantics as third party libraries >>>>> regarding the requirements of the Derived type to be complete, it >>>>> could be an internal class of SelectBaseClass. >>>>> >>>>> I?m not going to value the reusability vs LoC, I leave that decision >>>>> to you. >>>>> >>>>>> However, while I think there are other use cases for this specific >>>>>> utility, I still think the proposed change set as a whole is adding a >>>>>> lot of code just to avoid a a few lines of macro usage. That seems >>>>>> like a poor tradeoff to me. >>>>> >>>>> >>>>> Okay. >>>>> >>>>> In the end, we have to make a choice - what is more important, that >>>>> the traits resemble some external library or code size. Because last >>>>> time I proposed a small change and you wanted it to be closer to >>>>> C++11 behaviour and I made it closer to that at the expense of more >>>>> code. Now you are saying I have to make it even tighter to C++11 (or >>>>> boost?), but also don?t want more lines of code which I believe is >>>>> impossible given that I want to constrain both the class hierarchy to >>>>> be reliable and check the existence of the optional class. This >>>>> brings me to a few architectural questions which I may have (strong) >>>>> opinions about but are not up to me to decide. >>>>> >>>>> 1. Do we want to replace the macros with template metaprogramming? 2 >>>>> reviewers (and me) liked the metaprogramming approach and it was >>>>> implemented because they (like me) did not want the macros. But you >>>>> have a different opinion. I can?t judge who is right and wrong here. >>>>> >>>>> 2. Do we want to minimize the code size at the cost of reusability by >>>>> making contracts of dependent components weaker and putting them as >>>>> private classes in the SelectBaseClass? >>>>> >>>>> 3. If we do want template metaprogramming and want reusable >>>>> components, the question is, is it important that the specification >>>>> of completeness (which in general is almost impossible to implement >>>>> correctly in a portable way as you pointed out) of the template >>>>> arguments in IsBaseOf is identical as C++11/boost even though no code >>>>> needs it to be and on the contrary some code benefits from it not to >>>>> be? In that case which one of them do we, because they are already >>>>> different anyway? There is a tradeoff between code size and >>>>> complexity vs copying either one of the external libraries regarding >>>>> requirements of completeness of template parameters. >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>> > From david.holmes at oracle.com Thu Jan 8 03:06:01 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 08 Jan 2015 13:06:01 +1000 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> Message-ID: <54ADF419.4030206@oracle.com> On 8/01/2015 3:07 AM, Kim Barrett wrote: > Erik Helin pointed out to me that the change to the assertion text > from "assert(...)" to "vmassert(...)" might break some existing JBS > RULE comments for matching known failures. > > I'll look at how bad this really is, but it seems likely that the > thing to do is to revert the message text back to using "assert" > rather than "vmassert". That sounds like a good idea. Thanks, David From albert.noll at oracle.com Thu Jan 8 05:32:01 2015 From: albert.noll at oracle.com (Albert Noll) Date: Thu, 08 Jan 2015 06:32:01 +0100 Subject: RFR (XS): 8068540 - [TESTBUG] Exclude failing nightly tests In-Reply-To: <54ACF505.3070506@oracle.com> References: <54AC4A50.70101@oracle.com> <54AC4B16.7010201@oracle.com> <019601d02a0a$191b8d60$4b52a820$@oracle.com> <54AC894B.30903@oracle.com> <54ACF505.3070506@oracle.com> Message-ID: <54AE1651.4010408@oracle.com> So after this change, the tests are not run on platforms although they support NMT? If yes, wouldn't it be possible to just not run the test on a platforms where NMT is not supported? Albert On 01/07/2015 09:57 AM, Jesper Wilhelmsson wrote: > We are trying to get rid of all known failures so unless the test can > be modified to run only on the platforms that support NMT (which is my > preferred option), I think it is better to exclude the tests than to > let them fail on some platforms on a regular basis. > /Jesper > > > David Holmes skrev den 7/1/15 02:18: >> On 7/01/2015 9:40 AM, Christian Tornqvist wrote: >>> Hi Coleen and George, >>> >>> @Coleen: They fail because one of our internal platforms doesn't >>> support the >>> full feature set of NMT. >> >> Excluding them completely reduces test coverage. We do we need to >> exclude them? >> They simply turn up as known failures (of which there are dozens). >> >> ?? >> >> David >> >> >>> @George: The change looks good. >>> >>> Thanks, >>> Christian >>> >>> >>> >>> -----Original Message----- >>> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On >>> Behalf Of >>> Coleen Phillimore >>> Sent: Tuesday, January 6, 2015 3:53 PM >>> To: hotspot-dev at openjdk.java.net >>> Subject: Re: RFR (XS): 8068540 - [TESTBUG] Exclude failing nightly >>> tests >>> >>> >>> Why are these tests failing? The bug is not helpful. >>> Thanks, >>> Coleen >>> >>> On 1/6/15, 3:49 PM, George Triantafillou wrote: >>>> Please review this small fix for JDK-8068540. The tests were modified >>>> with the "@ignore" jtreg tag to exclude them from nightly testing. >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8068540 >>>> webrev: http://cr.openjdk.java.net/~gtriantafill/8068540/webrev.00/ >>>> >>>> >>>> The fix was tested locally on Linux with jtreg and the JPRT hotspot >>>> testset. >>>> >>>> -George >>> >>> From coleen.phillimore at oracle.com Thu Jan 8 05:36:10 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 08 Jan 2015 00:36:10 -0500 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> <54AA143B.2080900@oracle.com> <54ACB5A4.50603@oracle.com> <54ACBD00.6030501@oracle.com> Message-ID: <54AE174A.4050204@oracle.com> Hi Erik, On 1/7/15, 10:58 AM, Erik ?sterlund wrote: > Hi Coleen and David, > > I was under the impression that this change was wanted to remove the error prone conditional #define for supported specialized/generalized atomics at almost any cost. I was fine with the #define myself but implemented this conditional class injection since voices were raised by reviewers to find a better way, and the actual optimization (using #define) was only conditionally accepted, given that I would provide the template inheritance solution to get rid of the #define after. I'm sorry for coming in late to this thread and not agreeing with the code. It's really hard to have these discussions remotely and as a distraction to other work. I don't think those of us paying loose attention realized the extent of new template features were coming into the code as part of this clean up. One webrev I saw had a small template trait (which needed explaining to me at the time). I think using this particular clean up as an opportunity to add these bonus templates is not something that we want to do. > > Personally I don?t think the change is too complicated, especially not from the user?s point of view as you don?t really need to understand the internals to use the traits. For this specific change, only one trait is really used by atomics, the rest are dependencies designed to be reusable ?bonuses? if anyone (like myself) needs it later on (for instance for automatic closure specialization). If you're adding something general to utilities, I argue that it should be well understood so that it gets the most utilization. If such a mechanism is needed for something larger like automatic closure specialization (not sure what that is) then this may be greater motivation. It depends on the benefits. Generally, if it is a clean solution to a bunch of problems, I'd be all for it. I'd also like it to be as close to standard library code as possible and I didn't get the sense of that from my reading of the discussion. This is very cool stuff, but we need to proceed with caution and make sure we're all up to speed with the new C++ world and can use it effectively. > And the trait/metafunction actually used by Atomic is SelectBaseClass which simply selects among two classes the most specific class which is defined (a general, for the AtomicBase class and optionally AtomicPlatform if implemented, otherwise ignored and AtomicBase is used as fallback). > > But of course, if this fix is perceived as complicated and that this complexity is not motivated, then I understand and we can just close the issue and leave it the way it is I guess, if reviewers are now fine with that. Too complicated is subjective, certainly, but closing this issue and not adding this code for atomics is my preference. Thanks, Coleen > > Thanks, > /Erik > >> On 07 Jan 2015, at 05:58, David Holmes wrote: >> >> I'm inclined to agree with Coleen - while I originally lamented the use of platform specific ifdefs and wondered about using simple inheritance, this has gone off in a direction I never envisaged. I don't think the problem being attacked is sufficiently bad to warrant the complexity** of the solution. And it still adds a bunch of platform-specific ifdefs. :( >> >> Sorry. >> >> David >> ----- >> >> ** that is subjective of course - if you don't understand template meta-programming then of course it seems complex. I don't know if it remains complex if you do understand template meta-programming ? >> >> On 7/01/2015 2:27 PM, Coleen Phillimore wrote: >>> Thanks David, I'm working through this thread in reverse. Thank you >>> for the context. This helps a lot. >>> >>> I am not familiar with template meta-programming either. A long time >>> ago, I was a proponent of using templates but mostly as a type-checked >>> way of avoiding duplicate code. >>> >>> In this case of template meta-programming, I don't think the >>> intellectual load of the idioms offset the ugliness of the code that >>> they are replacing. Learning these idioms is not trivial. They are >>> really hard to read. Maybe if template classes had proper names rather >>> than X, Y, and Z it would be better. In any case, using this to >>> replace code isn't a clear win in clean code. There is still the >>> duplicate declarations of AtomicPlatform. There's also the unfortunate >>> conditional inclusions in shared code: >>> >>> +// Linux >>> +#ifdef TARGET_OS_ARCH_linux_x86 >>> +# include "atomic_linux_x86.hpp" >>> +#endif >>> + >>> +// Solaris >>> +#ifdef TARGET_OS_ARCH_solaris_x86 >>> +# include "atomic_solaris_x86.hpp" >>> +#endif >>> + >>> +// Windows >>> +#ifdef TARGET_OS_ARCH_windows_x86 >>> +# include "atomic_windows_x86.hpp" >>> +#endif >>> + >>> +// BSD >>> +#ifdef TARGET_OS_ARCH_bsd_x86 >>> +# include "atomic_bsd_x86.hpp" >>> +#endif >>> >>> >>> I don't think this aids maintainability or cleanliness of the code for >>> this use case. I don't think this should be added to the code base at >>> this time. >>> >>> Thanks, >>> Coleen >>> >>> On 1/4/15, 11:34 PM, David Holmes wrote: >>>> Sorry for the top-post but just wanted to reset the context a little >>>> here. Originally [1] I made the comment: >>>> >>>> "It's fine to have generic shared approaches but there really needs to >>>> be a way to allow platform specific "overrides"." >>>> >>>> For the actual original RFR I said [2]: >>>> >>>> "Can we pause and give some more thought to a clean mechanism for >>>> allowing a shared implementation if desired with the ability to >>>> override if desired. I really do not like to see CPU specific ifdefs >>>> being added to shared code. (And I would also not like to see all >>>> platforms being forced to reimplement this natively). >>>> >>>> I'm not saying we will find a simple solution, but it would be nice if >>>> we could get a few folk to think about it before proceeding with the >>>> ifdefs :)" >>>> >>>> Erik then proposed three alternative approaches [3] and the simple >>>> variant was chosen [4] and presented for review. However Paul Hohensee >>>> also made comments about an inheritance-based approach and Erik >>>> floated the first template-based variant [5] and there was some >>>> discussion with Paul and Kim. But we then ended up with the simple >>>> solution, leaving an inheritance-based solution for future work [6] >>>> (which is what is what is being discussed now). >>>> >>>> This template-based meta-programming is not something I have any >>>> familiarity with. My initial thought is this seems overkill for the >>>> situation we have with the Atomic class - but as I said I'm ignorant >>>> of the technique being used here. >>>> >>>> David >>>> ----- >>>> >>>> [1] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015292.html >>>> >>>> [2] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015303.html >>>> >>>> [3] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015362.html >>>> >>>> [4] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015401.html >>>> >>>> [5] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-October/015586.html >>>> >>>> [6] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-November/015889.html >>>> >>>> >>>> On 2/01/2015 12:07 AM, Erik ?sterlund wrote: >>>>> Hi Kim, >>>>> >>>>>> On 01 Jan 2015, at 07:42, Kim Barrett wrote: >>>>>> >>>>>> On Dec 31, 2014, at 6:12 AM, Erik ?sterlund >>>>>> wrote: >>>>>>>> The rest of these comments only matter if, contrary to my suggestion >>>>>>>> above, some additional infrastructure is thought necessary. >>>>>>>> >>>>>>> This is what I originally thought too and hence the current >>>>>>> solution. But voices said that using macros for this is, I quote, >>>>>>> "that bad". >>>>>>> >>>>>>>> [?] >>>>>>>> >>>>>>>> This scales with the number of AtomicPlatform definitions, rather >>>>>>>> than >>>>>>>> the number of specialized functions as happens with the existing >>>>>>>> code. >>>>>>> So your proposal is to still use ugly macros but move it to the >>>>>>> class level instead, even though at the moment there is really only >>>>>>> one member function that needs it. That feels a bit meh to be >>>>>>> honest. IMO if we want to do this right, why go half the way and >>>>>>> still rely on ugly macros? >>>>>> Please lay off with the perjorative "ugly macros". The preprocessor >>>>>> is part of C++, and isn't going away. It's a tool that can be used, >>>>>> or abused, and I think this is a fine use. To me, "ugly" is adding >>>>>> 800+ lines of code / tests / comments to eliminate a dozen lines of >>>>>> flag macro definitions and associated #ifdef?s. >>>>> Well people are entitled to have different opinions of course. As I >>>>> already mentioned, I did this because other reviewers (as well as me) >>>>> found it ugly to use macros for conditionally specializing, which is >>>>> the whole motivation of doing this, and was well understood from the >>>>> beginning. But I?m fine with leaving it as macros if this is in >>>>> general preferred and opinions have suddenly changed in this matter. >>>>> >>>>> Personally I don?t see how the number of rows of code define >>>>> uglyness. Reusability, testing, documentation and well definedness in >>>>> contracts/behaviours all lead to more lines of code, but I don?t >>>>> think that equals uglyness. So while I disagree more code is uglier >>>>> because it?s more, it?s all up to opinion and it?s quite pointless to >>>>> discuss; you are entitled to think that is ugly. >>>>> >>>>>> If that additional infrastructure were amortized across more use-cases >>>>>> then it might be more appealing. I might even have some additional >>>>>> use-cases for something along that line, though not in its present >>>>>> form. (More on that below.) >>>>> Yeah I specifically had in mind the wish to make a split between >>>>> commercial extensions to open source code actually. This pattern >>>>> allows easy optional class inheritance injection as a general design >>>>> pattern to achieve this. And for my own purposes the ability to >>>>> define asymmetric dekker synchronization to elide storeload fences >>>>> when the platform (e.g. TSO) supports it but otherwise revert to a >>>>> generalized implementation (using storeload fences). This is why I >>>>> think a clean reusable strategy is almost always better because it is >>>>> widely applicable, even if there would not be obvious other examples >>>>> where it is useful. >>>>> >>>>>>>> I think the name "IsBaseOfAndDerived" would be better as >>>>>>>> "IsBaseAndDerived?. [?] >>>>>>>> >>>>>>>> The current implementation of IsBaseOfAndDerived (and therefore >>>>>>>> IsBaseOf) may (silently) provide incorrect answers when called with >>>>>>>> incomplete types. [?] >>>>>>>> >>>>>>>> The actual usage in this proposed change-set even relies on this >>>>>>>> flawed behavior of these metafunctions, since AtomicPlatform isn't >>>>>>>> always defined. [?] >>>>>>>> >>>>>>> So what you are saying here is that I should rename >>>>>>> IsBaseOfAndDerived to IsBaseAndDerived to more closely resemble >>>>>>> other external libraries we do not use. But if I do that my >>>>>>> behaviour is "incorrect" because it is not identical to that of the >>>>>>> external library. And therefore my change would not work if it was >>>>>>> made "correctly?. >>>>>> The proposed IsBaseOfAndDerived and IsBaseOf are being described and >>>>>> placed as general utilities. Their names are strongly reminiscent of >>>>>> existing facilities in other libraries, including the (C++11) standard >>>>>> library. Conforming to these external standards will avoid surprises, >>>>>> especially when the proposed facilities are even described as being >>>>>> the same (except for a noted corner case that is both hard to deal >>>>>> with in a portable manner and not actually very interesting, so I'm >>>>>> not complaining about that) in the review request email. >>>>>> >>>>>> Additionally, I think quietly providing a potentially incorrect answer >>>>>> in the case of incomplete types renders the offered code quite broken >>>>>> for use as a general utility. I think the requirement for complete >>>>>> types by the well-known / standard facilities is well justified. >>>>>> >>>>> While I disagree it is universally incorrect (as long as it is well >>>>> defined and tested) to not have the exact same requirement on >>>>> completeness on the parameters as C++11/boost, who also already have >>>>> different behaviour in this exact regard as pointed out by yourself, >>>>> I can try to fix it if it bugs you, but it will make the >>>>> infrastructure even larger which seems to be a concern. >>>>> >>>>>>> On the contrary this is a well tested feature on all our supported >>>>>>> compilers and I did not intend to engineer it to be identical to >>>>>>> some external library if it only causes trouble rather than >>>>>>> helping. This is our own class and it can do what we want it to do >>>>>>> and be called what we want it to be called, and I think that is >>>>>>> fine as long as it is well tested, which it is. At least this is my >>>>>>> opinion. Anyone else agree? >>>>>> Problem is, I think it's well tested code to solve the wrong problem. >>>>>> >>>>>> I suggest that what we're really looking for is not a base/derived >>>>>> relationship between a pair of classes, but that we actually want to >>>>>> determine whether a platform-specific class exists. >>>>>> >>>>>> As a result, I think there is more infrastructure in this change set >>>>>> than is actually needed. The present implementation of >>>>>> SelectBaseClass uses IsBaseOf, but doesn't need a lot of the >>>>>> functionality of that metafunction. There is no actual need for a >>>>>> Base/Derived relationship between the (possibly not defined) >>>>>> platform-specific class and the default platform-generic class, so the >>>>>> use of IsBaseOf is an unnecessary restriction that only >>>>>> serendipitously tests for defined or not, due to the previously >>>>>> discussed defect in this particular implementation. I also think the >>>>>> name SelectBaseClass is perhaps overly general. >>>>> The main idea of SelectBaseClass is to inject an optional class into >>>>> the class hierarchy for specialization of generalized behaviour. It >>>>> is absolutely critical (and intended) that this class is constrained >>>>> to be a derived class of the fall-back class - it is not an artifact >>>>> nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, >>>>> A must be able to rely on the superclass being at least C-like so >>>>> that the behaviour of the superclass can be used reliably and >>>>> transparently regardless of the existence of the optional B. If this >>>>> contract would not hold, I think it could lead to some really weird >>>>> and unreliable code which should not be encouraged (at least not >>>>> according to the Liskov substitution principle etc). >>>>> >>>>> So what I need in the contract is a check if the optional class is >>>>> defined and if it fits in the inheritance hierarchy: they are both >>>>> important IMO. >>>>> >>>>> Yes it could have been less code. From the beginning I had a single >>>>> trait class that checked for both the inheritance and whether the >>>>> class is defined or not but you said you wanted the behaviour to more >>>>> closely match that of third party libraries. This is why there is now >>>>> a lot more code to more closely (but still not exactly) match that >>>>> behaviour. >>>>> >>>>>> I might be able to use a similar approach in some code I've been >>>>>> working on as a background task. And while writing this reply I've >>>>>> thought of another place that might benefit from something along those >>>>>> lines. Thinking about the code under review in that context, I think >>>>>> some changes would make these other possible use-cases easier and >>>>>> clearer. >>>>>> >>>>>> I believe what is actually needed is a metatfunction something like: >>>>>> >>>>>> /** >>>>>> * Metafunction whose nested type member is Default if Specific is >>>>>> * incomplete, otherwise Specific. >>>>>> * >>>>>> * Specific and Default must both be non-cv qualified class types, and >>>>>> * must not be the same type. >>>>>> * >>>>>> * If Specific is incomplete at the point where this metafunction is >>>>>> * invoked, it must never be defined elsewhere in the program. >>>>>> */ >>>>>> template >>>>>> struct SelectPlatformBase; >>>>>> >>>>>> [Note: I'm not wedded to that name.] >>>>>> >>>>>> Now, it turns out to be hard / impossible to write a generic >>>>>> is_complete metafunction, due to ODR. Whether a type is complete >>>>>> can vary between translation units, and even within a translation >>>>>> unit. Having is_complete::value have different values depending on >>>>>> where it is instantiated is an ODR violation. (One can make progress >>>>>> by using anonymous namespaces and macro wrappers with __LINE__ >>>>>> concatenation, but there is still the fundamental question of whether >>>>>> is_complete really even makes semantic sense.) >>>>> Yes and that makes me wonder if we care enough about the completeness >>>>> semantics being the same as third party libraries (who are already >>>>> inconsistent) if we do not need them to be, and on the contrary >>>>> benefit from them not to be. >>>>> >>>>>> However, we don't need a fully general is_complete utility. We have a >>>>>> much more restricted use-case, where we want to select an optionally >>>>>> defined platform-specific class if it exists, and otherwise fall back >>>>>> to a more generic class. And we're not using this in arbitrary >>>>>> contexts, but only to select a platform-specialized implementation if >>>>>> such exists. >>>>>> >>>>>> Here's a lightly tested implementation of SelectPlatformBase: >>>>>> >>>>>> // --- SelectPlatformBase >>>>>> >>>>>> template >>>>>> struct EnableIf { typedef T type; }; >>>>>> >>>>>> template >>>>>> struct EnableIf { }; >>>>>> >>>>>> template >>>>>> struct If { typedef Then type; }; >>>>>> >>>>>> template >>>>>> struct If { typedef Else type; }; >>>>>> >>>>>> // Metafunction whose nested value member is true if T is defined >>>>>> // (complete), and false if T is incomplete. T must be a non-cv >>>>>> // qualified class type. If T is incomplete at the point where this >>>>>> // metafunction is invoked, it must never be defined elsewhere in the >>>>>> // program. >>>>>> template >>>>>> class IsClassDefined { >>>>>> typedef char yes[1]; >>>>>> typedef char no[2]; >>>>>> >>>>>> template >>>>>> static typename EnableIf::type & >>>>>> check(U*); >>>>>> static no& check(...); >>>>>> >>>>>> public: >>>>>> static const bool value = sizeof(check((T*)0)) == sizeof(yes); >>>>>> }; >>>>>> >>>>>> template >>>>>> struct SelectPlatformBase { >>>>>> typedef typename If< >>>>>> IsClassDefined::value, Specific, Default>::type type; >>>>>> }; >>>>>> >>>>>> // --- end SelectPlatformBase >>>>>> >>>>>> That's ~30 lines of code (+ tests and comments TBD) to do precisely >>>>>> what we need, replacing ~100 lines (+ tests and comments) that have >>>>>> various issues. (If and EnableIf should probably be hoisted out as >>>>>> separate utilities.) We don't actually need SelectPlatformBase, and >>>>>> could instead just directly use the If and IsClassDefined >>>>>> metafunctions, but giving this functionality a name might be clearer. >>>>> As stated, I think it?s dangerous to allow conditional inheritance >>>>> where there is no inheritance constraint on the optional class. >>>>> Therefore I do not think this is better, although I agree it is smaller. >>>>> >>>>> I agree that our use case here is smaller. If the amount of code is >>>>> the problem and less code (and hence fewer reusable components) is >>>>> preferred, then the original implementation with a simple >>>>> IsBaseOfAndDerived does the trick (without checking types/cv >>>>> qualifiers) and could be done in just a few lines, and this is what I >>>>> originally did before you told me to expand it to more closely >>>>> resemble external libraries: >>>>> >>>>> template >>>>> class IsBaseOfAndDerived { >>>>> typedef char yes[1]; >>>>> typedef char no[2]; >>>>> >>>>> template >>>>> static yes &check(Derived*, T); >>>>> static no &check(Base*, int); >>>>> >>>>> template >>>>> struct IsBaseOfHost { >>>>> operator B*() const; >>>>> operator D*(); >>>>> }; >>>>> public: >>>>> static const bool value = sizeof(check(IsBaseOfHost>>>> Derived>(), int())) == sizeof(yes); >>>>> }; >>>>> >>>>> It checks the inheritance and existence of the Derived class which is >>>>> exactly the constraints I need. >>>>> >>>>> If you do not want to expose it publicly and remove the reusability >>>>> because it does not have identical semantics as third party libraries >>>>> regarding the requirements of the Derived type to be complete, it >>>>> could be an internal class of SelectBaseClass. >>>>> >>>>> I?m not going to value the reusability vs LoC, I leave that decision >>>>> to you. >>>>> >>>>>> However, while I think there are other use cases for this specific >>>>>> utility, I still think the proposed change set as a whole is adding a >>>>>> lot of code just to avoid a a few lines of macro usage. That seems >>>>>> like a poor tradeoff to me. >>>>> >>>>> Okay. >>>>> >>>>> In the end, we have to make a choice - what is more important, that >>>>> the traits resemble some external library or code size. Because last >>>>> time I proposed a small change and you wanted it to be closer to >>>>> C++11 behaviour and I made it closer to that at the expense of more >>>>> code. Now you are saying I have to make it even tighter to C++11 (or >>>>> boost?), but also don?t want more lines of code which I believe is >>>>> impossible given that I want to constrain both the class hierarchy to >>>>> be reliable and check the existence of the optional class. This >>>>> brings me to a few architectural questions which I may have (strong) >>>>> opinions about but are not up to me to decide. >>>>> >>>>> 1. Do we want to replace the macros with template metaprogramming? 2 >>>>> reviewers (and me) liked the metaprogramming approach and it was >>>>> implemented because they (like me) did not want the macros. But you >>>>> have a different opinion. I can?t judge who is right and wrong here. >>>>> >>>>> 2. Do we want to minimize the code size at the cost of reusability by >>>>> making contracts of dependent components weaker and putting them as >>>>> private classes in the SelectBaseClass? >>>>> >>>>> 3. If we do want template metaprogramming and want reusable >>>>> components, the question is, is it important that the specification >>>>> of completeness (which in general is almost impossible to implement >>>>> correctly in a portable way as you pointed out) of the template >>>>> arguments in IsBaseOf is identical as C++11/boost even though no code >>>>> needs it to be and on the contrary some code benefits from it not to >>>>> be? In that case which one of them do we, because they are already >>>>> different anyway? There is a tradeoff between code size and >>>>> complexity vs copying either one of the external libraries regarding >>>>> requirements of completeness of template parameters. >>>>> >>>>> Thanks, >>>>> /Erik >>>>> From david.holmes at oracle.com Thu Jan 8 05:36:12 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 08 Jan 2015 15:36:12 +1000 Subject: RFR (XS): 8068540 - [TESTBUG] Exclude failing nightly tests In-Reply-To: <54AE1651.4010408@oracle.com> References: <54AC4A50.70101@oracle.com> <54AC4B16.7010201@oracle.com> <019601d02a0a$191b8d60$4b52a820$@oracle.com> <54AC894B.30903@oracle.com> <54ACF505.3070506@oracle.com> <54AE1651.4010408@oracle.com> Message-ID: <54AE174C.9000704@oracle.com> On 8/01/2015 3:32 PM, Albert Noll wrote: > So after this change, the tests are not run on platforms although they > support NMT? If yes, wouldn't it be possible to just not run the test on > a platforms where NMT is not supported? That is the crux of the problem to solve: identifying which platforms the test can run on and which they can't. At present there are three possibilities: no NMT, NMT summary only, full NMT (NMT detail). The middle case is the problem. Current proposal is to remove the middle case. David ----- > Albert > > > On 01/07/2015 09:57 AM, Jesper Wilhelmsson wrote: >> We are trying to get rid of all known failures so unless the test can >> be modified to run only on the platforms that support NMT (which is my >> preferred option), I think it is better to exclude the tests than to >> let them fail on some platforms on a regular basis. >> /Jesper >> >> >> David Holmes skrev den 7/1/15 02:18: >>> On 7/01/2015 9:40 AM, Christian Tornqvist wrote: >>>> Hi Coleen and George, >>>> >>>> @Coleen: They fail because one of our internal platforms doesn't >>>> support the >>>> full feature set of NMT. >>> >>> Excluding them completely reduces test coverage. We do we need to >>> exclude them? >>> They simply turn up as known failures (of which there are dozens). >>> >>> ?? >>> >>> David >>> >>> >>>> @George: The change looks good. >>>> >>>> Thanks, >>>> Christian >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On >>>> Behalf Of >>>> Coleen Phillimore >>>> Sent: Tuesday, January 6, 2015 3:53 PM >>>> To: hotspot-dev at openjdk.java.net >>>> Subject: Re: RFR (XS): 8068540 - [TESTBUG] Exclude failing nightly >>>> tests >>>> >>>> >>>> Why are these tests failing? The bug is not helpful. >>>> Thanks, >>>> Coleen >>>> >>>> On 1/6/15, 3:49 PM, George Triantafillou wrote: >>>>> Please review this small fix for JDK-8068540. The tests were modified >>>>> with the "@ignore" jtreg tag to exclude them from nightly testing. >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8068540 >>>>> webrev: http://cr.openjdk.java.net/~gtriantafill/8068540/webrev.00/ >>>>> >>>>> >>>>> The fix was tested locally on Linux with jtreg and the JPRT hotspot >>>>> testset. >>>>> >>>>> -George >>>> >>>> > From tobias.hartmann at oracle.com Thu Jan 8 09:07:29 2015 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 08 Jan 2015 10:07:29 +0100 Subject: RFR: 8066875: VirtualSpace does not use large pages In-Reply-To: <20141215143649.GB24542@ehelin.jrpg.bea.com> References: <54874F17.7070303@oracle.com> <548814C1.8010905@oracle.com> <548AC362.8010800@oracle.com> <20141215143649.GB24542@ehelin.jrpg.bea.com> Message-ID: <54AE48D1.2060902@oracle.com> Hi Erik, looks good (not a reviewer). Thanks, Tobias On 15.12.2014 15:36, Erik Helin wrote: > On 2014-12-12, Albert Noll wrote: >> Hi Erik, >> >> thanks a lot for taking care of this. I have one minor comment (not a >> reviewer): >> >> 1418 size_t os::largest_page_size_less_than(size_t sz) { >> 1419 if (UseLargePages) { >> 1420 // The page sizes are sorted descendingly. >> 1421 for (size_t i = 0; _page_sizes[i] != 0; ++i) { >> 1422 if (_page_sizes[i] <= sz) { >> 1423 return _page_sizes[i]; >> 1424 } >> 1425 } >> 1426 } >> 1427 return vm_page_size(); >> 1428 } >> >> The function name suggests that the function returns a page size that is >> *less* than the value in the argument (sz). >> However, in line 1422 you check for '<=' . I think you should either fix the >> name of the function or the check in line 1422. > > Yeah, I wasn't too fond of that name either. I discussed some potential > names with the guys in the office and ended up with: > - os::page_size_for_region_aligned > - os::page_size_for_region_unaligned > > os::page_size_for_region_aligned and os::page_size_for_region_unaligned > takes the same number of parameters, a region size and minimum number of > pages. The difference is that page_size_for_region_aligned guarantees > that returned page size divides the given region size, whereas > page_size_for_region_unaligned does not guarantee this. > > These names were chosen because a call to page_size_for_region_unaligned > indicates that the surrounding code must handle any alignment on its > own. > > Webrevs: > - incremental: http://cr.openjdk.java.net/~ehelin/8066875/webrev.00-01/ > - full: http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/ > > Testing: > - JPRT > - Verified that the code cache now uses large pages even if > ReservedCodeCacheSize is 241 MB (see bug for more details). > - Added new internal vm tests (also run on SPARC machine with large > pages) > - Run hotspot jtreg tests on SPARC machine with large pages > > Thanks, > Erik > >> Best, >> Albert >> >> >> On 12/10/2014 10:39 AM, Tobias Hartmann wrote: >>> Hi Erik, >>> >>> looks good (not a reviewer). >>> >>> Thanks, >>> Tobias >>> >>> On 09.12.2014 20:35, Erik Helin wrote: >>>> Hi all, >>>> >>>> the fix for JDK-8049864 [0] made os::page_size_for_region slightly more strict >>>> since the function now demands that the given region_size is size aligned with >>>> respect to the chosen page_size. The reason for doing this was that >>>> os::page_size_for_region was used in two ways: >>>> 1. Give me a suitable page size for this amount of memory >>>> 2. Give me the largest page size for this amount of memory >>>> The fix for JDK-8049864 fixed os::page_size_for_region for the "suitable page >>>> size" scenario, but is too strict for the "largest page size" scenario. This >>>> caused a regression in VirtualSpace::initialize, which only needs the largest >>>> possible page size, since VirtualSpace::initialize_with_granularity takes care >>>> of any alignment issues. >>>> >>>> This patch adds the function os::largest_page_size_less_than and updates >>>> VirtualSpace::initialize to use this new function instead of >>>> os::page_size_for_region. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~ehelin/8066875/webrev.00/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8066875 >>>> >>>> Testing: >>>> - JPRT >>>> - Verified that the code cache now uses large pages even if >>>> ReservedCodeCacheSize is 241 MB (see bug for more details). >>>> - Added new internal vm tests (also run on SPARC machine with large >>>> pages) >>>> >>>> Thanks, >>>> Erik >>>> >>>> [0]: http://hg.openjdk.java.net/jdk9/hs-gc/hotspot/rev/b326a3e8dcab >> From goetz.lindenmaier at sap.com Thu Jan 8 10:02:00 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 8 Jan 2015 10:02:00 +0000 Subject: RFR(M): 8068503: ppc64: Encode/Decode nodes for disjoint cOops mode In-Reply-To: <54AD4885.9080700@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF69B22@DEWDFEMB12A.global.corp.sap> <54AD4885.9080700@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF6C3A9@DEWDFEMB12A.global.corp.sap> Hi Coleen, thanks for this hint! We never added the optimization there because it was hardly used when we had the cppInterpreter. C2 doesn't call it, and there is no C1. But now it should be there with us having ported the template interpreter, and the C1 port is planned for a long time now ;) Unfortunately it's not easy to add in the decodes, where it's most useful, as it requires that src and dst registers are different. But I'll implement it for the encode and make a new webrev after testing it tonight. Best regards, Goetz. -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore Sent: Mittwoch, 7. Januar 2015 15:54 To: hotspot-dev at openjdk.java.net Subject: Re: RFR(M): 8068503: ppc64: Encode/Decode nodes for disjoint cOops mode Don't you have to make (or want to make) changes to MacroAssembler::decode_heap_oop_not_null in the macro assembler files? Coleen On 1/7/15, 3:26 AM, Lindenmaier, Goetz wrote: > Hi, > > this change contains the implementation (activation) of the > encode/decode nodes exploiting the disjoint heap base > available since "8064457: Introduce compressed oops mode disjoint base..." > > Please review this change. > http://cr.openjdk.java.net/~goetz/webrevs/8068503-ppcDisjoint/ > > This is ppc64-only, so I don't need a sponsor ;) > > Best regards, > Goetz. > From erik.osterlund at lnu.se Thu Jan 8 10:41:21 2015 From: erik.osterlund at lnu.se (=?Windows-1252?Q?Erik_=D6sterlund?=) Date: Thu, 8 Jan 2015 10:41:21 +0000 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <54ADF341.2070802@oracle.com> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> <54AA143B.2080900@oracle.com> <54ACB5A4.50603@oracle.com> <54ACBD00.6030501@oracle.com> <54ADF341.2070802@oracle.com> Message-ID: <99A78AA0-6E54-4595-9B82-AEF050B2990C@lnu.se> Hi David, No problem, I must have misunderstood the situation too. Sorry about that. /Erik > On 08 Jan 2015, at 04:02, David Holmes wrote: > > On 8/01/2015 1:58 AM, Erik ?sterlund wrote: >> Hi Coleen and David, >> >> I was under the impression that this change was wanted to remove the >> error prone conditional #define for supported specialized/generalized >> atomics at almost any cost. > > Not at "almost any cost". :) As I said this went off in a different direction to what I had envisaged when considering inheritance. > >> I was fine with the #define myself but >> implemented this conditional class injection since voices were raised by >> reviewers to find a better way, and the actual optimization (using >> #define) was only conditionally accepted, given that I would provide the >> template inheritance solution to get rid of the #define after. > > I'm sorry you wasted your time on this, the template solution was not something I advocated. > > David > >> >> Personally I don?t think the change is too complicated, especially not from the user?s point of view as you don?t really need to understand the internals to use the traits. For this specific change, only one trait is really used by atomics, the rest are dependencies designed to be reusable ?bonuses? if anyone (like myself) needs it later on (for instance for automatic closure specialization). And the trait/metafunction actually used by Atomic is SelectBaseClass which simply selects among two classes the most specific class which is defined (a general, for the AtomicBase class and optionally AtomicPlatform if implemented, otherwise ignored and AtomicBase is used as fallback). >> >> But of course, if this fix is perceived as complicated and that this complexity is not motivated, then I understand and we can just close the issue and leave it the way it is I guess, if reviewers are now fine with that. >> >> Thanks, >> /Erik >> >>> On 07 Jan 2015, at 05:58, David Holmes wrote: >>> >>> I'm inclined to agree with Coleen - while I originally lamented the use of platform specific ifdefs and wondered about using simple inheritance, this has gone off in a direction I never envisaged. I don't think the problem being attacked is sufficiently bad to warrant the complexity** of the solution. And it still adds a bunch of platform-specific ifdefs. :( >>> >>> Sorry. >>> >>> David >>> ----- >>> >>> ** that is subjective of course - if you don't understand template meta-programming then of course it seems complex. I don't know if it remains complex if you do understand template meta-programming ? >>> >>> On 7/01/2015 2:27 PM, Coleen Phillimore wrote: >>>> >>>> Thanks David, I'm working through this thread in reverse. Thank you >>>> for the context. This helps a lot. >>>> >>>> I am not familiar with template meta-programming either. A long time >>>> ago, I was a proponent of using templates but mostly as a type-checked >>>> way of avoiding duplicate code. >>>> >>>> In this case of template meta-programming, I don't think the >>>> intellectual load of the idioms offset the ugliness of the code that >>>> they are replacing. Learning these idioms is not trivial. They are >>>> really hard to read. Maybe if template classes had proper names rather >>>> than X, Y, and Z it would be better. In any case, using this to >>>> replace code isn't a clear win in clean code. There is still the >>>> duplicate declarations of AtomicPlatform. There's also the unfortunate >>>> conditional inclusions in shared code: >>>> >>>> +// Linux >>>> +#ifdef TARGET_OS_ARCH_linux_x86 >>>> +# include "atomic_linux_x86.hpp" >>>> +#endif >>>> + >>>> +// Solaris >>>> +#ifdef TARGET_OS_ARCH_solaris_x86 >>>> +# include "atomic_solaris_x86.hpp" >>>> +#endif >>>> + >>>> +// Windows >>>> +#ifdef TARGET_OS_ARCH_windows_x86 >>>> +# include "atomic_windows_x86.hpp" >>>> +#endif >>>> + >>>> +// BSD >>>> +#ifdef TARGET_OS_ARCH_bsd_x86 >>>> +# include "atomic_bsd_x86.hpp" >>>> +#endif >>>> >>>> >>>> I don't think this aids maintainability or cleanliness of the code for >>>> this use case. I don't think this should be added to the code base at >>>> this time. >>>> >>>> Thanks, >>>> Coleen >>>> >>>> On 1/4/15, 11:34 PM, David Holmes wrote: >>>>> Sorry for the top-post but just wanted to reset the context a little >>>>> here. Originally [1] I made the comment: >>>>> >>>>> "It's fine to have generic shared approaches but there really needs to >>>>> be a way to allow platform specific "overrides"." >>>>> >>>>> For the actual original RFR I said [2]: >>>>> >>>>> "Can we pause and give some more thought to a clean mechanism for >>>>> allowing a shared implementation if desired with the ability to >>>>> override if desired. I really do not like to see CPU specific ifdefs >>>>> being added to shared code. (And I would also not like to see all >>>>> platforms being forced to reimplement this natively). >>>>> >>>>> I'm not saying we will find a simple solution, but it would be nice if >>>>> we could get a few folk to think about it before proceeding with the >>>>> ifdefs :)" >>>>> >>>>> Erik then proposed three alternative approaches [3] and the simple >>>>> variant was chosen [4] and presented for review. However Paul Hohensee >>>>> also made comments about an inheritance-based approach and Erik >>>>> floated the first template-based variant [5] and there was some >>>>> discussion with Paul and Kim. But we then ended up with the simple >>>>> solution, leaving an inheritance-based solution for future work [6] >>>>> (which is what is what is being discussed now). >>>>> >>>>> This template-based meta-programming is not something I have any >>>>> familiarity with. My initial thought is this seems overkill for the >>>>> situation we have with the Atomic class - but as I said I'm ignorant >>>>> of the technique being used here. >>>>> >>>>> David >>>>> ----- >>>>> >>>>> [1] >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015292.html >>>>> >>>>> [2] >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015303.html >>>>> >>>>> [3] >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015362.html >>>>> >>>>> [4] >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-September/015401.html >>>>> >>>>> [5] >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-October/015586.html >>>>> >>>>> [6] >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2014-November/015889.html >>>>> >>>>> >>>>> On 2/01/2015 12:07 AM, Erik ?sterlund wrote: >>>>>> Hi Kim, >>>>>> >>>>>>> On 01 Jan 2015, at 07:42, Kim Barrett wrote: >>>>>>> >>>>>>> On Dec 31, 2014, at 6:12 AM, Erik ?sterlund >>>>>>> wrote: >>>>>>>> >>>>>>>>> The rest of these comments only matter if, contrary to my suggestion >>>>>>>>> above, some additional infrastructure is thought necessary. >>>>>>>>> >>>>>>>> >>>>>>>> This is what I originally thought too and hence the current >>>>>>>> solution. But voices said that using macros for this is, I quote, >>>>>>>> "that bad". >>>>>>>> >>>>>>>>> [?] >>>>>>>>> >>>>>>>>> This scales with the number of AtomicPlatform definitions, rather >>>>>>>>> than >>>>>>>>> the number of specialized functions as happens with the existing >>>>>>>>> code. >>>>>>>> >>>>>>>> So your proposal is to still use ugly macros but move it to the >>>>>>>> class level instead, even though at the moment there is really only >>>>>>>> one member function that needs it. That feels a bit meh to be >>>>>>>> honest. IMO if we want to do this right, why go half the way and >>>>>>>> still rely on ugly macros? >>>>>>> >>>>>>> Please lay off with the perjorative "ugly macros". The preprocessor >>>>>>> is part of C++, and isn't going away. It's a tool that can be used, >>>>>>> or abused, and I think this is a fine use. To me, "ugly" is adding >>>>>>> 800+ lines of code / tests / comments to eliminate a dozen lines of >>>>>>> flag macro definitions and associated #ifdef?s. >>>>>> >>>>>> Well people are entitled to have different opinions of course. As I >>>>>> already mentioned, I did this because other reviewers (as well as me) >>>>>> found it ugly to use macros for conditionally specializing, which is >>>>>> the whole motivation of doing this, and was well understood from the >>>>>> beginning. But I?m fine with leaving it as macros if this is in >>>>>> general preferred and opinions have suddenly changed in this matter. >>>>>> >>>>>> Personally I don?t see how the number of rows of code define >>>>>> uglyness. Reusability, testing, documentation and well definedness in >>>>>> contracts/behaviours all lead to more lines of code, but I don?t >>>>>> think that equals uglyness. So while I disagree more code is uglier >>>>>> because it?s more, it?s all up to opinion and it?s quite pointless to >>>>>> discuss; you are entitled to think that is ugly. >>>>>> >>>>>>> If that additional infrastructure were amortized across more use-cases >>>>>>> then it might be more appealing. I might even have some additional >>>>>>> use-cases for something along that line, though not in its present >>>>>>> form. (More on that below.) >>>>>> >>>>>> Yeah I specifically had in mind the wish to make a split between >>>>>> commercial extensions to open source code actually. This pattern >>>>>> allows easy optional class inheritance injection as a general design >>>>>> pattern to achieve this. And for my own purposes the ability to >>>>>> define asymmetric dekker synchronization to elide storeload fences >>>>>> when the platform (e.g. TSO) supports it but otherwise revert to a >>>>>> generalized implementation (using storeload fences). This is why I >>>>>> think a clean reusable strategy is almost always better because it is >>>>>> widely applicable, even if there would not be obvious other examples >>>>>> where it is useful. >>>>>> >>>>>>> >>>>>>>>> I think the name "IsBaseOfAndDerived" would be better as >>>>>>>>> "IsBaseAndDerived?. [?] >>>>>>>>> >>>>>>>>> The current implementation of IsBaseOfAndDerived (and therefore >>>>>>>>> IsBaseOf) may (silently) provide incorrect answers when called with >>>>>>>>> incomplete types. [?] >>>>>>>>> >>>>>>>>> The actual usage in this proposed change-set even relies on this >>>>>>>>> flawed behavior of these metafunctions, since AtomicPlatform isn't >>>>>>>>> always defined. [?] >>>>>>>>> >>>>>>>> >>>>>>>> So what you are saying here is that I should rename >>>>>>>> IsBaseOfAndDerived to IsBaseAndDerived to more closely resemble >>>>>>>> other external libraries we do not use. But if I do that my >>>>>>>> behaviour is "incorrect" because it is not identical to that of the >>>>>>>> external library. And therefore my change would not work if it was >>>>>>>> made "correctly?. >>>>>>> >>>>>>> The proposed IsBaseOfAndDerived and IsBaseOf are being described and >>>>>>> placed as general utilities. Their names are strongly reminiscent of >>>>>>> existing facilities in other libraries, including the (C++11) standard >>>>>>> library. Conforming to these external standards will avoid surprises, >>>>>>> especially when the proposed facilities are even described as being >>>>>>> the same (except for a noted corner case that is both hard to deal >>>>>>> with in a portable manner and not actually very interesting, so I'm >>>>>>> not complaining about that) in the review request email. >>>>>>> >>>>>>> Additionally, I think quietly providing a potentially incorrect answer >>>>>>> in the case of incomplete types renders the offered code quite broken >>>>>>> for use as a general utility. I think the requirement for complete >>>>>>> types by the well-known / standard facilities is well justified. >>>>>>> >>>>>> >>>>>> While I disagree it is universally incorrect (as long as it is well >>>>>> defined and tested) to not have the exact same requirement on >>>>>> completeness on the parameters as C++11/boost, who also already have >>>>>> different behaviour in this exact regard as pointed out by yourself, >>>>>> I can try to fix it if it bugs you, but it will make the >>>>>> infrastructure even larger which seems to be a concern. >>>>>> >>>>>>>> On the contrary this is a well tested feature on all our supported >>>>>>>> compilers and I did not intend to engineer it to be identical to >>>>>>>> some external library if it only causes trouble rather than >>>>>>>> helping. This is our own class and it can do what we want it to do >>>>>>>> and be called what we want it to be called, and I think that is >>>>>>>> fine as long as it is well tested, which it is. At least this is my >>>>>>>> opinion. Anyone else agree? >>>>>>> >>>>>>> Problem is, I think it's well tested code to solve the wrong problem. >>>>>>> >>>>>>> I suggest that what we're really looking for is not a base/derived >>>>>>> relationship between a pair of classes, but that we actually want to >>>>>>> determine whether a platform-specific class exists. >>>>>>> >>>>>>> As a result, I think there is more infrastructure in this change set >>>>>>> than is actually needed. The present implementation of >>>>>>> SelectBaseClass uses IsBaseOf, but doesn't need a lot of the >>>>>>> functionality of that metafunction. There is no actual need for a >>>>>>> Base/Derived relationship between the (possibly not defined) >>>>>>> platform-specific class and the default platform-generic class, so the >>>>>>> use of IsBaseOf is an unnecessary restriction that only >>>>>>> serendipitously tests for defined or not, due to the previously >>>>>>> discussed defect in this particular implementation. I also think the >>>>>>> name SelectBaseClass is perhaps overly general. >>>>>> >>>>>> The main idea of SelectBaseClass is to inject an optional class into >>>>>> the class hierarchy for specialization of generalized behaviour. It >>>>>> is absolutely critical (and intended) that this class is constrained >>>>>> to be a derived class of the fall-back class - it is not an artifact >>>>>> nor an accidental behaviour. In the hierarchy A is-maybe-a B is-a C, >>>>>> A must be able to rely on the superclass being at least C-like so >>>>>> that the behaviour of the superclass can be used reliably and >>>>>> transparently regardless of the existence of the optional B. If this >>>>>> contract would not hold, I think it could lead to some really weird >>>>>> and unreliable code which should not be encouraged (at least not >>>>>> according to the Liskov substitution principle etc). >>>>>> >>>>>> So what I need in the contract is a check if the optional class is >>>>>> defined and if it fits in the inheritance hierarchy: they are both >>>>>> important IMO. >>>>>> >>>>>> Yes it could have been less code. From the beginning I had a single >>>>>> trait class that checked for both the inheritance and whether the >>>>>> class is defined or not but you said you wanted the behaviour to more >>>>>> closely match that of third party libraries. This is why there is now >>>>>> a lot more code to more closely (but still not exactly) match that >>>>>> behaviour. >>>>>> >>>>>>> I might be able to use a similar approach in some code I've been >>>>>>> working on as a background task. And while writing this reply I've >>>>>>> thought of another place that might benefit from something along those >>>>>>> lines. Thinking about the code under review in that context, I think >>>>>>> some changes would make these other possible use-cases easier and >>>>>>> clearer. >>>>>>> >>>>>>> I believe what is actually needed is a metatfunction something like: >>>>>>> >>>>>>> /** >>>>>>> * Metafunction whose nested type member is Default if Specific is >>>>>>> * incomplete, otherwise Specific. >>>>>>> * >>>>>>> * Specific and Default must both be non-cv qualified class types, and >>>>>>> * must not be the same type. >>>>>>> * >>>>>>> * If Specific is incomplete at the point where this metafunction is >>>>>>> * invoked, it must never be defined elsewhere in the program. >>>>>>> */ >>>>>>> template >>>>>>> struct SelectPlatformBase; >>>>>>> >>>>>>> [Note: I'm not wedded to that name.] >>>>>>> >>>>>>> Now, it turns out to be hard / impossible to write a generic >>>>>>> is_complete metafunction, due to ODR. Whether a type is complete >>>>>>> can vary between translation units, and even within a translation >>>>>>> unit. Having is_complete::value have different values depending on >>>>>>> where it is instantiated is an ODR violation. (One can make progress >>>>>>> by using anonymous namespaces and macro wrappers with __LINE__ >>>>>>> concatenation, but there is still the fundamental question of whether >>>>>>> is_complete really even makes semantic sense.) >>>>>> >>>>>> Yes and that makes me wonder if we care enough about the completeness >>>>>> semantics being the same as third party libraries (who are already >>>>>> inconsistent) if we do not need them to be, and on the contrary >>>>>> benefit from them not to be. >>>>>> >>>>>>> However, we don't need a fully general is_complete utility. We have a >>>>>>> much more restricted use-case, where we want to select an optionally >>>>>>> defined platform-specific class if it exists, and otherwise fall back >>>>>>> to a more generic class. And we're not using this in arbitrary >>>>>>> contexts, but only to select a platform-specialized implementation if >>>>>>> such exists. >>>>>>> >>>>>>> Here's a lightly tested implementation of SelectPlatformBase: >>>>>>> >>>>>>> // --- SelectPlatformBase >>>>>>> >>>>>>> template >>>>>>> struct EnableIf { typedef T type; }; >>>>>>> >>>>>>> template >>>>>>> struct EnableIf { }; >>>>>>> >>>>>>> template >>>>>>> struct If { typedef Then type; }; >>>>>>> >>>>>>> template >>>>>>> struct If { typedef Else type; }; >>>>>>> >>>>>>> // Metafunction whose nested value member is true if T is defined >>>>>>> // (complete), and false if T is incomplete. T must be a non-cv >>>>>>> // qualified class type. If T is incomplete at the point where this >>>>>>> // metafunction is invoked, it must never be defined elsewhere in the >>>>>>> // program. >>>>>>> template >>>>>>> class IsClassDefined { >>>>>>> typedef char yes[1]; >>>>>>> typedef char no[2]; >>>>>>> >>>>>>> template >>>>>>> static typename EnableIf::type & >>>>>>> check(U*); >>>>>>> static no& check(...); >>>>>>> >>>>>>> public: >>>>>>> static const bool value = sizeof(check((T*)0)) == sizeof(yes); >>>>>>> }; >>>>>>> >>>>>>> template >>>>>>> struct SelectPlatformBase { >>>>>>> typedef typename If< >>>>>>> IsClassDefined::value, Specific, Default>::type type; >>>>>>> }; >>>>>>> >>>>>>> // --- end SelectPlatformBase >>>>>>> >>>>>>> That's ~30 lines of code (+ tests and comments TBD) to do precisely >>>>>>> what we need, replacing ~100 lines (+ tests and comments) that have >>>>>>> various issues. (If and EnableIf should probably be hoisted out as >>>>>>> separate utilities.) We don't actually need SelectPlatformBase, and >>>>>>> could instead just directly use the If and IsClassDefined >>>>>>> metafunctions, but giving this functionality a name might be clearer. >>>>>> >>>>>> As stated, I think it?s dangerous to allow conditional inheritance >>>>>> where there is no inheritance constraint on the optional class. >>>>>> Therefore I do not think this is better, although I agree it is smaller. >>>>>> >>>>>> I agree that our use case here is smaller. If the amount of code is >>>>>> the problem and less code (and hence fewer reusable components) is >>>>>> preferred, then the original implementation with a simple >>>>>> IsBaseOfAndDerived does the trick (without checking types/cv >>>>>> qualifiers) and could be done in just a few lines, and this is what I >>>>>> originally did before you told me to expand it to more closely >>>>>> resemble external libraries: >>>>>> >>>>>> template >>>>>> class IsBaseOfAndDerived { >>>>>> typedef char yes[1]; >>>>>> typedef char no[2]; >>>>>> >>>>>> template >>>>>> static yes &check(Derived*, T); >>>>>> static no &check(Base*, int); >>>>>> >>>>>> template >>>>>> struct IsBaseOfHost { >>>>>> operator B*() const; >>>>>> operator D*(); >>>>>> }; >>>>>> public: >>>>>> static const bool value = sizeof(check(IsBaseOfHost>>>>> Derived>(), int())) == sizeof(yes); >>>>>> }; >>>>>> >>>>>> It checks the inheritance and existence of the Derived class which is >>>>>> exactly the constraints I need. >>>>>> >>>>>> If you do not want to expose it publicly and remove the reusability >>>>>> because it does not have identical semantics as third party libraries >>>>>> regarding the requirements of the Derived type to be complete, it >>>>>> could be an internal class of SelectBaseClass. >>>>>> >>>>>> I?m not going to value the reusability vs LoC, I leave that decision >>>>>> to you. >>>>>> >>>>>>> However, while I think there are other use cases for this specific >>>>>>> utility, I still think the proposed change set as a whole is adding a >>>>>>> lot of code just to avoid a a few lines of macro usage. That seems >>>>>>> like a poor tradeoff to me. >>>>>> >>>>>> >>>>>> Okay. >>>>>> >>>>>> In the end, we have to make a choice - what is more important, that >>>>>> the traits resemble some external library or code size. Because last >>>>>> time I proposed a small change and you wanted it to be closer to >>>>>> C++11 behaviour and I made it closer to that at the expense of more >>>>>> code. Now you are saying I have to make it even tighter to C++11 (or >>>>>> boost?), but also don?t want more lines of code which I believe is >>>>>> impossible given that I want to constrain both the class hierarchy to >>>>>> be reliable and check the existence of the optional class. This >>>>>> brings me to a few architectural questions which I may have (strong) >>>>>> opinions about but are not up to me to decide. >>>>>> >>>>>> 1. Do we want to replace the macros with template metaprogramming? 2 >>>>>> reviewers (and me) liked the metaprogramming approach and it was >>>>>> implemented because they (like me) did not want the macros. But you >>>>>> have a different opinion. I can?t judge who is right and wrong here. >>>>>> >>>>>> 2. Do we want to minimize the code size at the cost of reusability by >>>>>> making contracts of dependent components weaker and putting them as >>>>>> private classes in the SelectBaseClass? >>>>>> >>>>>> 3. If we do want template metaprogramming and want reusable >>>>>> components, the question is, is it important that the specification >>>>>> of completeness (which in general is almost impossible to implement >>>>>> correctly in a portable way as you pointed out) of the template >>>>>> arguments in IsBaseOf is identical as C++11/boost even though no code >>>>>> needs it to be and on the contrary some code benefits from it not to >>>>>> be? In that case which one of them do we, because they are already >>>>>> different anyway? There is a tradeoff between code size and >>>>>> complexity vs copying either one of the external libraries regarding >>>>>> requirements of completeness of template parameters. >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>> >> From rkennke at redhat.com Thu Jan 8 11:01:21 2015 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 08 Jan 2015 12:01:21 +0100 Subject: [PATCH] Fix Shark build in JDK9 In-Reply-To: <54AE560B.3060200@oracle.com> References: <1420641903.6838.9.camel@localhost> <54AD4C6B.1070702@oracle.com> <1420645301.6838.14.camel@localhost> <54AD5675.7010101@oracle.com> <1420647107.6838.15.camel@localhost> <54AD5BDF.1060401@oracle.com> <1420648161.6838.17.camel@localhost> <54AD639A.2060200@oracle.com> <1420662102.6838.23.camel@localhost> <54AE560B.3060200@oracle.com> Message-ID: <1420714881.6838.25.camel@localhost> Hi Erik, I'm CC-ing hotspot-dev for review of Hotspot code related changes. Yes, some additional changes to Hotspot are required. This is the full set of changes needed to build and run Shark: http://cr.openjdk.java.net/~rkennke/shark-build-hotspot/webrev.01/ In detail: - In the Makefile fix a typo to be able to build unzipped debuginfo. - In ciTypeFlow.cpp only include some files and code only when building C2. I don't think that code makes sense outside of C2. (That's the issue that you've seen). - In systemDictionary.cpp, exclude some code for Shark that creates and checks native wrappers for method handle intrinsics. Invokedynamic stuff is not yet implemented in Shark, so we can't do this. - In allocation.hpp, exclude the operator overrides for new etc, LLVM does allocations itself, and this would blow up. - In handles.hpp and handles.inline.hpp, I added an argument check_in_stack so that I can disable the in-stack check for the SharkNativeWrapper. The SharkNativeWrapper is allocated on-heap and the methodHandle inside the SharkNativeWrapper. I could have excluded that check altogther using an #ifndef SHARK, but the way I implemented it seemed more finegrained. - In SharkCompiler, I pulled out the code to initialize LLVM into its own method, and the call to set_state(initialized) out of the execution-engine-lock. This would otherwise complain about wrong lock-ordering. - In SharkRuntime, I changed the cast from intptr_t to oop to work with the new picky compilers ;-) Please review. Thanks and best regards, Roman Am Donnerstag, den 08.01.2015 um 11:03 +0100 schrieb Erik Joelsson: > Hello Roman, > > This addition looks good to me. > > Thinking about what the others said, it might be inconvenient to have > all this be pushed to different forests. I tried applying all the > changes to jdk9/hs-rt, but I can't seem to build zeroshark.. Did you > have more hotspot changes to be able to finish the build? > > My failure is: > > ciTypeFlow.o > /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp > In file included from > /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/regmask.hpp:29:0, > from > /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/compile.hpp:40, > from > /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp:38: > /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/optoreg.hpp:40:39: > fatal error: adfiles/adGlobals_zero.hpp: No such file or directory > > From what I can see, adfiles are not generated for zero or zeroshark > builds, so the include should probably be removed. > > Would you still like me to push what you currently have to hs-rt? > > /Erik > > On 2015-01-07 21:21, Roman Kennke wrote: > > Hi Erik, > > > > When I built Zero and Shark on my Raspberry Pi, I noticed another > > problem when copying jvm.cfg into the right places. I fixed it in a > > similar way as I did for the SA stuff: > > > > http://cr.openjdk.java.net/~rkennke/shark-build-jdk/webrev.02/ > > > > I think that should be all for now. > > > > Please push that into JDK9 if you think that's fine. > > > > Best regards, > > Roman > > > > Am Mittwoch, den 07.01.2015 um 17:49 +0100 schrieb Erik Joelsson: > >> On 2015-01-07 17:29, Roman Kennke wrote: > >>> Am Mittwoch, den 07.01.2015 um 17:16 +0100 schrieb Erik Joelsson: > >>>> On 2015-01-07 17:11, Roman Kennke wrote: > >>>>> Hi Erik, > >>>>> > >>>>> Do you have a bug for this? > >>>>> No. > >>>>> > >>>>> I haven't pushed any changes to JDK in a while. Is it possible in the > >>>>> meantime for me to create my own bugs? Otherwise, please file one for > >>>>> me :-) > >>>> You should be able to log in to https://bugs.openjdk.java.net and create > >>>> bugs since you have an OpenJDK identity. > >>> Done: > >>> > >>> https://bugs.openjdk.java.net/browse/JDK-8068598 > >>> > >>> While I'm at it, is it possible for me to push my own changes (except > >>> hotspot of course)? If yes, what needs to be done for regenerating the > >>> configure files? Simply run autogen.sh in common/autoconf with whatever > >>> version of autotools I have? Or doesn't it make sense at all b/c you > >>> need to regenerate your closed scripts? > >> It requires you to run common/autogen.sh yes, and that will require you > >> to have autoconf 2.69 installed. But since we also need to regenerate > >> the closed version, I can take care of the push for you. Will do it > >> tomorrow if that's ok? > >> > >> /Erik > >>> Roman > >>> > > > From rkennke at redhat.com Thu Jan 8 11:05:46 2015 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 08 Jan 2015 12:05:46 +0100 Subject: [PATCH] Fix Shark build in JDK9 In-Reply-To: <54AE560B.3060200@oracle.com> References: <1420641903.6838.9.camel@localhost> <54AD4C6B.1070702@oracle.com> <1420645301.6838.14.camel@localhost> <54AD5675.7010101@oracle.com> <1420647107.6838.15.camel@localhost> <54AD5BDF.1060401@oracle.com> <1420648161.6838.17.camel@localhost> <54AD639A.2060200@oracle.com> <1420662102.6838.23.camel@localhost> <54AE560B.3060200@oracle.com> Message-ID: <1420715146.6838.27.camel@localhost> Oh and notice that if you try to build it yourself, use a version of LLVM < 3.5. In 3.5, C++11 is used/required, and OpenJDK doesn't support C++11 yet. (are there any plans about this?) I'd recommend LLVM 3.4.2. Roman Am Donnerstag, den 08.01.2015 um 11:03 +0100 schrieb Erik Joelsson: > Hello Roman, > > This addition looks good to me. > > Thinking about what the others said, it might be inconvenient to have > all this be pushed to different forests. I tried applying all the > changes to jdk9/hs-rt, but I can't seem to build zeroshark.. Did you > have more hotspot changes to be able to finish the build? > > My failure is: > > ciTypeFlow.o > /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp > In file included from > /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/regmask.hpp:29:0, > from > /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/compile.hpp:40, > from > /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp:38: > /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/optoreg.hpp:40:39: > fatal error: adfiles/adGlobals_zero.hpp: No such file or directory > > From what I can see, adfiles are not generated for zero or zeroshark > builds, so the include should probably be removed. > > Would you still like me to push what you currently have to hs-rt? > > /Erik > > On 2015-01-07 21:21, Roman Kennke wrote: > > Hi Erik, > > > > When I built Zero and Shark on my Raspberry Pi, I noticed another > > problem when copying jvm.cfg into the right places. I fixed it in a > > similar way as I did for the SA stuff: > > > > http://cr.openjdk.java.net/~rkennke/shark-build-jdk/webrev.02/ > > > > I think that should be all for now. > > > > Please push that into JDK9 if you think that's fine. > > > > Best regards, > > Roman > > > > Am Mittwoch, den 07.01.2015 um 17:49 +0100 schrieb Erik Joelsson: > >> On 2015-01-07 17:29, Roman Kennke wrote: > >>> Am Mittwoch, den 07.01.2015 um 17:16 +0100 schrieb Erik Joelsson: > >>>> On 2015-01-07 17:11, Roman Kennke wrote: > >>>>> Hi Erik, > >>>>> > >>>>> Do you have a bug for this? > >>>>> No. > >>>>> > >>>>> I haven't pushed any changes to JDK in a while. Is it possible in the > >>>>> meantime for me to create my own bugs? Otherwise, please file one for > >>>>> me :-) > >>>> You should be able to log in to https://bugs.openjdk.java.net and create > >>>> bugs since you have an OpenJDK identity. > >>> Done: > >>> > >>> https://bugs.openjdk.java.net/browse/JDK-8068598 > >>> > >>> While I'm at it, is it possible for me to push my own changes (except > >>> hotspot of course)? If yes, what needs to be done for regenerating the > >>> configure files? Simply run autogen.sh in common/autoconf with whatever > >>> version of autotools I have? Or doesn't it make sense at all b/c you > >>> need to regenerate your closed scripts? > >> It requires you to run common/autogen.sh yes, and that will require you > >> to have autoconf 2.69 installed. But since we also need to regenerate > >> the closed version, I can take care of the push for you. Will do it > >> tomorrow if that's ok? > >> > >> /Erik > >>> Roman > >>> > > > From david.holmes at oracle.com Thu Jan 8 11:20:55 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 08 Jan 2015 21:20:55 +1000 Subject: [PATCH] Fix Shark build in JDK9 In-Reply-To: <1420715146.6838.27.camel@localhost> References: <1420641903.6838.9.camel@localhost> <54AD4C6B.1070702@oracle.com> <1420645301.6838.14.camel@localhost> <54AD5675.7010101@oracle.com> <1420647107.6838.15.camel@localhost> <54AD5BDF.1060401@oracle.com> <1420648161.6838.17.camel@localhost> <54AD639A.2060200@oracle.com> <1420662102.6838.23.camel@localhost> <54AE560B.3060200@oracle.com> <1420715146.6838.27.camel@localhost> Message-ID: <54AE6817.1080206@oracle.com> On 8/01/2015 9:05 PM, Roman Kennke wrote: > Oh and notice that if you try to build it yourself, use a version of > LLVM < 3.5. In 3.5, C++11 is used/required, and OpenJDK doesn't support > C++11 yet. (are there any plans about this?) I'd recommend LLVM 3.4.2. C++11? We still have workarounds for compilers that don't support C99 :( David > Roman > > Am Donnerstag, den 08.01.2015 um 11:03 +0100 schrieb Erik Joelsson: >> Hello Roman, >> >> This addition looks good to me. >> >> Thinking about what the others said, it might be inconvenient to have >> all this be pushed to different forests. I tried applying all the >> changes to jdk9/hs-rt, but I can't seem to build zeroshark.. Did you >> have more hotspot changes to be able to finish the build? >> >> My failure is: >> >> ciTypeFlow.o >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp >> In file included from >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/regmask.hpp:29:0, >> from >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/compile.hpp:40, >> from >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp:38: >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/optoreg.hpp:40:39: >> fatal error: adfiles/adGlobals_zero.hpp: No such file or directory >> >> From what I can see, adfiles are not generated for zero or zeroshark >> builds, so the include should probably be removed. >> >> Would you still like me to push what you currently have to hs-rt? >> >> /Erik >> >> On 2015-01-07 21:21, Roman Kennke wrote: >>> Hi Erik, >>> >>> When I built Zero and Shark on my Raspberry Pi, I noticed another >>> problem when copying jvm.cfg into the right places. I fixed it in a >>> similar way as I did for the SA stuff: >>> >>> http://cr.openjdk.java.net/~rkennke/shark-build-jdk/webrev.02/ >>> >>> I think that should be all for now. >>> >>> Please push that into JDK9 if you think that's fine. >>> >>> Best regards, >>> Roman >>> >>> Am Mittwoch, den 07.01.2015 um 17:49 +0100 schrieb Erik Joelsson: >>>> On 2015-01-07 17:29, Roman Kennke wrote: >>>>> Am Mittwoch, den 07.01.2015 um 17:16 +0100 schrieb Erik Joelsson: >>>>>> On 2015-01-07 17:11, Roman Kennke wrote: >>>>>>> Hi Erik, >>>>>>> >>>>>>> Do you have a bug for this? >>>>>>> No. >>>>>>> >>>>>>> I haven't pushed any changes to JDK in a while. Is it possible in the >>>>>>> meantime for me to create my own bugs? Otherwise, please file one for >>>>>>> me :-) >>>>>> You should be able to log in to https://bugs.openjdk.java.net and create >>>>>> bugs since you have an OpenJDK identity. >>>>> Done: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8068598 >>>>> >>>>> While I'm at it, is it possible for me to push my own changes (except >>>>> hotspot of course)? If yes, what needs to be done for regenerating the >>>>> configure files? Simply run autogen.sh in common/autoconf with whatever >>>>> version of autotools I have? Or doesn't it make sense at all b/c you >>>>> need to regenerate your closed scripts? >>>> It requires you to run common/autogen.sh yes, and that will require you >>>> to have autoconf 2.69 installed. But since we also need to regenerate >>>> the closed version, I can take care of the push for you. Will do it >>>> tomorrow if that's ok? >>>> >>>> /Erik >>>>> Roman >>>>> >>> >> > > From rkennke at redhat.com Thu Jan 8 11:31:59 2015 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 08 Jan 2015 12:31:59 +0100 Subject: [PATCH] Fix Shark build in JDK9 In-Reply-To: <54AE6817.1080206@oracle.com> References: <1420641903.6838.9.camel@localhost> <54AD4C6B.1070702@oracle.com> <1420645301.6838.14.camel@localhost> <54AD5675.7010101@oracle.com> <1420647107.6838.15.camel@localhost> <54AD5BDF.1060401@oracle.com> <1420648161.6838.17.camel@localhost> <54AD639A.2060200@oracle.com> <1420662102.6838.23.camel@localhost> <54AE560B.3060200@oracle.com> <1420715146.6838.27.camel@localhost> <54AE6817.1080206@oracle.com> Message-ID: <1420716719.6838.30.camel@localhost> Am Donnerstag, den 08.01.2015 um 21:20 +1000 schrieb David Holmes: > On 8/01/2015 9:05 PM, Roman Kennke wrote: > > Oh and notice that if you try to build it yourself, use a version of > > LLVM < 3.5. In 3.5, C++11 is used/required, and OpenJDK doesn't support > > C++11 yet. (are there any plans about this?) I'd recommend LLVM 3.4.2. > > C++11? We still have workarounds for compilers that don't support C99 :( Well ok, fair enough ;-) I guess for my purposes it's good enough to pass some GCC flag to turn on C++11 support, so that the new LLVM headers can be used. That's all that I need. Roman > > David > > > Roman > > > > Am Donnerstag, den 08.01.2015 um 11:03 +0100 schrieb Erik Joelsson: > >> Hello Roman, > >> > >> This addition looks good to me. > >> > >> Thinking about what the others said, it might be inconvenient to have > >> all this be pushed to different forests. I tried applying all the > >> changes to jdk9/hs-rt, but I can't seem to build zeroshark.. Did you > >> have more hotspot changes to be able to finish the build? > >> > >> My failure is: > >> > >> ciTypeFlow.o > >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp > >> In file included from > >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/regmask.hpp:29:0, > >> from > >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/compile.hpp:40, > >> from > >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp:38: > >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/optoreg.hpp:40:39: > >> fatal error: adfiles/adGlobals_zero.hpp: No such file or directory > >> > >> From what I can see, adfiles are not generated for zero or zeroshark > >> builds, so the include should probably be removed. > >> > >> Would you still like me to push what you currently have to hs-rt? > >> > >> /Erik > >> > >> On 2015-01-07 21:21, Roman Kennke wrote: > >>> Hi Erik, > >>> > >>> When I built Zero and Shark on my Raspberry Pi, I noticed another > >>> problem when copying jvm.cfg into the right places. I fixed it in a > >>> similar way as I did for the SA stuff: > >>> > >>> http://cr.openjdk.java.net/~rkennke/shark-build-jdk/webrev.02/ > >>> > >>> I think that should be all for now. > >>> > >>> Please push that into JDK9 if you think that's fine. > >>> > >>> Best regards, > >>> Roman > >>> > >>> Am Mittwoch, den 07.01.2015 um 17:49 +0100 schrieb Erik Joelsson: > >>>> On 2015-01-07 17:29, Roman Kennke wrote: > >>>>> Am Mittwoch, den 07.01.2015 um 17:16 +0100 schrieb Erik Joelsson: > >>>>>> On 2015-01-07 17:11, Roman Kennke wrote: > >>>>>>> Hi Erik, > >>>>>>> > >>>>>>> Do you have a bug for this? > >>>>>>> No. > >>>>>>> > >>>>>>> I haven't pushed any changes to JDK in a while. Is it possible in the > >>>>>>> meantime for me to create my own bugs? Otherwise, please file one for > >>>>>>> me :-) > >>>>>> You should be able to log in to https://bugs.openjdk.java.net and create > >>>>>> bugs since you have an OpenJDK identity. > >>>>> Done: > >>>>> > >>>>> https://bugs.openjdk.java.net/browse/JDK-8068598 > >>>>> > >>>>> While I'm at it, is it possible for me to push my own changes (except > >>>>> hotspot of course)? If yes, what needs to be done for regenerating the > >>>>> configure files? Simply run autogen.sh in common/autoconf with whatever > >>>>> version of autotools I have? Or doesn't it make sense at all b/c you > >>>>> need to regenerate your closed scripts? > >>>> It requires you to run common/autogen.sh yes, and that will require you > >>>> to have autoconf 2.69 installed. But since we also need to regenerate > >>>> the closed version, I can take care of the push for you. Will do it > >>>> tomorrow if that's ok? > >>>> > >>>> /Erik > >>>>> Roman > >>>>> > >>> > >> > > > > From aph at redhat.com Thu Jan 8 11:33:19 2015 From: aph at redhat.com (Andrew Haley) Date: Thu, 08 Jan 2015 11:33:19 +0000 Subject: [PATCH] Fix Shark build in JDK9 In-Reply-To: <1420716719.6838.30.camel@localhost> References: <1420641903.6838.9.camel@localhost> <54AD4C6B.1070702@oracle.com> <1420645301.6838.14.camel@localhost> <54AD5675.7010101@oracle.com> <1420647107.6838.15.camel@localhost> <54AD5BDF.1060401@oracle.com> <1420648161.6838.17.camel@localhost> <54AD639A.2060200@oracle.com> <1420662102.6838.23.camel@localhost> <54AE560B.3060200@oracle.com> <1420715146.6838.27.camel@localhost> <54AE6817.1080206@oracle.com> <1420716719.6838.30.camel@localhost> Message-ID: <54AE6AFF.4020209@redhat.com> On 01/08/2015 11:31 AM, Roman Kennke wrote: > I guess for my purposes it's good enough to pass some GCC flag to turn > on C++11 support, so that the new LLVM headers can be used. That's all > that I need. But does that work without further changes? Andrew. From erik.joelsson at oracle.com Thu Jan 8 11:38:54 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 08 Jan 2015 12:38:54 +0100 Subject: [PATCH] Fix Shark build in JDK9 In-Reply-To: <1420714881.6838.25.camel@localhost> References: <1420641903.6838.9.camel@localhost> <54AD4C6B.1070702@oracle.com> <1420645301.6838.14.camel@localhost> <54AD5675.7010101@oracle.com> <1420647107.6838.15.camel@localhost> <54AD5BDF.1060401@oracle.com> <1420648161.6838.17.camel@localhost> <54AD639A.2060200@oracle.com> <1420662102.6838.23.camel@localhost> <54AE560B.3060200@oracle.com> <1420714881.6838.25.camel@localhost> Message-ID: <54AE6C4E.9070605@oracle.com> Hello Roman, Thanks for the full patch! I discovered a problem with the top repo patch. When autoconf (using m4) evaluates your change, the brackets disappear. To fix this, more brackets need to be added. Here is a version that works for me that will make the correct sed expression appear in the generated file: diff -r 7063bdada583 common/autoconf/libraries.m4 --- a/common/autoconf/libraries.m4 +++ b/common/autoconf/libraries.m4 @@ -1080,7 +1080,7 @@ fi fi done - llvm_version=$("${LLVM_CONFIG}" --version | sed 's/\.//; s/svn.*//') + llvm_version=$("${LLVM_CONFIG}" --version | sed [ 's/\(^[0-9]\)*.\([0-9]*\).*/\1\2/; s/svn.*//' ]) LLVM_CFLAGS="${LLVM_CFLAGS} -DSHARK_LLVM_VERSION=${llvm_version}" unset LLVM_LDFLAGS I tried building locally (Ubuntu 14.04 with llvm 3.4) but it failed with undefined references at link time: /usr/lib/llvm-3.4/lib/libLLVMSupport.a(Process.o): In function `llvm::sys::Process::FileDescriptorHasColors(int)': (.text+0x5b7): undefined reference to `setupterm' /usr/lib/llvm-3.4/lib/libLLVMSupport.a(Process.o): In function `llvm::sys::Process::FileDescriptorHasColors(int)': (.text+0x5e0): undefined reference to `tigetnum' /usr/lib/llvm-3.4/lib/libLLVMSupport.a(Process.o): In function `llvm::sys::Process::FileDescriptorHasColors(int)': (.text+0x5e9): undefined reference to `set_curterm' /usr/lib/llvm-3.4/lib/libLLVMSupport.a(Process.o): In function `llvm::sys::Process::FileDescriptorHasColors(int)': (.text+0x5f1): undefined reference to `del_curterm' collect2: error: ld returned 1 exit status I'm not familiar enough with the hotspot source changes to provide a meaningful review, but when it's approved I could still push the whole thing. /Erik On 2015-01-08 12:01, Roman Kennke wrote: > Hi Erik, > > I'm CC-ing hotspot-dev for review of Hotspot code related changes. > > Yes, some additional changes to Hotspot are required. This is the full > set of changes needed to build and run Shark: > > http://cr.openjdk.java.net/~rkennke/shark-build-hotspot/webrev.01/ > > In detail: > > - In the Makefile fix a typo to be able to build unzipped debuginfo. > - In ciTypeFlow.cpp only include some files and code only when building > C2. I don't think that code makes sense outside of C2. (That's the issue > that you've seen). > - In systemDictionary.cpp, exclude some code for Shark that creates and > checks native wrappers for method handle intrinsics. Invokedynamic stuff > is not yet implemented in Shark, so we can't do this. > - In allocation.hpp, exclude the operator overrides for new etc, LLVM > does allocations itself, and this would blow up. > - In handles.hpp and handles.inline.hpp, I added an argument > check_in_stack so that I can disable the in-stack check for the > SharkNativeWrapper. The SharkNativeWrapper is allocated on-heap and the > methodHandle inside the SharkNativeWrapper. I could have excluded that > check altogther using an #ifndef SHARK, but the way I implemented it > seemed more finegrained. > - In SharkCompiler, I pulled out the code to initialize LLVM into its > own method, and the call to set_state(initialized) out of the > execution-engine-lock. This would otherwise complain about wrong > lock-ordering. > - In SharkRuntime, I changed the cast from intptr_t to oop to work with > the new picky compilers ;-) > > Please review. > > Thanks and best regards, > Roman > > Am Donnerstag, den 08.01.2015 um 11:03 +0100 schrieb Erik Joelsson: >> Hello Roman, >> >> This addition looks good to me. >> >> Thinking about what the others said, it might be inconvenient to have >> all this be pushed to different forests. I tried applying all the >> changes to jdk9/hs-rt, but I can't seem to build zeroshark.. Did you >> have more hotspot changes to be able to finish the build? >> >> My failure is: >> >> ciTypeFlow.o >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp >> In file included from >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/regmask.hpp:29:0, >> from >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/compile.hpp:40, >> from >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp:38: >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/optoreg.hpp:40:39: >> fatal error: adfiles/adGlobals_zero.hpp: No such file or directory >> >> From what I can see, adfiles are not generated for zero or zeroshark >> builds, so the include should probably be removed. >> >> Would you still like me to push what you currently have to hs-rt? >> >> /Erik >> >> On 2015-01-07 21:21, Roman Kennke wrote: >>> Hi Erik, >>> >>> When I built Zero and Shark on my Raspberry Pi, I noticed another >>> problem when copying jvm.cfg into the right places. I fixed it in a >>> similar way as I did for the SA stuff: >>> >>> http://cr.openjdk.java.net/~rkennke/shark-build-jdk/webrev.02/ >>> >>> I think that should be all for now. >>> >>> Please push that into JDK9 if you think that's fine. >>> >>> Best regards, >>> Roman >>> >>> Am Mittwoch, den 07.01.2015 um 17:49 +0100 schrieb Erik Joelsson: >>>> On 2015-01-07 17:29, Roman Kennke wrote: >>>>> Am Mittwoch, den 07.01.2015 um 17:16 +0100 schrieb Erik Joelsson: >>>>>> On 2015-01-07 17:11, Roman Kennke wrote: >>>>>>> Hi Erik, >>>>>>> >>>>>>> Do you have a bug for this? >>>>>>> No. >>>>>>> >>>>>>> I haven't pushed any changes to JDK in a while. Is it possible in the >>>>>>> meantime for me to create my own bugs? Otherwise, please file one for >>>>>>> me :-) >>>>>> You should be able to log in to https://bugs.openjdk.java.net and create >>>>>> bugs since you have an OpenJDK identity. >>>>> Done: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8068598 >>>>> >>>>> While I'm at it, is it possible for me to push my own changes (except >>>>> hotspot of course)? If yes, what needs to be done for regenerating the >>>>> configure files? Simply run autogen.sh in common/autoconf with whatever >>>>> version of autotools I have? Or doesn't it make sense at all b/c you >>>>> need to regenerate your closed scripts? >>>> It requires you to run common/autogen.sh yes, and that will require you >>>> to have autoconf 2.69 installed. But since we also need to regenerate >>>> the closed version, I can take care of the push for you. Will do it >>>> tomorrow if that's ok? >>>> >>>> /Erik >>>>> Roman >>>>> > From aph at redhat.com Thu Jan 8 11:54:11 2015 From: aph at redhat.com (Andrew Haley) Date: Thu, 08 Jan 2015 11:54:11 +0000 Subject: [PATCH] Fix Shark build in JDK9 In-Reply-To: <54AE6C4E.9070605@oracle.com> References: <1420641903.6838.9.camel@localhost> <54AD4C6B.1070702@oracle.com> <1420645301.6838.14.camel@localhost> <54AD5675.7010101@oracle.com> <1420647107.6838.15.camel@localhost> <54AD5BDF.1060401@oracle.com> <1420648161.6838.17.camel@localhost> <54AD639A.2060200@oracle.com> <1420662102.6838.23.camel@localhost> <54AE560B.3060200@oracle.com> <1420714881.6838.25.camel@localhost> <54AE6C4E.9070605@oracle.com> Message-ID: <54AE6FE3.5090504@redhat.com> On 01/08/2015 11:38 AM, Erik Joelsson wrote: > Hello Roman, > > Thanks for the full patch! > > I discovered a problem with the top repo patch. When autoconf (using m4) > evaluates your change, the brackets disappear. To fix this, more > brackets need to be added. Here is a version that works for me that will > make the correct sed expression appear in the generated file: > > diff -r 7063bdada583 common/autoconf/libraries.m4 > --- a/common/autoconf/libraries.m4 > +++ b/common/autoconf/libraries.m4 > @@ -1080,7 +1080,7 @@ > fi > fi > done > - llvm_version=$("${LLVM_CONFIG}" --version | sed 's/\.//; s/svn.*//') > + llvm_version=$("${LLVM_CONFIG}" --version | sed [ > 's/\(^[0-9]\)*.\([0-9]*\).*/\1\2/; s/svn.*//' ]) > LLVM_CFLAGS="${LLVM_CFLAGS} -DSHARK_LLVM_VERSION=${llvm_version}" > > unset LLVM_LDFLAGS > > I tried building locally (Ubuntu 14.04 with llvm 3.4) but it failed with > undefined references at link time: > > /usr/lib/llvm-3.4/lib/libLLVMSupport.a(Process.o): In function > `llvm::sys::Process::FileDescriptorHasColors(int)': > (.text+0x5b7): undefined reference to `setupterm' > /usr/lib/llvm-3.4/lib/libLLVMSupport.a(Process.o): In function > `llvm::sys::Process::FileDescriptorHasColors(int)': > (.text+0x5e0): undefined reference to `tigetnum' > /usr/lib/llvm-3.4/lib/libLLVMSupport.a(Process.o): In function > `llvm::sys::Process::FileDescriptorHasColors(int)': > (.text+0x5e9): undefined reference to `set_curterm' > /usr/lib/llvm-3.4/lib/libLLVMSupport.a(Process.o): In function > `llvm::sys::Process::FileDescriptorHasColors(int)': > (.text+0x5f1): undefined reference to `del_curterm' > collect2: error: ld returned 1 exit status That should be in the ncurses-devel package (or whatever it's called on Ubuntu) which I thought was an OpenJDK requirement anyway. Maybe the only problem is that we're not linking with -lncurses. Andrew. From erik.osterlund at lnu.se Thu Jan 8 13:12:38 2015 From: erik.osterlund at lnu.se (=?Windows-1252?Q?Erik_=D6sterlund?=) Date: Thu, 8 Jan 2015 13:12:38 +0000 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <54AE174A.4050204@oracle.com> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> <54AA143B.2080900@oracle.com> <54ACB5A4.50603@oracle.com> <54ACBD00.6030501@oracle.com> <54AE174A.4050204@oracle.com> Message-ID: <5749133C-5087-457B-A0E1-7F21CF1825BD@lnu.se> Hi Coleen, > On 08 Jan 2015, at 06:36, Coleen Phillimore wrote: > > I'm sorry for coming in late to this thread and not agreeing with the code. It's really hard to have these discussions remotely and as a distraction to other work. I completely understand, no worries. > I don't think those of us paying loose attention realized the extent of new template features were coming into the code as part of this clean up. One webrev I saw had a small template trait (which needed explaining to me at the time). I think using this particular clean up as an opportunity to add these bonus templates is not something that we want to do. As pointed out by Kim earlier, the webrev massively exaggerates ?the extent?. It?s really only ~100 LoC template stuff with a bunch of copyright boilerplate and testing, which are not negative maintenance overheads (we can all agree it?s a bad idea to remove testing and bunch everything together in one file to make it look less extensive right?). Actually this whole code size thing has been a bit confusing. First Kim said it looked rather extensive too. So I made a code minimal version (20 LoC only to get the job done but less reusability) and a reuse max version (~100 LoC). Then after looking closer, Kim concluded code size probably was not a big issue after closer inspection (copyrights, testing). So I dropped the code minimum version after that was ruled out as an issue. And now other voices are raised again about the perceived extensiveness of the solution. > If you're adding something general to utilities, I argue that it should be well understood so that it gets the most utilization. If such a mechanism is needed for something larger like automatic closure specialization (not sure what that is) then this may be greater motivation. It depends on the benefits. I thought the decision was already made to not go ahead with this. But if you want to know, I can give a few examples of things that benefits from the proposed template infrastructure with brief descriptions. The uninterested reader can skip ahead. ;) ** BEGIN BENEFITS ** 1) Automatic Closure Specialization: In the GC code, OopClosures and ExtendedOopClosures do arbitrary stuff with oops. It?s a neat abstraction but since the stuff to be done is arbitrary, it results in a virtual call. To work around this, mad people driven by dark forces invented a manual closure specialization mechanism to remove the virtual calls to the closures. It comes with a threefold of problems: 1) It requires closures to be manually added to a macro to be specialized (and hence get rid of the virtual call), 2) This technique fundamentally cannot handle composite closure types (closure filtering oop then calling another closure). It loses type information and knowledge of the composite structure is lost. 3) It smells. Using template metaprogramming (and some of the traits in this change) all those problems can be resolved: automatic specialization without manual labour, full type awareness meaning composite closure types can be specialized too, much cleaner, and we won?t have to define both do_oop and do_oop_nv all over the place. No problem for me to add stuff needed here at a later time though. 2) OrderAccess refactoring: OrderAccess has a number of problems in terms of maintainability and correctness, and I?m currently cleaning it up. There are two main issues, but I think they are related. (I suspect correctness is inconsistent due to poor maintainability) 2.a) OrderAccess Maintainability: Different concerns are now joined to a big blob: Which barrier to use (acquire, release, fence) and where to put it w.r.t. to memory accesses, how to implement those barriers, how to make atomic memory accesses, how to specialize more efficient variants are all different concerns, now flattened out to be done over and over again for each platform even though the behaviour is very general. I intend to move everything except specialization and barrier implementation to shared code and separate their concerns, i.e. how to make atomic memory accesses and where to put the barriers is the same for everyone except the specialized variants. This removes most of the code for most of the individual platforms and makes it easier to reason about the different concerns in isolation without changing stuff all over the place. To still allow specializations with inline assembly that deviate from the norm, this conditional class selection is once again needed and could benefit from the exact same pattern I intended to push here for specialization and generalization. 2.b) OrderAccess Correctness: Correctness concerns are inconsistent over different platforms and dangerous which is my second issue with OrderAccess. Specifically, for TSO architectures like SPARC and x86, OrderAccess relies on C++ volatile for acquire release semantics, which in most cases is wrong. In every compiler I know of except MSVC 2008, this can not be relied upon as the C++ standard section 1.9 is vague whether volatiles may be reordered w.r.t. non-volatiles and has been interpreted differently by compiler implementors. It only really promises reordering constraints between volatiles which is not enough for acquire release semantics. As a matter of fact, GCC explicitly states they may be arbitrarily reordered at will, and therefore we need compiler barriers to prevent this. This problem was observed on linux and fixed by Mikael, on Linux. But the exact same problem (relying on volatile to have acquire release semantics) still lurks around on the other TSO architectures, waiting for the right moment to strike. With this refactoring, correcting such an issue (which I also intend to do because I think the current implementation is flawed) can be easily done with 3 LoC per platform - the barrier selection for memory accesses, instead of for each type of memory access on nearly every TSO platform. This will also make a later transition to C++11/14 easier when compilers are ready, also with only a few LoC. So in summary the provided metatemplating stuff could help automating closure specialization, it could help generalize/specialize Atomics (in review now), and it could help generalize/specialize OrderedAccess which IMO needs refactoring. There are also some others on the top of my head as well like removing the fence in G1?s post reference write barrier using asymmetric dekker synchronization when the platform supports it using the same reusable generalization/specialization pattern specifying which fencing mechanism to use, if we decide we want to do that, and potentially using it as a straight forward pattern for building commercial variation points to the open source code without affecting the open source code base. And Kim had some additional uses too. Of course, #2 can use conditional #define too to get the job done which is what I?m doing now since this idea was not appreciated in the end, just saying there are more uses as you asked. ** END BENEFITS ** I did not elect to go for a reusable trait variant only because metaprogramming is fun. I genuinely think they are powerful tools that we could benefit from in different areas in runtime and gc. :) > Generally, if it is a clean solution to a bunch of problems, I'd be all for it. I'd also like it to be as close to standard library code as possible and I didn't get the sense of that from my reading of the discussion. I disagreed with Kim at one point (details not necessary I think) about the necessity for compliance with standard library code at a specific point, but that has already been fixed in favour of maximum compliance, which in the end everyone were happy with. > This is very cool stuff, but we need to proceed with caution and make sure we're all up to speed with the new C++ world and can use it effectively. I think this looks like a chicken and egg problem here. We need the tools to be available to learn how to use them and integrate them to our work flow. But we won?t make the tools available unless we know how to use them in our work flow. :/ > Too complicated is subjective, certainly, but closing this issue and not adding this code for atomics is my preference. 2 reviewers accepted (Volker and Paul), 2 reviewers rejected (you I take it, and David), 1 neutral (Kim) but willing to accept after an argument about whether the specialized and generalized classes should be connected by inheritance by contract (decided to remove the check and with it some traits. I'm in the middle, optimistic about the solution but confused as to how this discussion got so long it broke my email client. ;) I am fine with this decision. I never imagined this (~100 LoC metaprogramming) would lead to such a long discussion. If I knew the extent of the discussion, I would never have suggested it. I?m perfectly fine with leaving this idea behind if opinions are a bit split on this one. Thanks, /Erik From erik.joelsson at oracle.com Thu Jan 8 13:35:32 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 08 Jan 2015 14:35:32 +0100 Subject: [PATCH] Fix Shark build in JDK9 In-Reply-To: <54AE6FE3.5090504@redhat.com> References: <1420641903.6838.9.camel@localhost> <54AD4C6B.1070702@oracle.com> <1420645301.6838.14.camel@localhost> <54AD5675.7010101@oracle.com> <1420647107.6838.15.camel@localhost> <54AD5BDF.1060401@oracle.com> <1420648161.6838.17.camel@localhost> <54AD639A.2060200@oracle.com> <1420662102.6838.23.camel@localhost> <54AE560B.3060200@oracle.com> <1420714881.6838.25.camel@localhost> <54AE6C4E.9070605@oracle.com> <54AE6FE3.5090504@redhat.com> Message-ID: <54AE87A4.1060703@oracle.com> On 2015-01-08 12:54, Andrew Haley wrote: > On 01/08/2015 11:38 AM, Erik Joelsson wrote: >> I tried building locally (Ubuntu 14.04 with llvm 3.4) but it failed with >> undefined references at link time: >> >> /usr/lib/llvm-3.4/lib/libLLVMSupport.a(Process.o): In function >> `llvm::sys::Process::FileDescriptorHasColors(int)': >> (.text+0x5b7): undefined reference to `setupterm' >> /usr/lib/llvm-3.4/lib/libLLVMSupport.a(Process.o): In function >> `llvm::sys::Process::FileDescriptorHasColors(int)': >> (.text+0x5e0): undefined reference to `tigetnum' >> /usr/lib/llvm-3.4/lib/libLLVMSupport.a(Process.o): In function >> `llvm::sys::Process::FileDescriptorHasColors(int)': >> (.text+0x5e9): undefined reference to `set_curterm' >> /usr/lib/llvm-3.4/lib/libLLVMSupport.a(Process.o): In function >> `llvm::sys::Process::FileDescriptorHasColors(int)': >> (.text+0x5f1): undefined reference to `del_curterm' >> collect2: error: ld returned 1 exit status > That should be in the ncurses-devel package (or whatever it's called on > Ubuntu) which I thought was an OpenJDK requirement anyway. Maybe the > only problem is that we're not linking with -lncurses. On my machine, "llvm-config --ldflags" outputs: -L/usr/lib/llvm-3.4/lib -lpthread -lffi -ltinfo -ldl -lm But configure filters only -L arguments into LLVM_LDFLAGS. The rest of the -l flags above are already present on the libjvm link line, but before $(LLVM_LIBS). If I explicitly add -ltinfo last on the link line (after LLVM_LIBS), my build succeeds. /Erik From stefan.karlsson at oracle.com Thu Jan 8 13:47:32 2015 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 08 Jan 2015 14:47:32 +0100 Subject: RFR: 8066875: VirtualSpace does not use large pages In-Reply-To: <20141215143649.GB24542@ehelin.jrpg.bea.com> References: <54874F17.7070303@oracle.com> <548814C1.8010905@oracle.com> <548AC362.8010800@oracle.com> <20141215143649.GB24542@ehelin.jrpg.bea.com> Message-ID: <54AE8A74.2080304@oracle.com> Hi Erik, On 2014-12-15 15:36, Erik Helin wrote: > On 2014-12-12, Albert Noll wrote: >> Hi Erik, >> >> thanks a lot for taking care of this. I have one minor comment (not a >> reviewer): >> >> 1418 size_t os::largest_page_size_less_than(size_t sz) { >> 1419 if (UseLargePages) { >> 1420 // The page sizes are sorted descendingly. >> 1421 for (size_t i = 0; _page_sizes[i] != 0; ++i) { >> 1422 if (_page_sizes[i] <= sz) { >> 1423 return _page_sizes[i]; >> 1424 } >> 1425 } >> 1426 } >> 1427 return vm_page_size(); >> 1428 } >> >> The function name suggests that the function returns a page size that is >> *less* than the value in the argument (sz). >> However, in line 1422 you check for '<=' . I think you should either fix the >> name of the function or the check in line 1422. > Yeah, I wasn't too fond of that name either. I discussed some potential > names with the guys in the office and ended up with: > - os::page_size_for_region_aligned > - os::page_size_for_region_unaligned > > os::page_size_for_region_aligned and os::page_size_for_region_unaligned > takes the same number of parameters, a region size and minimum number of > pages. The difference is that page_size_for_region_aligned guarantees > that returned page size divides the given region size, whereas > page_size_for_region_unaligned does not guarantee this. > > These names were chosen because a call to page_size_for_region_unaligned > indicates that the surrounding code must handle any alignment on its > own. > > Webrevs: > - incremental: http://cr.openjdk.java.net/~ehelin/8066875/webrev.00-01/ > - full: http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/ http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/src/share/vm/runtime/virtualspace.cpp.udiff.html I think this code should use the unaligned version: ReservedSpace::ReservedSpace(size_t size) { - size_t page_size = os::page_size_for_region(size, 1); + size_t page_size = os::page_size_for_region_aligned(size, 1); bool large_pages = page_size != (size_t)os::vm_page_size(); // Don't force the alignment to be large page aligned, // since that will waste memory. size_t alignment = os::vm_allocation_granularity(); initialize(size, alignment, large_pages, NULL, 0, false); As the comment hints, 'size' might not be large pages aligned, but we might still want to use large pages for the middle part of the reserved memory. http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/src/share/vm/runtime/os.cpp.udiff.html Could this code: - if (page_size <= max_page_size && is_size_aligned(region_size, page_size)) { + if (page_size <= max_page_size) { + if (!must_be_aligned) { return page_size; } + if (is_size_aligned(region_size, page_size)) { + return page_size; + } + } be changed into one return statement?: if (page_size <= max_page_size) { if (!must_be_aligned || is_size_aligned(region_size, page_size)) { return page_size; } } http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/src/share/vm/runtime/os.cpp.udiff.html The comments in the tests would love some periods. :) Thanks, StefanK > > Testing: > - JPRT > - Verified that the code cache now uses large pages even if > ReservedCodeCacheSize is 241 MB (see bug for more details). > - Added new internal vm tests (also run on SPARC machine with large > pages) > - Run hotspot jtreg tests on SPARC machine with large pages > > Thanks, > Erik > >> Best, >> Albert >> >> >> On 12/10/2014 10:39 AM, Tobias Hartmann wrote: >>> Hi Erik, >>> >>> looks good (not a reviewer). >>> >>> Thanks, >>> Tobias >>> >>> On 09.12.2014 20:35, Erik Helin wrote: >>>> Hi all, >>>> >>>> the fix for JDK-8049864 [0] made os::page_size_for_region slightly more strict >>>> since the function now demands that the given region_size is size aligned with >>>> respect to the chosen page_size. The reason for doing this was that >>>> os::page_size_for_region was used in two ways: >>>> 1. Give me a suitable page size for this amount of memory >>>> 2. Give me the largest page size for this amount of memory >>>> The fix for JDK-8049864 fixed os::page_size_for_region for the "suitable page >>>> size" scenario, but is too strict for the "largest page size" scenario. This >>>> caused a regression in VirtualSpace::initialize, which only needs the largest >>>> possible page size, since VirtualSpace::initialize_with_granularity takes care >>>> of any alignment issues. >>>> >>>> This patch adds the function os::largest_page_size_less_than and updates >>>> VirtualSpace::initialize to use this new function instead of >>>> os::page_size_for_region. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~ehelin/8066875/webrev.00/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8066875 >>>> >>>> Testing: >>>> - JPRT >>>> - Verified that the code cache now uses large pages even if >>>> ReservedCodeCacheSize is 241 MB (see bug for more details). >>>> - Added new internal vm tests (also run on SPARC machine with large >>>> pages) >>>> >>>> Thanks, >>>> Erik >>>> >>>> [0]: http://hg.openjdk.java.net/jdk9/hs-gc/hotspot/rev/b326a3e8dcab From coleen.phillimore at oracle.com Thu Jan 8 14:02:55 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 08 Jan 2015 09:02:55 -0500 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> Message-ID: <54AE8E0F.6050104@oracle.com> This change looks good to me. Pending changing "vmassert" in the message back to "assert". I'm not sure what compatibility risk this has but it would be safe. Can you find one of the internal tests that uses assert and change to use vmassert as a test? I think when we do the full change from assert -> vmassert that with the compatibility line, changes should export cleanly to 8u-dev. They'll just look like we changes vmassert too. It would be useful to try this as an experiment though. Thanks for moving this forward. There's another thread about C++11 use and it's only a matter of time before this really bites us. Coleen On 12/30/14, 7:47 PM, Kim Barrett wrote: > Please review this change to rename the assert macro to vmassert, as a > step toward eliminating the name clash between the VM's assert() macro > and the standard ( / ) assert() macro. > > [Coleen has volunteered sponsorship, since she's been prodding me to > make this change for months. Changes are based on jdk9/hs-rt repo.] > > CR: > https://bugs.openjdk.java.net/browse/JDK-8068396 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8068396/webrev > > Testing: > jprt. Also manually verified asserts occur as expected in fastdebug > build and not in release build. > > Details of the changes are: > > * Renamed the assert macro to vmassert. Also renamed the assert_status > macro to vmassert_status, for name consistency. > > * Added temporary synonym macros of assert => vmassert and > assert_status => vmassert_status, for backward compatibility. This > change does not modify the (18K+) occurrences of the old macro names > to use the new names. The synonyms allow that renaming to occur > incrementally, with removal of the synonyms once renaming is complete. > > * Simplified the definition of vmassert_status to just use vmassert, > rather than duplicating much of the implementation of vmassert. While > I was at it, added back a comma in the message that was (probably > accidentally) removed by 6888954: argument formatting for assert() and > friends (hg:1410 4/22/2010). > > * Moved the assert_if_no_error macro to utilities/xmlstream.cpp, which > is the only place it is used. Also simplified the definition to just > use vmassert, rather than duplicating much of the implementation of > vmassert. > > * Changed shark/llvmHeaders.hpp to just define assert as a synonym of > vmassert, rather than duplicating the definition of [vm]assert. [This > is an ugly hack, but still an improvement on what was there. This is a > place that could quickly benefit from updating uses.] > > * Removed the non-product AssertRepeat option and the associated > support by the [vm]assert macro. Support for this option has been > broken since 6888954: argument formatting for assert() and friends > (hg:1410 4/22/2010), due to a missing line continuation in the > corresponding definition of the assert macro. Since the feature > doesn't look very interesting, has been broken for nearly 5 years, and > can easily be resurrected if needed, I chose to simplify by removal. > From george.triantafillou at oracle.com Thu Jan 8 15:49:18 2015 From: george.triantafillou at oracle.com (George Triantafillou) Date: Thu, 08 Jan 2015 10:49:18 -0500 Subject: Closed - RFR (XS): 8068540 - [TESTBUG] Exclude failing nightly tests Message-ID: <54AEA6FE.8060202@oracle.com> Please review the closed portion for JDK-8068540. The tests were modified with the "@ignore" jtreg tag to exclude them from nightly testing. JBS: https://bugs.openjdk.java.net/browse/JDK-8068540 webrev: http://cr.openjdk.java.net/~gtriantafill/8068540-closed/webrev.00/ The fix was tested locally on Linux with jtreg and the JPRT hotspot testset. -George From harold.seigel at oracle.com Thu Jan 8 16:34:50 2015 From: harold.seigel at oracle.com (harold seigel) Date: Thu, 08 Jan 2015 11:34:50 -0500 Subject: Closed - RFR (XS): 8068540 - [TESTBUG] Exclude failing nightly tests In-Reply-To: <54AEA6FE.8060202@oracle.com> References: <54AEA6FE.8060202@oracle.com> Message-ID: <54AEB1AA.9030909@oracle.com> Looks good George! Thanks, Harold On 1/8/2015 10:49 AM, George Triantafillou wrote: > Please review the closed portion for JDK-8068540. The tests were > modified with the "@ignore" jtreg tag to exclude them from nightly > testing. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8068540 > webrev: > http://cr.openjdk.java.net/~gtriantafill/8068540-closed/webrev.00/ > > > The fix was tested locally on Linux with jtreg and the JPRT hotspot > testset. > > -George From kim.barrett at oracle.com Thu Jan 8 23:57:59 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 8 Jan 2015 18:57:59 -0500 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> Message-ID: On Jan 7, 2015, at 12:07 PM, Kim Barrett wrote: > > Erik Helin pointed out to me that the change to the assertion text > from "assert(...)" to "vmassert(...)" might break some existing JBS > RULE comments for matching known failures. > > I'll look at how bad this really is, but it seems likely that the > thing to do is to revert the message text back to using "assert" > rather than "vmassert". I've reverted the message string back to saying "assert(" rather than "vmassert(". I'm working on getting an aurora test with the "vmassert(" string to see how bad that change might be, but I'm not sure I've done that properly yet. Regardless, the argument for reverting that string change seems pretty compelling. I looked at what will happen with [vm]assert_status, which was simplified to just expand into a vmassert rather than duplicating the implementation of that macro. The assert_status predicate expression gets included in the message string in the same way under the new as the old, so with vmassert message string reversion there won't be any change here either. I looked at assert_if_no_error similarly; it used to just expand into assert and now expands into vmassert, so again no change in the message. For testing, I did builds with and without these changes, with temporary code inserted to always fail an assertion, and verified that the relevant message texts were identical. CR: https://bugs.openjdk.java.net/browse/JDK-8068396 New webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.02 Incremental webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.02.incr The new webrev contains copyright updates, since its a new year. I left the copyright updates out of the incremental webrev. From coleen.phillimore at oracle.com Fri Jan 9 00:43:43 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 08 Jan 2015 19:43:43 -0500 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> Message-ID: <54AF243F.7050001@oracle.com> My prior email asked if you could test this change by changing an assert to vmassert. The assert in debug.cpp test_error_handler to vmassert would be a good one to test this change. thanks, Coleen On 1/8/15, 6:57 PM, Kim Barrett wrote: > On Jan 7, 2015, at 12:07 PM, Kim Barrett wrote: >> Erik Helin pointed out to me that the change to the assertion text >> from "assert(...)" to "vmassert(...)" might break some existing JBS >> RULE comments for matching known failures. >> >> I'll look at how bad this really is, but it seems likely that the >> thing to do is to revert the message text back to using "assert" >> rather than "vmassert". > I've reverted the message string back to saying "assert(" rather than > "vmassert(". I'm working on getting an aurora test with the > "vmassert(" string to see how bad that change might be, but I'm not > sure I've done that properly yet. Regardless, the argument for > reverting that string change seems pretty compelling. > > I looked at what will happen with [vm]assert_status, which was > simplified to just expand into a vmassert rather than duplicating the > implementation of that macro. The assert_status predicate expression > gets included in the message string in the same way under the new as > the old, so with vmassert message string reversion there won't be any > change here either. > > I looked at assert_if_no_error similarly; it used to just expand into > assert and now expands into vmassert, so again no change in the > message. > > For testing, I did builds with and without these changes, with > temporary code inserted to always fail an assertion, and verified that > the relevant message texts were identical. > > CR: https://bugs.openjdk.java.net/browse/JDK-8068396 > New webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.02 > Incremental webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.02.incr > > The new webrev contains copyright updates, since its a new year. I > left the copyright updates out of the incremental webrev. > > From kim.barrett at oracle.com Fri Jan 9 01:05:51 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 8 Jan 2015 20:05:51 -0500 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: <54AF243F.7050001@oracle.com> References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> <54AF243F.7050001@oracle.com> Message-ID: <22C65166-ABB5-408E-8A7C-5DDE6D61BAEE@oracle.com> On Jan 8, 2015, at 7:43 PM, Coleen Phillimore wrote: > > > My prior email asked if you could test this change by changing an assert to vmassert. The assert in debug.cpp test_error_handler to vmassert would be a good one to test this change. Ah, now I understand what you were asking for. OK, I?ll do that and post a new webrev later this evening. From david.holmes at oracle.com Fri Jan 9 02:29:27 2015 From: david.holmes at oracle.com (David Holmes) Date: Fri, 09 Jan 2015 12:29:27 +1000 Subject: RFR: 8067790: Better support for native implementations of Atomic::cmpxchg In-Reply-To: <5749133C-5087-457B-A0E1-7F21CF1825BD@lnu.se> References: <0F0D9F35-148D-4C62-8065-7E58F5EC76D1@lnu.se> <, <8DCE9938-C537-48AA-B27E-77E12279EA86@oracle.com> <>> <1D2B176D-67FA-43C0-96CA-2D33BEA16853@lnu.se> <72ACD7B2-BB22-49CA-8854-96CD12FFF64C@lnu.se> <54AA143B.2080900@oracle.com> <54ACB5A4.50603@oracle.com> <54ACBD00.6030501@oracle.com> <54AE174A.4050204@oracle.com> <5749133C-5087-457B-A0E1-7F21CF1825BD@lnu.se> Message-ID: <54AF3D07.9060900@oracle.com> Hi Erik, I don't want to get into it on this thread but please start a new thread on the OrderAccess changes you are proposing. Bear in mind our official compilers are never the most recent (by the time we ship) and we're unlikely to be able to use C++11 specifics in the near term, even if we wanted too (which we may not). Thanks, David On 8/01/2015 11:12 PM, Erik ?sterlund wrote: > Hi Coleen, > >> On 08 Jan 2015, at 06:36, Coleen Phillimore wrote: >> >> I'm sorry for coming in late to this thread and not agreeing with the code. It's really hard to have these discussions remotely and as a distraction to other work. > > I completely understand, no worries. > >> I don't think those of us paying loose attention realized the extent of new template features were coming into the code as part of this clean up. One webrev I saw had a small template trait (which needed explaining to me at the time). I think using this particular clean up as an opportunity to add these bonus templates is not something that we want to do. > > As pointed out by Kim earlier, the webrev massively exaggerates ?the extent?. It?s really only ~100 LoC template stuff with a bunch of copyright boilerplate and testing, which are not negative maintenance overheads (we can all agree it?s a bad idea to remove testing and bunch everything together in one file to make it look less extensive right?). > > Actually this whole code size thing has been a bit confusing. First Kim said it looked rather extensive too. So I made a code minimal version (20 LoC only to get the job done but less reusability) and a reuse max version (~100 LoC). Then after looking closer, Kim concluded code size probably was not a big issue after closer inspection (copyrights, testing). So I dropped the code minimum version after that was ruled out as an issue. And now other voices are raised again about the perceived extensiveness of the solution. > >> If you're adding something general to utilities, I argue that it should be well understood so that it gets the most utilization. If such a mechanism is needed for something larger like automatic closure specialization (not sure what that is) then this may be greater motivation. It depends on the benefits. > > I thought the decision was already made to not go ahead with this. > > But if you want to know, I can give a few examples of things that benefits from the proposed template infrastructure with brief descriptions. The uninterested reader can skip ahead. ;) > > ** BEGIN BENEFITS ** > > 1) Automatic Closure Specialization: In the GC code, OopClosures and ExtendedOopClosures do arbitrary stuff with oops. It?s a neat abstraction but since the stuff to be done is arbitrary, it results in a virtual call. To work around this, mad people driven by dark forces invented a manual closure specialization mechanism to remove the virtual calls to the closures. It comes with a threefold of problems: 1) It requires closures to be manually added to a macro to be specialized (and hence get rid of the virtual call), 2) This technique fundamentally cannot handle composite closure types (closure filtering oop then calling another closure). It loses type information and knowledge of the composite structure is lost. 3) It smells. Using template metaprogramming (and some of the traits in this change) all those problems can be resolved: automatic specialization without manual labour, full type awareness meaning composite closure types can be specialized too, much cleaner, and we won?t h ave to define both do_oop and do_oop_nv all over the place. No problem for me to add stuff needed here at a later time though. > > 2) OrderAccess refactoring: OrderAccess has a number of problems in terms of maintainability and correctness, and I?m currently cleaning it up. There are two main issues, but I think they are related. (I suspect correctness is inconsistent due to poor maintainability) > > 2.a) OrderAccess Maintainability: Different concerns are now joined to a big blob: Which barrier to use (acquire, release, fence) and where to put it w.r.t. to memory accesses, how to implement those barriers, how to make atomic memory accesses, how to specialize more efficient variants are all different concerns, now flattened out to be done over and over again for each platform even though the behaviour is very general. I intend to move everything except specialization and barrier implementation to shared code and separate their concerns, i.e. how to make atomic memory accesses and where to put the barriers is the same for everyone except the specialized variants. This removes most of the code for most of the individual platforms and makes it easier to reason about the different concerns in isolation without changing stuff all over the place. To still allow specializations with inline assembly that deviate from the norm, this conditional class selection is once again needed and could benefit from the exact same pattern I intended to push here for specialization and generalization. > > 2.b) OrderAccess Correctness: Correctness concerns are inconsistent over different platforms and dangerous which is my second issue with OrderAccess. Specifically, for TSO architectures like SPARC and x86, OrderAccess relies on C++ volatile for acquire release semantics, which in most cases is wrong. In every compiler I know of except MSVC 2008, this can not be relied upon as the C++ standard section 1.9 is vague whether volatiles may be reordered w.r.t. non-volatiles and has been interpreted differently by compiler implementors. It only really promises reordering constraints between volatiles which is not enough for acquire release semantics. As a matter of fact, GCC explicitly states they may be arbitrarily reordered at will, and therefore we need compiler barriers to prevent this. This problem was observed on linux and fixed by Mikael, on Linux. But the exact same problem (relying on volatile to have acquire release semantics) still lurks around on the other TSO architectures, waiting for the right moment to strike. With this refactoring, correcting such an issue (which I also intend to do because I think the current implementation is flawed) can be easily done with 3 LoC per platform - the barrier selection for memory accesses, instead of for each type of memory access on nearly every TSO platform. This will also make a later transition to C++11/14 easier when compilers are ready, also with only a few LoC. > > So in summary the provided metatemplating stuff could help automating closure specialization, it could help generalize/specialize Atomics (in review now), and it could help generalize/specialize OrderedAccess which IMO needs refactoring. There are also some others on the top of my head as well like removing the fence in G1?s post reference write barrier using asymmetric dekker synchronization when the platform supports it using the same reusable generalization/specialization pattern specifying which fencing mechanism to use, if we decide we want to do that, and potentially using it as a straight forward pattern for building commercial variation points to the open source code without affecting the open source code base. And Kim had some additional uses too. > > Of course, #2 can use conditional #define too to get the job done which is what I?m doing now since this idea was not appreciated in the end, just saying there are more uses as you asked. > > ** END BENEFITS ** > > I did not elect to go for a reusable trait variant only because metaprogramming is fun. I genuinely think they are powerful tools that we could benefit from in different areas in runtime and gc. :) > >> Generally, if it is a clean solution to a bunch of problems, I'd be all for it. I'd also like it to be as close to standard library code as possible and I didn't get the sense of that from my reading of the discussion. > > I disagreed with Kim at one point (details not necessary I think) about the necessity for compliance with standard library code at a specific point, but that has already been fixed in favour of maximum compliance, which in the end everyone were happy with. > >> This is very cool stuff, but we need to proceed with caution and make sure we're all up to speed with the new C++ world and can use it effectively. > > I think this looks like a chicken and egg problem here. We need the tools to be available to learn how to use them and integrate them to our work flow. But we won?t make the tools available unless we know how to use them in our work flow. :/ > >> Too complicated is subjective, certainly, but closing this issue and not adding this code for atomics is my preference. > > 2 reviewers accepted (Volker and Paul), 2 reviewers rejected (you I take it, and David), 1 neutral (Kim) but willing to accept after an argument about whether the specialized and generalized classes should be connected by inheritance by contract (decided to remove the check and with it some traits. I'm in the middle, optimistic about the solution but confused as to how this discussion got so long it broke my email client. ;) > > I am fine with this decision. I never imagined this (~100 LoC metaprogramming) would lead to such a long discussion. If I knew the extent of the discussion, I would never have suggested it. I?m perfectly fine with leaving this idea behind if opinions are a bit split on this one. > > Thanks, > /Erik > From david.holmes at oracle.com Fri Jan 9 05:27:21 2015 From: david.holmes at oracle.com (David Holmes) Date: Fri, 09 Jan 2015 15:27:21 +1000 Subject: [PATCH] Fix Shark build in JDK9 In-Reply-To: <1420714881.6838.25.camel@localhost> References: <1420641903.6838.9.camel@localhost> <54AD4C6B.1070702@oracle.com> <1420645301.6838.14.camel@localhost> <54AD5675.7010101@oracle.com> <1420647107.6838.15.camel@localhost> <54AD5BDF.1060401@oracle.com> <1420648161.6838.17.camel@localhost> <54AD639A.2060200@oracle.com> <1420662102.6838.23.camel@localhost> <54AE560B.3060200@oracle.com> <1420714881.6838.25.camel@localhost> Message-ID: <54AF66B9.8060900@oracle.com> Hi Roman, Commenting on the hotspot changes ... On 8/01/2015 9:01 PM, Roman Kennke wrote: > Hi Erik, > > I'm CC-ing hotspot-dev for review of Hotspot code related changes. > > Yes, some additional changes to Hotspot are required. This is the full > set of changes needed to build and run Shark: > > http://cr.openjdk.java.net/~rkennke/shark-build-hotspot/webrev.01/ > > In detail: > > - In the Makefile fix a typo to be able to build unzipped debuginfo. Ok. > - In ciTypeFlow.cpp only include some files and code only when building > C2. I don't think that code makes sense outside of C2. (That's the issue > that you've seen). Looks okay but someone from compiler team needs to comment. There may be other code that need adjusting. > - In systemDictionary.cpp, exclude some code for Shark that creates and > checks native wrappers for method handle intrinsics. Invokedynamic stuff > is not yet implemented in Shark, so we can't do this. Ok. > - In allocation.hpp, exclude the operator overrides for new etc, LLVM > does allocations itself, and this would blow up. I'm intrigued as the allocation strategy is not tied to the compiler but pervasive to the whole VM runtime. > - In handles.hpp and handles.inline.hpp, I added an argument > check_in_stack so that I can disable the in-stack check for the > SharkNativeWrapper. The SharkNativeWrapper is allocated on-heap and the > methodHandle inside the SharkNativeWrapper. I could have excluded that > check altogther using an #ifndef SHARK, but the way I implemented it > seemed more finegrained. I'd prefer an ifndef SHARK rather than change the API. Thanks, David > - In SharkCompiler, I pulled out the code to initialize LLVM into its > own method, and the call to set_state(initialized) out of the > execution-engine-lock. This would otherwise complain about wrong > lock-ordering. > - In SharkRuntime, I changed the cast from intptr_t to oop to work with > the new picky compilers ;-) > > Please review. > > Thanks and best regards, > Roman > > Am Donnerstag, den 08.01.2015 um 11:03 +0100 schrieb Erik Joelsson: >> Hello Roman, >> >> This addition looks good to me. >> >> Thinking about what the others said, it might be inconvenient to have >> all this be pushed to different forests. I tried applying all the >> changes to jdk9/hs-rt, but I can't seem to build zeroshark.. Did you >> have more hotspot changes to be able to finish the build? >> >> My failure is: >> >> ciTypeFlow.o >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp >> In file included from >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/regmask.hpp:29:0, >> from >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/compile.hpp:40, >> from >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp:38: >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/optoreg.hpp:40:39: >> fatal error: adfiles/adGlobals_zero.hpp: No such file or directory >> >> From what I can see, adfiles are not generated for zero or zeroshark >> builds, so the include should probably be removed. >> >> Would you still like me to push what you currently have to hs-rt? >> >> /Erik >> >> On 2015-01-07 21:21, Roman Kennke wrote: >>> Hi Erik, >>> >>> When I built Zero and Shark on my Raspberry Pi, I noticed another >>> problem when copying jvm.cfg into the right places. I fixed it in a >>> similar way as I did for the SA stuff: >>> >>> http://cr.openjdk.java.net/~rkennke/shark-build-jdk/webrev.02/ >>> >>> I think that should be all for now. >>> >>> Please push that into JDK9 if you think that's fine. >>> >>> Best regards, >>> Roman >>> >>> Am Mittwoch, den 07.01.2015 um 17:49 +0100 schrieb Erik Joelsson: >>>> On 2015-01-07 17:29, Roman Kennke wrote: >>>>> Am Mittwoch, den 07.01.2015 um 17:16 +0100 schrieb Erik Joelsson: >>>>>> On 2015-01-07 17:11, Roman Kennke wrote: >>>>>>> Hi Erik, >>>>>>> >>>>>>> Do you have a bug for this? >>>>>>> No. >>>>>>> >>>>>>> I haven't pushed any changes to JDK in a while. Is it possible in the >>>>>>> meantime for me to create my own bugs? Otherwise, please file one for >>>>>>> me :-) >>>>>> You should be able to log in to https://bugs.openjdk.java.net and create >>>>>> bugs since you have an OpenJDK identity. >>>>> Done: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8068598 >>>>> >>>>> While I'm at it, is it possible for me to push my own changes (except >>>>> hotspot of course)? If yes, what needs to be done for regenerating the >>>>> configure files? Simply run autogen.sh in common/autoconf with whatever >>>>> version of autotools I have? Or doesn't it make sense at all b/c you >>>>> need to regenerate your closed scripts? >>>> It requires you to run common/autogen.sh yes, and that will require you >>>> to have autoconf 2.69 installed. But since we also need to regenerate >>>> the closed version, I can take care of the push for you. Will do it >>>> tomorrow if that's ok? >>>> >>>> /Erik >>>>> Roman >>>>> >>> >> > > From erik.osterlund at lnu.se Fri Jan 9 10:32:14 2015 From: erik.osterlund at lnu.se (=?utf-8?B?RXJpayDDlnN0ZXJsdW5k?=) Date: Fri, 9 Jan 2015 10:32:14 +0000 Subject: OrderAccess Refactoring Message-ID: <78E41D90-81F0-4526-B7A3-A02A6B030515@lnu.se> Hi, I have some proposed changes to OrderAccess, thought I?d check if they are wanted. It comes in two parts: consistency fixing, and refactoring the code to a generalization/specialization approach instead to eliminate lots of redundant platform specific code which is not platform specific. It?s possible to pick only one if both are not wanted. 2.a) OrderAccess Maintainability Summary: Remove platform specific code which is not platform specific (most of it) and make it shared, and separate different concerns in the code. Different concerns are now joined to a big blob: Which barrier to use (acquire, release, fence) and where to put it w.r.t. to memory accesses, how to implement those barriers, how to make atomic memory accesses, how to specialize more efficient variants are all different concerns, now flattened out to be done over and over again for each platform even though the behaviour is very general. I propose to move everything except specialization and barrier implementation to shared code to separate their concerns, i.e. how to make atomic memory accesses and where to put the barriers w.r.t. accesses is the same for everyone except the few specialized variants with inline asm. This removes most of the code for most of the individual platforms and makes it easier to reason about the different concerns in isolation without changing stuff all over the place. The idea is to split the class into 3 classes using inheritance: OrderAccessBase, OrderAccessPlatform and OrderAccess (of course). Most of the stuff goes in OrderAccessBase, but platforms with inline assembly can put their specialized variants in OrderAccessPlatform. 2.b) OrderAccess Correctness Summary: Relying on volatile to provide acquire/release on TSO cannot generally be guaranteed, must be fixed on all TSO except x86 linux and windows. Correctness concerns are inconsistent over different platforms and dangerous which is my second issue with OrderAccess. Specifically, for TSO architectures like SPARC and x86, OrderAccess relies on C++ volatile for acquire release semantics, which in most cases is wrong. In every compiler I know of except MSVC 2008, this can not be relied upon as the C++ standard section 1.9 is vague whether volatiles may be reordered w.r.t. non-volatiles and has been interpreted differently by compiler implementors. It only really promises reordering constraints between volatiles which is not enough for acquire release semantics. As a matter of fact, GCC explicitly states they may be arbitrarily reordered at will, and therefore we need compiler barriers to prevent this. This problem was observed on linux and fixed by Mikael, on Linux. But the exact same problem (relying on volatile to have acquire release semantics) still lurks around on the other TSO architectures, waiting for the right moment to strike. With the refactoring, correcting such an issue (which I also intend to do because I think the current implementation is flawed) can be easily done with 3 LoC per platform - the barrier selection for memory accesses, instead of for each type of memory access on nearly every TSO platform. The refactoring will as a bonus make a later transition to C++11/14 easier when compilers are ready, also with only a few LoC. Since we are now forced to use a compiler barrier which can be a bit too restrictive for the intended semantics, a more modern compiler can instead relax this a bit without sacrificing correctness and give us the exact semantics we want. Any idea if this is wanted, and if so both a and b or only b? IMO it?s much easier to fix the correctness issues on different platforms with that extra bit of refactoring too. ;) Thanks, /Erik From karen.kinnear at oracle.com Fri Jan 9 14:18:15 2015 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Fri, 9 Jan 2015 09:18:15 -0500 Subject: OrderAccess Refactoring In-Reply-To: <78E41D90-81F0-4526-B7A3-A02A6B030515@lnu.se> References: <78E41D90-81F0-4526-B7A3-A02A6B030515@lnu.se> Message-ID: Erik, I am delighted to have you looking at this code in detail to make sure it is accurate and maintainable. > On Jan 9, 2015, at 5:32 AM, Erik ?sterlund wrote: > > Hi, > > I have some proposed changes to OrderAccess, thought I?d check if they are wanted. It comes in two parts: consistency fixing, and refactoring the code to a generalization/specialization approach instead to eliminate lots of redundant platform specific code which is not platform specific. It?s possible to pick only one if both are not wanted. > > 2.a) OrderAccess Maintainability > Summary: Remove platform specific code which is not platform specific (most of it) and make it shared, and separate different concerns in the code. > > Different concerns are now joined to a big blob: Which barrier to use (acquire, release, fence) and where to put it w.r.t. to memory accesses, how to implement those barriers, how to make atomic memory accesses, how to specialize more efficient variants are all different concerns, now flattened out to be done over and over again for each platform even though the behaviour is very general. I propose to move everything except specialization and barrier implementation to shared code to separate their concerns, i.e. how to make atomic memory accesses and where to put the barriers w.r.t. accesses is the same for everyone except the few specialized variants with inline asm. This removes most of the code for most of the individual platforms and makes it easier to reason about the different concerns in isolation without changing stuff all over the place. The idea is to split the class into 3 classes using inheritance: OrderAccessBase, OrderAccessPlatform and OrderAccess (of course). Most of the stuff goes in OrderAccessBase, but platforms with inline assembly can put their specialized variants in OrderAccessPlatform. In theory I like refactoring. In practice I hope there is a very clear way to specify the per-platform assumptions and to make it very obvious what the end results are for a given platform and API. > > 2.b) OrderAccess Correctness > Summary: Relying on volatile to provide acquire/release on TSO cannot generally be guaranteed, must be fixed on all TSO except x86 linux and windows. > > Correctness concerns are inconsistent over different platforms and dangerous which is my second issue with OrderAccess. Specifically, for TSO architectures like SPARC and x86, OrderAccess relies on C++ volatile for acquire release semantics, which in most cases is wrong. In every compiler I know of except MSVC 2008, this can not be relied upon as the C++ standard section 1.9 is vague whether volatiles may be reordered w.r.t. non-volatiles and has been interpreted differently by compiler implementors. It only really promises reordering constraints between volatiles which is not enough for acquire release semantics. As a matter of fact, GCC explicitly states they may be arbitrarily reordered at will, and therefore we need compiler barriers to prevent this. This problem was observed on linux and fixed by Mikael, on Linux. But the exact same problem (relying on volatile to have acquire release semantics) still lurks around on the other TSO architectures, waiting for the right moment to strike. With the refactoring, correcting such an issue (which I also intend to do because I think the current implementation is flawed) can be easily done with 3 LoC per platform - the barrier selection for memory accesses, instead of for each type of memory access on nearly every TSO platform. Having code reviewed (and obviously missed things) earlier rounds of OrderAccess - I just wanted to note that we never thought that C++ volatile would provide acquire/release semantics. volatile has always been just a compiler directive, and yes volatile-volatile only. Acquire/release are instructions to the hardware - not what I would call compiler barriers. Personally I would do the correctness fixes first - and make sure they are really well tested and carefully code reviewed studying the cookbook and memory model. I have not been following the more recent java memory model evolution - I believe Aleksey Snipilev has - if you have references to any updated advice from Doug Lea - that would be very helpful for your reviewers to study. I know the last time I reviewed this I had to make my own small charts and walk it through based on per-platform guarantees. I would find it helpful if you are aware of specific bugs - if you could list the actual problems and call those out? It is very hard to get this right and hard to review - so if you were to have targeted reviews for specific fixes we could give the changes the depth of study that they need. Doing the refactoring afterwards allows us the easier exercise of ensuring that the refactoring produces the same code that we already have studied in depth and believe to be correct. So as you refactor please do find a place to clarify the per-platform assumptions so if we realize our assumptions are wrong we will re-review the code. > > The refactoring will as a bonus make a later transition to C++11/14 easier when compilers are ready, also with only a few LoC. Since we are now forced to use a compiler barrier which can be a bit too restrictive for the intended semantics, a more modern compiler can instead relax this a bit without sacrificing correctness and give us the exact semantics we want. > > Any idea if this is wanted, and if so both a and b or only b? IMO it?s much easier to fix the correctness issues on different platforms with that extra bit of refactoring too. ;) thank you for offering and for asking for opinions :-) Karen > > Thanks, > /Erik From erik.joelsson at oracle.com Fri Jan 9 14:34:52 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 09 Jan 2015 15:34:52 +0100 Subject: RFR: JDK-8067479: verify-modules fails in bootcycle build Message-ID: <54AFE70C.3070208@oracle.com> Hello, Please review this patch which fixes the verify-modules target when running bootcycle build, and also reenables verify-modules when running "make images". There were two problems: * The bootcycle build configuration was broken so that both the normal and the bootcycle build used the same HOTSPOT_DIST directory. The consequence of this was that verify-modules worked when run on its own, but not if bootcycle-images had been run before. This is fixed in bootcycle-spec.gmk.in. * Since javac in JDK 9 no longer emits classes for implicitly compiled sources, certain classes in sa-jdi.jar were not compiled during the bootcycle build. I fixed this by adding the missing classes to sa.files. Not having the classes there might have been intentional (in at least some cases), but since they were compiled anyway, I felt it safer to just add them to the list to fix this issue. If these classes shouldn't be included, then they need to be properly removed in a followup fix. Bug: https://bugs.openjdk.java.net/browse/JDK-8067479 Webrev: http://cr.openjdk.java.net/~erikj/8067479/webrev.01/ Since this is changing hotspot, I assume it will need to go in through a hotspot forest. Which one? /Erik From magnus.ihse.bursie at oracle.com Fri Jan 9 15:24:46 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 09 Jan 2015 16:24:46 +0100 Subject: RFR: JDK-8067479: verify-modules fails in bootcycle build In-Reply-To: <54AFE70C.3070208@oracle.com> References: <54AFE70C.3070208@oracle.com> Message-ID: <54AFF2BE.5050700@oracle.com> On 2015-01-09 15:34, Erik Joelsson wrote: > Hello, > > Please review this patch which fixes the verify-modules target when > running bootcycle build, and also reenables verify-modules when > running "make images". > > There were two problems: > > * The bootcycle build configuration was broken so that both the normal > and the bootcycle build used the same HOTSPOT_DIST directory. The > consequence of this was that verify-modules worked when run on its > own, but not if bootcycle-images had been run before. This is fixed in > bootcycle-spec.gmk.in. > > * Since javac in JDK 9 no longer emits classes for implicitly compiled > sources, certain classes in sa-jdi.jar were not compiled during the > bootcycle build. I fixed this by adding the missing classes to > sa.files. Not having the classes there might have been intentional (in > at least some cases), but since they were compiled anyway, I felt it > safer to just add them to the list to fix this issue. If these classes > shouldn't be included, then they need to be properly removed in a > followup fix. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8067479 > Webrev: http://cr.openjdk.java.net/~erikj/8067479/webrev.01/ > > Since this is changing hotspot, I assume it will need to go in through > a hotspot forest. Which one? Top level build changes look good to me. Can't say anything about the SA stuff. /Magnus From erik.osterlund at lnu.se Fri Jan 9 16:07:54 2015 From: erik.osterlund at lnu.se (=?utf-8?B?RXJpayDDlnN0ZXJsdW5k?=) Date: Fri, 9 Jan 2015 16:07:54 +0000 Subject: OrderAccess Refactoring In-Reply-To: References: <78E41D90-81F0-4526-B7A3-A02A6B030515@lnu.se> Message-ID: Hi Karen, Thanks for the reply! On 09 Jan 2015, at 15:18, Karen Kinnear > wrote: Erik, I am delighted to have you looking at this code in detail to make sure it is accurate and maintainable. :) In theory I like refactoring. In practice I hope there is a very clear way to specify the per-platform assumptions and to make it very obvious what the end results are for a given platform and API. For each platform you only have to specify two things: 1. What is needed for acquire, release and fence for memory accesses, for that platform (5 LoC per platform). The pattern of when to use them (semantics) is generalized in shared code. 2. To provide any specializations for optimization purposes when applicable (certain TSO platforms that can join the access and semantics to a single inline asm instruction). Note that this is optional, and only done to get faster accesses. Many platforms do not do it at all. I think it?s very intuitive! Will provide webrev if this seems worthwhile. Having code reviewed (and obviously missed things) earlier rounds of OrderAccess - I just wanted to note that we never thought that C++ volatile would provide acquire/release semantics. volatile has always been just a compiler directive, and yes volatile-volatile only. Acquire/release are instructions to the hardware - not what I would call compiler barriers. Acquire/release are semantics, not necessarily bound to neither compiler/hardware. It gives a certain contract to the user that expects these acquire/release semantics. Compiler barriers and fences is merely a means of providing these semantics, but the user should not have to care about such details. If what you are saying is that it was intended only as acquire/release for hardware but not constrain compiler reorderings to comply with the semantics, then the contract is broken (does in fact not promise acquire release, only on hardware level but compilers could do whatever with non-volatiles which is equally as bad). A store release should synchronize with an acquire load to the same location, like in this example: T1: store x1 T1: release store x2 T2: load acquire x2 T2: load x1 The semantics should guarantee that if load acquire x2 happens after store release x2 in total order, then it is guaranteed that store x1 will be observed by the load x1, and not some old value that was there before store x1. This contract does not currently hold since the normal non-volatile memory accesses may be reordered with volatile memory accesses; the stores and loads may be reordered in this example by the compiler at will. Therefore acquire release semantics are not guaranteed in general, but only w.r.t. volatiles. This is neither clearly documented nor used in such a way. OrderedAccess is used in a way that assumes acquire/release semantics. But for certain TSO platforms, the contract only provides volatile semantics, not acquire release semantics. On other platforms, it properly provides acquire release semantics, which is expected. It only makes sense to by contract promise acquire/release semantics (both for compiler and hardware reorderings) for uses of these methods, and consistently do so on all platforms. Personally I would do the correctness fixes first - and make sure they are really well tested and carefully code reviewed studying the cookbook and memory model. I have not been following the more recent java memory model evolution - I believe Aleksey Snipilev has - if you have references to any updated advice from Doug Lea - that would be very helpful for your reviewers to study. I know the last time I reviewed this I had to make my own small charts and walk it through based on per-platform guarantees. There is nothing wrong with the Java memory model. The only problem is that OrderAccess promises semantics that are only guaranteed on some platforms. I would find it helpful if you are aware of specific bugs - if you could list the actual problems and call those out? It is very hard to get this right and hard to review - so if you were to have targeted reviews for specific fixes we could give the changes the depth of study that they need. In for instance https://bugs.openjdk.java.net/browse/JDK-8061964 this was observed and caused trouble for G1 card refinement that went boom. In the discussion I pointed out the issue of relying on volatiles as a cause for the crash, and it was fixed, but not consistently for all platforms, but only for gcc on linux where the reordering was observed in the generated asm output. But it is equally as dangerous on other (TSO) platforms (except windows where the compiler actually explicitly promises not to reorder - their volatiles follow acquire/release in compiler reorderings). I think hoping for for compilers to not reorder without any guarantees, waiting for things to go boom where it has already been observed on other platforms seems dangerous. Better safe than sorry. :) One different solution is to redocument the contract clearly as being acquire/release w.r.t. volatile accesses, which is currently the actual contract provided by the implementation. But since bugs were already discovered (which is very understandable because no other atomic library has such semantics that I know of) and its code is still unchanged, it seems dangerous to do so without re-evaluating the use of OrderedAccess in hotspot, which seems a bit painful. Also such a revised contract would be a disaster when you need to copy loads of data and then store release for instance. This would force the copying (of potentially lots of data) to either be done using volatile accesses rather than in an accelerated fashion (e.g. SSE extensions, write combining), or add compiler barriers to such copying to give the release store actual release store semantics. This would also require fixing G1 where a bug was observed to not rely on the intuitive and expected contract, but on an unintuitive and provably error prone contract. Ugh!! The only solution that seems to make sense to me is to fix the semantics of OrderedAccess (that were already changed on Linux) to be what is expected, and that is acquire/release w.r.t. other accesses (volatile or non-volatile). Doing the refactoring afterwards allows us the easier exercise of ensuring that the refactoring produces the same code that we already have studied in depth and believe to be correct. So as you refactor please do find a place to clarify the per-platform assumptions so if we realize our assumptions are wrong we will re-review the code. I was hoping to do both at the same time, since there are many platforms to be changed, and making them over and over for every type of access on every platform to be changed is a bit tedious (and error prone) compared to doing it once per platform, reusing the shared pattern (to be reviewed once instead of once per platform) and review the choice of barriers once for each platform instead of for every memory access on every platform. Of course I don?t mind if we choose to do things in the other order though if that is preferred. :) If you compare the change in semantics I propose, without thinking about the refactoring, just compare x86 on bsd and linux, you see that on linux a compiler barrier is correctly used, but not on bsd. This is the change that needs to be added to all TSO platforms except linux and windows to ensure correctness. thank you for offering and for asking for opinions :-) Thank you for the good feedback! It looks like this change seems wanted, so I?ll go ahead and prepare a webrev with the proposed changes. :) Thanks, Erik From goetz.lindenmaier at sap.com Fri Jan 9 16:13:35 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 9 Jan 2015 16:13:35 +0000 Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF6AE2C@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF69B49@DEWDFEMB12A.global.corp.sap> <02ed01d02a97$9b445780$d1cd0680$@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF6AE2C@DEWDFEMB12A.global.corp.sap> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF6C863@DEWDFEMB12A.global.corp.sap> Christian, thanks for pushing it! Best regards, Goetz. -----Original Message----- From: goetz.lindenmaier at sap.com Sent: Mittwoch, 7. Januar 2015 22:32 To: 'Christian Tornqvist'; hotspot-dev at openjdk.java.net Subject: RE: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi Christian, It would be great if you sponsor this, thanks! Best regards, Goetz. -----Original Message----- From: Christian Tornqvist [mailto:christian.tornqvist at oracle.com] Sent: Wednesday, January 07, 2015 5:33 PM To: Lindenmaier, Goetz; hotspot-dev at openjdk.java.net Subject: RE: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi Goetz, This looks good, I can sponsor this for you. Thanks, Christian -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz Sent: Wednesday, January 7, 2015 5:14 AM To: 'hotspot-dev at openjdk.java.net' Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Hi, A row of hotspot jtreg tests fail because support to test for Aix is missing. To pass them, I would like to submit the following fixes: * runtime/6888954/vmerrors.sh: AIX can read from address '0', then throws SIGILL. * serviceability/dcmd/DynLibDcmdTest.java: Just add check for Aix. * test/test_env.sh: Add support for AIX. Sort OSes alphabetical. * test/testlibrary/com/oracle/java/testlibrary/Platform.java: add isAix(). Sort OSes alphabetical. Please review this change, and I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev-01/ Thanks and best regards, Goetz. From kim.barrett at oracle.com Fri Jan 9 18:11:00 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 9 Jan 2015 13:11:00 -0500 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: <22C65166-ABB5-408E-8A7C-5DDE6D61BAEE@oracle.com> References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> <54AF243F.7050001@oracle.com> <22C65166-ABB5-408E-8A7C-5DDE6D61BAEE@oracle.com> Message-ID: <3A9DFD5D-A70D-4CD3-9B56-88B75F771D0A@oracle.com> On Jan 8, 2015, at 8:05 PM, Kim Barrett wrote: > > On Jan 8, 2015, at 7:43 PM, Coleen Phillimore wrote: >> >> >> My prior email asked if you could test this change by changing an assert to vmassert. The assert in debug.cpp test_error_handler to vmassert would be a good one to test this change. > > Ah, now I understand what you were asking for. OK, I?ll do that and post a new webrev later this evening. New webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03/ Incremental webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03.incr/ As before, only the full webrev contains copyright updates. I ran jtreg locally and verified test/runtime/6888954/vmerrors.sh ran successfully. From aph at redhat.com Fri Jan 9 18:30:01 2015 From: aph at redhat.com (Andrew Haley) Date: Fri, 09 Jan 2015 18:30:01 +0000 Subject: RFR: 6584008: jvmtiStringPrimitiveCallback should not be invoked when string value is null Message-ID: <54B01E29.2090309@redhat.com> Webrev at http://cr.openjdk.java.net/~aph/JDK-6584008/ I have attached a reporducer to https://bugs.openjdk.java.net/browse/JDK-6584008 but I'm not sure how to turn it into a proper JTREG test. Andrew. From coleen.phillimore at oracle.com Fri Jan 9 19:21:19 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 09 Jan 2015 14:21:19 -0500 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: <3A9DFD5D-A70D-4CD3-9B56-88B75F771D0A@oracle.com> References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> <54AF243F.7050001@oracle.com> <22C65166-ABB5-408E-8A7C-5DDE6D61BAEE@oracle.com> <3A9DFD5D-A70D-4CD3-9B56-88B75F771D0A@oracle.com> Message-ID: <54B02A2F.6010208@oracle.com> Looks great. thank you! Coleen On 1/9/15, 1:11 PM, Kim Barrett wrote: > On Jan 8, 2015, at 8:05 PM, Kim Barrett wrote: >> On Jan 8, 2015, at 7:43 PM, Coleen Phillimore wrote: >>> >>> My prior email asked if you could test this change by changing an assert to vmassert. The assert in debug.cpp test_error_handler to vmassert would be a good one to test this change. >> Ah, now I understand what you were asking for. OK, I?ll do that and post a new webrev later this evening. > New webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03/ > Incremental webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03.incr/ > > As before, only the full webrev contains copyright updates. > > I ran jtreg locally and verified test/runtime/6888954/vmerrors.sh ran successfully. > From erik.osterlund at lnu.se Fri Jan 9 21:17:59 2015 From: erik.osterlund at lnu.se (=?Windows-1252?Q?Erik_=D6sterlund?=) Date: Fri, 9 Jan 2015 21:17:59 +0000 Subject: OrderAccess Refactoring References: <78E41D90-81F0-4526-B7A3-A02A6B030515@lnu.se> Message-ID: [Hoping to fix nesting of comments removed by server so you can actually see who said what in previous post :-|] Hi Karen, Thanks for the reply! On 09 Jan 2015, at 15:18, Karen Kinnear > wrote: > Erik, > > I am delighted to have you looking at this code in detail to make sure it is accurate and maintainable. :) > In theory I like refactoring. In practice I hope there is a very clear way to specify the per-platform assumptions and to make it very obvious what the end results are > for a given platform and API. For each platform you only have to specify two things: 1. What is needed for acquire, release and fence for memory accesses, for that platform (5 LoC per platform). The pattern of when to use them (semantics) is generalized in shared code. 2. To provide any specializations for optimization purposes when applicable (certain TSO platforms that can join the access and semantics to a single inline asm instruction). Note that this is optional, and only done to get faster accesses. Many platforms do not do it at all. I think it?s very intuitive! Will provide webrev if this seems worthwhile. > Having code reviewed (and obviously missed things) earlier rounds of OrderAccess - I just wanted to note that we never thought that C++ volatile would provide > acquire/release semantics. volatile has always been just a compiler directive, and yes volatile-volatile only. Acquire/release are instructions to the hardware - not > what I would call compiler barriers. Acquire/release are semantics, not necessarily bound to neither compiler/hardware. It gives a certain contract to the user that expects these acquire/release semantics. Compiler barriers and fences is merely a means of providing these semantics, but the user should not have to care about such details. If what you are saying is that it was intended only as acquire/release for hardware but not constrain compiler reorderings to comply with the semantics, then the contract is broken (does in fact not promise acquire release, only on hardware level but compilers could do whatever with non-volatiles which is equally as bad). A store release should synchronize with an acquire load to the same location, like in this example: T1: store x1 T1: release store x2 T2: load acquire x2 T2: load x1 The semantics should guarantee that if load acquire x2 happens after store release x2 in total order, then it is guaranteed that store x1 will be observed by the load x1, and not some old value that was there before store x1. This contract does not currently hold since the normal non-volatile memory accesses may be reordered with volatile memory accesses; the stores and loads may be reordered in this example by the compiler at will. Therefore acquire release semantics are not guaranteed in general, but only w.r.t. volatiles. This is neither clearly documented nor used in such a way. OrderedAccess is used in a way that assumes acquire/release semantics. But for certain TSO platforms, the contract only provides volatile semantics, not acquire release semantics. On other platforms, it properly provides acquire release semantics, which is expected. It only makes sense to by contract promise acquire/release semantics (both for compiler and hardware reorderings) for uses of these methods, and consistently do so on all platforms. > Personally I would do the correctness fixes first - and make sure they are really well tested and carefully code reviewed studying the cookbook and memory model. > I have not been following the more recent java memory model evolution - I believe Aleksey Snipilev has - if you have references to any updated advice > from Doug Lea - that would be very helpful for your reviewers to study. I know the last time I reviewed this I had to make my own small charts and walk it > through based on per-platform guarantees. There is nothing wrong with the Java memory model. The only problem is that OrderAccess promises semantics that are only guaranteed on some platforms. > I would find it helpful if you are aware of specific bugs - if you could list the actual problems and call those out? It is very hard to get this right and > hard to review - so if you were to have targeted reviews for specific fixes we could give the changes the depth of study that they need. In for instance https://bugs.openjdk.java.net/browse/JDK-8061964 this was observed and caused trouble for G1 card refinement that went boom. In the discussion I pointed out the issue of relying on volatiles as a cause for the crash, and it was fixed, but not consistently for all platforms, but only for gcc on linux where the reordering was observed in the generated asm output. But it is equally as dangerous on other (TSO) platforms (except windows where the compiler actually explicitly promises not to reorder - their volatiles follow acquire/release in compiler reorderings). I think hoping for for compilers to not reorder without any guarantees, waiting for things to go boom where it has already been observed on other platforms seems dangerous. Better safe than sorry. :) One different solution is to redocument the contract clearly as being acquire/release w.r.t. volatile accesses, which is currently the actual contract provided by the implementation. But since bugs were already discovered (which is very understandable because no other atomic library has such semantics that I know of) and its code is still unchanged, it seems dangerous to do so without re-evaluating the use of OrderedAccess in hotspot, which seems a bit painful. Also such a revised contract would be a disaster when you need to copy loads of data and then store release for instance. This would force the copying (of potentially lots of data) to either be done using volatile accesses rather than in an accelerated fashion (e.g. SSE extensions, write combining), or add compiler barriers to such copying to give the release store actual release store semantics. This would also require fixing G1 where a bug was observed to not rely on the intuitive and expected contract, but on an unintuitive and provably error prone contract. Ugh!! The only solution that seems to make sense to me is to fix the semantics of OrderedAccess (that were already changed on Linux) to be what is expected, and that is acquire/release w.r.t. other accesses (volatile or non-volatile). > Doing the refactoring afterwards allows us the easier exercise of ensuring that the refactoring produces the same code that we already have studied > in depth and believe to be correct. > > So as you refactor please do find a place to clarify the per-platform assumptions so if we realize our assumptions are wrong we will re-review the code. I was hoping to do both at the same time, since there are many platforms to be changed, and making them over and over for every type of access on every platform to be changed is a bit tedious (and error prone) compared to doing it once per platform, reusing the shared pattern (to be reviewed once instead of once per platform) and review the choice of barriers once for each platform instead of for every memory access on every platform. Of course I don?t mind if we choose to do things in the other order though if that is preferred. :) If you compare the change in semantics I propose, without thinking about the refactoring, just compare x86 on bsd and linux, you see that on linux a compiler barrier is correctly used, but not on bsd. This is the change that needs to be added to all TSO platforms except linux and windows to ensure correctness. thank you for offering and for asking for opinions :-) Thank you for the good feedback! It looks like this change seems wanted, so I?ll go ahead and prepare a webrev with the proposed changes. :) Thanks, Erik From kim.barrett at oracle.com Fri Jan 9 21:48:10 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 9 Jan 2015 16:48:10 -0500 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: <54B02A2F.6010208@oracle.com> References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> <54AF243F.7050001@oracle.com> <22C65166-ABB5-408E-8A7C-5DDE6D61BAEE@oracle.com> <3A9DFD5D-A70D-4CD3-9B56-88B75F771D0A@oracle.com> <54B02A2F.6010208@oracle.com> Message-ID: <43D5737F-E8FC-476A-AB2C-72B89BFF6131@oracle.com> On Jan 9, 2015, at 2:21 PM, Coleen Phillimore wrote: > > > Looks great. > thank you! > Coleen Thank you for your review, and for putting up with my denseness regarding the test you wanted. From paul.hohensee at gmail.com Fri Jan 9 22:39:06 2015 From: paul.hohensee at gmail.com (Paul Hohensee) Date: Fri, 9 Jan 2015 17:39:06 -0500 Subject: OrderAccess Refactoring Message-ID: Hi Erik, Being the person who put OrderAccess into its present form :), here's some background/comments. First, I'm happy that someone's looking at this. I wanted to make available both barriers and acquire/release functionality, and also provide common composition methods. In retrospect, I didn't separate the two concepts well enough in the interface (the work was done as part of an ia64 port, so I was kinda fixated on ia64 concepts at the time), so that's definitely something that should be fixed. We understood that volatile semantics wouldn't be enough by themselves to implement the semantics we wanted ordering on most platforms, but they *are* necessary to prevent compilers (not the hardware though) from reordering volatile accesses with respect to each other. Whether or not the volatile keyword prevents non-volatile memory accesses from being reordered wrt volatile ones seemed a bit fuzzy in the C++ standard at the time (and still does, at least to me), but we figured we could handle that in the os_cpu files by checking which compiler was being used. The code that implemented the *hardware* semantics also went into the os_cpu file(s). That's why there's some seemingly silly code there, including assembly that executes what amount to nops. I initially wrote the code in factored form, but changed to the current scheme because I found it difficult to see all the code for a single platform in one place (I wasn't using an IDE :) ). In retrospect, an unwise decision due to the usual code duplication problems. Mea culpa. :) Looking forward to your webrev. Thanks, Paul --------- Hi Karen, Thanks for the reply! On 09 Jan 2015, at 15:18, Karen Kinnear > wrote: > Erik, > > I am delighted to have you looking at this code in detail to make sure it is accurate and maintainable. :) > In theory I like refactoring. In practice I hope there is a very clear way to specify the per-platform assumptions and to make it very obvious what the end results are > for a given platform and API. For each platform you only have to specify two things: 1. What is needed for acquire, release and fence for memory accesses, for that platform (5 LoC per platform). The pattern of when to use them (semantics) is generalized in shared code. 2. To provide any specializations for optimization purposes when applicable (certain TSO platforms that can join the access and semantics to a single inline asm instruction). Note that this is optional, and only done to get faster accesses. Many platforms do not do it at all. I think it?s very intuitive! Will provide webrev if this seems worthwhile. > Having code reviewed (and obviously missed things) earlier rounds of OrderAccess - I just wanted to note that we never thought that C++ volatile would provide > acquire/release semantics. volatile has always been just a compiler directive, and yes volatile-volatile only. Acquire/release are instructions to the hardware - not > what I would call compiler barriers. Acquire/release are semantics, not necessarily bound to neither compiler/hardware. It gives a certain contract to the user that expects these acquire/release semantics. Compiler barriers and fences is merely a means of providing these semantics, but the user should not have to care about such details. If what you are saying is that it was intended only as acquire/release for hardware but not constrain compiler reorderings to comply with the semantics, then the contract is broken (does in fact not promise acquire release, only on hardware level but compilers could do whatever with non-volatiles which is equally as bad). A store release should synchronize with an acquire load to the same location, like in this example: T1: store x1 T1: release store x2 T2: load acquire x2 T2: load x1 The semantics should guarantee that if load acquire x2 happens after store release x2 in total order, then it is guaranteed that store x1 will be observed by the load x1, and not some old value that was there before store x1. This contract does not currently hold since the normal non-volatile memory accesses may be reordered with volatile memory accesses; the stores and loads may be reordered in this example by the compiler at will. Therefore acquire release semantics are not guaranteed in general, but only w.r.t. volatiles. This is neither clearly documented nor used in such a way. OrderedAccess is used in a way that assumes acquire/release semantics. But for certain TSO platforms, the contract only provides volatile semantics, not acquire release semantics. On other platforms, it properly provides acquire release semantics, which is expected. It only makes sense to by contract promise acquire/release semantics (both for compiler and hardware reorderings) for uses of these methods, and consistently do so on all platforms. > Personally I would do the correctness fixes first - and make sure they are really well tested and carefully code reviewed studying the cookbook and memory model. > I have not been following the more recent java memory model evolution - I believe Aleksey Snipilev has - if you have references to any updated advice > from Doug Lea - that would be very helpful for your reviewers to study. I know the last time I reviewed this I had to make my own small charts and walk it > through based on per-platform guarantees. There is nothing wrong with the Java memory model. The only problem is that OrderAccess promises semantics that are only guaranteed on some platforms. > I would find it helpful if you are aware of specific bugs - if you could list the actual problems and call those out? It is very hard to get this right and > hard to review - so if you were to have targeted reviews for specific fixes we could give the changes the depth of study that they need. In for instance https://bugs.openjdk.java.net/browse/JDK-8061964 this was observed and caused trouble for G1 card refinement that went boom. In the discussion I pointed out the issue of relying on volatiles as a cause for the crash, and it was fixed, but not consistently for all platforms, but only for gcc on linux where the reordering was observed in the generated asm output. But it is equally as dangerous on other (TSO) platforms (except windows where the compiler actually explicitly promises not to reorder - their volatiles follow acquire/release in compiler reorderings). I think hoping for for compilers to not reorder without any guarantees, waiting for things to go boom where it has already been observed on other platforms seems dangerous. Better safe than sorry. :) One different solution is to redocument the contract clearly as being acquire/release w.r.t. volatile accesses, which is currently the actual contract provided by the implementation. But since bugs were already discovered (which is very understandable because no other atomic library has such semantics that I know of) and its code is still unchanged, it seems dangerous to do so without re-evaluating the use of OrderedAccess in hotspot, which seems a bit painful. Also such a revised contract would be a disaster when you need to copy loads of data and then store release for instance. This would force the copying (of potentially lots of data) to either be done using volatile accesses rather than in an accelerated fashion (e.g. SSE extensions, write combining), or add compiler barriers to such copying to give the release store actual release store semantics. This would also require fixing G1 where a bug was observed to not rely on the intuitive and expected contract, but on an unintuitive and provably error prone contract. Ugh!! The only solution that seems to make sense to me is to fix the semantics of OrderedAccess (that were already changed on Linux) to be what is expected, and that is acquire/release w.r.t. other accesses (volatile or non-volatile). > Doing the refactoring afterwards allows us the easier exercise of ensuring that the refactoring produces the same code that we already have studied > in depth and believe to be correct. > > So as you refactor please do find a place to clarify the per-platform assumptions so if we realize our assumptions are wrong we will re-review the code. I was hoping to do both at the same time, since there are many platforms to be changed, and making them over and over for every type of access on every platform to be changed is a bit tedious (and error prone) compared to doing it once per platform, reusing the shared pattern (to be reviewed once instead of once per platform) and review the choice of barriers once for each platform instead of for every memory access on every platform. Of course I don?t mind if we choose to do things in the other order though if that is preferred. :) If you compare the change in semantics I propose, without thinking about the refactoring, just compare x86 on bsd and linux, you see that on linux a compiler barrier is correctly used, but not on bsd. This is the change that needs to be added to all TSO platforms except linux and windows to ensure correctness. thank you for offering and for asking for opinions :-) Thank you for the good feedback! It looks like this change seems wanted, so I?ll go ahead and prepare a webrev with the proposed changes. :) From kedar.mhaswade at gmail.com Fri Jan 9 23:50:55 2015 From: kedar.mhaswade at gmail.com (kedar mhaswade) Date: Fri, 9 Jan 2015 15:50:55 -0800 Subject: iinc_w instruction and hsdis build instructions ... Message-ID: Hello JVM developers! I hope I am posting to the right mailing list, if not, please let me know where I should direct these two questions. 1) When generating the bytecode by javac-7 and javac-8, I see iinc_w instruction (and javap shows subtle differences in disassembly on the same java class file). I don't find a direct reference to this instruction at: http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.html#jvms-6.5. Where should I be looking for the documentation? 2) I want to build hsdis on Ubuntu 14.04. I have tried with a bunch of binutils versions, but it fails rather obscurely [1]. Are there any instructions for a successful build on Ubuntu/Debian? Thanks! -Kedar 1- /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd -I. -I. -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/../include -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Werror -fPIC -O -c -o elf64-x86-64.lo /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c libtool: compile: gcc -DHAVE_CONFIG_H -I. -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd -I. -I. -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/../include -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Werror -fPIC -O -c /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c -o elf64-x86-64.o /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c: In function 'elf64_x86_64_relocate_section': /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:2394:16: error: variable 'warned' set but not used [-Werror=unused-but-set-variable] bfd_boolean warned; ^ /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:2870:29: error: variable 'type2' set but not used [-Werror=unused-but-set-variable] unsigned int val, type, type2; ^ /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3116:29: error: variable 'type2' set but not used [-Werror=unused-but-set-variable] unsigned int val, type, type2; ^ /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3116:23: error: variable 'type' set but not used [-Werror=unused-but-set-variable] unsigned int val, type, type2; ^ /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3116:18: error: variable 'val' set but not used [-Werror=unused-but-set-variable] unsigned int val, type, type2; ^ /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3147:23: error: variable 'type' set but not used [-Werror=unused-but-set-variable] unsigned int val, type; ^ /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3147:18: error: variable 'val' set but not used [-Werror=unused-but-set-variable] unsigned int val, type; ^ cc1: all warnings being treated as errors make[4]: *** [elf64-x86-64.lo] Error 1 make[4]: Leaving directory `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64/bfd' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64/bfd' make[2]: *** [all] Error 2 make[2]: Leaving directory `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64/bfd' make[1]: *** [all-bfd] Error 2 make[1]: Leaving directory `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64' make: *** [build/linux-amd64/bfd/libbfd.a] Error 2 From david.dehaven at oracle.com Sat Jan 10 04:45:32 2015 From: david.dehaven at oracle.com (David DeHaven) Date: Fri, 9 Jan 2015 20:45:32 -0800 Subject: [8u60] RFR: 8043340: [macosx] Fix hard-wired paths to JavaVM.framework Message-ID: <39719ED3-1F4E-4E6C-AA9F-A3C4766537B4@oracle.com> Please review the open source changes for 8043340. The goal here is to get jdk8u to build on Mac OS X 10.9+ systems where Xcode 5+ and Xcode 4 are co-installed, a configuration which is becoming more and more commonplace as more developers are focusing on JDK 9 now (which needs Xcode 5 installed), not to mention new systems ship with 10.10 which only supports the Xcode 5/6 command line tools. It's too much of a hassle to maintain separate systems for building jdk8u and JPRT isn't suitable for frequent changes so something must be done to address this. The jdk changes are similar to those done for JDK9, though I removed the changes for building with Xcode 5 (JDK-8043591) since we do not support building JDK 8 with clang. The hotspot changes are almost identical, the lack of SYSROOT_CFLAGS necessitated changing the logic in saproc.make a bit. It still builds with or without spec.gmk, though without it you will either need to define SDKPATH or have a sane Xcode 4 installation. For the top level build system: - most of the logic of sanitizing the Xcode build environment is in toolchain.m4 - the LIPO variable was removed since it was completely unused - OTOOL was moved to after the Xcode sanitizing so it can be picked up from DEVELOPER_DIR if needed - MACOSX_UNIVERSAL is now being set to false by default and ALT_MACOSX_UNIVERSAL was added to hotspot-spec.gmk.in so the hotspot build is in sync with the jdk build (this was a bug, IMHO) That last change removed any need to run lipo (only done in hotspot), so the fact that /usr/bin/lipo is broken with Xcode 4 is a non-issue. There is a weird case where some early versions of the Xcode 5 command line tools installed /usr/bin/{gcc|g++} as a symlink to {clang|clang++}, that case is handled by putting $DEVELOPER_DIR/usr/bin on the path so autoconf picks up the actual gcc executable. JBS Issue: https://bugs.openjdk.java.net/browse/JDK-8043340 Webrevs: http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v0/top http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v0/hotspot http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v0/jdk JPRT runs are being kicked off. I'll have one run from hotspot directly. I'll post results here. -DrD- From david.holmes at oracle.com Sat Jan 10 10:11:40 2015 From: david.holmes at oracle.com (David Holmes) Date: Sat, 10 Jan 2015 20:11:40 +1000 Subject: OrderAccess Refactoring In-Reply-To: References: <78E41D90-81F0-4526-B7A3-A02A6B030515@lnu.se> Message-ID: <54B0FADC.80600@oracle.com> Erik, Some very quick comments as there was a related meta-discussion on this very recently (core-libs-dev) in relation to some descriptions of the Unsafe memory barrier operations. storestore, loadload etc are just as important (more so in my opinion) as acquire/release. acquire/release semantics are problematic because there is no single common accepted meaning. Even talking about acquire() and release() as independent operations is somewhat meaningless - it is only when they are paired, and fixed in place that they start to give meaning to things. load_acquire/store_release don't necessarily map cleanly to acquire()/release() as currently described in orderAccess.hpp; nor do they necessarily map cleanly to the same named operations on eg ARMv8; nor to similar named entities in C++11. As Paul stated these were introduced to map to the ia64 architecture - which I believe had somewhat different semantics. This whole area is a minefield because there are different concepts of what "acquire" and "release" may mean - so you need to start with detailed definitions. Personally, and I know I am not alone, I abhor the acquire/release terminology and usage, I much prefer the unambiguous storeload, storestore etc. :) The current code often expresses things in a semantically incorrect way eg that: store_release(addr, val) == store(addr, val); release(); but in practice the end code provides what is required (often providing a stronger barrier because that's all that is available on a platform). I also agree that these primitives have to manage both compiler reorderings and hardware reorderings. And I agree with Karen - tackle correctness first then refactoring. IT will be too hard to track correctness otherwise. Personally, given I work in this code quite a bit, I like to see the complete set of definitions for a platform in one place. The duplication is something I can live with when it is mostly one-liners. Cheers, David On 10/01/2015 7:17 AM, Erik ?sterlund wrote: > [Hoping to fix nesting of comments removed by server so you can actually > see who said what in previous post :-|] > > Hi Karen, > > Thanks for the reply! > > On 09 Jan 2015, at 15:18, Karen Kinnear > > wrote: > >> Erik, >> >> I am delighted to have you looking at this code in detail to make sure it is accurate and maintainable. > > :) > >> In theory I like refactoring. In practice I hope there is a very clear way to specify the per-platform assumptions and to make it very obvious what the end results are >> for a given platform and API. > > For each platform you only have to specify two things: > > 1. What is needed for acquire, release and fence for memory accesses, > for that platform (5 LoC per platform). The pattern of when to use them > (semantics) is generalized in shared code. > > 2. To provide any specializations for optimization purposes when > applicable (certain TSO platforms that can join the access and semantics > to a single inline asm instruction). Note that this is optional, and > only done to get faster accesses. Many platforms do not do it at all. > > I think it?s very intuitive! Will provide webrev if this seems worthwhile. > >> Having code reviewed (and obviously missed things) earlier rounds of OrderAccess - I just wanted to note that we never thought that C++ volatile would provide >> acquire/release semantics. volatile has always been just a compiler directive, and yes volatile-volatile only. Acquire/release are instructions to the hardware - not >> what I would call compiler barriers. > > Acquire/release are semantics, not necessarily bound to neither > compiler/hardware. It gives a certain contract to the user that expects > these acquire/release semantics. Compiler barriers and fences is merely > a means of providing these semantics, but the user should not have to > care about such details. > > If what you are saying is that it was intended only as acquire/release > for hardware but not constrain compiler reorderings to comply with the > semantics, then the contract is broken (does in fact not promise acquire > release, only on hardware level but compilers could do whatever with > non-volatiles which is equally as bad). > > A store release should synchronize with an acquire load to the same > location, like in this example: > > T1: store x1 > T1: release store x2 > > T2: load acquire x2 > T2: load x1 > > The semantics should guarantee that if load acquire x2 happens after > store release x2 in total order, then it is guaranteed that store x1 > will be observed by the load x1, and not some old value that was there > before store x1. This contract does not currently hold since the normal > non-volatile memory accesses may be reordered with volatile memory > accesses; the stores and loads may be reordered in this example by the > compiler at will. > > Therefore acquire release semantics are not guaranteed in general, but > only w.r.t. volatiles. This is neither clearly documented nor used in > such a way. > > OrderedAccess is used in a way that assumes acquire/release semantics. > But for certain TSO platforms, the contract only provides volatile > semantics, not acquire release semantics. On other platforms, it > properly provides acquire release semantics, which is expected. > > It only makes sense to by contract promise acquire/release semantics > (both for compiler and hardware reorderings) for uses of these methods, > and consistently do so on all platforms. > >> Personally I would do the correctness fixes first - and make sure they are really well tested and carefully code reviewed studying the cookbook and memory model. >> I have not been following the more recent java memory model evolution - I believe Aleksey Snipilev has - if you have references to any updated advice >> from Doug Lea - that would be very helpful for your reviewers to study. I know the last time I reviewed this I had to make my own small charts and walk it >> through based on per-platform guarantees. > > There is nothing wrong with the Java memory model. The only problem is > that OrderAccess promises semantics that are only guaranteed on some > platforms. > >> I would find it helpful if you are aware of specific bugs - if you could list the actual problems and call those out? It is very hard to get this right and >> hard to review - so if you were to have targeted reviews for specific fixes we could give the changes the depth of study that they need. > > In for instance https://bugs.openjdk.java.net/browse/JDK-8061964 this > was observed and caused trouble for G1 card refinement that went boom. > In the discussion I pointed out the issue of relying on volatiles as a > cause for the crash, and it was fixed, but not consistently for all > platforms, but only for gcc on linux where the reordering was observed > in the generated asm output. But it is equally as dangerous on other > (TSO) platforms (except windows where the compiler actually explicitly > promises not to reorder - their volatiles follow acquire/release in > compiler reorderings). I think hoping for for compilers to not reorder > without any guarantees, waiting for things to go boom where it has > already been observed on other platforms seems dangerous. Better safe > than sorry. :) > > One different solution is to redocument the contract clearly as being > acquire/release w.r.t. volatile accesses, which is currently the actual > contract provided by the implementation. > But since bugs were already discovered (which is very understandable > because no other atomic library has such semantics that I know of) and > its code is still unchanged, it seems dangerous to do so without > re-evaluating the use of OrderedAccess in hotspot, which seems a bit > painful. Also such a revised contract would be a disaster when you need > to copy loads of data and then store release for instance. This would > force the copying (of potentially lots of data) to either be done using > volatile accesses rather than in an accelerated fashion (e.g. SSE > extensions, write combining), or add compiler barriers to such copying > to give the release store actual release store semantics. This would > also require fixing G1 where a bug was observed to not rely on the > intuitive and expected contract, but on an unintuitive and provably > error prone contract. Ugh!! > > The only solution that seems to make sense to me is to fix the semantics > of OrderedAccess (that were already changed on Linux) to be what is > expected, and that is acquire/release w.r.t. other accesses (volatile or > non-volatile). > >> Doing the refactoring afterwards allows us the easier exercise of ensuring that the refactoring produces the same code that we already have studied >> in depth and believe to be correct. >> >> So as you refactor please do find a place to clarify the per-platform assumptions so if we realize our assumptions are wrong we will re-review the code. > > I was hoping to do both at the same time, since there are many platforms > to be changed, and making them over and over for every type of access on > every platform to be changed is a bit tedious (and error prone) compared > to doing it once per platform, reusing the shared pattern (to be > reviewed once instead of once per platform) and review the choice of > barriers once for each platform instead of for every memory access on > every platform. Of course I don?t mind if we choose to do things in the > other order though if that is preferred. :) > > If you compare the change in semantics I propose, without thinking about > the refactoring, just compare x86 on bsd and linux, you see that on > linux a compiler barrier is correctly used, but not on bsd. This is the > change that needs to be added to all TSO platforms except linux and > windows to ensure correctness. > > thank you for offering and for asking for opinions :-) > > Thank you for the good feedback! It looks like this change seems wanted, > so I?ll go ahead and prepare a webrev with the proposed changes. :) > > Thanks, > Erik > From christian.tornqvist at oracle.com Sat Jan 10 13:42:46 2015 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Sat, 10 Jan 2015 08:42:46 -0500 Subject: RFR(M): 8067941: [TESTBUG] Fix tests for OS with 64K page size. In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF69221@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF43394@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CF69221@DEWDFEMB12A.global.corp.sap> Message-ID: <036a01d02cdb$51989b00$f4c9d100$@oracle.com> Hi Goetz, The WBStackSize.java change looks fine, I can sponsor the change if you still need a sponsor. Thanks, Christian -----Original Message----- From: hotspot-runtime-dev [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz Sent: Monday, January 5, 2015 4:36 AM To: 'hotspot-dev at openjdk.java.net'; hotspot-runtime-dev at openjdk.java.net Subject: RE: RFR(M): 8067941: [TESTBUG] Fix tests for OS with 64K page size. Hi, could someone from runtime please have a look at the last of these fixes? runtime/whitebox/WBStackSize.java http://cr.openjdk.java.net/~goetz/webrevs/8067941-64K/webrev.01/test/runtime /whitebox/WBStackSize.java.udiff.html I get the shadow page size from WB to compute the space wasted on the stack so far. And I please need a sponsor for this! Best regards, Goetz. From: Lindenmaier, Goetz Sent: Freitag, 19. Dezember 2014 17:06 To: 'hotspot-dev at openjdk.java.net' Subject: RFR(M): 8067941: [TESTBUG] Fix tests for OS with 64K page size. Hi, please review this simple change. I please need a sponsor. The change should be downported to 8u60. It applies nicely except for some missing files. http://cr.openjdk.java.net/~goetz/webrevs/8067941-64K/webrev.01/ The issue is that we have linux ppc64 machines that have default page size of 64K. This imposes bigger limits on stack and heap sizes than if running with 4k pages. Ergonomics increase some flags if they are too small. Increasing the sizes used in the test slightly helps in most cases. Details: runtime/whitebox/WBStackSize.java The constant esimating the so far occupied stack is too small. Fix: Consider size of shadow pages compiler/uncommontrap/TestStackBangRbp.java: compiler/uncommontrap/TestStackBangMonitorOwned.java compiler/uncommontrap/StackOverflowGuardPagesOff.java compiler/uncommontrap/8009761/Test8009761.java compiler/runtime/6865265/StackOverflowBug.java compiler/exceptions/TestRecursiveReplacedException.java Increase stack size to 392K, need at least 328K. sanity/ExecuteInternalVMTests.java - Assertion in metaspace.cpp simly must fail with vm_page_size == 64K as it basically asserts vm_page_size < 16K - With 64K pages, heap sizes are computed differently starting from the alignment which is card_size * vm_page_size. gc/arguments/TestMaxHeapSizeTools.java Sizes are alinged up being equal in the end. Choose bigger sizes. gc/g1/TestHumongousAllocInitialMark.java Test computes some size based on the -Xmx value it uses. Heap size is increased slightly with 64K making the computation wrong and the test fail. Choose heap size that needs not be aligned. gc/g1/TestGCLogMessages.java Heap of 10M is increased to 32M because of the alignment with 64K pages. This makes the evacuation succeed. Choose 32M from the beginning and adapt size of huge object. Best regards, Goetz. From david.holmes at oracle.com Sun Jan 11 23:20:50 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Jan 2015 09:20:50 +1000 Subject: iinc_w instruction and hsdis build instructions ... In-Reply-To: References: Message-ID: <54B30552.60707@oracle.com> On 10/01/2015 9:50 AM, kedar mhaswade wrote: > Hello JVM developers! > I hope I am posting to the right mailing list, if not, please let me know > where I should direct these two questions. > > 1) When generating the bytecode by javac-7 and javac-8, I see iinc_w > instruction (and javap shows subtle differences in disassembly on the same > java class file). I don't find a direct reference to this instruction at: > http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.html#jvms-6.5. Javac questions should be asked on compiler-dev at openjdk.java.net. But I think iinc_w is just a mnemonic for using iinc+wide as per the docs you reference: "The iinc opcode can be used in conjunction with the wide instruction (?wide) to access a local variable using a two-byte unsigned index and to increment it by a two-byte immediate signed value. " > Where should I be looking for the documentation? > > 2) I want to build hsdis on Ubuntu 14.04. I have tried with a bunch of > binutils versions, but it fails rather obscurely [1]. Are there any > instructions for a successful build on Ubuntu/Debian? Right list for this one but I don't have an answer sorry. David > Thanks! > -Kedar > > 1- /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd > -I. -I. > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/../include > -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Werror -fPIC -O -c > -o elf64-x86-64.lo > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c > libtool: compile: gcc -DHAVE_CONFIG_H -I. > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd > -I. -I. > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/../include > -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Werror -fPIC -O -c > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c > -o elf64-x86-64.o > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c: > In function 'elf64_x86_64_relocate_section': > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:2394:16: > error: variable 'warned' set but not used [-Werror=unused-but-set-variable] > bfd_boolean warned; > ^ > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:2870:29: > error: variable 'type2' set but not used [-Werror=unused-but-set-variable] > unsigned int val, type, type2; > ^ > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3116:29: > error: variable 'type2' set but not used [-Werror=unused-but-set-variable] > unsigned int val, type, type2; > ^ > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3116:23: > error: variable 'type' set but not used [-Werror=unused-but-set-variable] > unsigned int val, type, type2; > ^ > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3116:18: > error: variable 'val' set but not used [-Werror=unused-but-set-variable] > unsigned int val, type, type2; > ^ > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3147:23: > error: variable 'type' set but not used [-Werror=unused-but-set-variable] > unsigned int val, type; > ^ > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3147:18: > error: variable 'val' set but not used [-Werror=unused-but-set-variable] > unsigned int val, type; > ^ > cc1: all warnings being treated as errors > make[4]: *** [elf64-x86-64.lo] Error 1 > make[4]: Leaving directory > `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64/bfd' > make[3]: *** [all-recursive] Error 1 > make[3]: Leaving directory > `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64/bfd' > make[2]: *** [all] Error 2 > make[2]: Leaving directory > `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64/bfd' > make[1]: *** [all-bfd] Error 2 > make[1]: Leaving directory > `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64' > make: *** [build/linux-amd64/bfd/libbfd.a] Error 2 > From david.holmes at oracle.com Mon Jan 12 02:33:20 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Jan 2015 12:33:20 +1000 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: <3A9DFD5D-A70D-4CD3-9B56-88B75F771D0A@oracle.com> References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> <54AF243F.7050001@oracle.com> <22C65166-ABB5-408E-8A7C-5DDE6D61BAEE@oracle.com> <3A9DFD5D-A70D-4CD3-9B56-88B75F771D0A@oracle.com> Message-ID: <54B33270.4070400@oracle.com> On 10/01/2015 4:11 AM, Kim Barrett wrote: > On Jan 8, 2015, at 8:05 PM, Kim Barrett wrote: >> >> On Jan 8, 2015, at 7:43 PM, Coleen Phillimore wrote: >>> >>> >>> My prior email asked if you could test this change by changing an assert to vmassert. The assert in debug.cpp test_error_handler to vmassert would be a good one to test this change. >> >> Ah, now I understand what you were asking for. OK, I?ll do that and post a new webrev later this evening. > > New webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03/ > Incremental webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03.incr/ Webrev looks fine but for some reason you patch file did not update: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03/hotspot.changeset still seems to be version 1. Thanks, David > As before, only the full webrev contains copyright updates. > > I ran jtreg locally and verified test/runtime/6888954/vmerrors.sh ran successfully. > From david.holmes at oracle.com Mon Jan 12 02:45:41 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Jan 2015 12:45:41 +1000 Subject: RFR: JDK-8067479: verify-modules fails in bootcycle build In-Reply-To: <54AFE70C.3070208@oracle.com> References: <54AFE70C.3070208@oracle.com> Message-ID: <54B33555.8030206@oracle.com> Hi Erik, On 10/01/2015 12:34 AM, Erik Joelsson wrote: > Hello, > > Please review this patch which fixes the verify-modules target when > running bootcycle build, and also reenables verify-modules when running > "make images". > > There were two problems: > > * The bootcycle build configuration was broken so that both the normal > and the bootcycle build used the same HOTSPOT_DIST directory. The > consequence of this was that verify-modules worked when run on its own, > but not if bootcycle-images had been run before. This is fixed in > bootcycle-spec.gmk.in. > > * Since javac in JDK 9 no longer emits classes for implicitly compiled > sources, certain classes in sa-jdi.jar were not compiled during the > bootcycle build. I fixed this by adding the missing classes to sa.files. > Not having the classes there might have been intentional (in at least > some cases), but since they were compiled anyway, I felt it safer to > just add them to the list to fix this issue. If these classes shouldn't > be included, then they need to be properly removed in a followup fix. SA is owned by serviceability - cc'd. Changes seem okay as a solution to immediate problem, but I don't think anyone expects the IA64 stuff to still be needed. It is on the todo list to eradicate IA64 IIRC. Looks like there is limited awareness of the need to keep sa.files up to date. :( Thanks, David > Bug: https://bugs.openjdk.java.net/browse/JDK-8067479 > Webrev: http://cr.openjdk.java.net/~erikj/8067479/webrev.01/ > > Since this is changing hotspot, I assume it will need to go in through a > hotspot forest. Which one? > > /Erik From dean.long at oracle.com Mon Jan 12 04:31:28 2015 From: dean.long at oracle.com (Dean Long) Date: Sun, 11 Jan 2015 20:31:28 -0800 Subject: RFR: AARCH64: Top-level JDK changes In-Reply-To: References: <545CFFA9.4070107@redhat.com> <545D0290.5080307@oracle.com> Message-ID: <54B34E20.5030506@oracle.com> I found a small problem with the new config.sub wrapper. It works with the bash shell but not with the dash shell. The problem seems to be with this line: result=`. $DIR/autoconf-config.sub $sub_args "$@"` "dash" doesn't seem to support args passed with ".", so $sub_args "$@" are ignored. dl From goetz.lindenmaier at sap.com Mon Jan 12 07:01:51 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 12 Jan 2015 07:01:51 +0000 Subject: RFR(M): 8067941: [TESTBUG] Fix tests for OS with 64K page size. In-Reply-To: <036a01d02cdb$51989b00$f4c9d100$@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF43394@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CF69221@DEWDFEMB12A.global.corp.sap> <036a01d02cdb$51989b00$f4c9d100$@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF6DBA4@DEWDFEMB12A.global.corp.sap> Hi Christian, it would be great if you could push this change. The webrev is fine from my side. Thanks a lot for your help! Best regards, Goetz. -----Original Message----- From: Christian Tornqvist [mailto:christian.tornqvist at oracle.com] Sent: Saturday, January 10, 2015 2:43 PM To: Lindenmaier, Goetz; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: RE: RFR(M): 8067941: [TESTBUG] Fix tests for OS with 64K page size. Hi Goetz, The WBStackSize.java change looks fine, I can sponsor the change if you still need a sponsor. Thanks, Christian -----Original Message----- From: hotspot-runtime-dev [mailto:hotspot-runtime-dev-bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz Sent: Monday, January 5, 2015 4:36 AM To: 'hotspot-dev at openjdk.java.net'; hotspot-runtime-dev at openjdk.java.net Subject: RE: RFR(M): 8067941: [TESTBUG] Fix tests for OS with 64K page size. Hi, could someone from runtime please have a look at the last of these fixes? runtime/whitebox/WBStackSize.java http://cr.openjdk.java.net/~goetz/webrevs/8067941-64K/webrev.01/test/runtime /whitebox/WBStackSize.java.udiff.html I get the shadow page size from WB to compute the space wasted on the stack so far. And I please need a sponsor for this! Best regards, Goetz. From: Lindenmaier, Goetz Sent: Freitag, 19. Dezember 2014 17:06 To: 'hotspot-dev at openjdk.java.net' Subject: RFR(M): 8067941: [TESTBUG] Fix tests for OS with 64K page size. Hi, please review this simple change. I please need a sponsor. The change should be downported to 8u60. It applies nicely except for some missing files. http://cr.openjdk.java.net/~goetz/webrevs/8067941-64K/webrev.01/ The issue is that we have linux ppc64 machines that have default page size of 64K. This imposes bigger limits on stack and heap sizes than if running with 4k pages. Ergonomics increase some flags if they are too small. Increasing the sizes used in the test slightly helps in most cases. Details: runtime/whitebox/WBStackSize.java The constant esimating the so far occupied stack is too small. Fix: Consider size of shadow pages compiler/uncommontrap/TestStackBangRbp.java: compiler/uncommontrap/TestStackBangMonitorOwned.java compiler/uncommontrap/StackOverflowGuardPagesOff.java compiler/uncommontrap/8009761/Test8009761.java compiler/runtime/6865265/StackOverflowBug.java compiler/exceptions/TestRecursiveReplacedException.java Increase stack size to 392K, need at least 328K. sanity/ExecuteInternalVMTests.java - Assertion in metaspace.cpp simly must fail with vm_page_size == 64K as it basically asserts vm_page_size < 16K - With 64K pages, heap sizes are computed differently starting from the alignment which is card_size * vm_page_size. gc/arguments/TestMaxHeapSizeTools.java Sizes are alinged up being equal in the end. Choose bigger sizes. gc/g1/TestHumongousAllocInitialMark.java Test computes some size based on the -Xmx value it uses. Heap size is increased slightly with 64K making the computation wrong and the test fail. Choose heap size that needs not be aligned. gc/g1/TestGCLogMessages.java Heap of 10M is increased to 32M because of the alignment with 64K pages. This makes the evacuation succeed. Choose 32M from the beginning and adapt size of huge object. Best regards, Goetz. From staffan.larsen at oracle.com Mon Jan 12 07:51:13 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 12 Jan 2015 08:51:13 +0100 Subject: RFR: 6584008: jvmtiStringPrimitiveCallback should not be invoked when string value is null In-Reply-To: <54B01E29.2090309@redhat.com> References: <54B01E29.2090309@redhat.com> Message-ID: Looks good. But I would appreciate if you could add braces to the if-statement. There is no simple way to compile and run JVMTI agents in jtreg today, although we have started to add support for it. Thanks, /Staffan > On 9 jan 2015, at 19:30, Andrew Haley wrote: > > Webrev at http://cr.openjdk.java.net/~aph/JDK-6584008/ > > I have attached a reporducer to > https://bugs.openjdk.java.net/browse/JDK-6584008 but I'm not sure how > to turn it into a proper JTREG test. > > Andrew. From goetz.lindenmaier at sap.com Mon Jan 12 07:57:31 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 12 Jan 2015 07:57:31 +0000 Subject: RFR(M): 8068503: ppc64: Encode/Decode nodes for disjoint cOops mode In-Reply-To: <54AD4885.9080700@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF69B22@DEWDFEMB12A.global.corp.sap> <54AD4885.9080700@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF6DDED@DEWDFEMB12A.global.corp.sap> Hi Coleen, Now I implemented the optimization in macroAssembler, also for the decode case, anyways. Unfortunately it's only used in the methodHandles code. Testing took a bit because the results were spoiled by unrelated problems. I uploaded a new webrev: http://cr.openjdk.java.net/~goetz/webrevs/8068503-ppcDisjoint/webrev.02/ Best regards, Goetz. -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore Sent: Wednesday, January 07, 2015 3:54 PM To: hotspot-dev at openjdk.java.net Subject: Re: RFR(M): 8068503: ppc64: Encode/Decode nodes for disjoint cOops mode Don't you have to make (or want to make) changes to MacroAssembler::decode_heap_oop_not_null in the macro assembler files? Coleen On 1/7/15, 3:26 AM, Lindenmaier, Goetz wrote: > Hi, > > this change contains the implementation (activation) of the > encode/decode nodes exploiting the disjoint heap base > available since "8064457: Introduce compressed oops mode disjoint base..." > > Please review this change. > http://cr.openjdk.java.net/~goetz/webrevs/8068503-ppcDisjoint/ > > This is ppc64-only, so I don't need a sponsor ;) > > Best regards, > Goetz. > From volker.simonis at gmail.com Mon Jan 12 08:14:01 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 12 Jan 2015 09:14:01 +0100 Subject: iinc_w instruction and hsdis build instructions ... In-Reply-To: References: Message-ID: Hi Kedar, the hsdis build error is probably because of a new warning in recent versions of GCC. See https://gcc.gnu.org/gcc-4.6/porting_to.html Quoted from there: New warnings for unused variables and parameters The behavior of -Wall has changed and now includes the new warning flags -Wunused-but-set-variable and (with -Wall -Wextra) -Wunused-but-set-parameter. This may result in new warnings in code that compiled cleanly with previous versions of GCC. For example, void fn (void) { int foo; foo = bar (); /* foo is never used. */ } Gives the following diagnostic: warning: variable "foo" set but not used [-Wunused-but-set-variable] Although these warnings will not result in compilation failure, often -Wall is used in conjunction with -Werror and as a result, new warnings are turned into new errors. To fix, first see if the unused variable or parameter can be removed without changing the result or logic of the surrounding code. If not, annotate it with __attribute__((__unused__)). As a workaround, add -Wno-error=unused-but-set-variable or -Wno-error=unused-but-set-parameter. Regards, Volker On Sat, Jan 10, 2015 at 12:50 AM, kedar mhaswade wrote: > Hello JVM developers! > I hope I am posting to the right mailing list, if not, please let me know > where I should direct these two questions. > > 1) When generating the bytecode by javac-7 and javac-8, I see iinc_w > instruction (and javap shows subtle differences in disassembly on the same > java class file). I don't find a direct reference to this instruction at: > http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.html#jvms-6.5. > > Where should I be looking for the documentation? > > 2) I want to build hsdis on Ubuntu 14.04. I have tried with a bunch of > binutils versions, but it fails rather obscurely [1]. Are there any > instructions for a successful build on Ubuntu/Debian? > > Thanks! > -Kedar > > 1- /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd > -I. -I. > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/../include > -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Werror -fPIC -O -c > -o elf64-x86-64.lo > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c > libtool: compile: gcc -DHAVE_CONFIG_H -I. > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd > -I. -I. > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/../include > -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Werror -fPIC -O -c > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c > -o elf64-x86-64.o > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c: > In function 'elf64_x86_64_relocate_section': > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:2394:16: > error: variable 'warned' set but not used [-Werror=unused-but-set-variable] > bfd_boolean warned; > ^ > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:2870:29: > error: variable 'type2' set but not used [-Werror=unused-but-set-variable] > unsigned int val, type, type2; > ^ > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3116:29: > error: variable 'type2' set but not used [-Werror=unused-but-set-variable] > unsigned int val, type, type2; > ^ > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3116:23: > error: variable 'type' set but not used [-Werror=unused-but-set-variable] > unsigned int val, type, type2; > ^ > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3116:18: > error: variable 'val' set but not used [-Werror=unused-but-set-variable] > unsigned int val, type, type2; > ^ > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3147:23: > error: variable 'type' set but not used [-Werror=unused-but-set-variable] > unsigned int val, type; > ^ > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3147:18: > error: variable 'val' set but not used [-Werror=unused-but-set-variable] > unsigned int val, type; > ^ > cc1: all warnings being treated as errors > make[4]: *** [elf64-x86-64.lo] Error 1 > make[4]: Leaving directory > `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64/bfd' > make[3]: *** [all-recursive] Error 1 > make[3]: Leaving directory > `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64/bfd' > make[2]: *** [all] Error 2 > make[2]: Leaving directory > `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64/bfd' > make[1]: *** [all-bfd] Error 2 > make[1]: Leaving directory > `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64' > make: *** [build/linux-amd64/bfd/libbfd.a] Error 2 From goetz.lindenmaier at sap.com Mon Jan 12 08:22:50 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 12 Jan 2015 08:22:50 +0000 Subject: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available Message-ID: <4295855A5C1DE049A61835A1887419CC2CF6DE0B@DEWDFEMB12A.global.corp.sap> Hi, This test uses SA, thus fails on platforms where that's not implemented. I see problems on mac and aix. Call Platform.shouldSAAttach() to check whether SA should work. Please review this small fix. I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8068778-jtregSA/webrev.01/ Best regards, Goetz. From erik.helin at oracle.com Mon Jan 12 09:02:46 2015 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 12 Jan 2015 10:02:46 +0100 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: <54B33270.4070400@oracle.com> References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> <54AF243F.7050001@oracle.com> <22C65166-ABB5-408E-8A7C-5DDE6D61BAEE@oracle.com> <3A9DFD5D-A70D-4CD3-9B56-88B75F771D0A@oracle.com> <54B33270.4070400@oracle.com> Message-ID: <20150112090246.GI3035@ehelin.jrpg.bea.com> On 2015-01-12, David Holmes wrote: > On 10/01/2015 4:11 AM, Kim Barrett wrote: > >On Jan 8, 2015, at 8:05 PM, Kim Barrett wrote: > >> > >>On Jan 8, 2015, at 7:43 PM, Coleen Phillimore wrote: > >>> > >>> > >>>My prior email asked if you could test this change by changing an assert to vmassert. The assert in debug.cpp test_error_handler to vmassert would be a good one to test this change. > >> > >>Ah, now I understand what you were asking for. OK, I?ll do that and post a new webrev later this evening. > > > >New webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03/ > >Incremental webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03.incr/ > > Webrev looks fine but for some reason you patch file did not update: > > http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03/hotspot.changeset > > still seems to be version 1. I suspect that Kim is using `webrev -r` to create the webrev, this is known to produce broken patch files: https://bugs.openjdk.java.net/browse/CODETOOLS-7901115 Thanks, Erik > Thanks, > David > > >As before, only the full webrev contains copyright updates. > > > >I ran jtreg locally and verified test/runtime/6888954/vmerrors.sh ran successfully. > > From erik.helin at oracle.com Mon Jan 12 09:20:01 2015 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 12 Jan 2015 10:20:01 +0100 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: <3A9DFD5D-A70D-4CD3-9B56-88B75F771D0A@oracle.com> References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> <54AF243F.7050001@oracle.com> <22C65166-ABB5-408E-8A7C-5DDE6D61BAEE@oracle.com> <3A9DFD5D-A70D-4CD3-9B56-88B75F771D0A@oracle.com> Message-ID: <20150112092001.GJ3035@ehelin.jrpg.bea.com> On 2015-01-09, Kim Barrett wrote: > On Jan 8, 2015, at 8:05 PM, Kim Barrett wrote: > > > > On Jan 8, 2015, at 7:43 PM, Coleen Phillimore wrote: > >> > >> > >> My prior email asked if you could test this change by changing an assert to vmassert. The assert in debug.cpp test_error_handler to vmassert would be a good one to test this change. > > > > Ah, now I understand what you were asking for. OK, I?ll do that and post a new webrev later this evening. > > New webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03/ > Incremental webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03.incr/ Looks good, Reviewed. Thanks, Erik > As before, only the full webrev contains copyright updates. > > I ran jtreg locally and verified test/runtime/6888954/vmerrors.sh ran successfully. > From aph at redhat.com Mon Jan 12 10:25:12 2015 From: aph at redhat.com (Andrew Haley) Date: Mon, 12 Jan 2015 10:25:12 +0000 Subject: RFR: AARCH64: Top-level JDK changes In-Reply-To: <54B34E20.5030506@oracle.com> References: <545CFFA9.4070107@redhat.com> <545D0290.5080307@oracle.com> <54B34E20.5030506@oracle.com> Message-ID: <54B3A108.9070203@redhat.com> On 12/01/15 04:31, Dean Long wrote: > I found a small problem with the new config.sub wrapper. It works with > the bash shell but not with the dash shell. > The problem seems to be with this line: > > result=`. $DIR/autoconf-config.sub $sub_args "$@"` > > "dash" doesn't seem to support args passed with ".", so $sub_args "$@" > are ignored. Thank you. Sorry for the breakage; I didn't intend to use any non-portable shell idioms. I'm not expert enough to say whether this is a bug in dash, but I don't suppose that matters: we are where we are. There's no reason not to replace the "." with "bash". Do you want me to do anything? Andrew. From erik.joelsson at oracle.com Mon Jan 12 10:35:56 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 12 Jan 2015 11:35:56 +0100 Subject: [8u60] RFR: 8043340: [macosx] Fix hard-wired paths to JavaVM.framework In-Reply-To: <39719ED3-1F4E-4E6C-AA9F-A3C4766537B4@oracle.com> References: <39719ED3-1F4E-4E6C-AA9F-A3C4766537B4@oracle.com> Message-ID: <54B3A38C.10909@oracle.com> Hello, These changes look ok to me. With these changes, configure will unconditionally fail if trying to use XCode 5. I know we don't officially support using XCode 5 for JDK 8, but aren't people working around it with some patches? How hard would it be to make it at least build? /Erik On 2015-01-10 05:45, David DeHaven wrote: > Please review the open source changes for 8043340. The goal here is to get jdk8u to build on Mac OS X 10.9+ systems where Xcode 5+ and Xcode 4 are co-installed, a configuration which is becoming more and more commonplace as more developers are focusing on JDK 9 now (which needs Xcode 5 installed), not to mention new systems ship with 10.10 which only supports the Xcode 5/6 command line tools. It's too much of a hassle to maintain separate systems for building jdk8u and JPRT isn't suitable for frequent changes so something must be done to address this. > > The jdk changes are similar to those done for JDK9, though I removed the changes for building with Xcode 5 (JDK-8043591) since we do not support building JDK 8 with clang. > > The hotspot changes are almost identical, the lack of SYSROOT_CFLAGS necessitated changing the logic in saproc.make a bit. It still builds with or without spec.gmk, though without it you will either need to define SDKPATH or have a sane Xcode 4 installation. > > For the top level build system: > - most of the logic of sanitizing the Xcode build environment is in toolchain.m4 > - the LIPO variable was removed since it was completely unused > - OTOOL was moved to after the Xcode sanitizing so it can be picked up from DEVELOPER_DIR if needed > - MACOSX_UNIVERSAL is now being set to false by default and ALT_MACOSX_UNIVERSAL was added to hotspot-spec.gmk.in so the hotspot build is in sync with the jdk build (this was a bug, IMHO) > > That last change removed any need to run lipo (only done in hotspot), so the fact that /usr/bin/lipo is broken with Xcode 4 is a non-issue. > > There is a weird case where some early versions of the Xcode 5 command line tools installed /usr/bin/{gcc|g++} as a symlink to {clang|clang++}, that case is handled by putting $DEVELOPER_DIR/usr/bin on the path so autoconf picks up the actual gcc executable. > > JBS Issue: > https://bugs.openjdk.java.net/browse/JDK-8043340 > > Webrevs: > http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v0/top > http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v0/hotspot > http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v0/jdk > > JPRT runs are being kicked off. I'll have one run from hotspot directly. I'll post results here. > > -DrD- > From david.holmes at oracle.com Mon Jan 12 11:26:37 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Jan 2015 21:26:37 +1000 Subject: De-universalizing hotspot in jdk8u In-Reply-To: <54B3A4D0.3080604@oracle.com> References: <54B300FE.2000804@oracle.com> <54B3A4D0.3080604@oracle.com> Message-ID: <54B3AF6D.20607@oracle.com> cc'ing macosx-port-dev and hotspot-dev On 12/01/2015 8:41 PM, Erik Joelsson wrote: > > On 2015-01-12 00:02, David Holmes wrote: >> Hi David, >> >> On 10/01/2015 2:00 AM, David DeHaven wrote: >>> >>> We have this nice little comment in common/autoconf/jdk-options.m4: >>> # On Macosx universal binaries are produced, but they only contain >>> # 64 bit intel. This invalidates control of which jvms are built >>> # from configure, but only server is valid anyway. Fix this >>> # when hotspot makefiles are rewritten. >>> if test "x$MACOSX_UNIVERSAL" = xtrue; then >>> HOTSPOT_TARGET=universal_${HOTSPOT_EXPORT} >>> fi >>> >>> >>> So.. I turned it off: >>> if test "x$OPENJDK_TARGET_OS" = "xmacosx"; then >>> # MACOSX_UNIVERSAL="true" >>> MACOSX_UNIVERSAL="false" >>> fi >>> >>> And in hotspot/make/bsd/makefiles/defs.make: >>> # Universal build settings >>> ifeq ($(OS_VENDOR), Darwin) >>> # Build universal binaries by default on Mac OS X >>> # MACOSX_UNIVERSAL = true >>> MACOSX_UNIVERSAL = false >>> ifneq ($(ALT_MACOSX_UNIVERSAL),) >>> >>> hotspot still seems to build happily (I'm kicking off tests now). Is >>> there are particular reason this hasn't been done yet? I'd like to >>> know before I pursue this as a means of eliminating our dependency on >>> lipo which is causing an inordinate amount of grief when trying to >>> build on 10.9 and later when Xcode 5+ is coinstalled. I have some >>> logic to work around using the broken /usr/bin/lipo, but it doesn't >>> help later in some "other" build target that ends up using '-arch >>> i386 -arch x86_64'. It does not explicitly run lipo, it relies on gcc >>> to do the fattening itself and so it fails even though I've gone to >>> the effort of working around the broken lipo (and it isn't necessary >>> anyways because it doesn't even build the i386 binaries even though >>> it's supposed to be "universal"). >> >> The hotspot makefiles still haven't been rewritten (though Magnus had >> been working on it). >> >> Also I was under the impression that we used the universal stuff >> because we had to deliver in a particular way on OSX. But I don't know >> the details and the people who dealt with the original OSX port effort >> are no longer here. >> > I don't know why Hotspot is packaged as a universal binary. In the jdk, > only libJObjC was built as a universal binary and that lib was removed > before JDK 8 shipped. What part of Macosx would need to interact > directly with libjvm.dylib in a way that required a (fake) universal > format, but no other libs in the jdk? I would say go ahead and change it. In the spirit of "never take down a fence until you know why it was put up in the first place" I've cc'd macosx-port-dev to see if there is anyone around who recalls why hotspot is using the universal binary feature. David H. ------- > /Erik > >> David H. >> >>> >>> Quite frankly I'd rather just remove all the universal binary stuff >>> from hotspot, but I'm trying my best to minimize the changes there... >>> the "other" problem I can handle easily. >>> >>> -DrD- >>> > From magnus.ihse.bursie at oracle.com Mon Jan 12 11:49:34 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 12 Jan 2015 12:49:34 +0100 Subject: RFR: AARCH64: Top-level JDK changes In-Reply-To: <54B34E20.5030506@oracle.com> References: <545CFFA9.4070107@redhat.com> <545D0290.5080307@oracle.com> <54B34E20.5030506@oracle.com> Message-ID: <54B3B4CE.8060804@oracle.com> On 2015-01-12 05:31, Dean Long wrote: > I found a small problem with the new config.sub wrapper. It works > with the bash shell but not with the dash shell. > The problem seems to be with this line: > > result=`. $DIR/autoconf-config.sub $sub_args "$@"` > > "dash" doesn't seem to support args passed with ".", so $sub_args "$@" > are ignored. bash is the required shell for running configure. We do not support non-bash shells. In fact, we go to lengths to try to ensure that we are indeed running under bash. /Magnus From erik.osterlund at lnu.se Mon Jan 12 13:18:21 2015 From: erik.osterlund at lnu.se (=?iso-8859-1?Q?Erik_=D6sterlund?=) Date: Mon, 12 Jan 2015 13:18:21 +0000 Subject: OrderAccess Refactoring References: Message-ID: Hi Paul, Thank you for clearing things out, it really helps talking to the person who originally wrote this code! It seems to me that this code was written at a time when it was vaguely and incorrectly documented both what the hardware and compilers guarantee. Must have been a nightmare! I think it's still vague today. On 09/01/15 23:45, Paul Hohensee wrote > We understood that volatile semantics wouldn't be enough by themselves > to implement the semantics we wanted ordering on most platforms, but > they *are* necessary to prevent compilers (not the hardware though) from > reordering volatile accesses with respect to each other. Whether or not > the volatile keyword prevents non-volatile memory accesses from being > reordered wrt volatile ones seemed a bit fuzzy in the C++ standard at > the time (and still does, at least to me), but we figured we could > handle that in the os_cpu files by checking which compiler was being > used. The code that implemented the *hardware* semantics also went into > the os_cpu file(s). That's why there's some seemingly silly code there, > including assembly that executes what amount to nops. I agree it was and still is very vague which is why different compilers interpreted it differently. MSVC is explicit about using acquire release for volatiles (wanted) and GCC is explicit about not guaranteeing non-volatile w.r.t. volatile. I have not read anything explicit from the others so will assume it's unsafe to rely on anything unless we find explicit proof it is reliable. I saw the strange barriers with nops and dummy volatile writes you speak of, and intend to change these to normal compiler barriers instead. Because the dummy volatile memory accesses don't actually give us the guarantees we want for the same reason outlined before (volatile semantics not good enough), while also looking a bit strange at the same time haha! > I initially wrote the code in factored form, but changed to the current > scheme because I found it difficult to see all the code for a single > platform in one place (I wasn't using an IDE :) ). In retrospect, an > unwise decision due to the usual code duplication problems. Mea culpa. :) :) Thank you for the background and useful context Paul! /Erik From erik.osterlund at lnu.se Mon Jan 12 14:18:51 2015 From: erik.osterlund at lnu.se (=?iso-8859-1?Q?Erik_=D6sterlund?=) Date: Mon, 12 Jan 2015 14:18:51 +0000 Subject: OrderAccess Refactoring References: <78E41D90-81F0-4526-B7A3-A02A6B030515@lnu.se> <54B0FADC.80600@oracle.com> Message-ID: Hi David, Thank you for the feedback! :) On 10/01/15 11:11, David Holmes wrote: > acquire/release semantics are problematic because there is no single > common accepted meaning. Even talking about acquire() and release() as > independent operations is somewhat meaningless - it is only when they > are paired, and fixed in place that they start to give meaning to things. I agree it is a bit vague. The description in the comments about preventing all memory accesses from floating up and down is the one I recognize and it is conceptually good. But then it is later defined in a table as acquire = Load(Store|Load) and release = (Store|Load)Store. The reason is understandable - even though the description of acquire/release is quite general, they are only intended and designed to be used in certain contexts like release store and load acquire (with variants). Outside such similar contexts, it would not give the guarantees described. This gets more confusing when you go on reading that fence is conceptually acquire and release. It is true on the conceptual level using the conceptual descriptions of acquire/release, but the actual guarantees of release + acquire is StoreStore, LoadLoad and LoadStore according to the tables below, while fence() also provides StoreLoad constraints. I don't know about you but I find that a bit misleading. I mean, it might be quite obvious for anyone using it to only use acquire/release in the intended contexts where it makes sense and then there is no trouble, but I guess it doesn't hurt to add some comments making it explicit to avoid any confusion. We don't want confusion I think. > load_acquire/store_release don't necessarily map cleanly to > acquire()/release() as currently described in orderAccess.hpp; nor do > they necessarily map cleanly to the same named operations on eg ARMv8; > nor to similar named entities in C++11. As Paul stated these were > introduced to map to the ia64 architecture - which I believe had > somewhat different semantics. They do map cleanly after I change acquire() and release() to compiler barriers on TSO platforms (and fencing on PPC). Except some exceptions like load acquire on PPC that was specialized for speed using twi-isync and x86 on windows that don't need the stronger barriers since volatile is more accurate anyway. > This whole area is a minefield because there are different concepts of > what "acquire" and "release" may mean - so you need to start with > detailed definitions. Personally, and I know I am not alone, I abhor the > acquire/release terminology and usage, I much prefer the unambiguous > storeload, storestore etc. :) Yes I think we need a better concrete guarantee/contract. The current conceptual description of acquire/release is nice I think and useful also for understanding when to use it, but I want to make it explicit that release is only used in conjunction with writes and acquire only in conjunction with loads (and swapping instructions etc), and therefore do not give the same guarantees as described as the more general conceptual description promises. The concrete guarantee users can rely on is acquire = Load(Store|Load) and release = (Store|Load)Store, and that's consistent with the implementation and reasonable intended usage. Hence, it's fine to use them in the way acquire/release is described now, iff these barriers suffice to guarantee it, otherwise acquire/release should not be used in the given context. Does that make sense? > The current code often expresses things in a semantically incorrect way > eg that: > > store_release(addr, val) == store(addr, val); release(); > > but in practice the end code provides what is required (often providing > a stronger barrier because that's all that is available on a platform). I'm not sure I get what you mean by this being semantically incorrect. I get it's suboptimal and might be slower triggering compiler barriers that won't be necessary when joined, but semantically surely it should be the same, right? > And I agree with Karen - tackle correctness first then refactoring. IT > will be too hard to track correctness otherwise. Personally, given I > work in this code quite a bit, I like to see the complete set of > definitions for a platform in one place. The duplication is something I > can live with when it is mostly one-liners. Okay no problem. I built the general variant first (because it was easier since I don't have access to other platforms except mine haha). But I fully understand yours and Karen's motivation so I will get back to you when I manage to do it the code duplicated way so it's easier to follow the evolution. Oh and speaking of which, I find it very strange that store_fence and release_store_fence is duplicated on x86 because we don't want to cast away the volatile. This whole non-volatile thing seems a bit off to me - surely all calls to OrderedAccess should be expected to be volatile, nothing else would make any sense? And if so, why is store_fence the one exception that is non-volatile while none of the others are? Unless anyone knows there is a good reason for this, I'd like to make it consistent with the rest of the code and make it volatile as well (and hence get rid of some code duplication while at it). Permission granted? ;) Thank you David for a nice discussion about this! :) /Erik From kedar.mhaswade at gmail.com Mon Jan 12 14:19:14 2015 From: kedar.mhaswade at gmail.com (kedar mhaswade) Date: Mon, 12 Jan 2015 06:19:14 -0800 Subject: iinc_w instruction and hsdis build instructions ... In-Reply-To: References: Message-ID: Thank you! I tried the -Wno-error=unused-but-set-variable workaround and it seems to have worked after running into another error [1]. I now have hsdis-amd64.so. Regards, Kedar 1- make CFLAGS="-Wno-error=unused-but-set-variable" BINUTILS=binutils-2.23.2 ARCH=amd64 is the command which runs into linking statically with libbfd.a (/tmp/ccUxTKNw.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC). Instead of linking statically I chose to use the shared object, so, I installed the binutils-dev package that gives the symlink (/usr/lib/libbfd.so -> libbfd-2.24-system.so) and then used -lbfd like so: make CFLAGS="-Wno-error=unused-but-set-variable -lbfd" BINUTILS=binutils-2.23.2 ARCH=amd64. This seems to have fixed the build. Hopefully I built the right hsdis so :-). On Mon, Jan 12, 2015 at 12:14 AM, Volker Simonis wrote: > Hi Kedar, > > the hsdis build error is probably because of a new warning in recent > versions of GCC. See https://gcc.gnu.org/gcc-4.6/porting_to.html > > Quoted from there: > > New warnings for unused variables and parameters > > The behavior of -Wall has changed and now includes the new warning > flags -Wunused-but-set-variable and (with -Wall -Wextra) > -Wunused-but-set-parameter. This may result in new warnings in code > that compiled cleanly with previous versions of GCC. > > For example, > > void fn (void) > { > int foo; > foo = bar (); /* foo is never used. */ > } > > Gives the following diagnostic: > > warning: variable "foo" set but not used [-Wunused-but-set-variable] > > Although these warnings will not result in compilation failure, often > -Wall is used in conjunction with -Werror and as a result, new > warnings are turned into new errors. > > To fix, first see if the unused variable or parameter can be removed > without changing the result or logic of the surrounding code. If not, > annotate it with __attribute__((__unused__)). > > As a workaround, add -Wno-error=unused-but-set-variable or > -Wno-error=unused-but-set-parameter. > > Regards, > Volker > > > On Sat, Jan 10, 2015 at 12:50 AM, kedar mhaswade > wrote: > > Hello JVM developers! > > I hope I am posting to the right mailing list, if not, please let me > know > > where I should direct these two questions. > > > > 1) When generating the bytecode by javac-7 and javac-8, I see iinc_w > > instruction (and javap shows subtle differences in disassembly on the > same > > java class file). I don't find a direct reference to this instruction at: > > http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.html#jvms-6.5. > > > > Where should I be looking for the documentation? > > > > 2) I want to build hsdis on Ubuntu 14.04. I have tried with a bunch of > > binutils versions, but it fails rather obscurely [1]. Are there any > > instructions for a successful build on Ubuntu/Debian? > > > > Thanks! > > -Kedar > > > > 1- /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > > > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd > > -I. -I. > > > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd > > > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/../include > > -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Werror -fPIC -O -c > > -o elf64-x86-64.lo > > > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c > > libtool: compile: gcc -DHAVE_CONFIG_H -I. > > > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd > > -I. -I. > > > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd > > > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/../include > > -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Werror -fPIC -O -c > > > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c > > -o elf64-x86-64.o > > > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c: > > In function 'elf64_x86_64_relocate_section': > > > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:2394:16: > > error: variable 'warned' set but not used > [-Werror=unused-but-set-variable] > > bfd_boolean warned; > > ^ > > > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:2870:29: > > error: variable 'type2' set but not used > [-Werror=unused-but-set-variable] > > unsigned int val, type, type2; > > ^ > > > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3116:29: > > error: variable 'type2' set but not used > [-Werror=unused-but-set-variable] > > unsigned int val, type, type2; > > ^ > > > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3116:23: > > error: variable 'type' set but not used [-Werror=unused-but-set-variable] > > unsigned int val, type, type2; > > ^ > > > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3116:18: > > error: variable 'val' set but not used [-Werror=unused-but-set-variable] > > unsigned int val, type, type2; > > ^ > > > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3147:23: > > error: variable 'type' set but not used [-Werror=unused-but-set-variable] > > unsigned int val, type; > > ^ > > > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3147:18: > > error: variable 'val' set but not used [-Werror=unused-but-set-variable] > > unsigned int val, type; > > ^ > > cc1: all warnings being treated as errors > > make[4]: *** [elf64-x86-64.lo] Error 1 > > make[4]: Leaving directory > > > `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64/bfd' > > make[3]: *** [all-recursive] Error 1 > > make[3]: Leaving directory > > > `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64/bfd' > > make[2]: *** [all] Error 2 > > make[2]: Leaving directory > > > `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64/bfd' > > make[1]: *** [all-bfd] Error 2 > > make[1]: Leaving directory > > > `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64' > > make: *** [build/linux-amd64/bfd/libbfd.a] Error 2 > From volker.simonis at gmail.com Mon Jan 12 14:38:46 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 12 Jan 2015 15:38:46 +0100 Subject: iinc_w instruction and hsdis build instructions ... In-Reply-To: References: Message-ID: Glad you succeeded to build it. I dont think you have to set ARCH on the command line. If you only want to build the 64-bit version of hsdis it should be enough to build the target "all64" as described in the README file. Regards, Volker PS: you can easily check if your hsdis works by copying it into the the directory where you libjvm.so is located (i.e. jre/lib/amd64/server) or by exporting it's location in LD_LIBRARY_PATH and running "java -XX:+UnlockDiagnosticVMOptions -XX:+PrintInterpreter " On Mon, Jan 12, 2015 at 3:19 PM, kedar mhaswade wrote: > Thank you! > I tried the -Wno-error=unused-but-set-variable workaround and it seems to > have worked after running into another error [1]. > I now have hsdis-amd64.so. > > Regards, > Kedar > 1- make CFLAGS="-Wno-error=unused-but-set-variable" BINUTILS=binutils-2.23.2 > ARCH=amd64 is the command which runs into linking statically with libbfd.a > (/tmp/ccUxTKNw.o: relocation R_X86_64_32 against `.rodata' can not be used > when making a shared object; recompile with -fPIC). Instead of linking > statically I chose to use the shared object, so, I installed the > binutils-dev package that gives the symlink (/usr/lib/libbfd.so -> > libbfd-2.24-system.so) and then used -lbfd like so: make > CFLAGS="-Wno-error=unused-but-set-variable -lbfd" BINUTILS=binutils-2.23.2 > ARCH=amd64. This seems to have fixed the build. Hopefully I built the right > hsdis so :-). > > On Mon, Jan 12, 2015 at 12:14 AM, Volker Simonis > wrote: >> >> Hi Kedar, >> >> the hsdis build error is probably because of a new warning in recent >> versions of GCC. See https://gcc.gnu.org/gcc-4.6/porting_to.html >> >> Quoted from there: >> >> New warnings for unused variables and parameters >> >> The behavior of -Wall has changed and now includes the new warning >> flags -Wunused-but-set-variable and (with -Wall -Wextra) >> -Wunused-but-set-parameter. This may result in new warnings in code >> that compiled cleanly with previous versions of GCC. >> >> For example, >> >> void fn (void) >> { >> int foo; >> foo = bar (); /* foo is never used. */ >> } >> >> Gives the following diagnostic: >> >> warning: variable "foo" set but not used [-Wunused-but-set-variable] >> >> Although these warnings will not result in compilation failure, often >> -Wall is used in conjunction with -Werror and as a result, new >> warnings are turned into new errors. >> >> To fix, first see if the unused variable or parameter can be removed >> without changing the result or logic of the surrounding code. If not, >> annotate it with __attribute__((__unused__)). >> >> As a workaround, add -Wno-error=unused-but-set-variable or >> -Wno-error=unused-but-set-parameter. >> >> Regards, >> Volker >> >> >> On Sat, Jan 10, 2015 at 12:50 AM, kedar mhaswade >> wrote: >> > Hello JVM developers! >> > I hope I am posting to the right mailing list, if not, please let me >> > know >> > where I should direct these two questions. >> > >> > 1) When generating the bytecode by javac-7 and javac-8, I see iinc_w >> > instruction (and javap shows subtle differences in disassembly on the >> > same >> > java class file). I don't find a direct reference to this instruction >> > at: >> > http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.html#jvms-6.5. >> > >> > Where should I be looking for the documentation? >> > >> > 2) I want to build hsdis on Ubuntu 14.04. I have tried with a bunch of >> > binutils versions, but it fails rather obscurely [1]. Are there any >> > instructions for a successful build on Ubuntu/Debian? >> > >> > Thanks! >> > -Kedar >> > >> > 1- /bin/bash ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. >> > >> > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd >> > -I. -I. >> > >> > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd >> > >> > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/../include >> > -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Werror -fPIC -O >> > -c >> > -o elf64-x86-64.lo >> > >> > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c >> > libtool: compile: gcc -DHAVE_CONFIG_H -I. >> > >> > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd >> > -I. -I. >> > >> > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd >> > >> > -I/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/../include >> > -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Werror -fPIC -O -c >> > >> > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c >> > -o elf64-x86-64.o >> > >> > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c: >> > In function 'elf64_x86_64_relocate_section': >> > >> > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:2394:16: >> > error: variable 'warned' set but not used >> > [-Werror=unused-but-set-variable] >> > bfd_boolean warned; >> > ^ >> > >> > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:2870:29: >> > error: variable 'type2' set but not used >> > [-Werror=unused-but-set-variable] >> > unsigned int val, type, type2; >> > ^ >> > >> > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3116:29: >> > error: variable 'type2' set but not used >> > [-Werror=unused-but-set-variable] >> > unsigned int val, type, type2; >> > ^ >> > >> > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3116:23: >> > error: variable 'type' set but not used >> > [-Werror=unused-but-set-variable] >> > unsigned int val, type, type2; >> > ^ >> > >> > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3116:18: >> > error: variable 'val' set but not used [-Werror=unused-but-set-variable] >> > unsigned int val, type, type2; >> > ^ >> > >> > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3147:23: >> > error: variable 'type' set but not used >> > [-Werror=unused-but-set-variable] >> > unsigned int val, type; >> > ^ >> > >> > /media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/binutils-2.19.1/bfd/elf64-x86-64.c:3147:18: >> > error: variable 'val' set but not used [-Werror=unused-but-set-variable] >> > unsigned int val, type; >> > ^ >> > cc1: all warnings being treated as errors >> > make[4]: *** [elf64-x86-64.lo] Error 1 >> > make[4]: Leaving directory >> > >> > `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64/bfd' >> > make[3]: *** [all-recursive] Error 1 >> > make[3]: Leaving directory >> > >> > `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64/bfd' >> > make[2]: *** [all] Error 2 >> > make[2]: Leaving directory >> > >> > `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64/bfd' >> > make[1]: *** [all-bfd] Error 2 >> > make[1]: Leaving directory >> > >> > `/media/kedar/2tb1/Projects/hsdis/openjdk/hotspot/src/share/tools/hsdis/build/linux-amd64' >> > make: *** [build/linux-amd64/bfd/libbfd.a] Error 2 > > From david.dehaven at oracle.com Mon Jan 12 15:17:30 2015 From: david.dehaven at oracle.com (David DeHaven) Date: Mon, 12 Jan 2015 07:17:30 -0800 Subject: De-universalizing hotspot in jdk8u In-Reply-To: <54B3AF6D.20607@oracle.com> References: <54B300FE.2000804@oracle.com> <54B3A4D0.3080604@oracle.com> <54B3AF6D.20607@oracle.com> Message-ID: <825EBFC2-6DE1-4550-842B-217C06532C9B@oracle.com> >>>> We have this nice little comment in common/autoconf/jdk-options.m4: >>>> # On Macosx universal binaries are produced, but they only contain >>>> # 64 bit intel. This invalidates control of which jvms are built >>>> # from configure, but only server is valid anyway. Fix this >>>> # when hotspot makefiles are rewritten. >>>> if test "x$MACOSX_UNIVERSAL" = xtrue; then >>>> HOTSPOT_TARGET=universal_${HOTSPOT_EXPORT} >>>> fi >>>> >>>> >>>> So.. I turned it off: >>>> if test "x$OPENJDK_TARGET_OS" = "xmacosx"; then >>>> # MACOSX_UNIVERSAL="true" >>>> MACOSX_UNIVERSAL="false" >>>> fi >>>> >>>> And in hotspot/make/bsd/makefiles/defs.make: >>>> # Universal build settings >>>> ifeq ($(OS_VENDOR), Darwin) >>>> # Build universal binaries by default on Mac OS X >>>> # MACOSX_UNIVERSAL = true >>>> MACOSX_UNIVERSAL = false >>>> ifneq ($(ALT_MACOSX_UNIVERSAL),) >>>> >>>> hotspot still seems to build happily (I'm kicking off tests now). Is >>>> there are particular reason this hasn't been done yet? I'd like to >>>> know before I pursue this as a means of eliminating our dependency on >>>> lipo which is causing an inordinate amount of grief when trying to >>>> build on 10.9 and later when Xcode 5+ is coinstalled. I have some >>>> logic to work around using the broken /usr/bin/lipo, but it doesn't >>>> help later in some "other" build target that ends up using '-arch >>>> i386 -arch x86_64'. It does not explicitly run lipo, it relies on gcc >>>> to do the fattening itself and so it fails even though I've gone to >>>> the effort of working around the broken lipo (and it isn't necessary >>>> anyways because it doesn't even build the i386 binaries even though >>>> it's supposed to be "universal"). >>> >>> The hotspot makefiles still haven't been rewritten (though Magnus had >>> been working on it). >>> >>> Also I was under the impression that we used the universal stuff >>> because we had to deliver in a particular way on OSX. But I don't know >>> the details and the people who dealt with the original OSX port effort >>> are no longer here. >>> >> I don't know why Hotspot is packaged as a universal binary. In the jdk, >> only libJObjC was built as a universal binary and that lib was removed >> before JDK 8 shipped. What part of Macosx would need to interact >> directly with libjvm.dylib in a way that required a (fake) universal >> format, but no other libs in the jdk? I would say go ahead and change it. > > In the spirit of "never take down a fence until you know why it was put up in the first place" I've cc'd macosx-port-dev to see if there is anyone around who recalls why hotspot is using the universal binary feature. I hear you. My best guess is we'd initially planned on supporting 32 bit but (???) and the easiest way to deliver is in a universal binary. Note that we don't actually deliver universal binaries as the JVM is 64 bit only, or at least there are not nearly enough components there to run a full 32 bit JVM. -DrD- From david.dehaven at oracle.com Mon Jan 12 15:28:34 2015 From: david.dehaven at oracle.com (David DeHaven) Date: Mon, 12 Jan 2015 07:28:34 -0800 Subject: [8u60] RFR: 8043340: [macosx] Fix hard-wired paths to JavaVM.framework In-Reply-To: <54B3A38C.10909@oracle.com> References: <39719ED3-1F4E-4E6C-AA9F-A3C4766537B4@oracle.com> <54B3A38C.10909@oracle.com> Message-ID: <8B01B84D-5B6F-478E-ABF1-33F5ADE2F826@oracle.com> It won't build at all with Xcode 5, there is no gcc compiler and the clang changes were never backported to jdk8u. -DrD- > Hello, > > These changes look ok to me. > > With these changes, configure will unconditionally fail if trying to use XCode 5. I know we don't officially support using XCode 5 for JDK 8, but aren't people working around it with some patches? How hard would it be to make it at least build? > > /Erik > > On 2015-01-10 05:45, David DeHaven wrote: >> Please review the open source changes for 8043340. The goal here is to get jdk8u to build on Mac OS X 10.9+ systems where Xcode 5+ and Xcode 4 are co-installed, a configuration which is becoming more and more commonplace as more developers are focusing on JDK 9 now (which needs Xcode 5 installed), not to mention new systems ship with 10.10 which only supports the Xcode 5/6 command line tools. It's too much of a hassle to maintain separate systems for building jdk8u and JPRT isn't suitable for frequent changes so something must be done to address this. >> >> The jdk changes are similar to those done for JDK9, though I removed the changes for building with Xcode 5 (JDK-8043591) since we do not support building JDK 8 with clang. >> >> The hotspot changes are almost identical, the lack of SYSROOT_CFLAGS necessitated changing the logic in saproc.make a bit. It still builds with or without spec.gmk, though without it you will either need to define SDKPATH or have a sane Xcode 4 installation. >> >> For the top level build system: >> - most of the logic of sanitizing the Xcode build environment is in toolchain.m4 >> - the LIPO variable was removed since it was completely unused >> - OTOOL was moved to after the Xcode sanitizing so it can be picked up from DEVELOPER_DIR if needed >> - MACOSX_UNIVERSAL is now being set to false by default and ALT_MACOSX_UNIVERSAL was added to hotspot-spec.gmk.in so the hotspot build is in sync with the jdk build (this was a bug, IMHO) >> >> That last change removed any need to run lipo (only done in hotspot), so the fact that /usr/bin/lipo is broken with Xcode 4 is a non-issue. >> >> There is a weird case where some early versions of the Xcode 5 command line tools installed /usr/bin/{gcc|g++} as a symlink to {clang|clang++}, that case is handled by putting $DEVELOPER_DIR/usr/bin on the path so autoconf picks up the actual gcc executable. >> >> JBS Issue: >> https://bugs.openjdk.java.net/browse/JDK-8043340 >> >> Webrevs: >> http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v0/top >> http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v0/hotspot >> http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v0/jdk >> >> JPRT runs are being kicked off. I'll have one run from hotspot directly. I'll post results here. >> >> -DrD- >> > From kim.barrett at oracle.com Mon Jan 12 15:58:15 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 12 Jan 2015 10:58:15 -0500 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: <20150112090246.GI3035@ehelin.jrpg.bea.com> References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> <54AF243F.7050001@oracle.com> <22C65166-ABB5-408E-8A7C-5DDE6D61BAEE@oracle.com> <3A9DFD5D-A70D-4CD3-9B56-88B75F771D0A@oracle.com> <54B33270.4070400@oracle.com> <20150112090246.GI3035@ehelin.jrpg.bea.com> Message-ID: <55FE925E-BD23-48C8-BAA4-8365EDCBDB49@oracle.com> On Jan 12, 2015, at 4:02 AM, Erik Helin wrote: > > On 2015-01-12, David Holmes wrote: >> On 10/01/2015 4:11 AM, Kim Barrett wrote: >>> New webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03/ >>> Incremental webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03.incr/ >> >> Webrev looks fine but for some reason you patch file did not update: >> >> http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03/hotspot.changeset >> >> still seems to be version 1. > > I suspect that Kim is using `webrev -r` to create the webrev, this is > known to produce broken patch files: > > https://bugs.openjdk.java.net/browse/CODETOOLS-7901115 Using ?webrev -r? is indeed what I?m doing. I was unaware of that bug. Any suggestions for what else I should do? I?m using hg mq?s to manage the changes locally. That seems to require using -r when creating the webrev; ?-r qparent? for the full webrev, and ?-r ? for the incremental webrev. From david.dehaven at oracle.com Mon Jan 12 16:25:21 2015 From: david.dehaven at oracle.com (David DeHaven) Date: Mon, 12 Jan 2015 08:25:21 -0800 Subject: [8u60] RFR: 8043340: [macosx] Fix hard-wired paths to JavaVM.framework In-Reply-To: <8B01B84D-5B6F-478E-ABF1-33F5ADE2F826@oracle.com> References: <39719ED3-1F4E-4E6C-AA9F-A3C4766537B4@oracle.com> <54B3A38C.10909@oracle.com> <8B01B84D-5B6F-478E-ABF1-33F5ADE2F826@oracle.com> Message-ID: <08C63261-D594-404E-892B-BDE2898B505A@oracle.com> Or rather, the point of this exercise is to eliminate the hacks to get it to build with Xcode 5 (I'm not sure if anyone was truly successful with that). It's far better to just build with Xcode 4.6.3, and with these changes you don't even need to pre-sanitize your Xcode environment. A proper build setup would have Xcode 5 or 6 installed in /Applications/Xcode.app (generally MAS managed) and Xcode 4.6.3 (still available for download on ADC) somewhere NOT directly in /Applications (the Mac App Store has a nasty habit of "upgrading" it when it sees it there). I keep mine in /Applications/old along with a copy of Xcode 5.1.1 for test purposes. The --with-xcode-path argument is optional, you should also be able to build with Xcode 4 selected via "sudo xcode-select -switch /path/to/Xcode4.app". I leave MAS managed Xcode (currently 6) active as I'm constantly bouncing between projects and it's a hassle to have to remember to reset the active toolchain, so I thought it best to allow configure to be provided a path. I'm kind of bummed I didn't get to show my really nasty hack for working around the broken lipo stub tool, it was so horrible it could be considered artwork! :) -DrD- > It won't build at all with Xcode 5, there is no gcc compiler and the clang changes were never backported to jdk8u. > > -DrD- > >> Hello, >> >> These changes look ok to me. >> >> With these changes, configure will unconditionally fail if trying to use XCode 5. I know we don't officially support using XCode 5 for JDK 8, but aren't people working around it with some patches? How hard would it be to make it at least build? >> >> /Erik >> >> On 2015-01-10 05:45, David DeHaven wrote: >>> Please review the open source changes for 8043340. The goal here is to get jdk8u to build on Mac OS X 10.9+ systems where Xcode 5+ and Xcode 4 are co-installed, a configuration which is becoming more and more commonplace as more developers are focusing on JDK 9 now (which needs Xcode 5 installed), not to mention new systems ship with 10.10 which only supports the Xcode 5/6 command line tools. It's too much of a hassle to maintain separate systems for building jdk8u and JPRT isn't suitable for frequent changes so something must be done to address this. >>> >>> The jdk changes are similar to those done for JDK9, though I removed the changes for building with Xcode 5 (JDK-8043591) since we do not support building JDK 8 with clang. >>> >>> The hotspot changes are almost identical, the lack of SYSROOT_CFLAGS necessitated changing the logic in saproc.make a bit. It still builds with or without spec.gmk, though without it you will either need to define SDKPATH or have a sane Xcode 4 installation. >>> >>> For the top level build system: >>> - most of the logic of sanitizing the Xcode build environment is in toolchain.m4 >>> - the LIPO variable was removed since it was completely unused >>> - OTOOL was moved to after the Xcode sanitizing so it can be picked up from DEVELOPER_DIR if needed >>> - MACOSX_UNIVERSAL is now being set to false by default and ALT_MACOSX_UNIVERSAL was added to hotspot-spec.gmk.in so the hotspot build is in sync with the jdk build (this was a bug, IMHO) >>> >>> That last change removed any need to run lipo (only done in hotspot), so the fact that /usr/bin/lipo is broken with Xcode 4 is a non-issue. >>> >>> There is a weird case where some early versions of the Xcode 5 command line tools installed /usr/bin/{gcc|g++} as a symlink to {clang|clang++}, that case is handled by putting $DEVELOPER_DIR/usr/bin on the path so autoconf picks up the actual gcc executable. >>> >>> JBS Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8043340 >>> >>> Webrevs: >>> http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v0/top >>> http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v0/hotspot >>> http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v0/jdk >>> >>> JPRT runs are being kicked off. I'll have one run from hotspot directly. I'll post results here. >>> >>> -DrD- >>> >> > From erik.joelsson at oracle.com Mon Jan 12 16:30:08 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 12 Jan 2015 17:30:08 +0100 Subject: [8u60] RFR: 8043340: [macosx] Fix hard-wired paths to JavaVM.framework In-Reply-To: <08C63261-D594-404E-892B-BDE2898B505A@oracle.com> References: <39719ED3-1F4E-4E6C-AA9F-A3C4766537B4@oracle.com> <54B3A38C.10909@oracle.com> <8B01B84D-5B6F-478E-ABF1-33F5ADE2F826@oracle.com> <08C63261-D594-404E-892B-BDE2898B505A@oracle.com> Message-ID: <54B3F690.5090309@oracle.com> I'm happy with that answer. Thanks! /Erik On 2015-01-12 17:25, David DeHaven wrote: > Or rather, the point of this exercise is to eliminate the hacks to get it to build with Xcode 5 (I'm not sure if anyone was truly successful with that). It's far better to just build with Xcode 4.6.3, and with these changes you don't even need to pre-sanitize your Xcode environment. > > A proper build setup would have Xcode 5 or 6 installed in /Applications/Xcode.app (generally MAS managed) and Xcode 4.6.3 (still available for download on ADC) somewhere NOT directly in /Applications (the Mac App Store has a nasty habit of "upgrading" it when it sees it there). I keep mine in /Applications/old along with a copy of Xcode 5.1.1 for test purposes. > > The --with-xcode-path argument is optional, you should also be able to build with Xcode 4 selected via "sudo xcode-select -switch /path/to/Xcode4.app". I leave MAS managed Xcode (currently 6) active as I'm constantly bouncing between projects and it's a hassle to have to remember to reset the active toolchain, so I thought it best to allow configure to be provided a path. > > > I'm kind of bummed I didn't get to show my really nasty hack for working around the broken lipo stub tool, it was so horrible it could be considered artwork! :) > > -DrD- > >> It won't build at all with Xcode 5, there is no gcc compiler and the clang changes were never backported to jdk8u. >> >> -DrD- >> >>> Hello, >>> >>> These changes look ok to me. >>> >>> With these changes, configure will unconditionally fail if trying to use XCode 5. I know we don't officially support using XCode 5 for JDK 8, but aren't people working around it with some patches? How hard would it be to make it at least build? >>> >>> /Erik >>> >>> On 2015-01-10 05:45, David DeHaven wrote: >>>> Please review the open source changes for 8043340. The goal here is to get jdk8u to build on Mac OS X 10.9+ systems where Xcode 5+ and Xcode 4 are co-installed, a configuration which is becoming more and more commonplace as more developers are focusing on JDK 9 now (which needs Xcode 5 installed), not to mention new systems ship with 10.10 which only supports the Xcode 5/6 command line tools. It's too much of a hassle to maintain separate systems for building jdk8u and JPRT isn't suitable for frequent changes so something must be done to address this. >>>> >>>> The jdk changes are similar to those done for JDK9, though I removed the changes for building with Xcode 5 (JDK-8043591) since we do not support building JDK 8 with clang. >>>> >>>> The hotspot changes are almost identical, the lack of SYSROOT_CFLAGS necessitated changing the logic in saproc.make a bit. It still builds with or without spec.gmk, though without it you will either need to define SDKPATH or have a sane Xcode 4 installation. >>>> >>>> For the top level build system: >>>> - most of the logic of sanitizing the Xcode build environment is in toolchain.m4 >>>> - the LIPO variable was removed since it was completely unused >>>> - OTOOL was moved to after the Xcode sanitizing so it can be picked up from DEVELOPER_DIR if needed >>>> - MACOSX_UNIVERSAL is now being set to false by default and ALT_MACOSX_UNIVERSAL was added to hotspot-spec.gmk.in so the hotspot build is in sync with the jdk build (this was a bug, IMHO) >>>> >>>> That last change removed any need to run lipo (only done in hotspot), so the fact that /usr/bin/lipo is broken with Xcode 4 is a non-issue. >>>> >>>> There is a weird case where some early versions of the Xcode 5 command line tools installed /usr/bin/{gcc|g++} as a symlink to {clang|clang++}, that case is handled by putting $DEVELOPER_DIR/usr/bin on the path so autoconf picks up the actual gcc executable. >>>> >>>> JBS Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8043340 >>>> >>>> Webrevs: >>>> http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v0/top >>>> http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v0/hotspot >>>> http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v0/jdk >>>> >>>> JPRT runs are being kicked off. I'll have one run from hotspot directly. I'll post results here. >>>> >>>> -DrD- >>>> From david.dehaven at oracle.com Mon Jan 12 17:05:28 2015 From: david.dehaven at oracle.com (David DeHaven) Date: Mon, 12 Jan 2015 09:05:28 -0800 Subject: [8u60] RFR: 8043340: [macosx] Fix hard-wired paths to JavaVM.framework In-Reply-To: <08C63261-D594-404E-892B-BDE2898B505A@oracle.com> References: <39719ED3-1F4E-4E6C-AA9F-A3C4766537B4@oracle.com> <54B3A38C.10909@oracle.com> <8B01B84D-5B6F-478E-ABF1-33F5ADE2F826@oracle.com> <08C63261-D594-404E-892B-BDE2898B505A@oracle.com> Message-ID: > The --with-xcode-path argument is optional, you should also be able to build with Xcode 4 selected via "sudo xcode-select -switch /path/to/Xcode4.app". I leave MAS managed Xcode (currently 6) active as I'm constantly bouncing between projects and it's a hassle to have to remember to reset the active toolchain, so I thought it best to allow configure to be provided a path. Ugh. I broke something along the way, this doesn't *quite* work. xcrun complains with "xcrun: error: missing DEVELOPER_DIR path:" I think I'm exporting an empty DEVELOPER_DIR. I shall fix and update. -DrD- From daniel.daugherty at oracle.com Mon Jan 12 18:54:55 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 12 Jan 2015 11:54:55 -0700 Subject: De-universalizing hotspot in jdk8u In-Reply-To: <825EBFC2-6DE1-4550-842B-217C06532C9B@oracle.com> References: <54B300FE.2000804@oracle.com> <54B3A4D0.3080604@oracle.com> <54B3AF6D.20607@oracle.com> <825EBFC2-6DE1-4550-842B-217C06532C9B@oracle.com> Message-ID: <54B4187F.5060608@oracle.com> On 1/12/15 8:17 AM, David DeHaven wrote: >>>>> We have this nice little comment in common/autoconf/jdk-options.m4: >>>>> # On Macosx universal binaries are produced, but they only contain >>>>> # 64 bit intel. This invalidates control of which jvms are built >>>>> # from configure, but only server is valid anyway. Fix this >>>>> # when hotspot makefiles are rewritten. >>>>> if test "x$MACOSX_UNIVERSAL" = xtrue; then >>>>> HOTSPOT_TARGET=universal_${HOTSPOT_EXPORT} >>>>> fi >>>>> >>>>> >>>>> So.. I turned it off: >>>>> if test "x$OPENJDK_TARGET_OS" = "xmacosx"; then >>>>> # MACOSX_UNIVERSAL="true" >>>>> MACOSX_UNIVERSAL="false" >>>>> fi >>>>> >>>>> And in hotspot/make/bsd/makefiles/defs.make: >>>>> # Universal build settings >>>>> ifeq ($(OS_VENDOR), Darwin) >>>>> # Build universal binaries by default on Mac OS X >>>>> # MACOSX_UNIVERSAL = true >>>>> MACOSX_UNIVERSAL = false >>>>> ifneq ($(ALT_MACOSX_UNIVERSAL),) >>>>> >>>>> hotspot still seems to build happily (I'm kicking off tests now). Is >>>>> there are particular reason this hasn't been done yet? I'd like to >>>>> know before I pursue this as a means of eliminating our dependency on >>>>> lipo which is causing an inordinate amount of grief when trying to >>>>> build on 10.9 and later when Xcode 5+ is coinstalled. I have some >>>>> logic to work around using the broken /usr/bin/lipo, but it doesn't >>>>> help later in some "other" build target that ends up using '-arch >>>>> i386 -arch x86_64'. It does not explicitly run lipo, it relies on gcc >>>>> to do the fattening itself and so it fails even though I've gone to >>>>> the effort of working around the broken lipo (and it isn't necessary >>>>> anyways because it doesn't even build the i386 binaries even though >>>>> it's supposed to be "universal"). >>>> The hotspot makefiles still haven't been rewritten (though Magnus had >>>> been working on it). >>>> >>>> Also I was under the impression that we used the universal stuff >>>> because we had to deliver in a particular way on OSX. But I don't know >>>> the details and the people who dealt with the original OSX port effort >>>> are no longer here. >>>> >>> I don't know why Hotspot is packaged as a universal binary. In the jdk, >>> only libJObjC was built as a universal binary and that lib was removed >>> before JDK 8 shipped. What part of Macosx would need to interact >>> directly with libjvm.dylib in a way that required a (fake) universal >>> format, but no other libs in the jdk? I would say go ahead and change it. >> In the spirit of "never take down a fence until you know why it was put up in the first place" I've cc'd macosx-port-dev to see if there is anyone around who recalls why hotspot is using the universal binary feature. > I hear you. > > My best guess is we'd initially planned on supporting 32 bit but (???) and the easiest way to deliver is in a universal binary. Note that we don't actually deliver universal binaries as the JVM is 64 bit only, or at least there are not nearly enough components there to run a full 32 bit JVM. > > -DrD- We (Jim Melvin did the work) included universal support in HotSpot to not preclude someone else from doing the 32-bit MacOS X port and providing it via OpenJDK channels. The rest of the JDK did not include universal support, but the hotspot code would have served as an example way forward... However, we never heard from anyone willing to do the 32-bit port... Dan From david.dehaven at oracle.com Mon Jan 12 19:06:30 2015 From: david.dehaven at oracle.com (David DeHaven) Date: Mon, 12 Jan 2015 11:06:30 -0800 Subject: De-universalizing hotspot in jdk8u In-Reply-To: <54B4187F.5060608@oracle.com> References: <54B300FE.2000804@oracle.com> <54B3A4D0.3080604@oracle.com> <54B3AF6D.20607@oracle.com> <825EBFC2-6DE1-4550-842B-217C06532C9B@oracle.com> <54B4187F.5060608@oracle.com> Message-ID: >>>> I don't know why Hotspot is packaged as a universal binary. In the jdk, >>>> only libJObjC was built as a universal binary and that lib was removed >>>> before JDK 8 shipped. What part of Macosx would need to interact >>>> directly with libjvm.dylib in a way that required a (fake) universal >>>> format, but no other libs in the jdk? I would say go ahead and change it. >>> In the spirit of "never take down a fence until you know why it was put up in the first place" I've cc'd macosx-port-dev to see if there is anyone around who recalls why hotspot is using the universal binary feature. >> I hear you. >> >> My best guess is we'd initially planned on supporting 32 bit but (???) and the easiest way to deliver is in a universal binary. Note that we don't actually deliver universal binaries as the JVM is 64 bit only, or at least there are not nearly enough components there to run a full 32 bit JVM. > > We (Jim Melvin did the work) included universal support in HotSpot > to not preclude someone else from doing the 32-bit MacOS X port and > providing it via OpenJDK channels. The rest of the JDK did not > include universal support, but the hotspot code would have served > as an example way forward... > > However, we never heard from anyone willing to do the 32-bit port... Thanks for the explanation Daniel. I'm preserving what's there in the same spirit, but turning off universal by default since it's unnecessary. It sounds to me like this is the correct approach. -DrD- From astrange at apple.com Mon Jan 12 19:20:09 2015 From: astrange at apple.com (Alex Strange) Date: Mon, 12 Jan 2015 11:20:09 -0800 Subject: De-universalizing hotspot in jdk8u In-Reply-To: <54B3AF6D.20607@oracle.com> References: <54B300FE.2000804@oracle.com> <54B3A4D0.3080604@oracle.com> <54B3AF6D.20607@oracle.com> Message-ID: <9B1E95A7-8496-4E59-9B00-FCC1F4B780FA@apple.com> > On Jan 12, 2015, at 3:26 AM, David Holmes wrote: > > cc'ing macosx-port-dev and hotspot-dev > > On 12/01/2015 8:41 PM, Erik Joelsson wrote: >> >> On 2015-01-12 00:02, David Holmes wrote: >>> Hi David, >>> >>> On 10/01/2015 2:00 AM, David DeHaven wrote: >>>> >>>> We have this nice little comment in common/autoconf/jdk-options.m4: >>>> # On Macosx universal binaries are produced, but they only contain >>>> # 64 bit intel. This invalidates control of which jvms are built >>>> # from configure, but only server is valid anyway. Fix this >>>> # when hotspot makefiles are rewritten. >>>> if test "x$MACOSX_UNIVERSAL" = xtrue; then >>>> HOTSPOT_TARGET=universal_${HOTSPOT_EXPORT} >>>> fi >>>> >>>> >>>> So.. I turned it off: >>>> if test "x$OPENJDK_TARGET_OS" = "xmacosx"; then >>>> # MACOSX_UNIVERSAL="true" >>>> MACOSX_UNIVERSAL="false" >>>> fi >>>> >>>> And in hotspot/make/bsd/makefiles/defs.make: >>>> # Universal build settings >>>> ifeq ($(OS_VENDOR), Darwin) >>>> # Build universal binaries by default on Mac OS X >>>> # MACOSX_UNIVERSAL = true >>>> MACOSX_UNIVERSAL = false >>>> ifneq ($(ALT_MACOSX_UNIVERSAL),) >>>> >>>> hotspot still seems to build happily (I'm kicking off tests now). Is >>>> there are particular reason this hasn't been done yet? I'd like to >>>> know before I pursue this as a means of eliminating our dependency on >>>> lipo which is causing an inordinate amount of grief when trying to >>>> build on 10.9 and later when Xcode 5+ is coinstalled. I have some >>>> logic to work around using the broken /usr/bin/lipo, but it doesn't >>>> help later in some "other" build target that ends up using '-arch >>>> i386 -arch x86_64'. It does not explicitly run lipo, it relies on gcc >>>> to do the fattening itself and so it fails even though I've gone to >>>> the effort of working around the broken lipo (and it isn't necessary >>>> anyways because it doesn't even build the i386 binaries even though >>>> it's supposed to be "universal"). >>> >>> The hotspot makefiles still haven't been rewritten (though Magnus had >>> been working on it). >>> >>> Also I was under the impression that we used the universal stuff >>> because we had to deliver in a particular way on OSX. But I don't know >>> the details and the people who dealt with the original OSX port effort >>> are no longer here. >>> >> I don't know why Hotspot is packaged as a universal binary. In the jdk, >> only libJObjC was built as a universal binary and that lib was removed >> before JDK 8 shipped. What part of Macosx would need to interact >> directly with libjvm.dylib in a way that required a (fake) universal >> format, but no other libs in the jdk? I would say go ahead and change it. > > In the spirit of "never take down a fence until you know why it was put up in the first place" I've cc'd macosx-port-dev to see if there is anyone around who recalls why hotspot is using the universal binary feature. > > David H. > ------- I did the original universal binary work for hotspot when bringing up macosx-port. At the time we were building and testing JDK 7 as universal (i386+x86-64) since e.g. some apps had JNI code that was only built 32-bit. The other jdk components didn't need any makefile work, because the compiler can build them universal by itself, but hotspot autogenerates a lot of arch-specific code and it was easier to build it twice and glue them together in the makefile. As long as Java is only been shipped 64-bit these days, I personally don't see a reason to keep it. > /Erik >> >>> David H. >>> >>>> >>>> Quite frankly I'd rather just remove all the universal binary stuff >>>> from hotspot, but I'm trying my best to minimize the changes there... >>>> the "other" problem I can handle easily. >>>> >>>> -DrD- >>>> >> From kim.barrett at oracle.com Mon Jan 12 20:30:56 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 12 Jan 2015 15:30:56 -0500 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: <54B33270.4070400@oracle.com> References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> <54AF243F.7050001@oracle.com> <22C65166-ABB5-408E-8A7C-5DDE6D61BAEE@oracle.com> <3A9DFD5D-A70D-4CD3-9B56-88B75F771D0A@oracle.com> <54B33270.4070400@oracle.com> Message-ID: On Jan 11, 2015, at 9:33 PM, David Holmes wrote: > > Webrev looks fine but for some reason you patch file did not update: > > http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03/hotspot.changeset > > still seems to be version 1. David, Thanks for your review. webrev patch issue discussed elsewhere; I was planning to give Coleen an ?hg export? file for pushing anyway. From kim.barrett at oracle.com Mon Jan 12 20:31:52 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 12 Jan 2015 15:31:52 -0500 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: <20150112092001.GJ3035@ehelin.jrpg.bea.com> References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> <54AF243F.7050001@oracle.com> <22C65166-ABB5-408E-8A7C-5DDE6D61BAEE@oracle.com> <3A9DFD5D-A70D-4CD3-9B56-88B75F771D0A@oracle.com> <20150112092001.GJ3035@ehelin.jrpg.bea.com> Message-ID: On Jan 12, 2015, at 4:20 AM, Erik Helin wrote: > > On 2015-01-09, Kim Barrett wrote: >> On Jan 8, 2015, at 8:05 PM, Kim Barrett wrote: >>> >>> On Jan 8, 2015, at 7:43 PM, Coleen Phillimore wrote: >>>> >>>> >>>> My prior email asked if you could test this change by changing an assert to vmassert. The assert in debug.cpp test_error_handler to vmassert would be a good one to test this change. >>> >>> Ah, now I understand what you were asking for. OK, I?ll do that and post a new webrev later this evening. >> >> New webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03/ >> Incremental webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03.incr/ > > Looks good, Reviewed. Erik - thanks for your review. From david.holmes at oracle.com Tue Jan 13 01:40:33 2015 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Jan 2015 11:40:33 +1000 Subject: De-universalizing hotspot in jdk8u In-Reply-To: <9B1E95A7-8496-4E59-9B00-FCC1F4B780FA@apple.com> References: <54B300FE.2000804@oracle.com> <54B3A4D0.3080604@oracle.com> <54B3AF6D.20607@oracle.com> <9B1E95A7-8496-4E59-9B00-FCC1F4B780FA@apple.com> Message-ID: <54B47791.2040301@oracle.com> On 13/01/2015 5:20 AM, Alex Strange wrote: > >> On Jan 12, 2015, at 3:26 AM, David Holmes wrote: >> >> cc'ing macosx-port-dev and hotspot-dev >> >> On 12/01/2015 8:41 PM, Erik Joelsson wrote: >>> >>> On 2015-01-12 00:02, David Holmes wrote: >>>> Hi David, >>>> >>>> On 10/01/2015 2:00 AM, David DeHaven wrote: >>>>> >>>>> We have this nice little comment in common/autoconf/jdk-options.m4: >>>>> # On Macosx universal binaries are produced, but they only contain >>>>> # 64 bit intel. This invalidates control of which jvms are built >>>>> # from configure, but only server is valid anyway. Fix this >>>>> # when hotspot makefiles are rewritten. >>>>> if test "x$MACOSX_UNIVERSAL" = xtrue; then >>>>> HOTSPOT_TARGET=universal_${HOTSPOT_EXPORT} >>>>> fi >>>>> >>>>> >>>>> So.. I turned it off: >>>>> if test "x$OPENJDK_TARGET_OS" = "xmacosx"; then >>>>> # MACOSX_UNIVERSAL="true" >>>>> MACOSX_UNIVERSAL="false" >>>>> fi >>>>> >>>>> And in hotspot/make/bsd/makefiles/defs.make: >>>>> # Universal build settings >>>>> ifeq ($(OS_VENDOR), Darwin) >>>>> # Build universal binaries by default on Mac OS X >>>>> # MACOSX_UNIVERSAL = true >>>>> MACOSX_UNIVERSAL = false >>>>> ifneq ($(ALT_MACOSX_UNIVERSAL),) >>>>> >>>>> hotspot still seems to build happily (I'm kicking off tests now). Is >>>>> there are particular reason this hasn't been done yet? I'd like to >>>>> know before I pursue this as a means of eliminating our dependency on >>>>> lipo which is causing an inordinate amount of grief when trying to >>>>> build on 10.9 and later when Xcode 5+ is coinstalled. I have some >>>>> logic to work around using the broken /usr/bin/lipo, but it doesn't >>>>> help later in some "other" build target that ends up using '-arch >>>>> i386 -arch x86_64'. It does not explicitly run lipo, it relies on gcc >>>>> to do the fattening itself and so it fails even though I've gone to >>>>> the effort of working around the broken lipo (and it isn't necessary >>>>> anyways because it doesn't even build the i386 binaries even though >>>>> it's supposed to be "universal"). >>>> >>>> The hotspot makefiles still haven't been rewritten (though Magnus had >>>> been working on it). >>>> >>>> Also I was under the impression that we used the universal stuff >>>> because we had to deliver in a particular way on OSX. But I don't know >>>> the details and the people who dealt with the original OSX port effort >>>> are no longer here. >>>> >>> I don't know why Hotspot is packaged as a universal binary. In the jdk, >>> only libJObjC was built as a universal binary and that lib was removed >>> before JDK 8 shipped. What part of Macosx would need to interact >>> directly with libjvm.dylib in a way that required a (fake) universal >>> format, but no other libs in the jdk? I would say go ahead and change it. >> >> In the spirit of "never take down a fence until you know why it was put up in the first place" I've cc'd macosx-port-dev to see if there is anyone around who recalls why hotspot is using the universal binary feature. >> >> David H. >> ------- > > I did the original universal binary work for hotspot when bringing up macosx-port. > > At the time we were building and testing JDK 7 as universal (i386+x86-64) since e.g. some apps had JNI code that was only built 32-bit. > The other jdk components didn't need any makefile work, because the compiler can build them universal by itself, but hotspot autogenerates a lot of arch-specific code and it was easier to build it twice and glue them together in the makefile. > > As long as Java is only been shipped 64-bit these days, I personally don't see a reason to keep it. Thanks for the info Alex. Based on that and Dan's response this proposed change seems good to go ahead. David H. > >> /Erik >>> >>>> David H. >>>> >>>>> >>>>> Quite frankly I'd rather just remove all the universal binary stuff >>>>> from hotspot, but I'm trying my best to minimize the changes there... >>>>> the "other" problem I can handle easily. >>>>> >>>>> -DrD- >>>>> >>> > From david.holmes at oracle.com Tue Jan 13 02:37:36 2015 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Jan 2015 12:37:36 +1000 Subject: OrderAccess Refactoring In-Reply-To: References: <78E41D90-81F0-4526-B7A3-A02A6B030515@lnu.se> <54B0FADC.80600@oracle.com> Message-ID: <54B484F0.3060503@oracle.com> Hi Erik, Quick partial response ... On 13/01/2015 12:18 AM, Erik ?sterlund wrote: > Hi David, > > Thank you for the feedback! :) > > On 10/01/15 11:11, David Holmes wrote: >> acquire/release semantics are problematic because there is no single >> common accepted meaning. Even talking about acquire() and release() as >> independent operations is somewhat meaningless - it is only when they >> are paired, and fixed in place that they start to give meaning to things. > > I agree it is a bit vague. The description in the comments about > preventing all memory accesses from floating up and down is the one I > recognize and it is conceptually good. But then it is later defined in a > table as acquire = Load(Store|Load) and release = (Store|Load)Store. The table is not a definition, it is an example of how the acquire/release semantics can be implemented on the different architectures: membar loadStore|loadLoad is sufficient to implement acquire, but not exactly equivalent - acquire/release semantics can not be expressed exactly in terms of loadload|loadStore etc, as acquire/release define one-way barriers whereas loadload etc are bi-directional. The > reason is understandable - even though the description of > acquire/release is quite general, they are only intended and designed to > be used in certain contexts like release store and load acquire (with > variants). Outside such similar contexts, it would not give the > guarantees described. > > This gets more confusing when you go on reading that fence is > conceptually acquire and release. It is true on the conceptual level > using the conceptual descriptions of acquire/release, but the actual > guarantees of release + acquire is StoreStore, LoadLoad and LoadStore > according to the tables below, while fence() also provides StoreLoad > constraints. I don't know about you but I find that a bit misleading. Yes conceptually fence is "back-to-back acquire and release" but that doesn't mean fence can necessarily be implemented by concatenating the implementations of acquire and release on any given platform. The problem is the "back-to-back" part as you need something to stop loads from moving down past the acquire, and stores from moving up before the release. Hence to implement fence you need all 4 membars and you don't need the "dummy" load or store. > I mean, it might be quite obvious for anyone using it to only use > acquire/release in the intended contexts where it makes sense and then > there is no trouble, but I guess it doesn't hurt to add some comments > making it explicit to avoid any confusion. We don't want confusion I think. > >> load_acquire/store_release don't necessarily map cleanly to >> acquire()/release() as currently described in orderAccess.hpp; nor do >> they necessarily map cleanly to the same named operations on eg ARMv8; >> nor to similar named entities in C++11. As Paul stated these were >> introduced to map to the ia64 architecture - which I believe had >> somewhat different semantics. > > They do map cleanly after I change acquire() and release() to compiler > barriers on TSO platforms (and fencing on PPC). Except some exceptions > like load acquire on PPC that was specialized for speed using twi-isync > and x86 on windows that don't need the stronger barriers since volatile > is more accurate anyway. > >> This whole area is a minefield because there are different concepts of >> what "acquire" and "release" may mean - so you need to start with >> detailed definitions. Personally, and I know I am not alone, I abhor the >> acquire/release terminology and usage, I much prefer the unambiguous >> storeload, storestore etc. :) > > Yes I think we need a better concrete guarantee/contract. The current > conceptual description of acquire/release is nice I think and useful > also for understanding when to use it, but I want to make it explicit > that release is only used in conjunction with writes and acquire only in > conjunction with loads (and swapping instructions etc), and therefore do > not give the same guarantees as described as the more general conceptual > description promises. Or put another way there is no acquire()/release() only load_acquire()/store_release() ? > The concrete guarantee users can rely on is acquire = Load(Store|Load) > and release = (Store|Load)Store, and that's consistent with the Nope as per above that is not correct. > implementation and reasonable intended usage. Hence, it's fine to use > them in the way acquire/release is described now, iff these barriers > suffice to guarantee it, otherwise acquire/release should not be used in > the given context. Does that make sense? > >> The current code often expresses things in a semantically incorrect way >> eg that: >> >> store_release(addr, val) == store(addr, val); release(); >> >> but in practice the end code provides what is required (often providing >> a stronger barrier because that's all that is available on a platform). > > I'm not sure I get what you mean by this being semantically incorrect. I > get it's suboptimal and might be slower triggering compiler barriers > that won't be necessary when joined, but semantically surely it should > be the same, right? Sorry I got the API backwards. What we often see is: release_store(addr, val) == release(); store(addr,val); but semantically that is wrong because release() allows stores to move ahead of it so the RHS could be executed as: store(addr, val); release() which != release_store (which binds the store tightly to the 'release' part). But on a given platform release() might be implemented in such a way that the store can't move ahead of it - so the code is right, but the way it is written is potentially incorrect given the abstract specifications. Though another way to look at this, given these are all contained in per-platform files, is that when you see something like: release_store(addr, val) == release(); store(addr,val); it isn't a claim of general equivalence, but a statement that is accurate for the implementation on the current platform. (Though I'd still prefer not to see it written this way - and that is what I want to fix under https://bugs.openjdk.java.net/browse/JDK-7143664 ) BTW this is another point of confusion when we have an API like release_store() but instructions like st.rel - which side of the barrier is the store on, and does it matter? > >> And I agree with Karen - tackle correctness first then refactoring. IT >> will be too hard to track correctness otherwise. Personally, given I >> work in this code quite a bit, I like to see the complete set of >> definitions for a platform in one place. The duplication is something I >> can live with when it is mostly one-liners. > > Okay no problem. I built the general variant first (because it was > easier since I don't have access to other platforms except mine haha). > But I fully understand yours and Karen's motivation so I will get back > to you when I manage to do it the code duplicated way so it's easier to > follow the evolution. > > Oh and speaking of which, I find it very strange that store_fence and > release_store_fence is duplicated on x86 because we don't want to cast > away the volatile. This whole non-volatile thing seems a bit off to me - Very long, depressing history of issues with volatiles and casting with different compilers. So what you see is the result of what needed to be done to "make things work". > surely all calls to OrderedAccess should be expected to be volatile, I agree, and I've suggested this in the past too, but was told that making variables volatile incurs too much performance penalty in some cases, so we only cast them to volatile when needed for these ops. > nothing else would make any sense? And if so, why is store_fence the one > exception that is non-volatile while none of the others are? Unless > anyone knows there is a good reason for this, I'd like to make it > consistent with the rest of the code and make it volatile as well (and > hence get rid of some code duplication while at it). Permission granted? ;) That case would need to be examined in detail. :) > Thank you David for a nice discussion about this! :) I'd like to say it is fun, but it really isn't :) This stuff gives me continual headaches. Cheers, David > /Erik > From erik.osterlund at lnu.se Tue Jan 13 06:41:32 2015 From: erik.osterlund at lnu.se (=?iso-8859-1?Q?Erik_=D6sterlund?=) Date: Tue, 13 Jan 2015 06:41:32 +0000 Subject: OrderAccess Refactoring References: <78E41D90-81F0-4526-B7A3-A02A6B030515@lnu.se> <54B0FADC.80600@oracle.com> <54B484F0.3060503@oracle.com> Message-ID: Hi David, Thank you for your reply! :) On 13/01/15 03:37, David Holmes wrote: > The table is not a definition, it is an example of how the > acquire/release semantics can be implemented on the different architectures: > > membar loadStore|loadLoad > > is sufficient to implement acquire, but not exactly equivalent - > acquire/release semantics can not be expressed exactly in terms of > loadload|loadStore etc, as acquire/release define one-way barriers > whereas loadload etc are bi-directional. Understood, but I just think it is a bit more than just an example (agreeing it's an approximation of the intended semantics). This is in fact how it actually behaves concretely and is implemented on all platforms, as opposed to the conceptual description of acquire/release which in general can not be relied upon outside certain narrowed contexts which might do with being clarified. I personally think it's useful to point that context out where acquire release is defined, so that anyone in doubt can see what is actually guaranteed by the implementation and what is not and hence when such semantics can be relied upon and when they can't. If you disagree though, I'm fine with leaving the comment as it was. ;) > Yes conceptually fence is "back-to-back acquire and release" but that > doesn't mean fence can necessarily be implemented by concatenating the > implementations of acquire and release on any given platform. And that is exactly what I believe could be weird to somebody out there. If acquire() and release() had the semantics described, you would be able to call them both and get a fence, but since they are assumed to only be paired with loads and stores in a certain well defined order, you never need the StoreLoad, hence making them not a fence, and they shouldn't be of course. > and you don't need the "dummy" load or store. Agreed, I got rid of them all. :) > Or put another way there is no acquire()/release() only > load_acquire()/store_release() ? Yes these are the contexts where it gives the intended semantics in the implementation. And outside similar contexts, it makes no sense to use it. Hopefully this is clear enough to the user. > Sorry I got the API backwards. What we often see is: > > release_store(addr, val) == release(); store(addr,val); > > but semantically that is wrong because release() allows stores to move > ahead of it so the RHS could be executed as: > > store(addr, val); release() Actually that is semantically equivalent by all means both by definition, current implementation and intention. And this is why I'm stressing that acquire() and release() only make sense together with a load and a store. Whether they are separate methods or not is irrelevant, it has the same semantics anyway and it has to. To demonstrate my point, let's take a simple example. If your reasoning of splitting barriers from memory accesses is correct, then look what happens in the following example: 1.a release store 2.b load acquire 2.c load acquire ...should be equivalent to: 1.a release 1.b store 1.c load 1.d acquire 1.e load 1.f acquire ...and if it wasn't we would need a fence in acquire (cf. 1.d) to prevent reordering between 1.b and 1.e which the general definition of acquire guarantees if not constrained to its load. Ouch!! ;) But fear not, because when they are coupled with a store/load, regardless if you use release() store() or release_store(), they are equivalent and act /as if/ they were joined as a single instruction. And as you see, 1.b and 1.c may reorder without violating any rules so we don't have to do any expensive full memory barriers. This is why I'm saying that the description we have of acquire() and release() only make sense in a narrowed context: when acquire is only used after a load and release only before a store, then the semantics hold true to the definition, and whether you use the split or joined form you get the same observed results and semantics. And in this narrowed context where it makes sense, it's equivalent to the TSO memory ordering, and I know you didn't like it but that can be defined as acquire = Load(Load|Store) and release = (Load|Store)Store, in that narrowed context where it is valid w.r.t. the original definition. And these are also the actual guarantees in our implementation which is helpful. :) I know you didn't like this as a definition, but it solves the problem of equivalence here and perfectly defines the scope of when acquire() and release() can be used. And that is precisely when my definition is equal to the current definition. ;) > (Though I'd > still prefer not to see it written this way - and that is what I want to > fix under https://bugs.openjdk.java.net/browse/JDK-7143664 ) I do agree it's better to use the combined variant but maybe for a different reason. :) They are actually semantically equivalent so I disagree about that. However it is possible to have a better barrier for the combined variant. For instance PPC has optimized acquire load, x86 on Windows don't need compiler barriers because its volatiles have acquire/release compiler reordering semantics which is tighter etc. And then of course, the user of the API wouldn't have to read between the lines in the documentation about in which contexts the acquire() and release() barriers are actually working as defined. By removing the possibility of doing something potentially wrong, less errors might be made! ;) > BTW this is another point of confusion when we have an API like > release_store() but instructions like st.rel - which side of the barrier > is the store on, and does it matter? Yes it does matter. The release is always before the store and the acquire after the load - this is well defined. The reason for this is that it matches an abstract machine with a store buffer using a FIFO queue which is equivalent to TSO which is well understood and portable. If they were not in this order it would be useless for this usage: T1: write stuff T1: release store x1 T2: load acquire x1 T2: load stuff The release/acquire store/load to x1 should synchronize so that the loading of stuff in T2 will see the writing of stuff in T1. Note that if the release and acquire were on the wrong side, this would break. > Very long, depressing history of issues with volatiles and casting with > different compilers. So what you see is the result of what needed to be > done to "make things work". :) >> surely all calls to OrderedAccess should be expected to be volatile, > > I agree, and I've suggested this in the past too, but was told that > making variables volatile incurs too much performance penalty in some > cases, so we only cast them to volatile when needed for these ops. Performance is a non-argument IMO. Because it's only used for store_fence, and it always requires a compiler barrier to work at least for the fence. So volatile won't make a difference at all. Even if it would (which it doesn't), the real deal is the fence, not the volatile memory access (because C++ volatile is not Java volatile). If that was the only argument, I'd like to make this consistent! :) >> nothing else would make any sense? And if so, why is store_fence the one >> exception that is non-volatile while none of the others are? Unless >> anyone knows there is a good reason for this, I'd like to make it >> consistent with the rest of the code and make it volatile as well (and >> hence get rid of some code duplication while at it). Permission granted? ;) > > That case would need to be examined in detail. :) My assessment: 1. Volatile can't impact performance in store_fence (requiring compiler barrier anyway and volatile in C++ only does compiler level reordering, and fences would kill any other effect even if there was one which there is not). 2. Volatile won't break anything and create illegal reorderings, because it's stricter than non-volatile. On the contrary it's easy to make mistakes leading to trouble on for instance windows without it. 3. It makes the code more consistent, allows to remove several copy pastes, and generalize making the code more readable. If I get your permission, I'll add volatile! >> Thank you David for a nice discussion about this! :) > > I'd like to say it is fun, but it really isn't :) This stuff gives me > continual headaches. Sorry about the headaches, I feel a bit guilty! Hopefully with this cleanup, at least some future headache will be relieved though. :) Thanks for the discussion, and hope this email won't make the headache worse! /Erik From tobias.hartmann at oracle.com Tue Jan 13 06:53:37 2015 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 13 Jan 2015 07:53:37 +0100 Subject: [8u60] Backport RFR: 8066763: fatal error "assert(false) failed: unexpected yanked node" in postaloc.cpp:139 In-Reply-To: <54AD3C76.40601@oracle.com> References: <54AD3C76.40601@oracle.com> Message-ID: <54B4C0F1.6050107@oracle.com> Hi, can I get a review for this? Thanks, Tobias On 07.01.2015 15:02, Tobias Hartmann wrote: > Hi, > > please review the following backport request for 8u60. > > 8066763: fatal error "assert(false) failed: unexpected yanked node" in > postaloc.cpp:139 > https://bugs.openjdk.java.net/browse/JDK-8066763 > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/f1cb27c85c83 > > The changes were pushed to JDK 9 on Dec 12 and nightly testing showed no > problems. The patch applies cleanly to 8u60. > > Thanks, > Tobias > From erik.helin at oracle.com Tue Jan 13 08:01:40 2015 From: erik.helin at oracle.com (Erik Helin) Date: Tue, 13 Jan 2015 09:01:40 +0100 Subject: RFR: 8068396: Rename assert() to vmassert() In-Reply-To: <55FE925E-BD23-48C8-BAA4-8365EDCBDB49@oracle.com> References: <42C93786-4866-47D9-A14C-F99B3EF4F687@oracle.com> <54AF243F.7050001@oracle.com> <22C65166-ABB5-408E-8A7C-5DDE6D61BAEE@oracle.com> <3A9DFD5D-A70D-4CD3-9B56-88B75F771D0A@oracle.com> <54B33270.4070400@oracle.com> <20150112090246.GI3035@ehelin.jrpg.bea.com> <55FE925E-BD23-48C8-BAA4-8365EDCBDB49@oracle.com> Message-ID: <20150113080140.GB4167@ehelin.jrpg.bea.com> On 2015-01-12, Kim Barrett wrote: > On Jan 12, 2015, at 4:02 AM, Erik Helin wrote: > > > > On 2015-01-12, David Holmes wrote: > >> On 10/01/2015 4:11 AM, Kim Barrett wrote: > >>> New webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03/ > >>> Incremental webrev: http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03.incr/ > >> > >> Webrev looks fine but for some reason you patch file did not update: > >> > >> http://cr.openjdk.java.net/~kbarrett/8068396/webrev.03/hotspot.changeset > >> > >> still seems to be version 1. > > > > I suspect that Kim is using `webrev -r` to create the webrev, this is > > known to produce broken patch files: > > > > https://bugs.openjdk.java.net/browse/CODETOOLS-7901115 > > Using ?webrev -r? is indeed what I?m doing. I was unaware of that bug. > > Any suggestions for what else I should do? I?m using hg mq?s to manage the changes locally. > That seems to require using -r when creating the webrev; ?-r qparent? for the full webrev, and > ?-r ? for the incremental webrev. This is the way I manage my patches as well. Unfortunately, I don't know of any way to work around this bug in webrev :( Anyone here with ksh and hg knowledge that might be able to take a look at the bug? Thanks, Erik From dean.long at oracle.com Tue Jan 13 08:32:43 2015 From: dean.long at oracle.com (Dean Long) Date: Tue, 13 Jan 2015 00:32:43 -0800 Subject: RFR: AARCH64: Top-level JDK changes In-Reply-To: <54B3B4CE.8060804@oracle.com> References: <545CFFA9.4070107@redhat.com> <545D0290.5080307@oracle.com> <54B34E20.5030506@oracle.com> <54B3B4CE.8060804@oracle.com> Message-ID: <54B4D82B.3050608@oracle.com> On 1/12/2015 3:49 AM, Magnus Ihse Bursie wrote: > On 2015-01-12 05:31, Dean Long wrote: >> I found a small problem with the new config.sub wrapper. It works >> with the bash shell but not with the dash shell. >> The problem seems to be with this line: >> >> result=`. $DIR/autoconf-config.sub $sub_args "$@"` >> >> "dash" doesn't seem to support args passed with ".", so $sub_args >> "$@" are ignored. > > bash is the required shell for running configure. We do not support > non-bash shells. In fact, we go to lengths to try to ensure that we > are indeed running under bash. > > /Magnus I was thinking 'bash configure' was enough, but it turns out 'CONFIG_SHELL=bash bash configure' gives better results. dl From dean.long at oracle.com Tue Jan 13 08:44:04 2015 From: dean.long at oracle.com (Dean Long) Date: Tue, 13 Jan 2015 00:44:04 -0800 Subject: RFR: AARCH64: Top-level JDK changes In-Reply-To: <54B3A108.9070203@redhat.com> References: <545CFFA9.4070107@redhat.com> <545D0290.5080307@oracle.com> <54B34E20.5030506@oracle.com> <54B3A108.9070203@redhat.com> Message-ID: <54B4DAD4.5060904@oracle.com> On 1/12/2015 2:25 AM, Andrew Haley wrote: > On 12/01/15 04:31, Dean Long wrote: >> I found a small problem with the new config.sub wrapper. It works with >> the bash shell but not with the dash shell. >> The problem seems to be with this line: >> >> result=`. $DIR/autoconf-config.sub $sub_args "$@"` >> >> "dash" doesn't seem to support args passed with ".", so $sub_args "$@" >> are ignored. > Thank you. Sorry for the breakage; I didn't intend to use any > non-portable shell idioms. I'm not expert enough to say whether this > is a bug in dash, but I don't suppose that matters: we are where we > are. > > There's no reason not to replace the "." with "bash". Do you want me > to do anything? I guess not, since we only support bash. However, I did notice another problem with the same file. "aarch64-linux-gnu" comes out as "aarch64-linux-gnu" instead of "aarch64-unknown-linux-gnu". I came up with a simpler version, where I replace "aarch64-" with "arm-", run autoconf-config.sub, then replace "arm-" back to "aarch64-". dl > Andrew. From aph at redhat.com Tue Jan 13 09:08:19 2015 From: aph at redhat.com (Andrew Haley) Date: Tue, 13 Jan 2015 09:08:19 +0000 Subject: RFR: AARCH64: Top-level JDK changes In-Reply-To: <54B4DAD4.5060904@oracle.com> References: <545CFFA9.4070107@redhat.com> <545D0290.5080307@oracle.com> <54B34E20.5030506@oracle.com> <54B3A108.9070203@redhat.com> <54B4DAD4.5060904@oracle.com> Message-ID: <54B4E083.9040502@redhat.com> On 13/01/15 08:44, Dean Long wrote: > I came up with a simpler version, where I replace "aarch64-" with > "arm-", run autoconf-config.sub, then replace "arm-" back to > "aarch64-". Thanks. That sounds good to me. Andrew. From rkennke at redhat.com Tue Jan 13 13:23:17 2015 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 13 Jan 2015 14:23:17 +0100 Subject: [PATCH] Fix Shark build in JDK9 In-Reply-To: <54AF66B9.8060900@oracle.com> References: <1420641903.6838.9.camel@localhost> <54AD4C6B.1070702@oracle.com> <1420645301.6838.14.camel@localhost> <54AD5675.7010101@oracle.com> <1420647107.6838.15.camel@localhost> <54AD5BDF.1060401@oracle.com> <1420648161.6838.17.camel@localhost> <54AD639A.2060200@oracle.com> <1420662102.6838.23.camel@localhost> <54AE560B.3060200@oracle.com> <1420714881.6838.25.camel@localhost> <54AF66B9.8060900@oracle.com> Message-ID: <1421155397.6838.43.camel@localhost> Hi David, > > - In ciTypeFlow.cpp only include some files and code only when building > > C2. I don't think that code makes sense outside of C2. (That's the issue > > that you've seen). > > Looks okay but someone from compiler team needs to comment. There may be > other code that need adjusting. It might be less intrusive to guard this with #ifndef SHARK instead of #ifdef COMPILER2. I don't think it makes a difference though. > > - In allocation.hpp, exclude the operator overrides for new etc, LLVM > > does allocations itself, and this would blow up. > > I'm intrigued as the allocation strategy is not tied to the compiler but > pervasive to the whole VM runtime. In a comment it says this whole block is disabled in PRODUCT versions, because of 3rd party code that might be in use. This is basically the situation with Shark: we need to be able to do allocations in LLVM, and LLVM doesn't know about Hotspot's allocation safeguards. That's why I disable it for Shark builds. > > - In handles.hpp and handles.inline.hpp, I added an argument > > check_in_stack so that I can disable the in-stack check for the > > SharkNativeWrapper. The SharkNativeWrapper is allocated on-heap and the > > methodHandle inside the SharkNativeWrapper. I could have excluded that > > check altogther using an #ifndef SHARK, but the way I implemented it > > seemed more finegrained. > > I'd prefer an ifndef SHARK rather than change the API. Yeah ok, I understand that. However, I do not know how I can do an #ifdef inside a #define block. I'd need to duplicate the whole block, and put #ifdefs around those. This raises the question about code maintainability though, any changes in one block need to be done in the other block as well. I can do that if you prefer over the API change. That being said, multi-line-#defines are evil anyway ;-) Roman From vladimir.kozlov at oracle.com Tue Jan 13 16:11:17 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 13 Jan 2015 08:11:17 -0800 Subject: [8u60] Backport RFR: 8066763: fatal error "assert(false) failed: unexpected yanked node" in postaloc.cpp:139 In-Reply-To: <54B4C0F1.6050107@oracle.com> References: <54AD3C76.40601@oracle.com> <54B4C0F1.6050107@oracle.com> Message-ID: <54B543A5.9040906@oracle.com> Good. Thanks, Vladimir On 1/12/15 10:53 PM, Tobias Hartmann wrote: > Hi, > > can I get a review for this? > > Thanks, > Tobias > > On 07.01.2015 15:02, Tobias Hartmann wrote: >> Hi, >> >> please review the following backport request for 8u60. >> >> 8066763: fatal error "assert(false) failed: unexpected yanked node" in >> postaloc.cpp:139 >> https://bugs.openjdk.java.net/browse/JDK-8066763 >> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/f1cb27c85c83 >> >> The changes were pushed to JDK 9 on Dec 12 and nightly testing showed no >> problems. The patch applies cleanly to 8u60. >> >> Thanks, >> Tobias >> From tobias.hartmann at oracle.com Tue Jan 13 17:35:56 2015 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 13 Jan 2015 18:35:56 +0100 Subject: [8u60] Backport RFR: 8066763: fatal error "assert(false) failed: unexpected yanked node" in postaloc.cpp:139 In-Reply-To: <54B543A5.9040906@oracle.com> References: <54AD3C76.40601@oracle.com> <54B4C0F1.6050107@oracle.com> <54B543A5.9040906@oracle.com> Message-ID: <54B5577C.3070704@oracle.com> Thanks, Vladimir. Best, Tobias On 13.01.2015 17:11, Vladimir Kozlov wrote: > Good. > > Thanks, > Vladimir > > On 1/12/15 10:53 PM, Tobias Hartmann wrote: >> Hi, >> >> can I get a review for this? >> >> Thanks, >> Tobias >> >> On 07.01.2015 15:02, Tobias Hartmann wrote: >>> Hi, >>> >>> please review the following backport request for 8u60. >>> >>> 8066763: fatal error "assert(false) failed: unexpected yanked node" in >>> postaloc.cpp:139 >>> https://bugs.openjdk.java.net/browse/JDK-8066763 >>> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/f1cb27c85c83 >>> >>> The changes were pushed to JDK 9 on Dec 12 and nightly testing showed no >>> problems. The patch applies cleanly to 8u60. >>> >>> Thanks, >>> Tobias >>> From dean.long at oracle.com Tue Jan 13 18:56:21 2015 From: dean.long at oracle.com (Dean Long) Date: Tue, 13 Jan 2015 10:56:21 -0800 Subject: RFR: AARCH64: Top-level JDK changes In-Reply-To: <54B4E083.9040502@redhat.com> References: <545CFFA9.4070107@redhat.com> <545D0290.5080307@oracle.com> <54B34E20.5030506@oracle.com> <54B3A108.9070203@redhat.com> <54B4DAD4.5060904@oracle.com> <54B4E083.9040502@redhat.com> Message-ID: <54B56A55.3020803@oracle.com> On 1/13/2015 1:08 AM, Andrew Haley wrote: > On 13/01/15 08:44, Dean Long wrote: > >> I came up with a simpler version, where I replace "aarch64-" with >> "arm-", run autoconf-config.sub, then replace "arm-" back to >> "aarch64-". > Thanks. That sounds good to me. > > Andrew. Here's the patch. If it looks good, I can file a bug and push it to the staging repo. dl diff -r b052cb38b985 common/autoconf/build-aux/config.sub --- a/common/autoconf/build-aux/config.sub Thu Dec 11 15:05:06 2014 -0800 +++ b/common/autoconf/build-aux/config.sub Tue Jan 13 13:57:23 2015 -0500 @@ -41,25 +41,8 @@ case $1 in -- ) # Stop option processing shift; break ;; - aarch64-gnu ) - sub_args="$sub_args aarch64-unknown-gnu" - shift; ;; - aarch64-linux ) - sub_args="$sub_args aarch64-unknown-linux-gnu" - shift; ;; - aarch64-*-linux ) - os=`echo $1 | sed 's/aarch64-\(.*\)-linux/\1/'` - config="aarch64-unknown-linux-gnu" - sub_args="$sub_args $config" - shift; ;; - aarch64-*-gnu ) - os=`echo $1 | sed 's/aarch64-\(.*\)-gnu.*$/\1/'` - config="aarch64-unknown-gnu" - sub_args="$sub_args $config" - shift; ;; - aarch64-*-linux-* ) - os=`echo $1 | sed 's/aarch64-\(.*\)-linux-.*$/'` - config="aarch64-unknown-linux-gnu" + aarch64-* ) + config=`echo $1 | sed 's/^aarch64-/arm-/'` sub_args="$sub_args $config" shift; ;; - ) # Use stdin as input. @@ -74,9 +57,7 @@ result=`. $DIR/autoconf-config.sub $sub_args "$@"` exitcode=$? -if [ "x$os" != "x" ] ; then - result=`echo $result | sed "s/-unknown-/-$os-/"` -fi +result=`echo $result | sed "s/^arm-/aarch64-/"` echo $result exit $exitcode From coleen.phillimore at oracle.com Tue Jan 13 21:36:31 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 13 Jan 2015 16:36:31 -0500 Subject: RFR(M): 8068503: ppc64: Encode/Decode nodes for disjoint cOops mode In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF6DDED@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF69B22@DEWDFEMB12A.global.corp.sap> <54AD4885.9080700@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF6DDED@DEWDFEMB12A.global.corp.sap> Message-ID: <54B58FDF.7070506@oracle.com> Since this is mostly PPC and compiler code, I can't properly review it, so please do not wait for me. Coleen On 1/12/15, 2:57 AM, Lindenmaier, Goetz wrote: > Hi Coleen, > > Now I implemented the optimization in macroAssembler, also for the > decode case, anyways. > Unfortunately it's only used in the methodHandles code. > Testing took a bit because the results were spoiled by unrelated problems. > > I uploaded a new webrev: > http://cr.openjdk.java.net/~goetz/webrevs/8068503-ppcDisjoint/webrev.02/ > > Best regards, > Goetz. > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore > Sent: Wednesday, January 07, 2015 3:54 PM > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8068503: ppc64: Encode/Decode nodes for disjoint cOops mode > > > Don't you have to make (or want to make) changes to > MacroAssembler::decode_heap_oop_not_null in the macro assembler files? > > Coleen > > On 1/7/15, 3:26 AM, Lindenmaier, Goetz wrote: >> Hi, >> >> this change contains the implementation (activation) of the >> encode/decode nodes exploiting the disjoint heap base >> available since "8064457: Introduce compressed oops mode disjoint base..." >> >> Please review this change. >> http://cr.openjdk.java.net/~goetz/webrevs/8068503-ppcDisjoint/ >> >> This is ppc64-only, so I don't need a sponsor ;) >> >> Best regards, >> Goetz. >> From rkennke at redhat.com Tue Jan 13 21:41:04 2015 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 13 Jan 2015 22:41:04 +0100 Subject: [PATCH] Fix Shark build in JDK9 In-Reply-To: <54AF66B9.8060900@oracle.com> References: <1420641903.6838.9.camel@localhost> <54AD4C6B.1070702@oracle.com> <1420645301.6838.14.camel@localhost> <54AD5675.7010101@oracle.com> <1420647107.6838.15.camel@localhost> <54AD5BDF.1060401@oracle.com> <1420648161.6838.17.camel@localhost> <54AD639A.2060200@oracle.com> <1420662102.6838.23.camel@localhost> <54AE560B.3060200@oracle.com> <1420714881.6838.25.camel@localhost> <54AF66B9.8060900@oracle.com> Message-ID: <1421185264.6838.46.camel@localhost> Ok I think I found a way to avoid copying the whole #define block just for one line. I define that line before (and empty in case of SHARK) and use that inside the big #define. http://cr.openjdk.java.net/~rkennke/shark-build-hotspot/webrev.02/ Please review and push if ok. Roman Am Freitag, den 09.01.2015 um 15:27 +1000 schrieb David Holmes: > Hi Roman, > > Commenting on the hotspot changes ... > > On 8/01/2015 9:01 PM, Roman Kennke wrote: > > Hi Erik, > > > > I'm CC-ing hotspot-dev for review of Hotspot code related changes. > > > > Yes, some additional changes to Hotspot are required. This is the full > > set of changes needed to build and run Shark: > > > > http://cr.openjdk.java.net/~rkennke/shark-build-hotspot/webrev.01/ > > > > In detail: > > > > - In the Makefile fix a typo to be able to build unzipped debuginfo. > > Ok. > > > - In ciTypeFlow.cpp only include some files and code only when building > > C2. I don't think that code makes sense outside of C2. (That's the issue > > that you've seen). > > Looks okay but someone from compiler team needs to comment. There may be > other code that need adjusting. > > > - In systemDictionary.cpp, exclude some code for Shark that creates and > > checks native wrappers for method handle intrinsics. Invokedynamic stuff > > is not yet implemented in Shark, so we can't do this. > > Ok. > > > - In allocation.hpp, exclude the operator overrides for new etc, LLVM > > does allocations itself, and this would blow up. > > I'm intrigued as the allocation strategy is not tied to the compiler but > pervasive to the whole VM runtime. > > > - In handles.hpp and handles.inline.hpp, I added an argument > > check_in_stack so that I can disable the in-stack check for the > > SharkNativeWrapper. The SharkNativeWrapper is allocated on-heap and the > > methodHandle inside the SharkNativeWrapper. I could have excluded that > > check altogther using an #ifndef SHARK, but the way I implemented it > > seemed more finegrained. > > I'd prefer an ifndef SHARK rather than change the API. > > Thanks, > David > > > - In SharkCompiler, I pulled out the code to initialize LLVM into its > > own method, and the call to set_state(initialized) out of the > > execution-engine-lock. This would otherwise complain about wrong > > lock-ordering. > > - In SharkRuntime, I changed the cast from intptr_t to oop to work with > > the new picky compilers ;-) > > > > Please review. > > > > Thanks and best regards, > > Roman > > > > Am Donnerstag, den 08.01.2015 um 11:03 +0100 schrieb Erik Joelsson: > >> Hello Roman, > >> > >> This addition looks good to me. > >> > >> Thinking about what the others said, it might be inconvenient to have > >> all this be pushed to different forests. I tried applying all the > >> changes to jdk9/hs-rt, but I can't seem to build zeroshark.. Did you > >> have more hotspot changes to be able to finish the build? > >> > >> My failure is: > >> > >> ciTypeFlow.o > >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp > >> In file included from > >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/regmask.hpp:29:0, > >> from > >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/compile.hpp:40, > >> from > >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp:38: > >> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/optoreg.hpp:40:39: > >> fatal error: adfiles/adGlobals_zero.hpp: No such file or directory > >> > >> From what I can see, adfiles are not generated for zero or zeroshark > >> builds, so the include should probably be removed. > >> > >> Would you still like me to push what you currently have to hs-rt? > >> > >> /Erik > >> > >> On 2015-01-07 21:21, Roman Kennke wrote: > >>> Hi Erik, > >>> > >>> When I built Zero and Shark on my Raspberry Pi, I noticed another > >>> problem when copying jvm.cfg into the right places. I fixed it in a > >>> similar way as I did for the SA stuff: > >>> > >>> http://cr.openjdk.java.net/~rkennke/shark-build-jdk/webrev.02/ > >>> > >>> I think that should be all for now. > >>> > >>> Please push that into JDK9 if you think that's fine. > >>> > >>> Best regards, > >>> Roman > >>> > >>> Am Mittwoch, den 07.01.2015 um 17:49 +0100 schrieb Erik Joelsson: > >>>> On 2015-01-07 17:29, Roman Kennke wrote: > >>>>> Am Mittwoch, den 07.01.2015 um 17:16 +0100 schrieb Erik Joelsson: > >>>>>> On 2015-01-07 17:11, Roman Kennke wrote: > >>>>>>> Hi Erik, > >>>>>>> > >>>>>>> Do you have a bug for this? > >>>>>>> No. > >>>>>>> > >>>>>>> I haven't pushed any changes to JDK in a while. Is it possible in the > >>>>>>> meantime for me to create my own bugs? Otherwise, please file one for > >>>>>>> me :-) > >>>>>> You should be able to log in to https://bugs.openjdk.java.net and create > >>>>>> bugs since you have an OpenJDK identity. > >>>>> Done: > >>>>> > >>>>> https://bugs.openjdk.java.net/browse/JDK-8068598 > >>>>> > >>>>> While I'm at it, is it possible for me to push my own changes (except > >>>>> hotspot of course)? If yes, what needs to be done for regenerating the > >>>>> configure files? Simply run autogen.sh in common/autoconf with whatever > >>>>> version of autotools I have? Or doesn't it make sense at all b/c you > >>>>> need to regenerate your closed scripts? > >>>> It requires you to run common/autogen.sh yes, and that will require you > >>>> to have autoconf 2.69 installed. But since we also need to regenerate > >>>> the closed version, I can take care of the push for you. Will do it > >>>> tomorrow if that's ok? > >>>> > >>>> /Erik > >>>>> Roman > >>>>> > >>> > >> > > > > From david.dehaven at oracle.com Wed Jan 14 03:09:06 2015 From: david.dehaven at oracle.com (David DeHaven) Date: Tue, 13 Jan 2015 19:09:06 -0800 Subject: [8u60] RFR: 8043340: [macosx] Fix hard-wired paths to JavaVM.framework In-Reply-To: References: <39719ED3-1F4E-4E6C-AA9F-A3C4766537B4@oracle.com> <54B3A38C.10909@oracle.com> <8B01B84D-5B6F-478E-ABF1-33F5ADE2F826@oracle.com> <08C63261-D594-404E-892B-BDE2898B505A@oracle.com> Message-ID: <4DC9B291-D302-428C-A5F7-177066C65991@oracle.com> >> The --with-xcode-path argument is optional, you should also be able to build with Xcode 4 selected via "sudo xcode-select -switch /path/to/Xcode4.app". I leave MAS managed Xcode (currently 6) active as I'm constantly bouncing between projects and it's a hassle to have to remember to reset the active toolchain, so I thought it best to allow configure to be provided a path. > > Ugh. I broke something along the way, this doesn't *quite* work. > > xcrun complains with "xcrun: error: missing DEVELOPER_DIR path:" > > I think I'm exporting an empty DEVELOPER_DIR. I shall fix and update. TL;DR: Please review round 2: http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v1/ (I removed generated-configure.sh to reduce the review size, it will be re-generated prior to pushing) I've tested the following configuration scenarios (output from a shell script I cobbled together..) field values: XC6 - Xcode 6 installed in /Applications/Xcode.app XC4 - Xcode 4 installed in some other dir (empty) - Argument not passed to configure Result meanings: DEV_DIR set - configure succeeded, DEVELOPER_DIR was properly exported in spec.gmk DEV_DIR NOT set - configure succeeded, DEVELOPER_DIR was not needed (XC4 must be selected to achieve this) Configure failed - Configure properly failed when it detected Xcode > 4 "Selected" Xcode means version reported by xcode-select -p | Xcode selected | --with-xcode-path | DEVELOPER_DIR | result | ----------------------------------------------------------------------------- | XC4 | | | DEV_DIR NOT set | ----------------------------------------------------------------------------- | XC4 | | XC4 | DEV_DIR set | ----------------------------------------------------------------------------- | XC4 | | XC6 | Configure failed | ----------------------------------------------------------------------------- | XC4 | XC4 | | DEV_DIR set | ----------------------------------------------------------------------------- | XC4 | XC4 | XC4 | DEV_DIR set | ----------------------------------------------------------------------------- | XC4 | XC4 | XC6 | DEV_DIR set | ----------------------------------------------------------------------------- | XC4 | XC6 | | Configure failed | ----------------------------------------------------------------------------- | XC4 | XC6 | XC4 | Configure failed | ----------------------------------------------------------------------------- | XC4 | XC6 | XC6 | Configure failed | ----------------------------------------------------------------------------- | XC6 | | | Configure failed | ----------------------------------------------------------------------------- | XC6 | | XC4 | DEV_DIR set | ----------------------------------------------------------------------------- | XC6 | | XC6 | Configure failed | ----------------------------------------------------------------------------- | XC6 | XC4 | | DEV_DIR set | ----------------------------------------------------------------------------- | XC6 | XC4 | XC4 | DEV_DIR set | ----------------------------------------------------------------------------- | XC6 | XC4 | XC6 | DEV_DIR set | ----------------------------------------------------------------------------- | XC6 | XC6 | | Configure failed | ----------------------------------------------------------------------------- | XC6 | XC6 | XC4 | Configure failed | ----------------------------------------------------------------------------- | XC6 | XC6 | XC6 | Configure failed | ----------------------------------------------------------------------------- All the results are as expected. Please note that --with-xcode-path overrides DEVELOPER_DIR, since that could be set in the environment. (yeah, I went a little OCD on this...) -DrD- From serguei.spitsyn at oracle.com Wed Jan 14 05:47:52 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 13 Jan 2015 21:47:52 -0800 Subject: 4-th round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found Message-ID: <54B60308.4000105@oracle.com> Please, review the fix for: https://bugs.openjdk.java.net/browse/JDK-8068162 Open webrevs: http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/jdk/8068162-Test-IsModifiableAgent/ Summary: The sun.misc.Unsafe:throwIllegalAccessError() method is used in place of a default interface method in the itable if a default method was not defined in the interface. In fact, it happens for two interfaces that purhaps are auto-generated: java/nio/CharBuffer java/nio/HeapCharBuffer This approach creates a problem when the class sun.misc.Unsafe is retransformed. The Method* pointer to the old (redefined) method in the itable triggers an assert (see the hs_err log in the bug report). Coleen told me that a similar approach is going to be implemented for some vtable entries. Coleen, thanks for suggesting a better fix for this issue! The fix is to replace the old Unsafe method in the itable/vtable with the latest method version. Testing: In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests Thanks, Serguei From david.holmes at oracle.com Wed Jan 14 07:22:02 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 14 Jan 2015 17:22:02 +1000 Subject: [PATCH] Fix Shark build in JDK9 In-Reply-To: <1421185264.6838.46.camel@localhost> References: <1420641903.6838.9.camel@localhost> <54AD4C6B.1070702@oracle.com> <1420645301.6838.14.camel@localhost> <54AD5675.7010101@oracle.com> <1420647107.6838.15.camel@localhost> <54AD5BDF.1060401@oracle.com> <1420648161.6838.17.camel@localhost> <54AD639A.2060200@oracle.com> <1420662102.6838.23.camel@localhost> <54AE560B.3060200@oracle.com> <1420714881.6838.25.camel@localhost> <54AF66B9.8060900@oracle.com> <1421185264.6838.46.camel@localhost> Message-ID: <54B6191A.9040800@oracle.com> Hi Roman, On 14/01/2015 7:41 AM, Roman Kennke wrote: > Ok I think I found a way to avoid copying the whole #define block just > for one line. I define that line before (and empty in case of SHARK) and > use that inside the big #define. > > http://cr.openjdk.java.net/~rkennke/shark-build-hotspot/webrev.02/ Can you use SHARK_ONLY or NOT_SHARK macros? I think that would be cleaner. Still need a second hotspot reviewer - preferably someone from compiler team. And was a bug filed for this? Thanks, David > Please review and push if ok. > > Roman > > Am Freitag, den 09.01.2015 um 15:27 +1000 schrieb David Holmes: >> Hi Roman, >> >> Commenting on the hotspot changes ... >> >> On 8/01/2015 9:01 PM, Roman Kennke wrote: >>> Hi Erik, >>> >>> I'm CC-ing hotspot-dev for review of Hotspot code related changes. >>> >>> Yes, some additional changes to Hotspot are required. This is the full >>> set of changes needed to build and run Shark: >>> >>> http://cr.openjdk.java.net/~rkennke/shark-build-hotspot/webrev.01/ >>> >>> In detail: >>> >>> - In the Makefile fix a typo to be able to build unzipped debuginfo. >> >> Ok. >> >>> - In ciTypeFlow.cpp only include some files and code only when building >>> C2. I don't think that code makes sense outside of C2. (That's the issue >>> that you've seen). >> >> Looks okay but someone from compiler team needs to comment. There may be >> other code that need adjusting. >> >>> - In systemDictionary.cpp, exclude some code for Shark that creates and >>> checks native wrappers for method handle intrinsics. Invokedynamic stuff >>> is not yet implemented in Shark, so we can't do this. >> >> Ok. >> >>> - In allocation.hpp, exclude the operator overrides for new etc, LLVM >>> does allocations itself, and this would blow up. >> >> I'm intrigued as the allocation strategy is not tied to the compiler but >> pervasive to the whole VM runtime. >> >>> - In handles.hpp and handles.inline.hpp, I added an argument >>> check_in_stack so that I can disable the in-stack check for the >>> SharkNativeWrapper. The SharkNativeWrapper is allocated on-heap and the >>> methodHandle inside the SharkNativeWrapper. I could have excluded that >>> check altogther using an #ifndef SHARK, but the way I implemented it >>> seemed more finegrained. >> >> I'd prefer an ifndef SHARK rather than change the API. >> >> Thanks, >> David >> >>> - In SharkCompiler, I pulled out the code to initialize LLVM into its >>> own method, and the call to set_state(initialized) out of the >>> execution-engine-lock. This would otherwise complain about wrong >>> lock-ordering. >>> - In SharkRuntime, I changed the cast from intptr_t to oop to work with >>> the new picky compilers ;-) >>> >>> Please review. >>> >>> Thanks and best regards, >>> Roman >>> >>> Am Donnerstag, den 08.01.2015 um 11:03 +0100 schrieb Erik Joelsson: >>>> Hello Roman, >>>> >>>> This addition looks good to me. >>>> >>>> Thinking about what the others said, it might be inconvenient to have >>>> all this be pushed to different forests. I tried applying all the >>>> changes to jdk9/hs-rt, but I can't seem to build zeroshark.. Did you >>>> have more hotspot changes to be able to finish the build? >>>> >>>> My failure is: >>>> >>>> ciTypeFlow.o >>>> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp >>>> In file included from >>>> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/regmask.hpp:29:0, >>>> from >>>> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/compile.hpp:40, >>>> from >>>> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp:38: >>>> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/optoreg.hpp:40:39: >>>> fatal error: adfiles/adGlobals_zero.hpp: No such file or directory >>>> >>>> From what I can see, adfiles are not generated for zero or zeroshark >>>> builds, so the include should probably be removed. >>>> >>>> Would you still like me to push what you currently have to hs-rt? >>>> >>>> /Erik >>>> >>>> On 2015-01-07 21:21, Roman Kennke wrote: >>>>> Hi Erik, >>>>> >>>>> When I built Zero and Shark on my Raspberry Pi, I noticed another >>>>> problem when copying jvm.cfg into the right places. I fixed it in a >>>>> similar way as I did for the SA stuff: >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/shark-build-jdk/webrev.02/ >>>>> >>>>> I think that should be all for now. >>>>> >>>>> Please push that into JDK9 if you think that's fine. >>>>> >>>>> Best regards, >>>>> Roman >>>>> >>>>> Am Mittwoch, den 07.01.2015 um 17:49 +0100 schrieb Erik Joelsson: >>>>>> On 2015-01-07 17:29, Roman Kennke wrote: >>>>>>> Am Mittwoch, den 07.01.2015 um 17:16 +0100 schrieb Erik Joelsson: >>>>>>>> On 2015-01-07 17:11, Roman Kennke wrote: >>>>>>>>> Hi Erik, >>>>>>>>> >>>>>>>>> Do you have a bug for this? >>>>>>>>> No. >>>>>>>>> >>>>>>>>> I haven't pushed any changes to JDK in a while. Is it possible in the >>>>>>>>> meantime for me to create my own bugs? Otherwise, please file one for >>>>>>>>> me :-) >>>>>>>> You should be able to log in to https://bugs.openjdk.java.net and create >>>>>>>> bugs since you have an OpenJDK identity. >>>>>>> Done: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8068598 >>>>>>> >>>>>>> While I'm at it, is it possible for me to push my own changes (except >>>>>>> hotspot of course)? If yes, what needs to be done for regenerating the >>>>>>> configure files? Simply run autogen.sh in common/autoconf with whatever >>>>>>> version of autotools I have? Or doesn't it make sense at all b/c you >>>>>>> need to regenerate your closed scripts? >>>>>> It requires you to run common/autogen.sh yes, and that will require you >>>>>> to have autoconf 2.69 installed. But since we also need to regenerate >>>>>> the closed version, I can take care of the push for you. Will do it >>>>>> tomorrow if that's ok? >>>>>> >>>>>> /Erik >>>>>>> Roman >>>>>>> >>>>> >>>> >>> >>> > > From erik.joelsson at oracle.com Wed Jan 14 08:33:28 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 14 Jan 2015 09:33:28 +0100 Subject: [8u60] RFR: 8043340: [macosx] Fix hard-wired paths to JavaVM.framework In-Reply-To: <4DC9B291-D302-428C-A5F7-177066C65991@oracle.com> References: <39719ED3-1F4E-4E6C-AA9F-A3C4766537B4@oracle.com> <54B3A38C.10909@oracle.com> <8B01B84D-5B6F-478E-ABF1-33F5ADE2F826@oracle.com> <08C63261-D594-404E-892B-BDE2898B505A@oracle.com> <4DC9B291-D302-428C-A5F7-177066C65991@oracle.com> Message-ID: <54B629D8.8070700@oracle.com> Hello, This looks good to me. Thanks for the detailed table! /Erik On 2015-01-14 04:09, David DeHaven wrote: >>> The --with-xcode-path argument is optional, you should also be able to build with Xcode 4 selected via "sudo xcode-select -switch /path/to/Xcode4.app". I leave MAS managed Xcode (currently 6) active as I'm constantly bouncing between projects and it's a hassle to have to remember to reset the active toolchain, so I thought it best to allow configure to be provided a path. >> Ugh. I broke something along the way, this doesn't *quite* work. >> >> xcrun complains with "xcrun: error: missing DEVELOPER_DIR path:" >> >> I think I'm exporting an empty DEVELOPER_DIR. I shall fix and update. > TL;DR: Please review round 2: > http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v1/ > > (I removed generated-configure.sh to reduce the review size, it will be re-generated prior to pushing) > > > I've tested the following configuration scenarios (output from a shell script I cobbled together..) > > field values: > XC6 - Xcode 6 installed in /Applications/Xcode.app > XC4 - Xcode 4 installed in some other dir > (empty) - Argument not passed to configure > > Result meanings: > DEV_DIR set - configure succeeded, DEVELOPER_DIR was properly exported in spec.gmk > DEV_DIR NOT set - configure succeeded, DEVELOPER_DIR was not needed (XC4 must be selected to achieve this) > Configure failed - Configure properly failed when it detected Xcode > 4 > > "Selected" Xcode means version reported by xcode-select -p > > > | Xcode selected | --with-xcode-path | DEVELOPER_DIR | result | > ----------------------------------------------------------------------------- > | XC4 | | | DEV_DIR NOT set | > ----------------------------------------------------------------------------- > | XC4 | | XC4 | DEV_DIR set | > ----------------------------------------------------------------------------- > | XC4 | | XC6 | Configure failed | > ----------------------------------------------------------------------------- > | XC4 | XC4 | | DEV_DIR set | > ----------------------------------------------------------------------------- > | XC4 | XC4 | XC4 | DEV_DIR set | > ----------------------------------------------------------------------------- > | XC4 | XC4 | XC6 | DEV_DIR set | > ----------------------------------------------------------------------------- > | XC4 | XC6 | | Configure failed | > ----------------------------------------------------------------------------- > | XC4 | XC6 | XC4 | Configure failed | > ----------------------------------------------------------------------------- > | XC4 | XC6 | XC6 | Configure failed | > ----------------------------------------------------------------------------- > | XC6 | | | Configure failed | > ----------------------------------------------------------------------------- > | XC6 | | XC4 | DEV_DIR set | > ----------------------------------------------------------------------------- > | XC6 | | XC6 | Configure failed | > ----------------------------------------------------------------------------- > | XC6 | XC4 | | DEV_DIR set | > ----------------------------------------------------------------------------- > | XC6 | XC4 | XC4 | DEV_DIR set | > ----------------------------------------------------------------------------- > | XC6 | XC4 | XC6 | DEV_DIR set | > ----------------------------------------------------------------------------- > | XC6 | XC6 | | Configure failed | > ----------------------------------------------------------------------------- > | XC6 | XC6 | XC4 | Configure failed | > ----------------------------------------------------------------------------- > | XC6 | XC6 | XC6 | Configure failed | > ----------------------------------------------------------------------------- > > All the results are as expected. Please note that --with-xcode-path overrides DEVELOPER_DIR, since that could be set in the environment. > > (yeah, I went a little OCD on this...) > > -DrD- > From erik.joelsson at oracle.com Wed Jan 14 08:35:17 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 14 Jan 2015 09:35:17 +0100 Subject: [PATCH] Fix Shark build in JDK9 In-Reply-To: <54B6191A.9040800@oracle.com> References: <1420641903.6838.9.camel@localhost> <54AD4C6B.1070702@oracle.com> <1420645301.6838.14.camel@localhost> <54AD5675.7010101@oracle.com> <1420647107.6838.15.camel@localhost> <54AD5BDF.1060401@oracle.com> <1420648161.6838.17.camel@localhost> <54AD639A.2060200@oracle.com> <1420662102.6838.23.camel@localhost> <54AE560B.3060200@oracle.com> <1420714881.6838.25.camel@localhost> <54AF66B9.8060900@oracle.com> <1421185264.6838.46.camel@localhost> <54B6191A.9040800@oracle.com> Message-ID: <54B62A45.2030600@oracle.com> On 2015-01-14 08:22, David Holmes wrote: > Hi Roman, > > On 14/01/2015 7:41 AM, Roman Kennke wrote: >> Ok I think I found a way to avoid copying the whole #define block just >> for one line. I define that line before (and empty in case of SHARK) and >> use that inside the big #define. >> >> http://cr.openjdk.java.net/~rkennke/shark-build-hotspot/webrev.02/ > > Can you use SHARK_ONLY or NOT_SHARK macros? I think that would be > cleaner. > > Still need a second hotspot reviewer - preferably someone from > compiler team. > > And was a bug filed for this? > A bug was filed at least: https://bugs.openjdk.java.net/browse/JDK-8068598 I promised to handle the push when it's all reviewed. /Erik > Thanks, > David > >> Please review and push if ok. >> >> Roman >> >> Am Freitag, den 09.01.2015 um 15:27 +1000 schrieb David Holmes: >>> Hi Roman, >>> >>> Commenting on the hotspot changes ... >>> >>> On 8/01/2015 9:01 PM, Roman Kennke wrote: >>>> Hi Erik, >>>> >>>> I'm CC-ing hotspot-dev for review of Hotspot code related changes. >>>> >>>> Yes, some additional changes to Hotspot are required. This is the full >>>> set of changes needed to build and run Shark: >>>> >>>> http://cr.openjdk.java.net/~rkennke/shark-build-hotspot/webrev.01/ >>>> >>>> In detail: >>>> >>>> - In the Makefile fix a typo to be able to build unzipped debuginfo. >>> >>> Ok. >>> >>>> - In ciTypeFlow.cpp only include some files and code only when >>>> building >>>> C2. I don't think that code makes sense outside of C2. (That's the >>>> issue >>>> that you've seen). >>> >>> Looks okay but someone from compiler team needs to comment. There >>> may be >>> other code that need adjusting. >>> >>>> - In systemDictionary.cpp, exclude some code for Shark that creates >>>> and >>>> checks native wrappers for method handle intrinsics. Invokedynamic >>>> stuff >>>> is not yet implemented in Shark, so we can't do this. >>> >>> Ok. >>> >>>> - In allocation.hpp, exclude the operator overrides for new etc, LLVM >>>> does allocations itself, and this would blow up. >>> >>> I'm intrigued as the allocation strategy is not tied to the compiler >>> but >>> pervasive to the whole VM runtime. >>> >>>> - In handles.hpp and handles.inline.hpp, I added an argument >>>> check_in_stack so that I can disable the in-stack check for the >>>> SharkNativeWrapper. The SharkNativeWrapper is allocated on-heap and >>>> the >>>> methodHandle inside the SharkNativeWrapper. I could have excluded that >>>> check altogther using an #ifndef SHARK, but the way I implemented it >>>> seemed more finegrained. >>> >>> I'd prefer an ifndef SHARK rather than change the API. >>> >>> Thanks, >>> David >>> >>>> - In SharkCompiler, I pulled out the code to initialize LLVM into its >>>> own method, and the call to set_state(initialized) out of the >>>> execution-engine-lock. This would otherwise complain about wrong >>>> lock-ordering. >>>> - In SharkRuntime, I changed the cast from intptr_t to oop to work >>>> with >>>> the new picky compilers ;-) >>>> >>>> Please review. >>>> >>>> Thanks and best regards, >>>> Roman >>>> >>>> Am Donnerstag, den 08.01.2015 um 11:03 +0100 schrieb Erik Joelsson: >>>>> Hello Roman, >>>>> >>>>> This addition looks good to me. >>>>> >>>>> Thinking about what the others said, it might be inconvenient to have >>>>> all this be pushed to different forests. I tried applying all the >>>>> changes to jdk9/hs-rt, but I can't seem to build zeroshark.. Did you >>>>> have more hotspot changes to be able to finish the build? >>>>> >>>>> My failure is: >>>>> >>>>> ciTypeFlow.o >>>>> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp >>>>> In file included from >>>>> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/regmask.hpp:29:0, >>>>> from >>>>> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/compile.hpp:40, >>>>> from >>>>> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/ci/ciTypeFlow.cpp:38: >>>>> /localhome/hg/jdk9-hs-rt/hotspot/src/share/vm/opto/optoreg.hpp:40:39: >>>>> fatal error: adfiles/adGlobals_zero.hpp: No such file or directory >>>>> >>>>> From what I can see, adfiles are not generated for zero or >>>>> zeroshark >>>>> builds, so the include should probably be removed. >>>>> >>>>> Would you still like me to push what you currently have to hs-rt? >>>>> >>>>> /Erik >>>>> >>>>> On 2015-01-07 21:21, Roman Kennke wrote: >>>>>> Hi Erik, >>>>>> >>>>>> When I built Zero and Shark on my Raspberry Pi, I noticed another >>>>>> problem when copying jvm.cfg into the right places. I fixed it in a >>>>>> similar way as I did for the SA stuff: >>>>>> >>>>>> http://cr.openjdk.java.net/~rkennke/shark-build-jdk/webrev.02/ >>>>>> >>>>>> I think that should be all for now. >>>>>> >>>>>> Please push that into JDK9 if you think that's fine. >>>>>> >>>>>> Best regards, >>>>>> Roman >>>>>> >>>>>> Am Mittwoch, den 07.01.2015 um 17:49 +0100 schrieb Erik Joelsson: >>>>>>> On 2015-01-07 17:29, Roman Kennke wrote: >>>>>>>> Am Mittwoch, den 07.01.2015 um 17:16 +0100 schrieb Erik Joelsson: >>>>>>>>> On 2015-01-07 17:11, Roman Kennke wrote: >>>>>>>>>> Hi Erik, >>>>>>>>>> >>>>>>>>>> Do you have a bug for this? >>>>>>>>>> No. >>>>>>>>>> >>>>>>>>>> I haven't pushed any changes to JDK in a while. Is it >>>>>>>>>> possible in the >>>>>>>>>> meantime for me to create my own bugs? Otherwise, please file >>>>>>>>>> one for >>>>>>>>>> me :-) >>>>>>>>> You should be able to log in to https://bugs.openjdk.java.net >>>>>>>>> and create >>>>>>>>> bugs since you have an OpenJDK identity. >>>>>>>> Done: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8068598 >>>>>>>> >>>>>>>> While I'm at it, is it possible for me to push my own changes >>>>>>>> (except >>>>>>>> hotspot of course)? If yes, what needs to be done for >>>>>>>> regenerating the >>>>>>>> configure files? Simply run autogen.sh in common/autoconf with >>>>>>>> whatever >>>>>>>> version of autotools I have? Or doesn't it make sense at all >>>>>>>> b/c you >>>>>>>> need to regenerate your closed scripts? >>>>>>> It requires you to run common/autogen.sh yes, and that will >>>>>>> require you >>>>>>> to have autoconf 2.69 installed. But since we also need to >>>>>>> regenerate >>>>>>> the closed version, I can take care of the push for you. Will do it >>>>>>> tomorrow if that's ok? >>>>>>> >>>>>>> /Erik >>>>>>>> Roman >>>>>>>> >>>>>> >>>>> >>>> >>>> >> >> From goetz.lindenmaier at sap.com Wed Jan 14 10:20:29 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 14 Jan 2015 10:20:29 +0000 Subject: RFR(M): 8068503: ppc64: Encode/Decode nodes for disjoint cOops mode In-Reply-To: <54B58FDF.7070506@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF69B22@DEWDFEMB12A.global.corp.sap> <54AD4885.9080700@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF6DDED@DEWDFEMB12A.global.corp.sap> <54B58FDF.7070506@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF6E83B@DEWDFEMB12A.global.corp.sap> Hi Coleen, ok, I understand. Thanks for the input on this one so far, it really was helpful! Best regards, Goetz. -----Original Message----- From: Coleen Phillimore [mailto:coleen.phillimore at oracle.com] Sent: Dienstag, 13. Januar 2015 22:37 To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' Subject: Re: RFR(M): 8068503: ppc64: Encode/Decode nodes for disjoint cOops mode Since this is mostly PPC and compiler code, I can't properly review it, so please do not wait for me. Coleen On 1/12/15, 2:57 AM, Lindenmaier, Goetz wrote: > Hi Coleen, > > Now I implemented the optimization in macroAssembler, also for the > decode case, anyways. > Unfortunately it's only used in the methodHandles code. > Testing took a bit because the results were spoiled by unrelated problems. > > I uploaded a new webrev: > http://cr.openjdk.java.net/~goetz/webrevs/8068503-ppcDisjoint/webrev.02/ > > Best regards, > Goetz. > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore > Sent: Wednesday, January 07, 2015 3:54 PM > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8068503: ppc64: Encode/Decode nodes for disjoint cOops mode > > > Don't you have to make (or want to make) changes to > MacroAssembler::decode_heap_oop_not_null in the macro assembler files? > > Coleen > > On 1/7/15, 3:26 AM, Lindenmaier, Goetz wrote: >> Hi, >> >> this change contains the implementation (activation) of the >> encode/decode nodes exploiting the disjoint heap base >> available since "8064457: Introduce compressed oops mode disjoint base..." >> >> Please review this change. >> http://cr.openjdk.java.net/~goetz/webrevs/8068503-ppcDisjoint/ >> >> This is ppc64-only, so I don't need a sponsor ;) >> >> Best regards, >> Goetz. >> From magnus.ihse.bursie at oracle.com Wed Jan 14 13:27:15 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 14 Jan 2015 14:27:15 +0100 Subject: RFR: AARCH64: Top-level JDK changes In-Reply-To: <54B4D82B.3050608@oracle.com> References: <545CFFA9.4070107@redhat.com> <545D0290.5080307@oracle.com> <54B34E20.5030506@oracle.com> <54B3B4CE.8060804@oracle.com> <54B4D82B.3050608@oracle.com> Message-ID: <54B66EB3.3020601@oracle.com> On 2015-01-13 09:32, Dean Long wrote: > On 1/12/2015 3:49 AM, Magnus Ihse Bursie wrote: >> On 2015-01-12 05:31, Dean Long wrote: >>> I found a small problem with the new config.sub wrapper. It works >>> with the bash shell but not with the dash shell. >>> The problem seems to be with this line: >>> >>> result=`. $DIR/autoconf-config.sub $sub_args "$@"` >>> >>> "dash" doesn't seem to support args passed with ".", so $sub_args >>> "$@" are ignored. >> >> bash is the required shell for running configure. We do not support >> non-bash shells. In fact, we go to lengths to try to ensure that we >> are indeed running under bash. >> >> /Magnus > I was thinking 'bash configure' was enough, but it turns out > 'CONFIG_SHELL=bash bash configure' gives better results. Hm, that's interesting. We were attempting to automatically use bash in the real configure script, regardless of what shell the user had to start the top-level configure wrapper. If you try the patch below, does it work better when you run "dash configure"? diff --git a/common/autoconf/configure b/common/autoconf/configure --- a/common/autoconf/configure +++ b/common/autoconf/configure @@ -36,6 +36,13 @@ shift fi +if test "x$BASH" = x; then + echo "Error: This script must be run using bash." 1>&2 + exit 1 +fi +# Force autoconf to use bash +export CONFIG_SHELL=$BASH + conf_script_dir="$TOPDIR/common/autoconf" if [ "$CUSTOM_CONFIG_DIR" = "" ]; then /Magnus From volker.simonis at gmail.com Wed Jan 14 13:42:55 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 14 Jan 2015 14:42:55 +0100 Subject: RFR(M): 8068503: ppc64: Encode/Decode nodes for disjoint cOops mode In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF6DDED@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF69B22@DEWDFEMB12A.global.corp.sap> <54AD4885.9080700@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF6DDED@DEWDFEMB12A.global.corp.sap> Message-ID: Hi Goetz, the change looks good. Thumbs up from my side. Regards, Volker On Mon, Jan 12, 2015 at 8:57 AM, Lindenmaier, Goetz wrote: > Hi Coleen, > > Now I implemented the optimization in macroAssembler, also for the > decode case, anyways. > Unfortunately it's only used in the methodHandles code. > Testing took a bit because the results were spoiled by unrelated problems. > > I uploaded a new webrev: > http://cr.openjdk.java.net/~goetz/webrevs/8068503-ppcDisjoint/webrev.02/ > > Best regards, > Goetz. > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore > Sent: Wednesday, January 07, 2015 3:54 PM > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8068503: ppc64: Encode/Decode nodes for disjoint cOops mode > > > Don't you have to make (or want to make) changes to > MacroAssembler::decode_heap_oop_not_null in the macro assembler files? > > Coleen > > On 1/7/15, 3:26 AM, Lindenmaier, Goetz wrote: >> Hi, >> >> this change contains the implementation (activation) of the >> encode/decode nodes exploiting the disjoint heap base >> available since "8064457: Introduce compressed oops mode disjoint base..." >> >> Please review this change. >> http://cr.openjdk.java.net/~goetz/webrevs/8068503-ppcDisjoint/ >> >> This is ppc64-only, so I don't need a sponsor ;) >> >> Best regards, >> Goetz. >> > From jaroslav.bachorik at oracle.com Wed Jan 14 14:05:41 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Wed, 14 Jan 2015 15:05:41 +0100 Subject: RFR 8068976: Remove JSDT implementation Message-ID: <54B677B5.3080809@oracle.com> Please, review the following removal of functionality. Issue : https://bugs.openjdk.java.net/browse/JDK-8068976 Webrev: http://cr.openjdk.java.net/~jbachorik/8068976/webrev.00 JDK-8062303 [1] requests the removal of com.sun.tracing API. This API was added as experimental in JDK7 [2][3] and haven't seen any significant uptake since then. JSDT builds on top of the com.sun.tracing API and provides the bridge to DTrace. It allows Java programs to fire custom DTrace probes. As an implementation of com.sun.tracing API it needs to be removed before proceeding with the removal of that API. This change is basically an anti-delta to [4] - minus the changes not related to JSDT and/or DTrace. This change passes all the regression and JCK tests. Thanks, -JB- [1] https://bugs.openjdk.java.net/browse/JDK-8062303 [2] https://bugs.openjdk.java.net/browse/JDK-6537506 [3] https://bugs.openjdk.java.net/browse/JDK-7037081 [4] http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/018d5b58dd4f From goetz.lindenmaier at sap.com Wed Jan 14 14:10:02 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 14 Jan 2015 14:10:02 +0000 Subject: RFR(M): 8068503: ppc64: Encode/Decode nodes for disjoint cOops mode In-Reply-To: References: <4295855A5C1DE049A61835A1887419CC2CF69B22@DEWDFEMB12A.global.corp.sap> <54AD4885.9080700@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF6DDED@DEWDFEMB12A.global.corp.sap> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF6E949@DEWDFEMB12A.global.corp.sap> Thanks Volker! -----Original Message----- From: Volker Simonis [mailto:volker.simonis at gmail.com] Sent: Mittwoch, 14. Januar 2015 14:43 To: Lindenmaier, Goetz Cc: Coleen Phillimore; hotspot-dev at openjdk.java.net Subject: Re: RFR(M): 8068503: ppc64: Encode/Decode nodes for disjoint cOops mode Hi Goetz, the change looks good. Thumbs up from my side. Regards, Volker On Mon, Jan 12, 2015 at 8:57 AM, Lindenmaier, Goetz wrote: > Hi Coleen, > > Now I implemented the optimization in macroAssembler, also for the > decode case, anyways. > Unfortunately it's only used in the methodHandles code. > Testing took a bit because the results were spoiled by unrelated problems. > > I uploaded a new webrev: > http://cr.openjdk.java.net/~goetz/webrevs/8068503-ppcDisjoint/webrev.02/ > > Best regards, > Goetz. > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore > Sent: Wednesday, January 07, 2015 3:54 PM > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8068503: ppc64: Encode/Decode nodes for disjoint cOops mode > > > Don't you have to make (or want to make) changes to > MacroAssembler::decode_heap_oop_not_null in the macro assembler files? > > Coleen > > On 1/7/15, 3:26 AM, Lindenmaier, Goetz wrote: >> Hi, >> >> this change contains the implementation (activation) of the >> encode/decode nodes exploiting the disjoint heap base >> available since "8064457: Introduce compressed oops mode disjoint base..." >> >> Please review this change. >> http://cr.openjdk.java.net/~goetz/webrevs/8068503-ppcDisjoint/ >> >> This is ppc64-only, so I don't need a sponsor ;) >> >> Best regards, >> Goetz. >> > From kmcguigan at twitter.com Wed Jan 14 14:13:23 2015 From: kmcguigan at twitter.com (Keith McGuigan) Date: Wed, 14 Jan 2015 09:13:23 -0500 Subject: RFR 8068976: Remove JSDT implementation In-Reply-To: <54B677B5.3080809@oracle.com> References: <54B677B5.3080809@oracle.com> Message-ID: What evidence do you have that no OpenJDK users are using this? On Wed, Jan 14, 2015 at 9:05 AM, Jaroslav Bachorik < jaroslav.bachorik at oracle.com> wrote: > Please, review the following removal of functionality. > > Issue : https://bugs.openjdk.java.net/browse/JDK-8068976 > Webrev: http://cr.openjdk.java.net/~jbachorik/8068976/webrev.00 > > JDK-8062303 [1] requests the removal of com.sun.tracing API. This API was > added as experimental in JDK7 [2][3] and haven't seen any significant > uptake since then. > > JSDT builds on top of the com.sun.tracing API and provides the bridge to > DTrace. It allows Java programs to fire custom DTrace probes. As an > implementation of com.sun.tracing API it needs to be removed before > proceeding with the removal of that API. > > This change is basically an anti-delta to [4] - minus the changes not > related to JSDT and/or DTrace. > > This change passes all the regression and JCK tests. > > Thanks, > > -JB- > > > [1] https://bugs.openjdk.java.net/browse/JDK-8062303 > [2] https://bugs.openjdk.java.net/browse/JDK-6537506 > [3] https://bugs.openjdk.java.net/browse/JDK-7037081 > [4] http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/018d5b58dd4f > -- [image: twitter-icon-large.png] Keith McGuigan @kamggg kmcguigan at twitter.com From jaroslav.bachorik at oracle.com Wed Jan 14 14:28:48 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Wed, 14 Jan 2015 15:28:48 +0100 Subject: RFR 8068976: Remove JSDT implementation In-Reply-To: References: <54B677B5.3080809@oracle.com> Message-ID: <54B67D20.4040406@oracle.com> On 14.1.2015 15:13, Keith McGuigan wrote: > What evidence do you have that no OpenJDK users are using this? A search from the snapshot of the maven central as of May 2014 finds only no reference to com.sun.tracing except it finds one artifact jsdt-compat (https://github.com/cmbntr/jsdt-compat) that redistributes com.sun.tracing APIs and implementation classes for pre-JDK7 usage. As there is no possible way to use JSDT without going through com.sun.tracing API the lack of references in maven central could be good enough indication of this functionality not being in use. AFAIK, there is no other reliable way to figure out whether an API is in active use or not :( -JB- > > On Wed, Jan 14, 2015 at 9:05 AM, Jaroslav Bachorik > > wrote: > > Please, review the following removal of functionality. > > Issue : https://bugs.openjdk.java.net/__browse/JDK-8068976 > > Webrev: http://cr.openjdk.java.net/~__jbachorik/8068976/webrev.00 > > > JDK-8062303 [1] requests the removal of com.sun.tracing API. This > API was added as experimental in JDK7 [2][3] and haven't seen any > significant uptake since then. > > JSDT builds on top of the com.sun.tracing API and provides the > bridge to DTrace. It allows Java programs to fire custom DTrace > probes. As an implementation of com.sun.tracing API it needs to be > removed before proceeding with the removal of that API. > > This change is basically an anti-delta to [4] - minus the changes > not related to JSDT and/or DTrace. > > This change passes all the regression and JCK tests. > > Thanks, > > -JB- > > > [1] https://bugs.openjdk.java.net/__browse/JDK-8062303 > > [2] https://bugs.openjdk.java.net/__browse/JDK-6537506 > > [3] https://bugs.openjdk.java.net/__browse/JDK-7037081 > > [4] > http://hg.openjdk.java.net/__jdk9/hs-rt/hotspot/rev/__018d5b58dd4f > > > > > > -- > > twitter-icon-large.png > > > > Keith McGuigan > > @kamggg > > kmcguigan at twitter.com > From stefan.johansson at oracle.com Wed Jan 14 15:34:22 2015 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 14 Jan 2015 16:34:22 +0100 Subject: RFR: 8067368: TestConcMarkCycleWB.java crashed at G1CollectedHeap::heap()+0xb Message-ID: <54B68C7E.2010707@oracle.com> Hi, Please review this small fix to add TestConcMarkCycleWB.java to the needs_g1gc test group. This should fix the nightly problem described in: https://bugs.openjdk.java.net/browse/JDK-8067368 Webrev: http://cr.openjdk.java.net/~sjohanss/8067368/hotspot.00/ Summary: The runs failing this test are executed with either the test-group "embedded_jdk" or "embedded_jre". Both those exclude tests defined in the group "needs_g1gc" and since TestConcMarkCycleWB.java is a G1 specific WhiteBox test it should be included in the "needs_g1gc" group. Test: Manually verified that a jtreg of execution for "embedded_jdk" or "embedded_jre" after the patch is applied don't include TestConcMarkCycleWB.java. Thanks, Stefan From jesper.wilhelmsson at oracle.com Wed Jan 14 16:23:50 2015 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 14 Jan 2015 17:23:50 +0100 Subject: RFR: 8067368: TestConcMarkCycleWB.java crashed at G1CollectedHeap::heap()+0xb In-Reply-To: <54B68C7E.2010707@oracle.com> References: <54B68C7E.2010707@oracle.com> Message-ID: <54B69816.40602@oracle.com> Looks good! /Jesper Stefan Johansson skrev den 14/1/15 16:34: > Hi, > > Please review this small fix to add TestConcMarkCycleWB.java to the needs_g1gc > test group. This should fix the nightly problem described in: > https://bugs.openjdk.java.net/browse/JDK-8067368 > > Webrev: > http://cr.openjdk.java.net/~sjohanss/8067368/hotspot.00/ > > Summary: > The runs failing this test are executed with either the test-group > "embedded_jdk" or "embedded_jre". Both those exclude tests defined in the group > "needs_g1gc" and since TestConcMarkCycleWB.java is a G1 specific WhiteBox test > it should be included in the "needs_g1gc" group. > > Test: > Manually verified that a jtreg of execution for "embedded_jdk" or "embedded_jre" > after the patch is applied don't include TestConcMarkCycleWB.java. > > Thanks, > Stefan From spullara+hotspot at gmail.com Wed Jan 14 17:23:41 2015 From: spullara+hotspot at gmail.com (Sam Pullara) Date: Wed, 14 Jan 2015 09:23:41 -0800 Subject: Invokedynamic deoptimization issues Message-ID: Hi, I've implemented a bunch of different ways for mustache.java to get data from names fields and methods. One thing I have noticed running benchmarks is that accessing the fields/methods in different ways can cause invokedynamic to either never be optimized or actually be deoptimized later during the running of the benchmarks (20x+ difference in performance). Here is the code I have been running: https://github.com/spullara/indybenchmark I was wondering what the best way to debug this would be? Thanks, Sam From vladimir.x.ivanov at oracle.com Wed Jan 14 18:40:04 2015 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 14 Jan 2015 21:40:04 +0300 Subject: Invokedynamic deoptimization issues In-Reply-To: References: Message-ID: <54B6B804.5020607@oracle.com> Sam, The first thing I usually do is run with: -XX:+PrintCompilation -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining I haven't looked at your benchmarks yet, but my main suspicion is that MethodHandles aren't compile-time constants, so inlining fails. Do you use invokedynamic bytecode or call MethodHandle.invokeExact()/invoke()? In the latter case, you should store MethodHandles in static final fields. Otherwise, JIT won't inline them. Best regards, Vladimir Ivanov On 1/14/15 8:23 PM, Sam Pullara wrote: > Hi, > > I've implemented a bunch of different ways for mustache.java to get data > from names fields and methods. One thing I have noticed running benchmarks > is that accessing the fields/methods in different ways can cause > invokedynamic to either never be optimized or actually be deoptimized later > during the running of the benchmarks (20x+ difference in performance). Here > is the code I have been running: > > https://github.com/spullara/indybenchmark > > I was wondering what the best way to debug this would be? > > Thanks, > Sam > From spullara+hotspot at gmail.com Wed Jan 14 18:57:36 2015 From: spullara+hotspot at gmail.com (Sam Pullara) Date: Wed, 14 Jan 2015 10:57:36 -0800 Subject: Invokedynamic deoptimization issues In-Reply-To: <54B6B804.5020607@oracle.com> References: <54B6B804.5020607@oracle.com> Message-ID: I'm using invokedynamic bytecode: https://github.com/spullara/mustache.java/blob/master/indy/src/main/java/com/github/mustachejava/indy/IndyWrapper.java If you run the benchmark with "-direct -indy -codegenReflectionOH -n 10" you will see interesting behavior. It is able to optimize indy and to be as fast as the direct call but around the 7th iteration it optimizes it again and makes it 10x slower. When I use your flags, I see that there is little difference in the optimization output however: samair11:indybenchmark sam$ diff indy1.txt indy2.txt 1c1,2 < 2408 396 % 4 indybench.Main::timeIndy @ 16 (73 bytes) --- > 6482 412 4 indybench.Main::timeIndy (73 bytes) > @ 0 java.lang.System::currentTimeMillis (0 bytes) (intrinsic) 4c5 < \-> TypeProfile (604309/604309 counts) = com/github/mustachejava/indy/W_Main_someMethod_a36851e6_61ca_4b15_b94e_7c73cdc71d1c --- > \-> TypeProfile (4204749/4204749 counts) = com/github/mustachejava/indy/W_Main_someMethod_a36851e6_61ca_4b15_b94e_7c73cdc71d1c 8,9c9,10 < \-> TypeProfile (6124/56502 counts) = com/github/mustachejava/codegen/CompiledGuards1 < \-> TypeProfile (50378/56502 counts) = com/github/mustachejava/codegen/CompiledGuards3 --- > \-> TypeProfile (6124/100293 counts) = com/github/mustachejava/codegen/CompiledGuards1 > \-> TypeProfile (94169/100293 counts) = com/github/mustachejava/codegen/CompiledGuards3 33a35 > @ 21 java.lang.Integer::intValue (5 bytes) accessor 55a58 > Sam Output: samair11:indybenchmark sam$ ./target/appassembler/bin/indybench -direct -indy -codegenReflectionOH -n 10 codegen reflection OH: 736 indy wrapper: 66 direct: 16 ----------------- codegen reflection OH: 1136 indy wrapper: 40 direct: 25 ----------------- codegen reflection OH: 577 indy wrapper: 4 direct: 5 ----------------- codegen reflection OH: 658 indy wrapper: 4 direct: 6 ----------------- codegen reflection OH: 567 indy wrapper: 3 direct: 6 ----------------- codegen reflection OH: 658 indy wrapper: 4 direct: 6 ----------------- codegen reflection OH: 625 indy wrapper: 5 direct: 6 ----------------- codegen reflection OH: 648 indy wrapper: 121 direct: 6 ----------------- codegen reflection OH: 618 indy wrapper: 65 direct: 7 ----------------- codegen reflection OH: 553 indy wrapper: 68 direct: 6 ----------------- On Wed, Jan 14, 2015 at 10:40 AM, Vladimir Ivanov < vladimir.x.ivanov at oracle.com> wrote: > Sam, > > The first thing I usually do is run with: > -XX:+PrintCompilation -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining > > I haven't looked at your benchmarks yet, but my main suspicion is that > MethodHandles aren't compile-time constants, so inlining fails. Do you use > invokedynamic bytecode or call MethodHandle.invokeExact()/invoke()? In > the latter case, you should store MethodHandles in static final fields. > Otherwise, JIT won't inline them. > > Best regards, > Vladimir Ivanov > > > On 1/14/15 8:23 PM, Sam Pullara wrote: > >> Hi, >> >> I've implemented a bunch of different ways for mustache.java to get data >> from names fields and methods. One thing I have noticed running benchmarks >> is that accessing the fields/methods in different ways can cause >> invokedynamic to either never be optimized or actually be deoptimized >> later >> during the running of the benchmarks (20x+ difference in performance). >> Here >> is the code I have been running: >> >> https://github.com/spullara/indybenchmark >> >> I was wondering what the best way to debug this would be? >> >> Thanks, >> Sam >> >> From vladimir.x.ivanov at oracle.com Wed Jan 14 19:28:32 2015 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 14 Jan 2015 22:28:32 +0300 Subject: Invokedynamic deoptimization issues In-Reply-To: References: <54B6B804.5020607@oracle.com> Message-ID: <54B6C360.9090609@oracle.com> Sam, I assume the scores are "lower/better". I tried to run your benchmark and here's what I see. On a sufficiently long run (-n >100) I observe that scores are equivalent: indy wrapper: 7 direct: 8 But there's a spike in indy score which is caused by method invalidation and concequent recompilation: 867 88 b indybench.Main::timeIndy (73 bytes) indy wrapper: 7 3276 88 indybench.Main::timeIndy (73 bytes) made not entrant indy wrapper: 114 3969 91 b indybench.Main::timeIndy (73 bytes) indy wrapper: 7 The cause of recompilation is a pruned branch in Integer.valueOf() during first compilation [1]. Can you confirm my observations that the score settles in a sufficiently long run? If not, please, tell JDK version you are using (java -version). I'd also be grateful for -XX:+LogCompilation logs for 2 separate runs (-indy & -direct, -n 1000). Best regards, Vladimir Ivanov [1] excerpt from "-XX:+UnlockDiagnosticVMOptions -XX:+LogCompilation -XX:LogFile=hotspot.log": On 1/14/15 9:57 PM, Sam Pullara wrote: > I'm using invokedynamic bytecode: > > https://github.com/spullara/mustache.java/blob/master/indy/src/main/java/com/github/mustachejava/indy/IndyWrapper.java > > If you run the benchmark with "-direct -indy -codegenReflectionOH -n 10" > you will see interesting behavior. It is able to optimize indy and to be > as fast as the direct call but around the 7th iteration it optimizes it > again and makes it 10x slower. When I use your flags, I see that there > is little difference in the optimization output however: > > samair11:indybenchmark sam$ diff indy1.txt indy2.txt > > 1c1,2 > > < 2408 396 % 4 indybench.Main::timeIndy @ 16 (73 bytes) > > --- > > > 6482 412 4 indybench.Main::timeIndy (73 bytes) > > > @ 0 > java.lang.System::currentTimeMillis (0 bytes) (intrinsic) > > 4c5 > > < \-> TypeProfile (604309/604309 counts) > = > com/github/mustachejava/indy/W_Main_someMethod_a36851e6_61ca_4b15_b94e_7c73cdc71d1c > > --- > > > \-> TypeProfile (4204749/4204749 > counts) = > com/github/mustachejava/indy/W_Main_someMethod_a36851e6_61ca_4b15_b94e_7c73cdc71d1c > > 8,9c9,10 > > < \-> TypeProfile (6124/56502 counts) > = com/github/mustachejava/codegen/CompiledGuards1 > > < \-> TypeProfile (50378/56502 > counts) = com/github/mustachejava/codegen/CompiledGuards3 > > --- > > > \-> TypeProfile (6124/100293 > counts) = com/github/mustachejava/codegen/CompiledGuards1 > > > \-> TypeProfile (94169/100293 > counts) = com/github/mustachejava/codegen/CompiledGuards3 > > 33a35 > > > @ 21 java.lang.Integer::intValue (5 > bytes) accessor > > 55a58 > > > > > > Sam > > Output: > > samair11:indybenchmark sam$ ./target/appassembler/bin/indybench -direct > -indy -codegenReflectionOH -n 10 > > codegen reflection OH: 736 > > indy wrapper: 66 > > direct: 16 > > ----------------- > > codegen reflection OH: 1136 > > indy wrapper: 40 > > direct: 25 > > ----------------- > > codegen reflection OH: 577 > > indy wrapper: 4 > > direct: 5 > > ----------------- > > codegen reflection OH: 658 > > indy wrapper: 4 > > direct: 6 > > ----------------- > > codegen reflection OH: 567 > > indy wrapper: 3 > > direct: 6 > > ----------------- > > codegen reflection OH: 658 > > indy wrapper: 4 > > direct: 6 > > ----------------- > > codegen reflection OH: 625 > > indy wrapper: 5 > > direct: 6 > > ----------------- > > codegen reflection OH: 648 > > indy wrapper: 121 > > direct: 6 > > ----------------- > > codegen reflection OH: 618 > > indy wrapper: 65 > > direct: 7 > > ----------------- > > codegen reflection OH: 553 > > indy wrapper: 68 > > direct: 6 > > ----------------- > > > > On Wed, Jan 14, 2015 at 10:40 AM, Vladimir Ivanov > > wrote: > > Sam, > > The first thing I usually do is run with: > -XX:+PrintCompilation -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining > > I haven't looked at your benchmarks yet, but my main suspicion is > that MethodHandles aren't compile-time constants, so inlining fails. > Do you use invokedynamic bytecode or call > MethodHandle.invokeExact()/__invoke()? In the latter case, you > should store MethodHandles in static final fields. Otherwise, JIT > won't inline them. > > Best regards, > Vladimir Ivanov > > > On 1/14/15 8:23 PM, Sam Pullara wrote: > > Hi, > > I've implemented a bunch of different ways for mustache.java to > get data > from names fields and methods. One thing I have noticed running > benchmarks > is that accessing the fields/methods in different ways can cause > invokedynamic to either never be optimized or actually be > deoptimized later > during the running of the benchmarks (20x+ difference in > performance). Here > is the code I have been running: > > https://github.com/spullara/__indybenchmark > > > I was wondering what the best way to debug this would be? > > Thanks, > Sam > > From david.dehaven at oracle.com Wed Jan 14 20:23:50 2015 From: david.dehaven at oracle.com (David DeHaven) Date: Wed, 14 Jan 2015 12:23:50 -0800 Subject: [8u60] RFR: 8043340: [macosx] Fix hard-wired paths to JavaVM.framework In-Reply-To: <54B629D8.8070700@oracle.com> References: <39719ED3-1F4E-4E6C-AA9F-A3C4766537B4@oracle.com> <54B3A38C.10909@oracle.com> <8B01B84D-5B6F-478E-ABF1-33F5ADE2F826@oracle.com> <08C63261-D594-404E-892B-BDE2898B505A@oracle.com> <4DC9B291-D302-428C-A5F7-177066C65991@oracle.com> <54B629D8.8070700@oracle.com> Message-ID: Can someone from hotspot-dev please look at the hotspot changes? -DrD- > Hello, > > This looks good to me. Thanks for the detailed table! > > /Erik > > On 2015-01-14 04:09, David DeHaven wrote: >>>> The --with-xcode-path argument is optional, you should also be able to build with Xcode 4 selected via "sudo xcode-select -switch /path/to/Xcode4.app". I leave MAS managed Xcode (currently 6) active as I'm constantly bouncing between projects and it's a hassle to have to remember to reset the active toolchain, so I thought it best to allow configure to be provided a path. >>> Ugh. I broke something along the way, this doesn't *quite* work. >>> >>> xcrun complains with "xcrun: error: missing DEVELOPER_DIR path:" >>> >>> I think I'm exporting an empty DEVELOPER_DIR. I shall fix and update. >> TL;DR: Please review round 2: >> http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v1/ >> >> (I removed generated-configure.sh to reduce the review size, it will be re-generated prior to pushing) >> >> >> I've tested the following configuration scenarios (output from a shell script I cobbled together..) >> >> field values: >> XC6 - Xcode 6 installed in /Applications/Xcode.app >> XC4 - Xcode 4 installed in some other dir >> (empty) - Argument not passed to configure >> >> Result meanings: >> DEV_DIR set - configure succeeded, DEVELOPER_DIR was properly exported in spec.gmk >> DEV_DIR NOT set - configure succeeded, DEVELOPER_DIR was not needed (XC4 must be selected to achieve this) >> Configure failed - Configure properly failed when it detected Xcode > 4 >> >> "Selected" Xcode means version reported by xcode-select -p >> >> >> | Xcode selected | --with-xcode-path | DEVELOPER_DIR | result | >> ----------------------------------------------------------------------------- >> | XC4 | | | DEV_DIR NOT set | >> ----------------------------------------------------------------------------- >> | XC4 | | XC4 | DEV_DIR set | >> ----------------------------------------------------------------------------- >> | XC4 | | XC6 | Configure failed | >> ----------------------------------------------------------------------------- >> | XC4 | XC4 | | DEV_DIR set | >> ----------------------------------------------------------------------------- >> | XC4 | XC4 | XC4 | DEV_DIR set | >> ----------------------------------------------------------------------------- >> | XC4 | XC4 | XC6 | DEV_DIR set | >> ----------------------------------------------------------------------------- >> | XC4 | XC6 | | Configure failed | >> ----------------------------------------------------------------------------- >> | XC4 | XC6 | XC4 | Configure failed | >> ----------------------------------------------------------------------------- >> | XC4 | XC6 | XC6 | Configure failed | >> ----------------------------------------------------------------------------- >> | XC6 | | | Configure failed | >> ----------------------------------------------------------------------------- >> | XC6 | | XC4 | DEV_DIR set | >> ----------------------------------------------------------------------------- >> | XC6 | | XC6 | Configure failed | >> ----------------------------------------------------------------------------- >> | XC6 | XC4 | | DEV_DIR set | >> ----------------------------------------------------------------------------- >> | XC6 | XC4 | XC4 | DEV_DIR set | >> ----------------------------------------------------------------------------- >> | XC6 | XC4 | XC6 | DEV_DIR set | >> ----------------------------------------------------------------------------- >> | XC6 | XC6 | | Configure failed | >> ----------------------------------------------------------------------------- >> | XC6 | XC6 | XC4 | Configure failed | >> ----------------------------------------------------------------------------- >> | XC6 | XC6 | XC6 | Configure failed | >> ----------------------------------------------------------------------------- >> >> All the results are as expected. Please note that --with-xcode-path overrides DEVELOPER_DIR, since that could be set in the environment. >> >> (yeah, I went a little OCD on this...) >> >> -DrD- >> > From dean.long at oracle.com Wed Jan 14 22:02:42 2015 From: dean.long at oracle.com (Dean Long) Date: Wed, 14 Jan 2015 14:02:42 -0800 Subject: RFR: AARCH64: Top-level JDK changes In-Reply-To: <54B66EB3.3020601@oracle.com> References: <545CFFA9.4070107@redhat.com> <545D0290.5080307@oracle.com> <54B34E20.5030506@oracle.com> <54B3B4CE.8060804@oracle.com> <54B4D82B.3050608@oracle.com> <54B66EB3.3020601@oracle.com> Message-ID: <54B6E782.3040405@oracle.com> On 1/14/2015 5:27 AM, Magnus Ihse Bursie wrote: > On 2015-01-13 09:32, Dean Long wrote: >> On 1/12/2015 3:49 AM, Magnus Ihse Bursie wrote: >>> On 2015-01-12 05:31, Dean Long wrote: >>>> I found a small problem with the new config.sub wrapper. It works >>>> with the bash shell but not with the dash shell. >>>> The problem seems to be with this line: >>>> >>>> result=`. $DIR/autoconf-config.sub $sub_args "$@"` >>>> >>>> "dash" doesn't seem to support args passed with ".", so $sub_args >>>> "$@" are ignored. >>> >>> bash is the required shell for running configure. We do not support >>> non-bash shells. In fact, we go to lengths to try to ensure that we >>> are indeed running under bash. >>> >>> /Magnus >> I was thinking 'bash configure' was enough, but it turns out >> 'CONFIG_SHELL=bash bash configure' gives better results. > > Hm, that's interesting. We were attempting to automatically use bash > in the real configure script, regardless of what shell the user had to > start the top-level configure wrapper. > > If you try the patch below, does it work better when you run "dash > configure"? > > diff --git a/common/autoconf/configure b/common/autoconf/configure > --- a/common/autoconf/configure > +++ b/common/autoconf/configure > @@ -36,6 +36,13 @@ > shift > fi > > +if test "x$BASH" = x; then > + echo "Error: This script must be run using bash." 1>&2 > + exit 1 > +fi > +# Force autoconf to use bash > +export CONFIG_SHELL=$BASH > + > conf_script_dir="$TOPDIR/common/autoconf" > > if [ "$CUSTOM_CONFIG_DIR" = "" ]; then > > /Magnus > Yes, that patch solves the problem. dl From dean.long at oracle.com Wed Jan 14 23:23:24 2015 From: dean.long at oracle.com (Dean Long) Date: Wed, 14 Jan 2015 15:23:24 -0800 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile Message-ID: <54B6FA6C.40202@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8031064 http://cr.openjdk.java.net/~dlong/8031064/webrev/ This change allows the build_vm_def.sh script to use the $NM chosen by configure rather than "nm", which makes things better if you're cross-compiling. dl From dean.long at oracle.com Wed Jan 14 23:34:15 2015 From: dean.long at oracle.com (Dean Long) Date: Wed, 14 Jan 2015 15:34:15 -0800 Subject: AARCH64: RFR (XS) 8068927: AARCH64: better handling of aarch64- triples (was RFR: AARCH64: Top-level JDK changes) In-Reply-To: <54B56A55.3020803@oracle.com> References: <545CFFA9.4070107@redhat.com> <545D0290.5080307@oracle.com> <54B34E20.5030506@oracle.com> <54B3A108.9070203@redhat.com> <54B4DAD4.5060904@oracle.com> <54B4E083.9040502@redhat.com> <54B56A55.3020803@oracle.com> Message-ID: <54B6FCF7.1010202@oracle.com> Can I get a review for this? https://bugs.openjdk.java.net/browse/JDK-8068927 http://cr.openjdk.java.net/~dlong/8068927/webrev/ thanks, dl On 1/13/2015 10:56 AM, Dean Long wrote: > On 1/13/2015 1:08 AM, Andrew Haley wrote: >> On 13/01/15 08:44, Dean Long wrote: >> >>> I came up with a simpler version, where I replace "aarch64-" with >>> "arm-", run autoconf-config.sub, then replace "arm-" back to >>> "aarch64-". >> Thanks. That sounds good to me. >> >> Andrew. > > Here's the patch. If it looks good, I can file a bug and push it to > the staging repo. > > dl > > > diff -r b052cb38b985 common/autoconf/build-aux/config.sub > --- a/common/autoconf/build-aux/config.sub Thu Dec 11 15:05:06 > 2014 -0800 > +++ b/common/autoconf/build-aux/config.sub Tue Jan 13 13:57:23 > 2015 -0500 > @@ -41,25 +41,8 @@ > case $1 in > -- ) # Stop option processing > shift; break ;; > - aarch64-gnu ) > - sub_args="$sub_args aarch64-unknown-gnu" > - shift; ;; > - aarch64-linux ) > - sub_args="$sub_args aarch64-unknown-linux-gnu" > - shift; ;; > - aarch64-*-linux ) > - os=`echo $1 | sed 's/aarch64-\(.*\)-linux/\1/'` > - config="aarch64-unknown-linux-gnu" > - sub_args="$sub_args $config" > - shift; ;; > - aarch64-*-gnu ) > - os=`echo $1 | sed 's/aarch64-\(.*\)-gnu.*$/\1/'` > - config="aarch64-unknown-gnu" > - sub_args="$sub_args $config" > - shift; ;; > - aarch64-*-linux-* ) > - os=`echo $1 | sed 's/aarch64-\(.*\)-linux-.*$/'` > - config="aarch64-unknown-linux-gnu" > + aarch64-* ) > + config=`echo $1 | sed 's/^aarch64-/arm-/'` > sub_args="$sub_args $config" > shift; ;; > - ) # Use stdin as input. > @@ -74,9 +57,7 @@ > result=`. $DIR/autoconf-config.sub $sub_args "$@"` > exitcode=$? > > -if [ "x$os" != "x" ] ; then > - result=`echo $result | sed "s/-unknown-/-$os-/"` > -fi > +result=`echo $result | sed "s/^arm-/aarch64-/"` > > echo $result > exit $exitcode > From spullara+hotspot at gmail.com Thu Jan 15 01:08:57 2015 From: spullara+hotspot at gmail.com (Sam Pullara) Date: Wed, 14 Jan 2015 17:08:57 -0800 Subject: Invokedynamic deoptimization issues In-Reply-To: <54B6C360.9090609@oracle.com> References: <54B6B804.5020607@oracle.com> <54B6C360.9090609@oracle.com> Message-ID: That one does indeed finally converge on an optimized version. If you add in another benchmark during the same run, it will optimize it, then deoptimize it and never return: ./target/appassembler/bin/indybench -reflectionOH -codegenReflectionOH -indy -indyOH -direct -n 100 reflection OH: 1200 codegen reflection OH: 1131 *indy wrapper: 58* indy OH: 1140 direct: 28 ----------------- reflection OH: 1818 codegen reflection OH: 1822 *indy wrapper: 1274* indy OH: 1169 direct: 24 ----------------- ....97 more... reflection OH: 1175 codegen reflection OH: 1679 *indy wrapper: 1010* indy OH: 990 direct: 7 ----------------- I feel like it is almost too random to depend on the optimization happening ? unless I can assume that this is due to some pathology in my test that doesn't come up in real life. Sam On Wed, Jan 14, 2015 at 11:28 AM, Vladimir Ivanov < vladimir.x.ivanov at oracle.com> wrote: > Sam, > > I assume the scores are "lower/better". > > I tried to run your benchmark and here's what I see. > > On a sufficiently long run (-n >100) I observe that scores are equivalent: > indy wrapper: 7 > direct: 8 > > But there's a spike in indy score which is caused by method invalidation > and concequent recompilation: > 867 88 b indybench.Main::timeIndy (73 bytes) > > indy wrapper: 7 > > 3276 88 indybench.Main::timeIndy (73 bytes) made not > entrant > > indy wrapper: 114 > > 3969 91 b indybench.Main::timeIndy (73 bytes) > > indy wrapper: 7 > > The cause of recompilation is a pruned branch in Integer.valueOf() during > first compilation [1]. > > Can you confirm my observations that the score settles in a sufficiently > long run? > > If not, please, tell JDK version you are using (java -version). I'd also > be grateful for -XX:+LogCompilation logs for 2 separate runs (-indy & > -direct, -n 1000). > > > Best regards, > Vladimir Ivanov > > [1] excerpt from "-XX:+UnlockDiagnosticVMOptions -XX:+LogCompilation > -XX:LogFile=hotspot.log": > > compile_id='88' compiler='C2' count='4' stamp='3.276'> > bytes='32' count='2048' backedge_count='1' iicount='16140' decompiles='1' > unstable_if_traps='4' overflow_recompiles='1'/> > > count='2048' iicount='16140'/> > bytes='55' count='2048' iicount='16140'/> > bytes='15' count='1' backedge_count='1' iicount='10000'/> > backedge_count='1' iicount='16141' decompiles='1'/> > bytes='73' count='5000' backedge_count='2049' iicount='3' decompiles='3' > unstable_if_traps='1'/> > > > > On 1/14/15 9:57 PM, Sam Pullara wrote: > >> I'm using invokedynamic bytecode: >> >> https://github.com/spullara/mustache.java/blob/master/ >> indy/src/main/java/com/github/mustachejava/indy/IndyWrapper.java >> >> If you run the benchmark with "-direct -indy -codegenReflectionOH -n 10" >> you will see interesting behavior. It is able to optimize indy and to be >> as fast as the direct call but around the 7th iteration it optimizes it >> again and makes it 10x slower. When I use your flags, I see that there >> is little difference in the optimization output however: >> >> samair11:indybenchmark sam$ diff indy1.txt indy2.txt >> >> 1c1,2 >> >> < 2408 396 % 4 indybench.Main::timeIndy @ 16 (73 bytes) >> >> --- >> >> > 6482 412 4 indybench.Main::timeIndy (73 bytes) >> >> > @ 0 >> java.lang.System::currentTimeMillis (0 bytes) (intrinsic) >> >> 4c5 >> >> < \-> TypeProfile (604309/604309 counts) >> = >> com/github/mustachejava/indy/W_Main_someMethod_a36851e6_ >> 61ca_4b15_b94e_7c73cdc71d1c >> >> --- >> >> > \-> TypeProfile (4204749/4204749 >> counts) = >> com/github/mustachejava/indy/W_Main_someMethod_a36851e6_ >> 61ca_4b15_b94e_7c73cdc71d1c >> >> 8,9c9,10 >> >> < \-> TypeProfile (6124/56502 counts) >> = com/github/mustachejava/codegen/CompiledGuards1 >> >> < \-> TypeProfile (50378/56502 >> counts) = com/github/mustachejava/codegen/CompiledGuards3 >> >> --- >> >> > \-> TypeProfile (6124/100293 >> counts) = com/github/mustachejava/codegen/CompiledGuards1 >> >> > \-> TypeProfile (94169/100293 >> counts) = com/github/mustachejava/codegen/CompiledGuards3 >> >> 33a35 >> >> > @ 21 java.lang.Integer::intValue (5 >> bytes) accessor >> >> 55a58 >> >> > >> >> >> Sam >> >> Output: >> >> samair11:indybenchmark sam$ ./target/appassembler/bin/indybench -direct >> -indy -codegenReflectionOH -n 10 >> >> codegen reflection OH: 736 >> >> indy wrapper: 66 >> >> direct: 16 >> >> ----------------- >> >> codegen reflection OH: 1136 >> >> indy wrapper: 40 >> >> direct: 25 >> >> ----------------- >> >> codegen reflection OH: 577 >> >> indy wrapper: 4 >> >> direct: 5 >> >> ----------------- >> >> codegen reflection OH: 658 >> >> indy wrapper: 4 >> >> direct: 6 >> >> ----------------- >> >> codegen reflection OH: 567 >> >> indy wrapper: 3 >> >> direct: 6 >> >> ----------------- >> >> codegen reflection OH: 658 >> >> indy wrapper: 4 >> >> direct: 6 >> >> ----------------- >> >> codegen reflection OH: 625 >> >> indy wrapper: 5 >> >> direct: 6 >> >> ----------------- >> >> codegen reflection OH: 648 >> >> indy wrapper: 121 >> >> direct: 6 >> >> ----------------- >> >> codegen reflection OH: 618 >> >> indy wrapper: 65 >> >> direct: 7 >> >> ----------------- >> >> codegen reflection OH: 553 >> >> indy wrapper: 68 >> >> direct: 6 >> >> ----------------- >> >> >> >> On Wed, Jan 14, 2015 at 10:40 AM, Vladimir Ivanov >> > >> wrote: >> >> Sam, >> >> The first thing I usually do is run with: >> -XX:+PrintCompilation -XX:+UnlockDiagnosticVMOptions >> -XX:+PrintInlining >> >> I haven't looked at your benchmarks yet, but my main suspicion is >> that MethodHandles aren't compile-time constants, so inlining fails. >> Do you use invokedynamic bytecode or call >> MethodHandle.invokeExact()/__invoke()? In the latter case, you >> should store MethodHandles in static final fields. Otherwise, JIT >> won't inline them. >> >> Best regards, >> Vladimir Ivanov >> >> >> On 1/14/15 8:23 PM, Sam Pullara wrote: >> >> Hi, >> >> I've implemented a bunch of different ways for mustache.java to >> get data >> from names fields and methods. One thing I have noticed running >> benchmarks >> is that accessing the fields/methods in different ways can cause >> invokedynamic to either never be optimized or actually be >> deoptimized later >> during the running of the benchmarks (20x+ difference in >> performance). Here >> is the code I have been running: >> >> https://github.com/spullara/__indybenchmark >> >> >> I was wondering what the best way to debug this would be? >> >> Thanks, >> Sam >> >> >> From david.holmes at oracle.com Thu Jan 15 05:58:25 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Jan 2015 15:58:25 +1000 Subject: [8u60] RFR: 8043340: [macosx] Fix hard-wired paths to JavaVM.framework In-Reply-To: References: <39719ED3-1F4E-4E6C-AA9F-A3C4766537B4@oracle.com> <54B3A38C.10909@oracle.com> <8B01B84D-5B6F-478E-ABF1-33F5ADE2F826@oracle.com> <08C63261-D594-404E-892B-BDE2898B505A@oracle.com> <4DC9B291-D302-428C-A5F7-177066C65991@oracle.com> <54B629D8.8070700@oracle.com> Message-ID: <54B75701.1080507@oracle.com> On 15/01/2015 6:23 AM, David DeHaven wrote: > > Can someone from hotspot-dev please look at the hotspot changes? Looks okay to me. Thanks, David H. > -DrD- > >> Hello, >> >> This looks good to me. Thanks for the detailed table! >> >> /Erik >> >> On 2015-01-14 04:09, David DeHaven wrote: >>>>> The --with-xcode-path argument is optional, you should also be able to build with Xcode 4 selected via "sudo xcode-select -switch /path/to/Xcode4.app". I leave MAS managed Xcode (currently 6) active as I'm constantly bouncing between projects and it's a hassle to have to remember to reset the active toolchain, so I thought it best to allow configure to be provided a path. >>>> Ugh. I broke something along the way, this doesn't *quite* work. >>>> >>>> xcrun complains with "xcrun: error: missing DEVELOPER_DIR path:" >>>> >>>> I think I'm exporting an empty DEVELOPER_DIR. I shall fix and update. >>> TL;DR: Please review round 2: >>> http://cr.openjdk.java.net/~ddehaven/8043340/jdk8u/v1/ >>> >>> (I removed generated-configure.sh to reduce the review size, it will be re-generated prior to pushing) >>> >>> >>> I've tested the following configuration scenarios (output from a shell script I cobbled together..) >>> >>> field values: >>> XC6 - Xcode 6 installed in /Applications/Xcode.app >>> XC4 - Xcode 4 installed in some other dir >>> (empty) - Argument not passed to configure >>> >>> Result meanings: >>> DEV_DIR set - configure succeeded, DEVELOPER_DIR was properly exported in spec.gmk >>> DEV_DIR NOT set - configure succeeded, DEVELOPER_DIR was not needed (XC4 must be selected to achieve this) >>> Configure failed - Configure properly failed when it detected Xcode > 4 >>> >>> "Selected" Xcode means version reported by xcode-select -p >>> >>> >>> | Xcode selected | --with-xcode-path | DEVELOPER_DIR | result | >>> ----------------------------------------------------------------------------- >>> | XC4 | | | DEV_DIR NOT set | >>> ----------------------------------------------------------------------------- >>> | XC4 | | XC4 | DEV_DIR set | >>> ----------------------------------------------------------------------------- >>> | XC4 | | XC6 | Configure failed | >>> ----------------------------------------------------------------------------- >>> | XC4 | XC4 | | DEV_DIR set | >>> ----------------------------------------------------------------------------- >>> | XC4 | XC4 | XC4 | DEV_DIR set | >>> ----------------------------------------------------------------------------- >>> | XC4 | XC4 | XC6 | DEV_DIR set | >>> ----------------------------------------------------------------------------- >>> | XC4 | XC6 | | Configure failed | >>> ----------------------------------------------------------------------------- >>> | XC4 | XC6 | XC4 | Configure failed | >>> ----------------------------------------------------------------------------- >>> | XC4 | XC6 | XC6 | Configure failed | >>> ----------------------------------------------------------------------------- >>> | XC6 | | | Configure failed | >>> ----------------------------------------------------------------------------- >>> | XC6 | | XC4 | DEV_DIR set | >>> ----------------------------------------------------------------------------- >>> | XC6 | | XC6 | Configure failed | >>> ----------------------------------------------------------------------------- >>> | XC6 | XC4 | | DEV_DIR set | >>> ----------------------------------------------------------------------------- >>> | XC6 | XC4 | XC4 | DEV_DIR set | >>> ----------------------------------------------------------------------------- >>> | XC6 | XC4 | XC6 | DEV_DIR set | >>> ----------------------------------------------------------------------------- >>> | XC6 | XC6 | | Configure failed | >>> ----------------------------------------------------------------------------- >>> | XC6 | XC6 | XC4 | Configure failed | >>> ----------------------------------------------------------------------------- >>> | XC6 | XC6 | XC6 | Configure failed | >>> ----------------------------------------------------------------------------- >>> >>> All the results are as expected. Please note that --with-xcode-path overrides DEVELOPER_DIR, since that could be set in the environment. >>> >>> (yeah, I went a little OCD on this...) >>> >>> -DrD- >>> >> > From david.holmes at oracle.com Thu Jan 15 06:31:17 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Jan 2015 16:31:17 +1000 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54B6FA6C.40202@oracle.com> References: <54B6FA6C.40202@oracle.com> Message-ID: <54B75EB5.3020407@oracle.com> Hi Dean, Code reviews don't go to jdk9-dev. Build-infra is not relevant to this either. You only need hotspot-dev for a hotspot build issue (though build-dev might be useful for more complex changes). On 15/01/2015 9:23 AM, Dean Long wrote: > https://bugs.openjdk.java.net/browse/JDK-8031064 > http://cr.openjdk.java.net/~dlong/8031064/webrev/ > > This change allows the build_vm_def.sh script to use the $NM chosen by > configure rather than "nm", > which makes things better if you're cross-compiling. I'm always wary of things that might be shell specific: vm.def: $(Res_Files) $(Obj_Files) ! NM=$(NM) sh $(GAMMADIR)/make/linux/makefiles/build_vm_def.sh *.o > $@ I use this style of shell script invocation a lot from bash, where it works, but I was under the impression not all shells support it. David > dl From dean.long at oracle.com Thu Jan 15 06:40:27 2015 From: dean.long at oracle.com (Dean Long) Date: Wed, 14 Jan 2015 22:40:27 -0800 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54B75EB5.3020407@oracle.com> References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> Message-ID: <54B760DB.70200@oracle.com> On 1/14/2015 10:31 PM, David Holmes wrote: > Hi Dean, > > Code reviews don't go to jdk9-dev. Build-infra is not relevant to this > either. You only need hotspot-dev for a hotspot build issue (though > build-dev might be useful for more complex changes). > OK. > On 15/01/2015 9:23 AM, Dean Long wrote: >> https://bugs.openjdk.java.net/browse/JDK-8031064 >> http://cr.openjdk.java.net/~dlong/8031064/webrev/ >> >> This change allows the build_vm_def.sh script to use the $NM chosen by >> configure rather than "nm", >> which makes things better if you're cross-compiling. > > I'm always wary of things that might be shell specific: > > vm.def: $(Res_Files) $(Obj_Files) > ! NM=$(NM) sh $(GAMMADIR)/make/linux/makefiles/build_vm_def.sh > *.o > $@ > > I use this style of shell script invocation a lot from bash, where it > works, but I was under the impression not all shells support it. > I thought we require bash, but maybe that's just for confignure. The above should work on all Bourne shell variants, but not csh variants. Any time we use something like '2> /dev/null', it's Bourne-shell specific, so I don't think our linux makefiles will work with csh. However, I can change it to any of the following: 1. env NM=$(NM) sh ... 2. NM=$(NM); export NM; sh ... 3. export NM vm.def: $(Res_Files) $(Obj_Files) sh ... I have another version that moves the awk code from build_vm_def.sh up into vm.make, but I wanted to keep the changes to a minimum. dl > David > >> dl From david.holmes at oracle.com Thu Jan 15 07:01:48 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Jan 2015 17:01:48 +1000 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54B760DB.70200@oracle.com> References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> <54B760DB.70200@oracle.com> Message-ID: <54B765DC.8050801@oracle.com> On 15/01/2015 4:40 PM, Dean Long wrote: > On 1/14/2015 10:31 PM, David Holmes wrote: >> Hi Dean, >> >> Code reviews don't go to jdk9-dev. Build-infra is not relevant to this >> either. You only need hotspot-dev for a hotspot build issue (though >> build-dev might be useful for more complex changes). >> > OK. That was also a hint to drop them from follow ups - which I now have but added build-dev. :) >> On 15/01/2015 9:23 AM, Dean Long wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8031064 >>> http://cr.openjdk.java.net/~dlong/8031064/webrev/ >>> >>> This change allows the build_vm_def.sh script to use the $NM chosen by >>> configure rather than "nm", >>> which makes things better if you're cross-compiling. >> >> I'm always wary of things that might be shell specific: >> >> vm.def: $(Res_Files) $(Obj_Files) >> ! NM=$(NM) sh $(GAMMADIR)/make/linux/makefiles/build_vm_def.sh >> *.o > $@ >> >> I use this style of shell script invocation a lot from bash, where it >> works, but I was under the impression not all shells support it. >> > I thought we require bash, but maybe that's just for confignure. The > above should work on all Bourne shell variants, > but not csh variants. Any time we use something like '2> /dev/null', > it's Bourne-shell specific, so I don't think our > linux makefiles will work with csh. However, I can change it to any of > the following: If the build-dev guys confirm we already assume bash that is fine. BTW the aix and bsd versions of this file have the same flawed logic, but probably don't expect to ever be cross-compiling. > 1. env NM=$(NM) sh ... > 2. NM=$(NM); export NM; sh ... > 3. > export NM > vm.def: $(Res_Files) $(Obj_Files) > sh ... > > I have another version that moves the awk code from build_vm_def.sh up > into vm.make, but I wanted to keep the changes to a minimum. I must admit I have no idea why this nm invocation has been farmed out to a shell script - perhaps the $ escaping became problematic ?? Thanks, David > > dl > >> David >> >>> dl > From david.holmes at oracle.com Thu Jan 15 07:37:20 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Jan 2015 17:37:20 +1000 Subject: RFR: 8067368: TestConcMarkCycleWB.java crashed at G1CollectedHeap::heap()+0xb In-Reply-To: <54B68C7E.2010707@oracle.com> References: <54B68C7E.2010707@oracle.com> Message-ID: <54B76E30.9040600@oracle.com> Looks good to me. Thanks, David On 15/01/2015 1:34 AM, Stefan Johansson wrote: > Hi, > > Please review this small fix to add TestConcMarkCycleWB.java to the > needs_g1gc test group. This should fix the nightly problem described in: > https://bugs.openjdk.java.net/browse/JDK-8067368 > > Webrev: > http://cr.openjdk.java.net/~sjohanss/8067368/hotspot.00/ > > Summary: > The runs failing this test are executed with either the test-group > "embedded_jdk" or "embedded_jre". Both those exclude tests defined in > the group "needs_g1gc" and since TestConcMarkCycleWB.java is a G1 > specific WhiteBox test it should be included in the "needs_g1gc" group. > > Test: > Manually verified that a jtreg of execution for "embedded_jdk" or > "embedded_jre" after the patch is applied don't include > TestConcMarkCycleWB.java. > > Thanks, > Stefan From dean.long at oracle.com Thu Jan 15 08:05:51 2015 From: dean.long at oracle.com (Dean Long) Date: Thu, 15 Jan 2015 00:05:51 -0800 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54B765DC.8050801@oracle.com> References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> <54B760DB.70200@oracle.com> <54B765DC.8050801@oracle.com> Message-ID: <54B774DF.6090505@oracle.com> On 1/14/2015 11:01 PM, David Holmes wrote: > On 15/01/2015 4:40 PM, Dean Long wrote: >> On 1/14/2015 10:31 PM, David Holmes wrote: >>> Hi Dean, >>> >>> Code reviews don't go to jdk9-dev. Build-infra is not relevant to this >>> either. You only need hotspot-dev for a hotspot build issue (though >>> build-dev might be useful for more complex changes). >>> >> OK. > > That was also a hint to drop them from follow ups - which I now have > but added build-dev. :) > >>> On 15/01/2015 9:23 AM, Dean Long wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8031064 >>>> http://cr.openjdk.java.net/~dlong/8031064/webrev/ >>>> >>>> This change allows the build_vm_def.sh script to use the $NM chosen by >>>> configure rather than "nm", >>>> which makes things better if you're cross-compiling. >>> >>> I'm always wary of things that might be shell specific: >>> >>> vm.def: $(Res_Files) $(Obj_Files) >>> ! NM=$(NM) sh $(GAMMADIR)/make/linux/makefiles/build_vm_def.sh >>> *.o > $@ >>> >>> I use this style of shell script invocation a lot from bash, where it >>> works, but I was under the impression not all shells support it. >>> >> I thought we require bash, but maybe that's just for confignure. The >> above should work on all Bourne shell variants, >> but not csh variants. Any time we use something like '2> /dev/null', >> it's Bourne-shell specific, so I don't think our >> linux makefiles will work with csh. However, I can change it to any of >> the following: > > If the build-dev guys confirm we already assume bash that is fine. > > BTW the aix and bsd versions of this file have the same flawed logic, > but probably don't expect to ever be cross-compiling. > > >> 1. env NM=$(NM) sh ... >> 2. NM=$(NM); export NM; sh ... >> 3. >> export NM >> vm.def: $(Res_Files) $(Obj_Files) >> sh ... >> >> I have another version that moves the awk code from build_vm_def.sh up >> into vm.make, but I wanted to keep the changes to a minimum. > > I must admit I have no idea why this nm invocation has been farmed out > to a shell script - perhaps the $ escaping became problematic ?? > That would be my guess. A compromise solution would have been to do the "nm" in vm.make, and just do the "awk" part in build_vm_def.sh (or perhaps build_vm_def.awk). dl > Thanks, > David >> >> dl >> >>> David >>> >>>> dl >> From jaroslav.bachorik at oracle.com Thu Jan 15 08:55:44 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Thu, 15 Jan 2015 09:55:44 +0100 Subject: RFR 8068976: Remove JSDT implementation In-Reply-To: <54B677B5.3080809@oracle.com> References: <54B677B5.3080809@oracle.com> Message-ID: <54B78090.1060706@oracle.com> On 14.1.2015 15:05, Jaroslav Bachorik wrote: > Please, review the following removal of functionality. > > Issue : https://bugs.openjdk.java.net/browse/JDK-8068976 > Webrev: http://cr.openjdk.java.net/~jbachorik/8068976/webrev.00 > > JDK-8062303 [1] requests the removal of com.sun.tracing API. This API > was added as experimental in JDK7 [2][3] and haven't seen any > significant uptake since then. Just to clarify - demoting this API to private internal (thus unsupported) was done in [3] -JB- > > JSDT builds on top of the com.sun.tracing API and provides the bridge to > DTrace. It allows Java programs to fire custom DTrace probes. As an > implementation of com.sun.tracing API it needs to be removed before > proceeding with the removal of that API. > > This change is basically an anti-delta to [4] - minus the changes not > related to JSDT and/or DTrace. > > This change passes all the regression and JCK tests. > > Thanks, > > -JB- > > > [1] https://bugs.openjdk.java.net/browse/JDK-8062303 > [2] https://bugs.openjdk.java.net/browse/JDK-6537506 > [3] https://bugs.openjdk.java.net/browse/JDK-7037081 > [4] http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/018d5b58dd4f From stefan.johansson at oracle.com Thu Jan 15 12:00:54 2015 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 15 Jan 2015 13:00:54 +0100 Subject: RFR: 8067368: TestConcMarkCycleWB.java crashed at G1CollectedHeap::heap()+0xb In-Reply-To: <54B69816.40602@oracle.com> References: <54B68C7E.2010707@oracle.com> <54B69816.40602@oracle.com> Message-ID: <54B7ABF6.8080303@oracle.com> Thanks for reviewing Jesper, Stefan On 2015-01-14 17:23, Jesper Wilhelmsson wrote: > Looks good! > /Jesper > > Stefan Johansson skrev den 14/1/15 16:34: >> Hi, >> >> Please review this small fix to add TestConcMarkCycleWB.java to the >> needs_g1gc >> test group. This should fix the nightly problem described in: >> https://bugs.openjdk.java.net/browse/JDK-8067368 >> >> Webrev: >> http://cr.openjdk.java.net/~sjohanss/8067368/hotspot.00/ >> >> Summary: >> The runs failing this test are executed with either the test-group >> "embedded_jdk" or "embedded_jre". Both those exclude tests defined in >> the group >> "needs_g1gc" and since TestConcMarkCycleWB.java is a G1 specific >> WhiteBox test >> it should be included in the "needs_g1gc" group. >> >> Test: >> Manually verified that a jtreg of execution for "embedded_jdk" or >> "embedded_jre" >> after the patch is applied don't include TestConcMarkCycleWB.java. >> >> Thanks, >> Stefan From stefan.johansson at oracle.com Thu Jan 15 12:01:13 2015 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 15 Jan 2015 13:01:13 +0100 Subject: RFR: 8067368: TestConcMarkCycleWB.java crashed at G1CollectedHeap::heap()+0xb In-Reply-To: <54B76E30.9040600@oracle.com> References: <54B68C7E.2010707@oracle.com> <54B76E30.9040600@oracle.com> Message-ID: <54B7AC09.4070002@oracle.com> Thanks for the review David, Stefan On 2015-01-15 08:37, David Holmes wrote: > Looks good to me. > > Thanks, > David > > On 15/01/2015 1:34 AM, Stefan Johansson wrote: >> Hi, >> >> Please review this small fix to add TestConcMarkCycleWB.java to the >> needs_g1gc test group. This should fix the nightly problem described in: >> https://bugs.openjdk.java.net/browse/JDK-8067368 >> >> Webrev: >> http://cr.openjdk.java.net/~sjohanss/8067368/hotspot.00/ >> >> Summary: >> The runs failing this test are executed with either the test-group >> "embedded_jdk" or "embedded_jre". Both those exclude tests defined in >> the group "needs_g1gc" and since TestConcMarkCycleWB.java is a G1 >> specific WhiteBox test it should be included in the "needs_g1gc" group. >> >> Test: >> Manually verified that a jtreg of execution for "embedded_jdk" or >> "embedded_jre" after the patch is applied don't include >> TestConcMarkCycleWB.java. >> >> Thanks, >> Stefan From volker.simonis at gmail.com Thu Jan 15 13:39:27 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 15 Jan 2015 14:39:27 +0100 Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF692D0@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF6922E@DEWDFEMB12A.global.corp.sap> <54AA6151.3040603@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF692D0@DEWDFEMB12A.global.corp.sap> Message-ID: Hi Goetz, I think the change is looking good now. Jesper, could you please be so kind to push this change for us? Thank you and best regards, Volker On Mon, Jan 5, 2015 at 12:48 PM, Lindenmaier, Goetz wrote: > Hi, > > I fixed it as proposed. > http://cr.openjdk.java.net/~goetz/webrevs/8068026-jtregClient/webrev.01/ > > I also assured the right exception is thrown, if I change the check for > the exit value to check for '1' (so that I get a test error), I now get: > java.lang.RuntimeException: Expected to get exit value of [1] > while before, it would have said the wrong message: > java.lang.RuntimeException: 'Unrecognized option: -client' missing from stdout/stderr > > Best regards, > Goetz. > > > -----Original Message----- > From: Jesper Wilhelmsson [mailto:jesper.wilhelmsson at oracle.com] > Sent: Montag, 5. Januar 2015 11:03 > To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' > Subject: Re: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java > > Hi, > > It seems your change introduces dead code: > > + if (output.shouldContain("Unrecognized option: -client") == null) { > + throw e; > + } > > If the string is found shouldContain will return a reference to 'output' (!= > null), and if the string is not found it will throw an exception, so the > introduced throw will never happen, right? > > This means the original exception 'e' is lost and a different one is thrown if > -client was accepted by the VM. > > /Jesper > > > Lindenmaier, Goetz skrev den 5/1/15 10:38: >> Just a little reminder ... this is really small! >> >> From: Lindenmaier, Goetz >> Sent: Montag, 22. Dezember 2014 14:08 >> To: 'hotspot-dev at openjdk.java.net' >> Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java >> >> Hi, >> >> I please need a review and a sponsor for this tiny change: >> >> http://cr.openjdk.java.net/~goetz/webrevs/8068026-jtregClient/webrev.01/ >> The test did not check whether -client is supported. >> >> The patch should also go to 8u60, please. >> >> Best regards, >> Goetz >> From magnus.ihse.bursie at oracle.com Thu Jan 15 13:41:46 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 15 Jan 2015 14:41:46 +0100 Subject: [aarch64-port-dev ] AARCH64: RFR (XS) 8068927: AARCH64: better handling of aarch64- triples (was RFR: AARCH64: Top-level JDK changes) In-Reply-To: <54B6FCF7.1010202@oracle.com> References: <545CFFA9.4070107@redhat.com> <545D0290.5080307@oracle.com> <54B34E20.5030506@oracle.com> <54B3A108.9070203@redhat.com> <54B4DAD4.5060904@oracle.com> <54B4E083.9040502@redhat.com> <54B56A55.3020803@oracle.com> <54B6FCF7.1010202@oracle.com> Message-ID: <54B7C39A.1050000@oracle.com> On 2015-01-15 00:34, Dean Long wrote: > Can I get a review for this? > > https://bugs.openjdk.java.net/browse/JDK-8068927 > http://cr.openjdk.java.net/~dlong/8068927/webrev/ Looks good to me. (However, I'm not a formal reviewer for the aarch64-port project) /Magnus From magnus.ihse.bursie at oracle.com Thu Jan 15 13:44:32 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 15 Jan 2015 14:44:32 +0100 Subject: RFR: AARCH64: Top-level JDK changes In-Reply-To: <54B6E782.3040405@oracle.com> References: <545CFFA9.4070107@redhat.com> <545D0290.5080307@oracle.com> <54B34E20.5030506@oracle.com> <54B3B4CE.8060804@oracle.com> <54B4D82B.3050608@oracle.com> <54B66EB3.3020601@oracle.com> <54B6E782.3040405@oracle.com> Message-ID: <54B7C440.7070807@oracle.com> On 2015-01-14 23:02, Dean Long wrote: > On 1/14/2015 5:27 AM, Magnus Ihse Bursie wrote: >> On 2015-01-13 09:32, Dean Long wrote: >>> On 1/12/2015 3:49 AM, Magnus Ihse Bursie wrote: >>>> On 2015-01-12 05:31, Dean Long wrote: >>>>> I found a small problem with the new config.sub wrapper. It works >>>>> with the bash shell but not with the dash shell. >>>>> The problem seems to be with this line: >>>>> >>>>> result=`. $DIR/autoconf-config.sub $sub_args "$@"` >>>>> >>>>> "dash" doesn't seem to support args passed with ".", so $sub_args >>>>> "$@" are ignored. >>>> >>>> bash is the required shell for running configure. We do not support >>>> non-bash shells. In fact, we go to lengths to try to ensure that we >>>> are indeed running under bash. >>>> >>>> /Magnus >>> I was thinking 'bash configure' was enough, but it turns out >>> 'CONFIG_SHELL=bash bash configure' gives better results. >> >> Hm, that's interesting. We were attempting to automatically use bash >> in the real configure script, regardless of what shell the user had >> to start the top-level configure wrapper. >> >> If you try the patch below, does it work better when you run "dash >> configure"? >> >> diff --git a/common/autoconf/configure b/common/autoconf/configure >> --- a/common/autoconf/configure >> +++ b/common/autoconf/configure >> @@ -36,6 +36,13 @@ >> shift >> fi >> >> +if test "x$BASH" = x; then >> + echo "Error: This script must be run using bash." 1>&2 >> + exit 1 >> +fi >> +# Force autoconf to use bash >> +export CONFIG_SHELL=$BASH >> + >> conf_script_dir="$TOPDIR/common/autoconf" >> >> if [ "$CUSTOM_CONFIG_DIR" = "" ]; then >> >> /Magnus >> > > Yes, that patch solves the problem. Thank you for testing it! I have opened https://bugs.openjdk.java.net/browse/JDK-8069057 and will integrate the fix to jdk9-dev. /Magnus From jesper.wilhelmsson at oracle.com Thu Jan 15 14:36:11 2015 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Thu, 15 Jan 2015 15:36:11 +0100 Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java In-Reply-To: References: <4295855A5C1DE049A61835A1887419CC2CF6922E@DEWDFEMB12A.global.corp.sap> <54AA6151.3040603@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF692D0@DEWDFEMB12A.global.corp.sap> Message-ID: <54B7D05B.8010203@oracle.com> Yes, I'll push it. /Jesper Volker Simonis skrev den 15/1/15 14:39: > Hi Goetz, > > I think the change is looking good now. > > Jesper, could you please be so kind to push this change for us? > > Thank you and best regards, > Volker > > > On Mon, Jan 5, 2015 at 12:48 PM, Lindenmaier, Goetz > wrote: >> Hi, >> >> I fixed it as proposed. >> http://cr.openjdk.java.net/~goetz/webrevs/8068026-jtregClient/webrev.01/ >> >> I also assured the right exception is thrown, if I change the check for >> the exit value to check for '1' (so that I get a test error), I now get: >> java.lang.RuntimeException: Expected to get exit value of [1] >> while before, it would have said the wrong message: >> java.lang.RuntimeException: 'Unrecognized option: -client' missing from stdout/stderr >> >> Best regards, >> Goetz. >> >> >> -----Original Message----- >> From: Jesper Wilhelmsson [mailto:jesper.wilhelmsson at oracle.com] >> Sent: Montag, 5. Januar 2015 11:03 >> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' >> Subject: Re: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java >> >> Hi, >> >> It seems your change introduces dead code: >> >> + if (output.shouldContain("Unrecognized option: -client") == null) { >> + throw e; >> + } >> >> If the string is found shouldContain will return a reference to 'output' (!= >> null), and if the string is not found it will throw an exception, so the >> introduced throw will never happen, right? >> >> This means the original exception 'e' is lost and a different one is thrown if >> -client was accepted by the VM. >> >> /Jesper >> >> >> Lindenmaier, Goetz skrev den 5/1/15 10:38: >>> Just a little reminder ... this is really small! >>> >>> From: Lindenmaier, Goetz >>> Sent: Montag, 22. Dezember 2014 14:08 >>> To: 'hotspot-dev at openjdk.java.net' >>> Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java >>> >>> Hi, >>> >>> I please need a review and a sponsor for this tiny change: >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8068026-jtregClient/webrev.01/ >>> The test did not check whether -client is supported. >>> >>> The patch should also go to 8u60, please. >>> >>> Best regards, >>> Goetz >>> From goetz.lindenmaier at sap.com Thu Jan 15 14:37:08 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 15 Jan 2015 14:37:08 +0000 Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java In-Reply-To: <54B7D05B.8010203@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF6922E@DEWDFEMB12A.global.corp.sap> <54AA6151.3040603@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF692D0@DEWDFEMB12A.global.corp.sap> <54B7D05B.8010203@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF6F75A@DEWDFEMB12A.global.corp.sap> That's great, thanks a lot, Jesper! Best regards, Goetz. -----Original Message----- From: Jesper Wilhelmsson [mailto:jesper.wilhelmsson at oracle.com] Sent: Donnerstag, 15. Januar 2015 15:36 To: Volker Simonis; Lindenmaier, Goetz Cc: hotspot-dev at openjdk.java.net Subject: Re: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java Yes, I'll push it. /Jesper Volker Simonis skrev den 15/1/15 14:39: > Hi Goetz, > > I think the change is looking good now. > > Jesper, could you please be so kind to push this change for us? > > Thank you and best regards, > Volker > > > On Mon, Jan 5, 2015 at 12:48 PM, Lindenmaier, Goetz > wrote: >> Hi, >> >> I fixed it as proposed. >> http://cr.openjdk.java.net/~goetz/webrevs/8068026-jtregClient/webrev.01/ >> >> I also assured the right exception is thrown, if I change the check for >> the exit value to check for '1' (so that I get a test error), I now get: >> java.lang.RuntimeException: Expected to get exit value of [1] >> while before, it would have said the wrong message: >> java.lang.RuntimeException: 'Unrecognized option: -client' missing from stdout/stderr >> >> Best regards, >> Goetz. >> >> >> -----Original Message----- >> From: Jesper Wilhelmsson [mailto:jesper.wilhelmsson at oracle.com] >> Sent: Montag, 5. Januar 2015 11:03 >> To: Lindenmaier, Goetz; 'hotspot-dev at openjdk.java.net' >> Subject: Re: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java >> >> Hi, >> >> It seems your change introduces dead code: >> >> + if (output.shouldContain("Unrecognized option: -client") == null) { >> + throw e; >> + } >> >> If the string is found shouldContain will return a reference to 'output' (!= >> null), and if the string is not found it will throw an exception, so the >> introduced throw will never happen, right? >> >> This means the original exception 'e' is lost and a different one is thrown if >> -client was accepted by the VM. >> >> /Jesper >> >> >> Lindenmaier, Goetz skrev den 5/1/15 10:38: >>> Just a little reminder ... this is really small! >>> >>> From: Lindenmaier, Goetz >>> Sent: Montag, 22. Dezember 2014 14:08 >>> To: 'hotspot-dev at openjdk.java.net' >>> Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java >>> >>> Hi, >>> >>> I please need a review and a sponsor for this tiny change: >>> >>> http://cr.openjdk.java.net/~goetz/webrevs/8068026-jtregClient/webrev.01/ >>> The test did not check whether -client is supported. >>> >>> The patch should also go to 8u60, please. >>> >>> Best regards, >>> Goetz >>> From erik.helin at oracle.com Thu Jan 15 15:36:14 2015 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 15 Jan 2015 16:36:14 +0100 Subject: RFR: 8066875: VirtualSpace does not use large pages In-Reply-To: <54AE8A74.2080304@oracle.com> References: <54874F17.7070303@oracle.com> <548814C1.8010905@oracle.com> <548AC362.8010800@oracle.com> <20141215143649.GB24542@ehelin.jrpg.bea.com> <54AE8A74.2080304@oracle.com> Message-ID: <20150115153614.GG4167@ehelin.jrpg.bea.com> On 2015-01-08, Stefan Karlsson wrote: > Hi Erik, Hi StefanK, thanks for reviewing! > On 2014-12-15 15:36, Erik Helin wrote: > >On 2014-12-12, Albert Noll wrote: > >>Hi Erik, > >> > >>thanks a lot for taking care of this. I have one minor comment (not a > >>reviewer): > >> > >>1418 size_t os::largest_page_size_less_than(size_t sz) { > >>1419 if (UseLargePages) { > >>1420 // The page sizes are sorted descendingly. > >>1421 for (size_t i = 0; _page_sizes[i] != 0; ++i) { > >>1422 if (_page_sizes[i] <= sz) { > >>1423 return _page_sizes[i]; > >>1424 } > >>1425 } > >>1426 } > >>1427 return vm_page_size(); > >>1428 } > >> > >>The function name suggests that the function returns a page size that is > >>*less* than the value in the argument (sz). > >>However, in line 1422 you check for '<=' . I think you should either fix the > >>name of the function or the check in line 1422. > >Yeah, I wasn't too fond of that name either. I discussed some potential > >names with the guys in the office and ended up with: > >- os::page_size_for_region_aligned > >- os::page_size_for_region_unaligned > > > >os::page_size_for_region_aligned and os::page_size_for_region_unaligned > >takes the same number of parameters, a region size and minimum number of > >pages. The difference is that page_size_for_region_aligned guarantees > >that returned page size divides the given region size, whereas > >page_size_for_region_unaligned does not guarantee this. > > > >These names were chosen because a call to page_size_for_region_unaligned > >indicates that the surrounding code must handle any alignment on its > >own. > > > >Webrevs: > >- incremental: http://cr.openjdk.java.net/~ehelin/8066875/webrev.00-01/ > >- full: http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/ New webrevs (with fixes to comments below): - incremental: http://cr.openjdk.java.net/~ehelin/8066875/webrev.01-02/ - full: http://cr.openjdk.java.net/~ehelin/8066875/webrev.02/ > http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/src/share/vm/runtime/virtualspace.cpp.udiff.html > > I think this code should use the unaligned version: > > ReservedSpace::ReservedSpace(size_t size) { > - size_t page_size = os::page_size_for_region(size, 1); > + size_t page_size = os::page_size_for_region_aligned(size, 1); > bool large_pages = page_size != (size_t)os::vm_page_size(); > // Don't force the alignment to be large page aligned, > // since that will waste memory. > size_t alignment = os::vm_allocation_granularity(); > initialize(size, alignment, large_pages, NULL, 0, false); > > As the comment hints, 'size' might not be large pages aligned, but we might still want to use large pages for the middle part of the reserved memory. Agree, fixed. > http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/src/share/vm/runtime/os.cpp.udiff.html > > Could this code: > > - if (page_size <= max_page_size && is_size_aligned(region_size, page_size)) { > + if (page_size <= max_page_size) { > + if (!must_be_aligned) { > return page_size; > } > + if (is_size_aligned(region_size, page_size)) { > + return page_size; > + } > + } > > be changed into one return statement?: > > if (page_size <= max_page_size) { > if (!must_be_aligned || is_size_aligned(region_size, page_size)) { > return page_size; > } > } Agree, this reads much nicer! > http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/src/share/vm/runtime/os.cpp.udiff.html > > The comments in the tests would love some periods. :) Sure, the comments now have periods :) Thanks, Erik > Thanks, > StefanK > > > > >Testing: > >- JPRT > >- Verified that the code cache now uses large pages even if > > ReservedCodeCacheSize is 241 MB (see bug for more details). > >- Added new internal vm tests (also run on SPARC machine with large > > pages) > >- Run hotspot jtreg tests on SPARC machine with large pages > > > >Thanks, > >Erik > > > >>Best, > >>Albert > >> > >> > >>On 12/10/2014 10:39 AM, Tobias Hartmann wrote: > >>>Hi Erik, > >>> > >>>looks good (not a reviewer). > >>> > >>>Thanks, > >>>Tobias > >>> > >>>On 09.12.2014 20:35, Erik Helin wrote: > >>>>Hi all, > >>>> > >>>>the fix for JDK-8049864 [0] made os::page_size_for_region slightly more strict > >>>>since the function now demands that the given region_size is size aligned with > >>>>respect to the chosen page_size. The reason for doing this was that > >>>>os::page_size_for_region was used in two ways: > >>>>1. Give me a suitable page size for this amount of memory > >>>>2. Give me the largest page size for this amount of memory > >>>>The fix for JDK-8049864 fixed os::page_size_for_region for the "suitable page > >>>>size" scenario, but is too strict for the "largest page size" scenario. This > >>>>caused a regression in VirtualSpace::initialize, which only needs the largest > >>>>possible page size, since VirtualSpace::initialize_with_granularity takes care > >>>>of any alignment issues. > >>>> > >>>>This patch adds the function os::largest_page_size_less_than and updates > >>>>VirtualSpace::initialize to use this new function instead of > >>>>os::page_size_for_region. > >>>> > >>>>Webrev: > >>>>http://cr.openjdk.java.net/~ehelin/8066875/webrev.00/ > >>>> > >>>>Bug: > >>>>https://bugs.openjdk.java.net/browse/JDK-8066875 > >>>> > >>>>Testing: > >>>>- JPRT > >>>>- Verified that the code cache now uses large pages even if > >>>> ReservedCodeCacheSize is 241 MB (see bug for more details). > >>>>- Added new internal vm tests (also run on SPARC machine with large > >>>> pages) > >>>> > >>>>Thanks, > >>>>Erik > >>>> > >>>>[0]: http://hg.openjdk.java.net/jdk9/hs-gc/hotspot/rev/b326a3e8dcab > From stefan.karlsson at oracle.com Thu Jan 15 15:42:28 2015 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 15 Jan 2015 16:42:28 +0100 Subject: RFR: 8066875: VirtualSpace does not use large pages In-Reply-To: <20150115153614.GG4167@ehelin.jrpg.bea.com> References: <54874F17.7070303@oracle.com> <548814C1.8010905@oracle.com> <548AC362.8010800@oracle.com> <20141215143649.GB24542@ehelin.jrpg.bea.com> <54AE8A74.2080304@oracle.com> <20150115153614.GG4167@ehelin.jrpg.bea.com> Message-ID: <54B7DFE4.9010503@oracle.com> On 2015-01-15 16:36, Erik Helin wrote: > On 2015-01-08, Stefan Karlsson wrote: >> Hi Erik, > Hi StefanK, > > thanks for reviewing! > >> On 2014-12-15 15:36, Erik Helin wrote: >>> On 2014-12-12, Albert Noll wrote: >>>> Hi Erik, >>>> >>>> thanks a lot for taking care of this. I have one minor comment (not a >>>> reviewer): >>>> >>>> 1418 size_t os::largest_page_size_less_than(size_t sz) { >>>> 1419 if (UseLargePages) { >>>> 1420 // The page sizes are sorted descendingly. >>>> 1421 for (size_t i = 0; _page_sizes[i] != 0; ++i) { >>>> 1422 if (_page_sizes[i] <= sz) { >>>> 1423 return _page_sizes[i]; >>>> 1424 } >>>> 1425 } >>>> 1426 } >>>> 1427 return vm_page_size(); >>>> 1428 } >>>> >>>> The function name suggests that the function returns a page size that is >>>> *less* than the value in the argument (sz). >>>> However, in line 1422 you check for '<=' . I think you should either fix the >>>> name of the function or the check in line 1422. >>> Yeah, I wasn't too fond of that name either. I discussed some potential >>> names with the guys in the office and ended up with: >>> - os::page_size_for_region_aligned >>> - os::page_size_for_region_unaligned >>> >>> os::page_size_for_region_aligned and os::page_size_for_region_unaligned >>> takes the same number of parameters, a region size and minimum number of >>> pages. The difference is that page_size_for_region_aligned guarantees >>> that returned page size divides the given region size, whereas >>> page_size_for_region_unaligned does not guarantee this. >>> >>> These names were chosen because a call to page_size_for_region_unaligned >>> indicates that the surrounding code must handle any alignment on its >>> own. >>> >>> Webrevs: >>> - incremental: http://cr.openjdk.java.net/~ehelin/8066875/webrev.00-01/ >>> - full: http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/ > New webrevs (with fixes to comments below): > - incremental: http://cr.openjdk.java.net/~ehelin/8066875/webrev.01-02/ Looks good. Thanks, StefanK > - full: http://cr.openjdk.java.net/~ehelin/8066875/webrev.02/ > >> http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/src/share/vm/runtime/virtualspace.cpp.udiff.html >> >> I think this code should use the unaligned version: >> >> ReservedSpace::ReservedSpace(size_t size) { >> - size_t page_size = os::page_size_for_region(size, 1); >> + size_t page_size = os::page_size_for_region_aligned(size, 1); >> bool large_pages = page_size != (size_t)os::vm_page_size(); >> // Don't force the alignment to be large page aligned, >> // since that will waste memory. >> size_t alignment = os::vm_allocation_granularity(); >> initialize(size, alignment, large_pages, NULL, 0, false); >> >> As the comment hints, 'size' might not be large pages aligned, but we might still want to use large pages for the middle part of the reserved memory. > Agree, fixed. > >> http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/src/share/vm/runtime/os.cpp.udiff.html >> >> Could this code: >> >> - if (page_size <= max_page_size && is_size_aligned(region_size, page_size)) { >> + if (page_size <= max_page_size) { >> + if (!must_be_aligned) { >> return page_size; >> } >> + if (is_size_aligned(region_size, page_size)) { >> + return page_size; >> + } >> + } >> >> be changed into one return statement?: >> >> if (page_size <= max_page_size) { >> if (!must_be_aligned || is_size_aligned(region_size, page_size)) { >> return page_size; >> } >> } > Agree, this reads much nicer! > >> http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/src/share/vm/runtime/os.cpp.udiff.html >> >> The comments in the tests would love some periods. :) > Sure, the comments now have periods :) > > Thanks, > Erik > >> Thanks, >> StefanK >> >>> Testing: >>> - JPRT >>> - Verified that the code cache now uses large pages even if >>> ReservedCodeCacheSize is 241 MB (see bug for more details). >>> - Added new internal vm tests (also run on SPARC machine with large >>> pages) >>> - Run hotspot jtreg tests on SPARC machine with large pages >>> >>> Thanks, >>> Erik >>> >>>> Best, >>>> Albert >>>> >>>> >>>> On 12/10/2014 10:39 AM, Tobias Hartmann wrote: >>>>> Hi Erik, >>>>> >>>>> looks good (not a reviewer). >>>>> >>>>> Thanks, >>>>> Tobias >>>>> >>>>> On 09.12.2014 20:35, Erik Helin wrote: >>>>>> Hi all, >>>>>> >>>>>> the fix for JDK-8049864 [0] made os::page_size_for_region slightly more strict >>>>>> since the function now demands that the given region_size is size aligned with >>>>>> respect to the chosen page_size. The reason for doing this was that >>>>>> os::page_size_for_region was used in two ways: >>>>>> 1. Give me a suitable page size for this amount of memory >>>>>> 2. Give me the largest page size for this amount of memory >>>>>> The fix for JDK-8049864 fixed os::page_size_for_region for the "suitable page >>>>>> size" scenario, but is too strict for the "largest page size" scenario. This >>>>>> caused a regression in VirtualSpace::initialize, which only needs the largest >>>>>> possible page size, since VirtualSpace::initialize_with_granularity takes care >>>>>> of any alignment issues. >>>>>> >>>>>> This patch adds the function os::largest_page_size_less_than and updates >>>>>> VirtualSpace::initialize to use this new function instead of >>>>>> os::page_size_for_region. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~ehelin/8066875/webrev.00/ >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8066875 >>>>>> >>>>>> Testing: >>>>>> - JPRT >>>>>> - Verified that the code cache now uses large pages even if >>>>>> ReservedCodeCacheSize is 241 MB (see bug for more details). >>>>>> - Added new internal vm tests (also run on SPARC machine with large >>>>>> pages) >>>>>> >>>>>> Thanks, >>>>>> Erik >>>>>> >>>>>> [0]: http://hg.openjdk.java.net/jdk9/hs-gc/hotspot/rev/b326a3e8dcab From aph at redhat.com Thu Jan 15 18:20:04 2015 From: aph at redhat.com (Andrew Haley) Date: Thu, 15 Jan 2015 18:20:04 +0000 Subject: RFR: 6584008: jvmtiStringPrimitiveCallback should not be invoked when string value is null In-Reply-To: References: <54B01E29.2090309@redhat.com> Message-ID: <54B804D4.7000907@redhat.com> On 01/12/2015 07:51 AM, Staffan Larsen wrote: > Looks good. But I would appreciate if you could add braces to the if-statement. Done. http://cr.openjdk.java.net/~aph/JDK-6584008-2/ I think I can't push this myself. Andrew. From aph at redhat.com Thu Jan 15 18:38:25 2015 From: aph at redhat.com (Andrew Haley) Date: Thu, 15 Jan 2015 18:38:25 +0000 Subject: 8068053: AARCH64: C1 and C2 compilers In-Reply-To: <54ABA2EA.4070802@redhat.com> References: <54A6A8F5.4040907@redhat.com> <928DC166-7256-4BCA-A9CA-C3ADB05834E5@oracle.com> <54AAD3FC.8000003@redhat.com> <54AAED1F.1000908@oracle.com> <54ABA2EA.4070802@redhat.com> Message-ID: <54B80921.50608@redhat.com> New webrev at http://cr.openjdk.java.net/~aph/aarch64-8068053-4/ Thanks, Andrew. From aph at redhat.com Thu Jan 15 18:40:05 2015 From: aph at redhat.com (Andrew Haley) Date: Thu, 15 Jan 2015 18:40:05 +0000 Subject: 8068055: AARCH64: os_cpu/ In-Reply-To: <54AAF87F.9090706@oracle.com> References: <54A6A977.4080405@redhat.com> <54AAF87F.9090706@oracle.com> Message-ID: <54B80985.2030606@redhat.com> On 01/05/2015 08:47 PM, Vladimir Kozlov wrote: > linux_aarch64.S is missing Copyright header. > > linux_aarch64.ad: "AMD64 Linux Architecture". > > os_linux_aarch64.cpp, print registers methods lists x86 registers. And #ifndef AMD64. > > The rest seems fine. New webrev at http://cr.openjdk.java.net/~aph/aarch64-8068055-2/ Thanks, Andrew. From aph at redhat.com Thu Jan 15 18:41:11 2015 From: aph at redhat.com (Andrew Haley) Date: Thu, 15 Jan 2015 18:41:11 +0000 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54AD4A98.9080501@redhat.com> References: <54A6A933.8000204@redhat.com> <7C4F2EEA-04D9-4421-8231-23E0A47A6502@oracle.com> <54AD4A98.9080501@redhat.com> Message-ID: <54B809C7.2040006@redhat.com> New webrev at http://cr.openjdk.java.net/~aph/aarch64-8068054-3/ Thanks, Andrew. From vladimir.kozlov at oracle.com Thu Jan 15 18:49:07 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 15 Jan 2015 10:49:07 -0800 Subject: 8068055: AARCH64: os_cpu/ In-Reply-To: <54B80985.2030606@redhat.com> References: <54A6A977.4080405@redhat.com> <54AAF87F.9090706@oracle.com> <54B80985.2030606@redhat.com> Message-ID: <54B80BA3.2070702@oracle.com> I started worry what is happening with this port. Glad to see it is still progressing :) This looks good. I will push it today through JPRT since it is new files. Thanks, Vladimir On 1/15/15 10:40 AM, Andrew Haley wrote: > On 01/05/2015 08:47 PM, Vladimir Kozlov wrote: >> linux_aarch64.S is missing Copyright header. >> >> linux_aarch64.ad: "AMD64 Linux Architecture". >> >> os_linux_aarch64.cpp, print registers methods lists x86 registers. And #ifndef AMD64. >> >> The rest seems fine. > > New webrev at http://cr.openjdk.java.net/~aph/aarch64-8068055-2/ > > Thanks, > Andrew. > From staffan.larsen at oracle.com Thu Jan 15 19:05:30 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Thu, 15 Jan 2015 20:05:30 +0100 Subject: RFR: 6584008: jvmtiStringPrimitiveCallback should not be invoked when string value is null In-Reply-To: <54B804D4.7000907@redhat.com> References: <54B01E29.2090309@redhat.com> <54B804D4.7000907@redhat.com> Message-ID: > On 15 jan 2015, at 19:20, Andrew Haley wrote: > > On 01/12/2015 07:51 AM, Staffan Larsen wrote: >> Looks good. But I would appreciate if you could add braces to the if-statement. > > Done. http://cr.openjdk.java.net/~aph/JDK-6584008-2/ > > I think I can't push this myself. I can push this for you if we get a second review. /Staffan From vladimir.kozlov at oracle.com Thu Jan 15 19:11:44 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 15 Jan 2015 11:11:44 -0800 Subject: RFR: 6584008: jvmtiStringPrimitiveCallback should not be invoked when string value is null In-Reply-To: References: <54B01E29.2090309@redhat.com> <54B804D4.7000907@redhat.com> Message-ID: <54B810F0.3050407@oracle.com> Reviewed. Thanks, Vladimir On 1/15/15 11:05 AM, Staffan Larsen wrote: > >> On 15 jan 2015, at 19:20, Andrew Haley wrote: >> >> On 01/12/2015 07:51 AM, Staffan Larsen wrote: >>> Looks good. But I would appreciate if you could add braces to the if-statement. >> >> Done. http://cr.openjdk.java.net/~aph/JDK-6584008-2/ >> >> I think I can't push this myself. > > I can push this for you if we get a second review. > > /Staffan > From vladimir.kozlov at oracle.com Thu Jan 15 19:26:17 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 15 Jan 2015 11:26:17 -0800 Subject: 8068053: AARCH64: C1 and C2 compilers In-Reply-To: <54B80921.50608@redhat.com> References: <54A6A8F5.4040907@redhat.com> <928DC166-7256-4BCA-A9CA-C3ADB05834E5@oracle.com> <54AAD3FC.8000003@redhat.com> <54AAED1F.1000908@oracle.com> <54ABA2EA.4070802@redhat.com> <54B80921.50608@redhat.com> Message-ID: <54B81459.8010809@oracle.com> Looks good. Thanks, Vladimir On 1/15/15 10:38 AM, Andrew Haley wrote: > New webrev at http://cr.openjdk.java.net/~aph/aarch64-8068053-4/ > > Thanks, > Andrew. > From vladimir.kozlov at oracle.com Thu Jan 15 19:49:14 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 15 Jan 2015 11:49:14 -0800 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54B809C7.2040006@redhat.com> References: <54A6A933.8000204@redhat.com> <7C4F2EEA-04D9-4421-8231-23E0A47A6502@oracle.com> <54AD4A98.9080501@redhat.com> <54B809C7.2040006@redhat.com> Message-ID: <54B819BA.5040804@oracle.com> Looks good to me. One thing caught my eye is relocInfo_aarch64.hpp: // FIXME for AARCH64 The values there are similar to relocInfo_x86.hpp. Do you still need to adjust them? Thanks, Vladimir On 1/15/15 10:41 AM, Andrew Haley wrote: > New webrev at http://cr.openjdk.java.net/~aph/aarch64-8068054-3/ > > Thanks, > Andrew. > From dean.long at oracle.com Thu Jan 15 20:04:51 2015 From: dean.long at oracle.com (Dean Long) Date: Thu, 15 Jan 2015 12:04:51 -0800 Subject: [aarch64-port-dev ] AARCH64: RFR (XS) 8068927: AARCH64: better handling of aarch64- triples (was RFR: AARCH64: Top-level JDK changes) In-Reply-To: <54B7C39A.1050000@oracle.com> References: <545CFFA9.4070107@redhat.com> <545D0290.5080307@oracle.com> <54B34E20.5030506@oracle.com> <54B3A108.9070203@redhat.com> <54B4DAD4.5060904@oracle.com> <54B4E083.9040502@redhat.com> <54B56A55.3020803@oracle.com> <54B6FCF7.1010202@oracle.com> <54B7C39A.1050000@oracle.com> Message-ID: <54B81D63.3050207@oracle.com> On 1/15/2015 5:41 AM, Magnus Ihse Bursie wrote: > On 2015-01-15 00:34, Dean Long wrote: >> Can I get a review for this? >> >> https://bugs.openjdk.java.net/browse/JDK-8068927 >> http://cr.openjdk.java.net/~dlong/8068927/webrev/ > > Looks good to me. (However, I'm not a formal reviewer for the > aarch64-port project) > > /Magnus > You're a Reviewer for jdk9, and this change will be going into 9 after it does its time in aarch64 staging, so I hope that's good enough. dl From aph at redhat.com Thu Jan 15 20:19:49 2015 From: aph at redhat.com (Andrew Haley) Date: Thu, 15 Jan 2015 20:19:49 +0000 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54B819BA.5040804@oracle.com> References: <54A6A933.8000204@redhat.com> <7C4F2EEA-04D9-4421-8231-23E0A47A6502@oracle.com> <54AD4A98.9080501@redhat.com> <54B809C7.2040006@redhat.com> <54B819BA.5040804@oracle.com> Message-ID: <54B820E5.1080304@redhat.com> On 01/15/2015 07:49 PM, Vladimir Kozlov wrote: > // FIXME for AARCH64 > > The values there are similar to relocInfo_x86.hpp. > > Do you still need to adjust them? Hmm, I suppose I should, yes. They don't seem to affect anything, though. Thanks, Andrew. From serguei.spitsyn at oracle.com Thu Jan 15 20:29:11 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 15 Jan 2015 12:29:11 -0800 Subject: RFR: 6584008: jvmtiStringPrimitiveCallback should not be invoked when string value is null In-Reply-To: <54B804D4.7000907@redhat.com> References: <54B01E29.2090309@redhat.com> <54B804D4.7000907@redhat.com> Message-ID: <54B82317.6040906@oracle.com> Looks good. Thanks, Serguei On 1/15/15 10:20 AM, Andrew Haley wrote: > On 01/12/2015 07:51 AM, Staffan Larsen wrote: >> Looks good. But I would appreciate if you could add braces to the if-statement. > Done. http://cr.openjdk.java.net/~aph/JDK-6584008-2/ > > I think I can't push this myself. > > Andrew. > From magnus.ihse.bursie at oracle.com Thu Jan 15 22:57:39 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 15 Jan 2015 23:57:39 +0100 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54B765DC.8050801@oracle.com> References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> <54B760DB.70200@oracle.com> <54B765DC.8050801@oracle.com> Message-ID: <54B845E3.7060000@oracle.com> On 2015-01-15 08:01, David Holmes wrote: > > If the build-dev guys confirm we already assume bash that is fine. For the rest of the world, we only use bash. For hotspot, we will use bash if called from the top-level Makefile. I can't say anything about what the convention have been for calling the hotspot makefiles directly, though. /Magnus From vladimir.kozlov at oracle.com Thu Jan 15 23:45:25 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 15 Jan 2015 15:45:25 -0800 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54B820E5.1080304@redhat.com> References: <54A6A933.8000204@redhat.com> <7C4F2EEA-04D9-4421-8231-23E0A47A6502@oracle.com> <54AD4A98.9080501@redhat.com> <54B809C7.2040006@redhat.com> <54B819BA.5040804@oracle.com> <54B820E5.1080304@redhat.com> Message-ID: <54B85115.8040100@oracle.com> They affect compression of relocation info. With offset_unit = 4 we can shift addresses to remove zero in low 2 bits. Also: static int addr_unit() { return offset_unit; } Vladimir On 1/15/15 12:19 PM, Andrew Haley wrote: > On 01/15/2015 07:49 PM, Vladimir Kozlov wrote: >> // FIXME for AARCH64 >> >> The values there are similar to relocInfo_x86.hpp. >> >> Do you still need to adjust them? > > Hmm, I suppose I should, yes. They don't seem to affect anything, though. > > Thanks, > Andrew. > From tobias.hartmann at oracle.com Fri Jan 16 08:09:33 2015 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 16 Jan 2015 09:09:33 +0100 Subject: RFR: 8066875: VirtualSpace does not use large pages In-Reply-To: <20150115153614.GG4167@ehelin.jrpg.bea.com> References: <54874F17.7070303@oracle.com> <548814C1.8010905@oracle.com> <548AC362.8010800@oracle.com> <20141215143649.GB24542@ehelin.jrpg.bea.com> <54AE8A74.2080304@oracle.com> <20150115153614.GG4167@ehelin.jrpg.bea.com> Message-ID: <54B8C73D.9010706@oracle.com> Hi Erik, looks good (not a reviewer). Best, Tobias On 15.01.2015 16:36, Erik Helin wrote: > On 2015-01-08, Stefan Karlsson wrote: >> Hi Erik, > > Hi StefanK, > > thanks for reviewing! > >> On 2014-12-15 15:36, Erik Helin wrote: >>> On 2014-12-12, Albert Noll wrote: >>>> Hi Erik, >>>> >>>> thanks a lot for taking care of this. I have one minor comment (not a >>>> reviewer): >>>> >>>> 1418 size_t os::largest_page_size_less_than(size_t sz) { >>>> 1419 if (UseLargePages) { >>>> 1420 // The page sizes are sorted descendingly. >>>> 1421 for (size_t i = 0; _page_sizes[i] != 0; ++i) { >>>> 1422 if (_page_sizes[i] <= sz) { >>>> 1423 return _page_sizes[i]; >>>> 1424 } >>>> 1425 } >>>> 1426 } >>>> 1427 return vm_page_size(); >>>> 1428 } >>>> >>>> The function name suggests that the function returns a page size that is >>>> *less* than the value in the argument (sz). >>>> However, in line 1422 you check for '<=' . I think you should either fix the >>>> name of the function or the check in line 1422. >>> Yeah, I wasn't too fond of that name either. I discussed some potential >>> names with the guys in the office and ended up with: >>> - os::page_size_for_region_aligned >>> - os::page_size_for_region_unaligned >>> >>> os::page_size_for_region_aligned and os::page_size_for_region_unaligned >>> takes the same number of parameters, a region size and minimum number of >>> pages. The difference is that page_size_for_region_aligned guarantees >>> that returned page size divides the given region size, whereas >>> page_size_for_region_unaligned does not guarantee this. >>> >>> These names were chosen because a call to page_size_for_region_unaligned >>> indicates that the surrounding code must handle any alignment on its >>> own. >>> >>> Webrevs: >>> - incremental: http://cr.openjdk.java.net/~ehelin/8066875/webrev.00-01/ >>> - full: http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/ > > New webrevs (with fixes to comments below): > - incremental: http://cr.openjdk.java.net/~ehelin/8066875/webrev.01-02/ > - full: http://cr.openjdk.java.net/~ehelin/8066875/webrev.02/ > >> http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/src/share/vm/runtime/virtualspace.cpp.udiff.html >> >> I think this code should use the unaligned version: >> >> ReservedSpace::ReservedSpace(size_t size) { >> - size_t page_size = os::page_size_for_region(size, 1); >> + size_t page_size = os::page_size_for_region_aligned(size, 1); >> bool large_pages = page_size != (size_t)os::vm_page_size(); >> // Don't force the alignment to be large page aligned, >> // since that will waste memory. >> size_t alignment = os::vm_allocation_granularity(); >> initialize(size, alignment, large_pages, NULL, 0, false); >> >> As the comment hints, 'size' might not be large pages aligned, but we might still want to use large pages for the middle part of the reserved memory. > > Agree, fixed. > >> http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/src/share/vm/runtime/os.cpp.udiff.html >> >> Could this code: >> >> - if (page_size <= max_page_size && is_size_aligned(region_size, page_size)) { >> + if (page_size <= max_page_size) { >> + if (!must_be_aligned) { >> return page_size; >> } >> + if (is_size_aligned(region_size, page_size)) { >> + return page_size; >> + } >> + } >> >> be changed into one return statement?: >> >> if (page_size <= max_page_size) { >> if (!must_be_aligned || is_size_aligned(region_size, page_size)) { >> return page_size; >> } >> } > > Agree, this reads much nicer! > >> http://cr.openjdk.java.net/~ehelin/8066875/webrev.01/src/share/vm/runtime/os.cpp.udiff.html >> >> The comments in the tests would love some periods. :) > > Sure, the comments now have periods :) > > Thanks, > Erik > >> Thanks, >> StefanK >> >>> >>> Testing: >>> - JPRT >>> - Verified that the code cache now uses large pages even if >>> ReservedCodeCacheSize is 241 MB (see bug for more details). >>> - Added new internal vm tests (also run on SPARC machine with large >>> pages) >>> - Run hotspot jtreg tests on SPARC machine with large pages >>> >>> Thanks, >>> Erik >>> >>>> Best, >>>> Albert >>>> >>>> >>>> On 12/10/2014 10:39 AM, Tobias Hartmann wrote: >>>>> Hi Erik, >>>>> >>>>> looks good (not a reviewer). >>>>> >>>>> Thanks, >>>>> Tobias >>>>> >>>>> On 09.12.2014 20:35, Erik Helin wrote: >>>>>> Hi all, >>>>>> >>>>>> the fix for JDK-8049864 [0] made os::page_size_for_region slightly more strict >>>>>> since the function now demands that the given region_size is size aligned with >>>>>> respect to the chosen page_size. The reason for doing this was that >>>>>> os::page_size_for_region was used in two ways: >>>>>> 1. Give me a suitable page size for this amount of memory >>>>>> 2. Give me the largest page size for this amount of memory >>>>>> The fix for JDK-8049864 fixed os::page_size_for_region for the "suitable page >>>>>> size" scenario, but is too strict for the "largest page size" scenario. This >>>>>> caused a regression in VirtualSpace::initialize, which only needs the largest >>>>>> possible page size, since VirtualSpace::initialize_with_granularity takes care >>>>>> of any alignment issues. >>>>>> >>>>>> This patch adds the function os::largest_page_size_less_than and updates >>>>>> VirtualSpace::initialize to use this new function instead of >>>>>> os::page_size_for_region. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~ehelin/8066875/webrev.00/ >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8066875 >>>>>> >>>>>> Testing: >>>>>> - JPRT >>>>>> - Verified that the code cache now uses large pages even if >>>>>> ReservedCodeCacheSize is 241 MB (see bug for more details). >>>>>> - Added new internal vm tests (also run on SPARC machine with large >>>>>> pages) >>>>>> >>>>>> Thanks, >>>>>> Erik >>>>>> >>>>>> [0]: http://hg.openjdk.java.net/jdk9/hs-gc/hotspot/rev/b326a3e8dcab >> From thomas.schatzl at oracle.com Fri Jan 16 08:40:01 2015 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 16 Jan 2015 09:40:01 +0100 Subject: RFR: 8066875: VirtualSpace does not use large pages In-Reply-To: <20150115153614.GG4167@ehelin.jrpg.bea.com> References: <54874F17.7070303@oracle.com> <548814C1.8010905@oracle.com> <548AC362.8010800@oracle.com> <20141215143649.GB24542@ehelin.jrpg.bea.com> <54AE8A74.2080304@oracle.com> <20150115153614.GG4167@ehelin.jrpg.bea.com> Message-ID: <1421397601.3106.4.camel@oracle.com> Hi Erik, looks good. Thomas From dean.long at oracle.com Fri Jan 16 10:47:27 2015 From: dean.long at oracle.com (Dean Long) Date: Fri, 16 Jan 2015 02:47:27 -0800 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54B845E3.7060000@oracle.com> References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> <54B760DB.70200@oracle.com> <54B765DC.8050801@oracle.com> <54B845E3.7060000@oracle.com> Message-ID: <54B8EC3F.5030408@oracle.com> On 1/15/2015 2:57 PM, Magnus Ihse Bursie wrote: > On 2015-01-15 08:01, David Holmes wrote: >> >> If the build-dev guys confirm we already assume bash that is fine. > > For the rest of the world, we only use bash. For hotspot, we will use > bash if called from the top-level Makefile. I can't say anything about > what the convention have been for calling the hotspot makefiles > directly, though. > > /Magnus "sh" or "bash", it shouldn't matter. Our makefiles are already using Bourne shell syntax, so if someone tries to use "make SHELL=csh", it will fail almost immediately. Therefore, I think "NM=$(NM) sh ..." should be safe. dl https://www.gnu.org/software/make/manual/make.html#Choosing-the-Shell From goetz.lindenmaier at sap.com Fri Jan 16 11:15:23 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 16 Jan 2015 11:15:23 +0000 Subject: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF6DE04@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF6DE04@DEWDFEMB12A.global.corp.sap> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF6FAFC@DEWDFEMB12A.global.corp.sap> Hi, could I please get a review for this small fix? I also please need a sponsor. Thanks, Goetz. From: goetz.lindenmaier at sap.com Sent: Montag, 12. Januar 2015 09:23 To: hotspot-dev at openjdk.java.net Subject: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available Hi, This test uses SA, thus fails on platforms where that's not implemented. I see problems on mac and aix. Call Platform.shouldSAAttach() to check whether SA should work. Please review this small fix. I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8068778-jtregSA/webrev.01/ Best regards, Goetz. From serguei.spitsyn at oracle.com Fri Jan 16 19:14:09 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 16 Jan 2015 11:14:09 -0800 Subject: 4-th round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54B60308.4000105@oracle.com> References: <54B60308.4000105@oracle.com> Message-ID: <54B96301.40108@oracle.com> Dan, David H. or David C., May I ask one of you to look at the webrev below? The issue itself is a little bit tricky, so it is not easy to review despite the small size. Coleen, Does the webrev matches what we discussed with you? Do you give me a thumbs up? Thanks, Serguei May I ask On 1/13/15 9:47 PM, serguei.spitsyn at oracle.com wrote: > Please, review the fix for: > https://bugs.openjdk.java.net/browse/JDK-8068162 > > > Open webrevs: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ > > http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/jdk/8068162-Test-IsModifiableAgent/ > > > > Summary: > > The sun.misc.Unsafe:throwIllegalAccessError() method is used in > place of a default > interface method in the itable if a default method was not defined > in the interface. > In fact, it happens for two interfaces that purhaps are > auto-generated: > java/nio/CharBuffer > java/nio/HeapCharBuffer > > This approach creates a problem when the class sun.misc.Unsafe is > retransformed. > The Method* pointer to the old (redefined) method in the itable > triggers an assert > (see the hs_err log in the bug report). > Coleen told me that a similar approach is going to be implemented > for some vtable entries. > Coleen, thanks for suggesting a better fix for this issue! > > The fix is to replace the old Unsafe method in the itable/vtable > with the latest method version. > > > Testing: > In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests > > > Thanks, > Serguei > From serguei.spitsyn at oracle.com Fri Jan 16 19:16:34 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 16 Jan 2015 11:16:34 -0800 Subject: 4-th round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54B96301.40108@oracle.com> References: <54B60308.4000105@oracle.com> <54B96301.40108@oracle.com> Message-ID: <54B96392.8090904@oracle.com> Sorry for the re-post. Forgot to add review candidates to the to-list. Thanks, Serguei On 1/16/15 11:14 AM, serguei.spitsyn at oracle.com wrote: > Dan, David H. or David C., > > May I ask one of you to look at the webrev below? > The issue itself is a little bit tricky, so it is not easy to review > despite the small size. > > Coleen, > > Does the webrev matches what we discussed with you? > Do you give me a thumbs up? > > Thanks, > Serguei > > May I ask > > On 1/13/15 9:47 PM, serguei.spitsyn at oracle.com wrote: >> Please, review the fix for: >> https://bugs.openjdk.java.net/browse/JDK-8068162 >> >> >> Open webrevs: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >> >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/jdk/8068162-Test-IsModifiableAgent/ >> >> >> >> Summary: >> >> The sun.misc.Unsafe:throwIllegalAccessError() method is used in >> place of a default >> interface method in the itable if a default method was not defined >> in the interface. >> In fact, it happens for two interfaces that purhaps are >> auto-generated: >> java/nio/CharBuffer >> java/nio/HeapCharBuffer >> >> This approach creates a problem when the class sun.misc.Unsafe is >> retransformed. >> The Method* pointer to the old (redefined) method in the itable >> triggers an assert >> (see the hs_err log in the bug report). >> Coleen told me that a similar approach is going to be implemented >> for some vtable entries. >> Coleen, thanks for suggesting a better fix for this issue! >> >> The fix is to replace the old Unsafe method in the itable/vtable >> with the latest method version. >> >> >> Testing: >> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >> >> >> Thanks, >> Serguei >> > From serguei.spitsyn at oracle.com Fri Jan 16 20:07:51 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 16 Jan 2015 12:07:51 -0800 Subject: 3-rd round RFR (S) 8008678: JSR 292: constant pool reconstitution must support pseudo strings In-Reply-To: <54934E73.4080808@oracle.com> References: <5475968C.40800@oracle.com> <54760B40.9040206@oracle.com> <54762FC9.2010800@oracle.com> <5491BCB9.10505@oracle.com> <5493299F.60005@oracle.com> <54934E73.4080808@oracle.com> Message-ID: <54B96F97.5070204@oracle.com> Please, review the fix for: https://bugs.openjdk.java.net/browse/JDK-8008678 Open webrev: http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8008678-JVMTI-pseudo.3/ Summary: Currently, a JVM_CONSTANT_String CP entry having a NULL reference to Symbol* indicates that it is a pseudo-string (patched string). This creates nasty issues for the constant pool reconstitution. Current suggestion is to avoid having a NULL reference and retain the original Symbol* reference for pseudo-strings. The pseudo-string indication will be if the Utf8 representation starts from "CONSTANT_PLACEHOLDER_". This approach makes the fix much simpler. I need a confirmation from the Compiler team that this won't break any assumptions or invariants. Big thanks to Coleen for previous round reviews and good advices! Testing: Run: - java/lang/instrument tests - new jtreg test (see webrev) that was written by Filipp Zhinkin Thanks, Serguei On 12/18/14 2:00 PM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > Thank you for reviewing! > > > On 12/18/14 11:23 AM, Coleen Phillimore wrote: >> >> Hi Serguei, >> >> Thank you for making the patches an optional field. >> >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/oops/constantPool.hpp.sdiff.html >> 198 if (!patched()) { >> 199 assert(false, "a pseudo-string map may exists for patched CP only"); >> 200 return 0; >> 201 } >> Why not >> assert(patched(), "a pseud-string map must exist for >> patched CP only"); > > Wanted it to be more reliable but it looks pointless. > Will make this change. > >> >> >> Why is this? Is this really a ShouldNotReachHere? should it be false? >> >> 215 assert(true, "not found a matching entry in pseudo-string map"); > > > A typo, must be false. > It is the last minute change. > Thanks for the catch! > > >> >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/prims/jvmtiRedefineClasses.cpp.udiff.html >> >> Don't you have to move the value of the patched field from the old >> constant pool to the new one? I hate to ask but is there merging >> that needs to be done? I don't know how to write this test case >> though. Is it possible to redefine a class with a constant pool >> patches with another that has constant pool patches? > > Thank you for asking this question. > If I understand correctly, the patching comes from the compiler side > for anonymous classes only. > I saw it for LambdaForm's only. > I think, the patching can not be changed with a retransformation. > But I'm not sure if it can not be changed with a redefinition. > > But if it can - then it would be safe to merge the 'patched' > condition, i.e. make it patched > if either the_class or scratch class is patched. > >> >> Somehow I thought you'd have to save the value of the cp_patches oops >> passed in. >> >> So I was wondering why you can't change this instead: >> >> bool is_pseudo_string_at(int which) { >> // A pseudo string is a string that doesn't have a symbol in the >> cpSlot >> - return unresolved_string_at(which) == NULL; >> + return (strncmp(unresolved_string_at(which)->as_utf8(), >> "CONSTANT_PLACEHOLDER_" , strlen("CONSTANT_PLACEHOLDER")) == 0); >> } > > I was thinking about the same but was not sure if it would work for > the compiler team. > We have to ask John about this (added John and Christian to the cc-list). > This question to John was in my plan! :) > > The beauty of the above approach is that there is no need to create an > intermediate > pseudo-string map and most of the code in from this webrev is not needed. > > > Thanks, > Serguei > >> >> And the asserts in the other functions below it. >> >> Coleen >> >> >> On 12/17/14, 12:26 PM, serguei.spitsyn at oracle.com wrote: >>> Please, review the second round fix for: >>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>> >>> Open webrev: >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/ >>> >>> >>> Summary: >>> >>> This fix implements a footprint saving approach suggested by Coleen. >>> To be able to reconstitute a class constant pool, an intermediate >>> pseudo-string map is used. >>> Now, this field is accounted optionally, only if the 'cp_patches' >>> is provided in the >>> ClassFileParser::parseClassFile() before ConstantPool is allocated. >>> This fix is not elegant, even a little bit ugly, but it is the >>> only way I see so far. >>> >>> Unfortunately, this approach did not help much to make some other >>> fields (eg., 'operands') optional. >>> The problem is that we have to account optional fields before >>> parsing, at the CP allocation time. >>> It is possible to re-allocate the ConstantPool when any >>> InvokeDynamic bytecode is discovered, >>> but it looks too complicated. >>> >>> Testing: >>> - the unit test from bug report >>> - nsk.jvmti,testlist, nsk.jdi.testlist, JTREG java/lang/instrument >>> - vm.mlvm.testlist, vm.quick.testlist, >>> vm.parallel_class_loading.testlist (in progress) >>> >>> >>> Thanks, >>> Serguei >>> >>> >>> On 11/26/14 11:53 AM, serguei.spitsyn at oracle.com wrote: >>>> Coleen, >>>> >>>> Thank you for looking at this! >>>> I'll check how this can be improved. >>>> It is my concern too. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> On 11/26/14 9:17 AM, Coleen Phillimore wrote: >>>>> >>>>> Serguei, >>>>> I had a quick look at this. I was wondering if we could make the >>>>> pseudo_string_map conditional in ConstantPool and not make all >>>>> classes pay in footprint for this field? The same thing probably >>>>> could be done for operands too. There are flags that you can set >>>>> to conditionally add a pointer to base() in this function. >>>>> >>>>> Typical C++ would subclass ConstantPool to add >>>>> InvokeDynamicConstantPool fields, but this is not typical C++ so >>>>> the trick we use is like the one in ConstMethod. I think it's >>>>> worth doing in this case. >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> On 11/26/14, 3:59 AM, serguei.spitsyn at oracle.com wrote: >>>>>> Please, review the fix for: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>>> >>>>>> >>>>>> Open webrev: >>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.1/ >>>>>> >>>>>> >>>>>> >>>>>> Summary: >>>>>> The pseudo-strings are currently not supported in >>>>>> reconstitution of constant pool. >>>>>> >>>>>> This is an explanation from John Rose about what the >>>>>> pseudo-strings are: >>>>>> >>>>>> "We still need "live" oop constants pre-linked into the >>>>>> constant pool of bytecodes which >>>>>> implement some method handles. We use the anonymous class >>>>>> pseudo-string feature for that. >>>>>> The relevant code is here: >>>>>> http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/tip/src/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java >>>>>> >>>>>> These oops are what "pseudo-strings" are. >>>>>> The odd name refers to the fact that, even though they are >>>>>> random oops, they appear in the constant pool >>>>>> where one would expect (because of class file syntax) to find >>>>>> a string." >>>>>> ... >>>>>> If you really wanted to reconstitute a class file for an >>>>>> anonymous class, and >>>>>> if that class has oop patching (pseudo-strings), you would >>>>>> need either to (a) reconstitute the patches array >>>>>> handed to Unsafe.defineAnonymousClass, or (b) accept whatever >>>>>> odd strings were there first, as an approximation. >>>>>> The "odd strings" are totally insignificant, and are >>>>>> typically something like "CONSTANT_PLACEHOLDER_42" >>>>>> (see java/lang/invoke/InvokerBytecodeGenerator.java)." >>>>>> >>>>>> >>>>>> Reconstitution of the ConstantPool is needed for both the >>>>>> JVMTI GetConstantPool() and RetransformClasses(). >>>>>> Finally, it goes to the ConstantPool::copy_cpool_bytes(). >>>>>> >>>>>> The problem is that a pseudo-string is a patched string that >>>>>> does not have >>>>>> a reference to the string symbol anymore: >>>>>> unresolved_string_at(idx) == NULL >>>>>> >>>>>> The fix is to create and fill in a map from >>>>>> JVM_CONSTANT_String cp index to the JVM_CONSTANT_Utf8 cp index >>>>>> to be able to restore this assotiation in the >>>>>> JvmtiConstantPoolReconstituter. >>>>>> >>>>>> Testing: >>>>>> Run: >>>>>> - java/lang/instrument tests >>>>>> - new jtreg test (see webrev) that was written by Filipp Zhinkin >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>> >>>> >>> >> > From serguei.spitsyn at oracle.com Fri Jan 16 22:38:33 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 16 Jan 2015 14:38:33 -0800 Subject: 4-th round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54B96392.8090904@oracle.com> References: <54B60308.4000105@oracle.com> <54B96301.40108@oracle.com> <54B96392.8090904@oracle.com> Message-ID: <54B992E9.4090904@oracle.com> Just wanted to tell that Dan is the second reviewer, so there is no pressure on others to review this. :) Thanks, Dan! Serguei On 1/16/15 11:16 AM, serguei.spitsyn at oracle.com wrote: > Sorry for the re-post. > Forgot to add review candidates to the to-list. > > Thanks, > Serguei > > On 1/16/15 11:14 AM, serguei.spitsyn at oracle.com wrote: >> Dan, David H. or David C., >> >> May I ask one of you to look at the webrev below? >> The issue itself is a little bit tricky, so it is not easy to review >> despite the small size. >> >> Coleen, >> >> Does the webrev matches what we discussed with you? >> Do you give me a thumbs up? >> >> Thanks, >> Serguei >> >> May I ask >> >> On 1/13/15 9:47 PM, serguei.spitsyn at oracle.com wrote: >>> Please, review the fix for: >>> https://bugs.openjdk.java.net/browse/JDK-8068162 >>> >>> >>> Open webrevs: >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>> >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/jdk/8068162-Test-IsModifiableAgent/ >>> >>> >>> >>> Summary: >>> >>> The sun.misc.Unsafe:throwIllegalAccessError() method is used in >>> place of a default >>> interface method in the itable if a default method was not >>> defined in the interface. >>> In fact, it happens for two interfaces that purhaps are >>> auto-generated: >>> java/nio/CharBuffer >>> java/nio/HeapCharBuffer >>> >>> This approach creates a problem when the class sun.misc.Unsafe is >>> retransformed. >>> The Method* pointer to the old (redefined) method in the itable >>> triggers an assert >>> (see the hs_err log in the bug report). >>> Coleen told me that a similar approach is going to be implemented >>> for some vtable entries. >>> Coleen, thanks for suggesting a better fix for this issue! >>> >>> The fix is to replace the old Unsafe method in the itable/vtable >>> with the latest method version. >>> >>> >>> Testing: >>> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >>> >>> >>> Thanks, >>> Serguei >>> >> > From coleen.phillimore at oracle.com Fri Jan 16 22:50:17 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 16 Jan 2015 17:50:17 -0500 Subject: 4-th round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54B992E9.4090904@oracle.com> References: <54B60308.4000105@oracle.com> <54B96301.40108@oracle.com> <54B96392.8090904@oracle.com> <54B992E9.4090904@oracle.com> Message-ID: <54B995A9.9030506@oracle.com> Hi Serguei, Sorry this got lost in my mailbox, twice. This change looks great! Thank you for re-enabling the test. Thanks, Coleen On 1/16/15, 5:38 PM, serguei.spitsyn at oracle.com wrote: > Just wanted to tell that Dan is the second reviewer, > so there is no pressure on others to review this. :) > > Thanks, Dan! > Serguei > > On 1/16/15 11:16 AM, serguei.spitsyn at oracle.com wrote: >> Sorry for the re-post. >> Forgot to add review candidates to the to-list. >> >> Thanks, >> Serguei >> >> On 1/16/15 11:14 AM, serguei.spitsyn at oracle.com wrote: >>> Dan, David H. or David C., >>> >>> May I ask one of you to look at the webrev below? >>> The issue itself is a little bit tricky, so it is not easy to review >>> despite the small size. >>> >>> Coleen, >>> >>> Does the webrev matches what we discussed with you? >>> Do you give me a thumbs up? >>> >>> Thanks, >>> Serguei >>> >>> May I ask >>> >>> On 1/13/15 9:47 PM, serguei.spitsyn at oracle.com wrote: >>>> Please, review the fix for: >>>> https://bugs.openjdk.java.net/browse/JDK-8068162 >>>> >>>> >>>> Open webrevs: >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>>> >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/jdk/8068162-Test-IsModifiableAgent/ >>>> >>>> >>>> >>>> Summary: >>>> >>>> The sun.misc.Unsafe:throwIllegalAccessError() method is used in >>>> place of a default >>>> interface method in the itable if a default method was not >>>> defined in the interface. >>>> In fact, it happens for two interfaces that purhaps are >>>> auto-generated: >>>> java/nio/CharBuffer >>>> java/nio/HeapCharBuffer >>>> >>>> This approach creates a problem when the class sun.misc.Unsafe >>>> is retransformed. >>>> The Method* pointer to the old (redefined) method in the itable >>>> triggers an assert >>>> (see the hs_err log in the bug report). >>>> Coleen told me that a similar approach is going to be >>>> implemented for some vtable entries. >>>> Coleen, thanks for suggesting a better fix for this issue! >>>> >>>> The fix is to replace the old Unsafe method in the itable/vtable >>>> with the latest method version. >>>> >>>> >>>> Testing: >>>> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >>> >> > From coleen.phillimore at oracle.com Fri Jan 16 22:53:54 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 16 Jan 2015 17:53:54 -0500 Subject: 3-rd round RFR (S) 8008678: JSR 292: constant pool reconstitution must support pseudo strings In-Reply-To: <54B96F97.5070204@oracle.com> References: <5475968C.40800@oracle.com> <54760B40.9040206@oracle.com> <54762FC9.2010800@oracle.com> <5491BCB9.10505@oracle.com> <5493299F.60005@oracle.com> <54934E73.4080808@oracle.com> <54B96F97.5070204@oracle.com> Message-ID: <54B99682.8070802@oracle.com> This change looks good to me also. Thanks, Coleen On 1/16/15, 3:07 PM, serguei.spitsyn at oracle.com wrote: > Please, review the fix for: > https://bugs.openjdk.java.net/browse/JDK-8008678 > > > Open webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8008678-JVMTI-pseudo.3/ > > > Summary: > Currently, a JVM_CONSTANT_String CP entry having a NULL reference > to Symbol* > indicates that it is a pseudo-string (patched string). > This creates nasty issues for the constant pool reconstitution. > > Current suggestion is to avoid having a NULL reference and retain > the original > Symbol* reference for pseudo-strings. The pseudo-string indication > will be > if the Utf8 representation starts from "CONSTANT_PLACEHOLDER_". > This approach makes the fix much simpler. > > I need a confirmation from the Compiler team that this won't break > any assumptions or invariants. > Big thanks to Coleen for previous round reviews and good advices! > > > Testing: > Run: > - java/lang/instrument tests > - new jtreg test (see webrev) that was written by Filipp Zhinkin > > > Thanks, > Serguei > > > On 12/18/14 2:00 PM, serguei.spitsyn at oracle.com wrote: >> Hi Coleen, >> >> Thank you for reviewing! >> >> >> On 12/18/14 11:23 AM, Coleen Phillimore wrote: >>> >>> Hi Serguei, >>> >>> Thank you for making the patches an optional field. >>> >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/oops/constantPool.hpp.sdiff.html >>> 198 if (!patched()) { >>> 199 assert(false, "a pseudo-string map may exists for patched CP only"); >>> 200 return 0; >>> 201 } >>> Why not >>> assert(patched(), "a pseud-string map must exist for >>> patched CP only"); >> >> Wanted it to be more reliable but it looks pointless. >> Will make this change. >> >>> >>> >>> Why is this? Is this really a ShouldNotReachHere? should it be false? >>> >>> 215 assert(true, "not found a matching entry in pseudo-string map"); >> >> >> A typo, must be false. >> It is the last minute change. >> Thanks for the catch! >> >> >>> >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/prims/jvmtiRedefineClasses.cpp.udiff.html >>> >>> Don't you have to move the value of the patched field from the old >>> constant pool to the new one? I hate to ask but is there merging >>> that needs to be done? I don't know how to write this test case >>> though. Is it possible to redefine a class with a constant pool >>> patches with another that has constant pool patches? >> >> Thank you for asking this question. >> If I understand correctly, the patching comes from the compiler side >> for anonymous classes only. >> I saw it for LambdaForm's only. >> I think, the patching can not be changed with a retransformation. >> But I'm not sure if it can not be changed with a redefinition. >> >> But if it can - then it would be safe to merge the 'patched' >> condition, i.e. make it patched >> if either the_class or scratch class is patched. >> >>> >>> Somehow I thought you'd have to save the value of the cp_patches >>> oops passed in. >>> >>> So I was wondering why you can't change this instead: >>> >>> bool is_pseudo_string_at(int which) { >>> // A pseudo string is a string that doesn't have a symbol in the >>> cpSlot >>> - return unresolved_string_at(which) == NULL; >>> + return (strncmp(unresolved_string_at(which)->as_utf8(), >>> "CONSTANT_PLACEHOLDER_" , strlen("CONSTANT_PLACEHOLDER")) == 0); >>> } >> >> I was thinking about the same but was not sure if it would work for >> the compiler team. >> We have to ask John about this (added John and Christian to the cc-list). >> This question to John was in my plan! :) >> >> The beauty of the above approach is that there is no need to create >> an intermediate >> pseudo-string map and most of the code in from this webrev is not needed. >> >> >> Thanks, >> Serguei >> >>> >>> And the asserts in the other functions below it. >>> >>> Coleen >>> >>> >>> On 12/17/14, 12:26 PM, serguei.spitsyn at oracle.com wrote: >>>> Please, review the second round fix for: >>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>> >>>> Open webrev: >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/ >>>> >>>> >>>> Summary: >>>> >>>> This fix implements a footprint saving approach suggested by Coleen. >>>> To be able to reconstitute a class constant pool, an intermediate >>>> pseudo-string map is used. >>>> Now, this field is accounted optionally, only if the 'cp_patches' >>>> is provided in the >>>> ClassFileParser::parseClassFile() before ConstantPool is allocated. >>>> This fix is not elegant, even a little bit ugly, but it is the >>>> only way I see so far. >>>> >>>> Unfortunately, this approach did not help much to make some other >>>> fields (eg., 'operands') optional. >>>> The problem is that we have to account optional fields before >>>> parsing, at the CP allocation time. >>>> It is possible to re-allocate the ConstantPool when any >>>> InvokeDynamic bytecode is discovered, >>>> but it looks too complicated. >>>> >>>> Testing: >>>> - the unit test from bug report >>>> - nsk.jvmti,testlist, nsk.jdi.testlist, JTREG java/lang/instrument >>>> - vm.mlvm.testlist, vm.quick.testlist, >>>> vm.parallel_class_loading.testlist (in progress) >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 11/26/14 11:53 AM, serguei.spitsyn at oracle.com wrote: >>>>> Coleen, >>>>> >>>>> Thank you for looking at this! >>>>> I'll check how this can be improved. >>>>> It is my concern too. >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> On 11/26/14 9:17 AM, Coleen Phillimore wrote: >>>>>> >>>>>> Serguei, >>>>>> I had a quick look at this. I was wondering if we could make the >>>>>> pseudo_string_map conditional in ConstantPool and not make all >>>>>> classes pay in footprint for this field? The same thing probably >>>>>> could be done for operands too. There are flags that you can set >>>>>> to conditionally add a pointer to base() in this function. >>>>>> >>>>>> Typical C++ would subclass ConstantPool to add >>>>>> InvokeDynamicConstantPool fields, but this is not typical C++ so >>>>>> the trick we use is like the one in ConstMethod. I think it's >>>>>> worth doing in this case. >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>> On 11/26/14, 3:59 AM, serguei.spitsyn at oracle.com wrote: >>>>>>> Please, review the fix for: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>>>> >>>>>>> >>>>>>> Open webrev: >>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.1/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> Summary: >>>>>>> The pseudo-strings are currently not supported in >>>>>>> reconstitution of constant pool. >>>>>>> >>>>>>> This is an explanation from John Rose about what the >>>>>>> pseudo-strings are: >>>>>>> >>>>>>> "We still need "live" oop constants pre-linked into the >>>>>>> constant pool of bytecodes which >>>>>>> implement some method handles. We use the anonymous class >>>>>>> pseudo-string feature for that. >>>>>>> The relevant code is here: >>>>>>> http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/tip/src/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java >>>>>>> >>>>>>> These oops are what "pseudo-strings" are. >>>>>>> The odd name refers to the fact that, even though they are >>>>>>> random oops, they appear in the constant pool >>>>>>> where one would expect (because of class file syntax) to >>>>>>> find a string." >>>>>>> ... >>>>>>> If you really wanted to reconstitute a class file for an >>>>>>> anonymous class, and >>>>>>> if that class has oop patching (pseudo-strings), you would >>>>>>> need either to (a) reconstitute the patches array >>>>>>> handed to Unsafe.defineAnonymousClass, or (b) accept >>>>>>> whatever odd strings were there first, as an approximation. >>>>>>> The "odd strings" are totally insignificant, and are >>>>>>> typically something like "CONSTANT_PLACEHOLDER_42" >>>>>>> (see java/lang/invoke/InvokerBytecodeGenerator.java)." >>>>>>> >>>>>>> >>>>>>> Reconstitution of the ConstantPool is needed for both the >>>>>>> JVMTI GetConstantPool() and RetransformClasses(). >>>>>>> Finally, it goes to the ConstantPool::copy_cpool_bytes(). >>>>>>> >>>>>>> The problem is that a pseudo-string is a patched string that >>>>>>> does not have >>>>>>> a reference to the string symbol anymore: >>>>>>> unresolved_string_at(idx) == NULL >>>>>>> >>>>>>> The fix is to create and fill in a map from >>>>>>> JVM_CONSTANT_String cp index to the JVM_CONSTANT_Utf8 cp index >>>>>>> to be able to restore this assotiation in the >>>>>>> JvmtiConstantPoolReconstituter. >>>>>>> >>>>>>> Testing: >>>>>>> Run: >>>>>>> - java/lang/instrument tests >>>>>>> - new jtreg test (see webrev) that was written by Filipp Zhinkin >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>> >>>>> >>>> >>> >> > From coleen.phillimore at oracle.com Fri Jan 16 22:56:11 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 16 Jan 2015 17:56:11 -0500 Subject: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF6FAFC@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF6DE04@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CF6FAFC@DEWDFEMB12A.global.corp.sap> Message-ID: <54B9970B.7040502@oracle.com> This seems fine. Someone from the serviceability group should probably sponsor. Thanks, Coleen On 1/16/15, 6:15 AM, Lindenmaier, Goetz wrote: > Hi, > > could I please get a review for this small fix? I also please need a sponsor. > > Thanks, > Goetz. > > From: goetz.lindenmaier at sap.com > Sent: Montag, 12. Januar 2015 09:23 > To: hotspot-dev at openjdk.java.net > Subject: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available > > Hi, > > This test uses SA, thus fails on platforms where that's not implemented. > I see problems on mac and aix. > Call Platform.shouldSAAttach() to check whether SA should work. > > Please review this small fix. I please need a sponsor. > http://cr.openjdk.java.net/~goetz/webrevs/8068778-jtregSA/webrev.01/ > > Best regards, > Goetz. From serguei.spitsyn at oracle.com Fri Jan 16 22:57:23 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 16 Jan 2015 14:57:23 -0800 Subject: 4-th round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54B995A9.9030506@oracle.com> References: <54B60308.4000105@oracle.com> <54B96301.40108@oracle.com> <54B96392.8090904@oracle.com> <54B992E9.4090904@oracle.com> <54B995A9.9030506@oracle.com> Message-ID: <54B99753.2070506@oracle.com> Thanks, Coleen! Serguei On 1/16/15 2:50 PM, Coleen Phillimore wrote: > > Hi Serguei, > > Sorry this got lost in my mailbox, twice. This change looks great! > Thank you for re-enabling the test. > > Thanks, > Coleen > > On 1/16/15, 5:38 PM, serguei.spitsyn at oracle.com wrote: >> Just wanted to tell that Dan is the second reviewer, >> so there is no pressure on others to review this. :) >> >> Thanks, Dan! >> Serguei >> >> On 1/16/15 11:16 AM, serguei.spitsyn at oracle.com wrote: >>> Sorry for the re-post. >>> Forgot to add review candidates to the to-list. >>> >>> Thanks, >>> Serguei >>> >>> On 1/16/15 11:14 AM, serguei.spitsyn at oracle.com wrote: >>>> Dan, David H. or David C., >>>> >>>> May I ask one of you to look at the webrev below? >>>> The issue itself is a little bit tricky, so it is not easy to >>>> review despite the small size. >>>> >>>> Coleen, >>>> >>>> Does the webrev matches what we discussed with you? >>>> Do you give me a thumbs up? >>>> >>>> Thanks, >>>> Serguei >>>> >>>> May I ask >>>> >>>> On 1/13/15 9:47 PM, serguei.spitsyn at oracle.com wrote: >>>>> Please, review the fix for: >>>>> https://bugs.openjdk.java.net/browse/JDK-8068162 >>>>> >>>>> >>>>> Open webrevs: >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>>>> >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/jdk/8068162-Test-IsModifiableAgent/ >>>>> >>>>> >>>>> >>>>> Summary: >>>>> >>>>> The sun.misc.Unsafe:throwIllegalAccessError() method is used in >>>>> place of a default >>>>> interface method in the itable if a default method was not >>>>> defined in the interface. >>>>> In fact, it happens for two interfaces that purhaps are >>>>> auto-generated: >>>>> java/nio/CharBuffer >>>>> java/nio/HeapCharBuffer >>>>> >>>>> This approach creates a problem when the class sun.misc.Unsafe >>>>> is retransformed. >>>>> The Method* pointer to the old (redefined) method in the itable >>>>> triggers an assert >>>>> (see the hs_err log in the bug report). >>>>> Coleen told me that a similar approach is going to be >>>>> implemented for some vtable entries. >>>>> Coleen, thanks for suggesting a better fix for this issue! >>>>> >>>>> The fix is to replace the old Unsafe method in the >>>>> itable/vtable with the latest method version. >>>>> >>>>> >>>>> Testing: >>>>> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>> >>> >> > From serguei.spitsyn at oracle.com Fri Jan 16 23:01:12 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 16 Jan 2015 15:01:12 -0800 Subject: 3-rd round RFR (S) 8008678: JSR 292: constant pool reconstitution must support pseudo strings In-Reply-To: <54B99682.8070802@oracle.com> References: <5475968C.40800@oracle.com> <54760B40.9040206@oracle.com> <54762FC9.2010800@oracle.com> <5491BCB9.10505@oracle.com> <5493299F.60005@oracle.com> <54934E73.4080808@oracle.com> <54B96F97.5070204@oracle.com> <54B99682.8070802@oracle.com> Message-ID: <54B99838.10601@oracle.com> John R. suggested to use the CPSlot(Symbol* ptr) to mark pseudo-strings. The updated webrev is going to be close to the .3 webrev. I will send it soon. Thanks, Serguei On 1/16/15 2:53 PM, Coleen Phillimore wrote: > > This change looks good to me also. > Thanks, > Coleen > > On 1/16/15, 3:07 PM, serguei.spitsyn at oracle.com wrote: >> Please, review the fix for: >> https://bugs.openjdk.java.net/browse/JDK-8008678 >> >> >> Open webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8008678-JVMTI-pseudo.3/ >> >> >> Summary: >> Currently, a JVM_CONSTANT_String CP entry having a NULL reference >> to Symbol* >> indicates that it is a pseudo-string (patched string). >> This creates nasty issues for the constant pool reconstitution. >> >> Current suggestion is to avoid having a NULL reference and retain >> the original >> Symbol* reference for pseudo-strings. The pseudo-string indication >> will be >> if the Utf8 representation starts from "CONSTANT_PLACEHOLDER_". >> This approach makes the fix much simpler. >> >> I need a confirmation from the Compiler team that this won't break >> any assumptions or invariants. >> Big thanks to Coleen for previous round reviews and good advices! >> >> >> Testing: >> Run: >> - java/lang/instrument tests >> - new jtreg test (see webrev) that was written by Filipp Zhinkin >> >> >> Thanks, >> Serguei >> >> >> On 12/18/14 2:00 PM, serguei.spitsyn at oracle.com wrote: >>> Hi Coleen, >>> >>> Thank you for reviewing! >>> >>> >>> On 12/18/14 11:23 AM, Coleen Phillimore wrote: >>>> >>>> Hi Serguei, >>>> >>>> Thank you for making the patches an optional field. >>>> >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/oops/constantPool.hpp.sdiff.html >>>> 198 if (!patched()) { >>>> 199 assert(false, "a pseudo-string map may exists for patched CP only"); >>>> 200 return 0; >>>> 201 } >>>> Why not >>>> assert(patched(), "a pseud-string map must exist >>>> for patched CP only"); >>> >>> Wanted it to be more reliable but it looks pointless. >>> Will make this change. >>> >>>> >>>> >>>> Why is this? Is this really a ShouldNotReachHere? should it be >>>> false? >>>> >>>> 215 assert(true, "not found a matching entry in pseudo-string map"); >>> >>> >>> A typo, must be false. >>> It is the last minute change. >>> Thanks for the catch! >>> >>> >>>> >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/prims/jvmtiRedefineClasses.cpp.udiff.html >>>> >>>> Don't you have to move the value of the patched field from the old >>>> constant pool to the new one? I hate to ask but is there merging >>>> that needs to be done? I don't know how to write this test case >>>> though. Is it possible to redefine a class with a constant pool >>>> patches with another that has constant pool patches? >>> >>> Thank you for asking this question. >>> If I understand correctly, the patching comes from the compiler side >>> for anonymous classes only. >>> I saw it for LambdaForm's only. >>> I think, the patching can not be changed with a retransformation. >>> But I'm not sure if it can not be changed with a redefinition. >>> >>> But if it can - then it would be safe to merge the 'patched' >>> condition, i.e. make it patched >>> if either the_class or scratch class is patched. >>> >>>> >>>> Somehow I thought you'd have to save the value of the cp_patches >>>> oops passed in. >>>> >>>> So I was wondering why you can't change this instead: >>>> >>>> bool is_pseudo_string_at(int which) { >>>> // A pseudo string is a string that doesn't have a symbol in >>>> the cpSlot >>>> - return unresolved_string_at(which) == NULL; >>>> + return (strncmp(unresolved_string_at(which)->as_utf8(), >>>> "CONSTANT_PLACEHOLDER_" , strlen("CONSTANT_PLACEHOLDER")) == 0); >>>> } >>> >>> I was thinking about the same but was not sure if it would work for >>> the compiler team. >>> We have to ask John about this (added John and Christian to the >>> cc-list). >>> This question to John was in my plan! :) >>> >>> The beauty of the above approach is that there is no need to create >>> an intermediate >>> pseudo-string map and most of the code in from this webrev is not >>> needed. >>> >>> >>> Thanks, >>> Serguei >>> >>>> >>>> And the asserts in the other functions below it. >>>> >>>> Coleen >>>> >>>> >>>> On 12/17/14, 12:26 PM, serguei.spitsyn at oracle.com wrote: >>>>> Please, review the second round fix for: >>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>> >>>>> Open webrev: >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/ >>>>> >>>>> >>>>> Summary: >>>>> >>>>> This fix implements a footprint saving approach suggested by Coleen. >>>>> To be able to reconstitute a class constant pool, an >>>>> intermediate pseudo-string map is used. >>>>> Now, this field is accounted optionally, only if the >>>>> 'cp_patches' is provided in the >>>>> ClassFileParser::parseClassFile() before ConstantPool is allocated. >>>>> This fix is not elegant, even a little bit ugly, but it is the >>>>> only way I see so far. >>>>> >>>>> Unfortunately, this approach did not help much to make some >>>>> other fields (eg., 'operands') optional. >>>>> The problem is that we have to account optional fields before >>>>> parsing, at the CP allocation time. >>>>> It is possible to re-allocate the ConstantPool when any >>>>> InvokeDynamic bytecode is discovered, >>>>> but it looks too complicated. >>>>> >>>>> Testing: >>>>> - the unit test from bug report >>>>> - nsk.jvmti,testlist, nsk.jdi.testlist, JTREG java/lang/instrument >>>>> - vm.mlvm.testlist, vm.quick.testlist, >>>>> vm.parallel_class_loading.testlist (in progress) >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 11/26/14 11:53 AM, serguei.spitsyn at oracle.com wrote: >>>>>> Coleen, >>>>>> >>>>>> Thank you for looking at this! >>>>>> I'll check how this can be improved. >>>>>> It is my concern too. >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> On 11/26/14 9:17 AM, Coleen Phillimore wrote: >>>>>>> >>>>>>> Serguei, >>>>>>> I had a quick look at this. I was wondering if we could make >>>>>>> the pseudo_string_map conditional in ConstantPool and not make >>>>>>> all classes pay in footprint for this field? The same thing >>>>>>> probably could be done for operands too. There are flags that >>>>>>> you can set to conditionally add a pointer to base() in this >>>>>>> function. >>>>>>> >>>>>>> Typical C++ would subclass ConstantPool to add >>>>>>> InvokeDynamicConstantPool fields, but this is not typical C++ so >>>>>>> the trick we use is like the one in ConstMethod. I think it's >>>>>>> worth doing in this case. >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> >>>>>>> On 11/26/14, 3:59 AM, serguei.spitsyn at oracle.com wrote: >>>>>>>> Please, review the fix for: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>>>>> >>>>>>>> >>>>>>>> Open webrev: >>>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.1/ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Summary: >>>>>>>> The pseudo-strings are currently not supported in >>>>>>>> reconstitution of constant pool. >>>>>>>> >>>>>>>> This is an explanation from John Rose about what the >>>>>>>> pseudo-strings are: >>>>>>>> >>>>>>>> "We still need "live" oop constants pre-linked into the >>>>>>>> constant pool of bytecodes which >>>>>>>> implement some method handles. We use the anonymous class >>>>>>>> pseudo-string feature for that. >>>>>>>> The relevant code is here: >>>>>>>> http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/tip/src/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java >>>>>>>> >>>>>>>> These oops are what "pseudo-strings" are. >>>>>>>> The odd name refers to the fact that, even though they are >>>>>>>> random oops, they appear in the constant pool >>>>>>>> where one would expect (because of class file syntax) to >>>>>>>> find a string." >>>>>>>> ... >>>>>>>> If you really wanted to reconstitute a class file for an >>>>>>>> anonymous class, and >>>>>>>> if that class has oop patching (pseudo-strings), you would >>>>>>>> need either to (a) reconstitute the patches array >>>>>>>> handed to Unsafe.defineAnonymousClass, or (b) accept >>>>>>>> whatever odd strings were there first, as an approximation. >>>>>>>> The "odd strings" are totally insignificant, and are >>>>>>>> typically something like "CONSTANT_PLACEHOLDER_42" >>>>>>>> (see java/lang/invoke/InvokerBytecodeGenerator.java)." >>>>>>>> >>>>>>>> >>>>>>>> Reconstitution of the ConstantPool is needed for both the >>>>>>>> JVMTI GetConstantPool() and RetransformClasses(). >>>>>>>> Finally, it goes to the ConstantPool::copy_cpool_bytes(). >>>>>>>> >>>>>>>> The problem is that a pseudo-string is a patched string that >>>>>>>> does not have >>>>>>>> a reference to the string symbol anymore: >>>>>>>> unresolved_string_at(idx) == NULL >>>>>>>> >>>>>>>> The fix is to create and fill in a map from >>>>>>>> JVM_CONSTANT_String cp index to the JVM_CONSTANT_Utf8 cp index >>>>>>>> to be able to restore this assotiation in the >>>>>>>> JvmtiConstantPoolReconstituter. >>>>>>>> >>>>>>>> Testing: >>>>>>>> Run: >>>>>>>> - java/lang/instrument tests >>>>>>>> - new jtreg test (see webrev) that was written by Filipp >>>>>>>> Zhinkin >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>> >>>>>> >>>>> >>>> >>> >> > From coleen.phillimore at oracle.com Fri Jan 16 23:03:51 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 16 Jan 2015 18:03:51 -0500 Subject: 3-rd round RFR (S) 8008678: JSR 292: constant pool reconstitution must support pseudo strings In-Reply-To: <54B99838.10601@oracle.com> References: <5475968C.40800@oracle.com> <54760B40.9040206@oracle.com> <54762FC9.2010800@oracle.com> <5491BCB9.10505@oracle.com> <5493299F.60005@oracle.com> <54934E73.4080808@oracle.com> <54B96F97.5070204@oracle.com> <54B99682.8070802@oracle.com> <54B99838.10601@oracle.com> Message-ID: <54B998D7.1020208@oracle.com> On 1/16/15, 6:01 PM, serguei.spitsyn at oracle.com wrote: > John R. suggested to use the CPSlot(Symbol* ptr) to mark pseudo-strings. I was sort of wondering about this along the same lines. You're setting the second bit, right? :) Coleen > The updated webrev is going to be close to the .3 webrev. > I will send it soon. > > Thanks, > Serguei > > On 1/16/15 2:53 PM, Coleen Phillimore wrote: >> >> This change looks good to me also. >> Thanks, >> Coleen >> >> On 1/16/15, 3:07 PM, serguei.spitsyn at oracle.com wrote: >>> Please, review the fix for: >>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>> >>> >>> Open webrev: >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8008678-JVMTI-pseudo.3/ >>> >>> >>> Summary: >>> Currently, a JVM_CONSTANT_String CP entry having a NULL reference >>> to Symbol* >>> indicates that it is a pseudo-string (patched string). >>> This creates nasty issues for the constant pool reconstitution. >>> >>> Current suggestion is to avoid having a NULL reference and retain >>> the original >>> Symbol* reference for pseudo-strings. The pseudo-string >>> indication will be >>> if the Utf8 representation starts from "CONSTANT_PLACEHOLDER_". >>> This approach makes the fix much simpler. >>> >>> I need a confirmation from the Compiler team that this won't >>> break any assumptions or invariants. >>> Big thanks to Coleen for previous round reviews and good advices! >>> >>> >>> Testing: >>> Run: >>> - java/lang/instrument tests >>> - new jtreg test (see webrev) that was written by Filipp Zhinkin >>> >>> >>> Thanks, >>> Serguei >>> >>> >>> On 12/18/14 2:00 PM, serguei.spitsyn at oracle.com wrote: >>>> Hi Coleen, >>>> >>>> Thank you for reviewing! >>>> >>>> >>>> On 12/18/14 11:23 AM, Coleen Phillimore wrote: >>>>> >>>>> Hi Serguei, >>>>> >>>>> Thank you for making the patches an optional field. >>>>> >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/oops/constantPool.hpp.sdiff.html >>>>> 198 if (!patched()) { >>>>> 199 assert(false, "a pseudo-string map may exists for patched CP only"); >>>>> 200 return 0; >>>>> 201 } >>>>> Why not >>>>> assert(patched(), "a pseud-string map must exist >>>>> for patched CP only"); >>>> >>>> Wanted it to be more reliable but it looks pointless. >>>> Will make this change. >>>> >>>>> >>>>> >>>>> Why is this? Is this really a ShouldNotReachHere? should it be >>>>> false? >>>>> >>>>> 215 assert(true, "not found a matching entry in pseudo-string map"); >>>> >>>> >>>> A typo, must be false. >>>> It is the last minute change. >>>> Thanks for the catch! >>>> >>>> >>>>> >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/prims/jvmtiRedefineClasses.cpp.udiff.html >>>>> >>>>> Don't you have to move the value of the patched field from the old >>>>> constant pool to the new one? I hate to ask but is there merging >>>>> that needs to be done? I don't know how to write this test case >>>>> though. Is it possible to redefine a class with a constant pool >>>>> patches with another that has constant pool patches? >>>> >>>> Thank you for asking this question. >>>> If I understand correctly, the patching comes from the compiler >>>> side for anonymous classes only. >>>> I saw it for LambdaForm's only. >>>> I think, the patching can not be changed with a retransformation. >>>> But I'm not sure if it can not be changed with a redefinition. >>>> >>>> But if it can - then it would be safe to merge the 'patched' >>>> condition, i.e. make it patched >>>> if either the_class or scratch class is patched. >>>> >>>>> >>>>> Somehow I thought you'd have to save the value of the cp_patches >>>>> oops passed in. >>>>> >>>>> So I was wondering why you can't change this instead: >>>>> >>>>> bool is_pseudo_string_at(int which) { >>>>> // A pseudo string is a string that doesn't have a symbol in >>>>> the cpSlot >>>>> - return unresolved_string_at(which) == NULL; >>>>> + return (strncmp(unresolved_string_at(which)->as_utf8(), >>>>> "CONSTANT_PLACEHOLDER_" , strlen("CONSTANT_PLACEHOLDER")) == 0); >>>>> } >>>> >>>> I was thinking about the same but was not sure if it would work for >>>> the compiler team. >>>> We have to ask John about this (added John and Christian to the >>>> cc-list). >>>> This question to John was in my plan! :) >>>> >>>> The beauty of the above approach is that there is no need to create >>>> an intermediate >>>> pseudo-string map and most of the code in from this webrev is not >>>> needed. >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>>> >>>>> And the asserts in the other functions below it. >>>>> >>>>> Coleen >>>>> >>>>> >>>>> On 12/17/14, 12:26 PM, serguei.spitsyn at oracle.com wrote: >>>>>> Please, review the second round fix for: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>>> >>>>>> Open webrev: >>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/ >>>>>> >>>>>> >>>>>> Summary: >>>>>> >>>>>> This fix implements a footprint saving approach suggested by >>>>>> Coleen. >>>>>> To be able to reconstitute a class constant pool, an >>>>>> intermediate pseudo-string map is used. >>>>>> Now, this field is accounted optionally, only if the >>>>>> 'cp_patches' is provided in the >>>>>> ClassFileParser::parseClassFile() before ConstantPool is allocated. >>>>>> This fix is not elegant, even a little bit ugly, but it is the >>>>>> only way I see so far. >>>>>> >>>>>> Unfortunately, this approach did not help much to make some >>>>>> other fields (eg., 'operands') optional. >>>>>> The problem is that we have to account optional fields before >>>>>> parsing, at the CP allocation time. >>>>>> It is possible to re-allocate the ConstantPool when any >>>>>> InvokeDynamic bytecode is discovered, >>>>>> but it looks too complicated. >>>>>> >>>>>> Testing: >>>>>> - the unit test from bug report >>>>>> - nsk.jvmti,testlist, nsk.jdi.testlist, JTREG java/lang/instrument >>>>>> - vm.mlvm.testlist, vm.quick.testlist, >>>>>> vm.parallel_class_loading.testlist (in progress) >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 11/26/14 11:53 AM, serguei.spitsyn at oracle.com wrote: >>>>>>> Coleen, >>>>>>> >>>>>>> Thank you for looking at this! >>>>>>> I'll check how this can be improved. >>>>>>> It is my concern too. >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> On 11/26/14 9:17 AM, Coleen Phillimore wrote: >>>>>>>> >>>>>>>> Serguei, >>>>>>>> I had a quick look at this. I was wondering if we could make >>>>>>>> the pseudo_string_map conditional in ConstantPool and not make >>>>>>>> all classes pay in footprint for this field? The same thing >>>>>>>> probably could be done for operands too. There are flags that >>>>>>>> you can set to conditionally add a pointer to base() in this >>>>>>>> function. >>>>>>>> >>>>>>>> Typical C++ would subclass ConstantPool to add >>>>>>>> InvokeDynamicConstantPool fields, but this is not typical C++ >>>>>>>> so the trick we use is like the one in ConstMethod. I think >>>>>>>> it's worth doing in this case. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>>>>> >>>>>>>> On 11/26/14, 3:59 AM, serguei.spitsyn at oracle.com wrote: >>>>>>>>> Please, review the fix for: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>>>>>> >>>>>>>>> >>>>>>>>> Open webrev: >>>>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.1/ >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Summary: >>>>>>>>> The pseudo-strings are currently not supported in >>>>>>>>> reconstitution of constant pool. >>>>>>>>> >>>>>>>>> This is an explanation from John Rose about what the >>>>>>>>> pseudo-strings are: >>>>>>>>> >>>>>>>>> "We still need "live" oop constants pre-linked into the >>>>>>>>> constant pool of bytecodes which >>>>>>>>> implement some method handles. We use the anonymous class >>>>>>>>> pseudo-string feature for that. >>>>>>>>> The relevant code is here: >>>>>>>>> http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/tip/src/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java >>>>>>>>> >>>>>>>>> These oops are what "pseudo-strings" are. >>>>>>>>> The odd name refers to the fact that, even though they are >>>>>>>>> random oops, they appear in the constant pool >>>>>>>>> where one would expect (because of class file syntax) to >>>>>>>>> find a string." >>>>>>>>> ... >>>>>>>>> If you really wanted to reconstitute a class file for an >>>>>>>>> anonymous class, and >>>>>>>>> if that class has oop patching (pseudo-strings), you would >>>>>>>>> need either to (a) reconstitute the patches array >>>>>>>>> handed to Unsafe.defineAnonymousClass, or (b) accept >>>>>>>>> whatever odd strings were there first, as an approximation. >>>>>>>>> The "odd strings" are totally insignificant, and are >>>>>>>>> typically something like "CONSTANT_PLACEHOLDER_42" >>>>>>>>> (see java/lang/invoke/InvokerBytecodeGenerator.java)." >>>>>>>>> >>>>>>>>> >>>>>>>>> Reconstitution of the ConstantPool is needed for both the >>>>>>>>> JVMTI GetConstantPool() and RetransformClasses(). >>>>>>>>> Finally, it goes to the ConstantPool::copy_cpool_bytes(). >>>>>>>>> >>>>>>>>> The problem is that a pseudo-string is a patched string >>>>>>>>> that does not have >>>>>>>>> a reference to the string symbol anymore: >>>>>>>>> unresolved_string_at(idx) == NULL >>>>>>>>> >>>>>>>>> The fix is to create and fill in a map from >>>>>>>>> JVM_CONSTANT_String cp index to the JVM_CONSTANT_Utf8 cp index >>>>>>>>> to be able to restore this assotiation in the >>>>>>>>> JvmtiConstantPoolReconstituter. >>>>>>>>> >>>>>>>>> Testing: >>>>>>>>> Run: >>>>>>>>> - java/lang/instrument tests >>>>>>>>> - new jtreg test (see webrev) that was written by Filipp >>>>>>>>> Zhinkin >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Serguei >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From serguei.spitsyn at oracle.com Fri Jan 16 23:10:13 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 16 Jan 2015 15:10:13 -0800 Subject: 3-rd round RFR (S) 8008678: JSR 292: constant pool reconstitution must support pseudo strings In-Reply-To: <54B998D7.1020208@oracle.com> References: <5475968C.40800@oracle.com> <54760B40.9040206@oracle.com> <54762FC9.2010800@oracle.com> <5491BCB9.10505@oracle.com> <5493299F.60005@oracle.com> <54934E73.4080808@oracle.com> <54B96F97.5070204@oracle.com> <54B99682.8070802@oracle.com> <54B99838.10601@oracle.com> <54B998D7.1020208@oracle.com> Message-ID: <54B99A55.6000002@oracle.com> On 1/16/15 3:03 PM, Coleen Phillimore wrote: > > On 1/16/15, 6:01 PM, serguei.spitsyn at oracle.com wrote: >> John R. suggested to use the CPSlot(Symbol* ptr) to mark pseudo-strings. > > I was sort of wondering about this along the same lines. You're > setting the second bit, right? :) I'm not sure yet. I'll check if setting the first bit would not create an ambiguity. Otherwise, will set the second one. Thanks, Serguei > Coleen > >> The updated webrev is going to be close to the .3 webrev. >> I will send it soon. >> >> Thanks, >> Serguei >> >> On 1/16/15 2:53 PM, Coleen Phillimore wrote: >>> >>> This change looks good to me also. >>> Thanks, >>> Coleen >>> >>> On 1/16/15, 3:07 PM, serguei.spitsyn at oracle.com wrote: >>>> Please, review the fix for: >>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>> >>>> >>>> Open webrev: >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8008678-JVMTI-pseudo.3/ >>>> >>>> >>>> Summary: >>>> Currently, a JVM_CONSTANT_String CP entry having a NULL >>>> reference to Symbol* >>>> indicates that it is a pseudo-string (patched string). >>>> This creates nasty issues for the constant pool reconstitution. >>>> >>>> Current suggestion is to avoid having a NULL reference and >>>> retain the original >>>> Symbol* reference for pseudo-strings. The pseudo-string >>>> indication will be >>>> if the Utf8 representation starts from "CONSTANT_PLACEHOLDER_". >>>> This approach makes the fix much simpler. >>>> >>>> I need a confirmation from the Compiler team that this won't >>>> break any assumptions or invariants. >>>> Big thanks to Coleen for previous round reviews and good advices! >>>> >>>> >>>> Testing: >>>> Run: >>>> - java/lang/instrument tests >>>> - new jtreg test (see webrev) that was written by Filipp Zhinkin >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 12/18/14 2:00 PM, serguei.spitsyn at oracle.com wrote: >>>>> Hi Coleen, >>>>> >>>>> Thank you for reviewing! >>>>> >>>>> >>>>> On 12/18/14 11:23 AM, Coleen Phillimore wrote: >>>>>> >>>>>> Hi Serguei, >>>>>> >>>>>> Thank you for making the patches an optional field. >>>>>> >>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/oops/constantPool.hpp.sdiff.html >>>>>> 198 if (!patched()) { >>>>>> 199 assert(false, "a pseudo-string map may exists for patched CP only"); >>>>>> 200 return 0; >>>>>> 201 } >>>>>> Why not >>>>>> assert(patched(), "a pseud-string map must exist >>>>>> for patched CP only"); >>>>> >>>>> Wanted it to be more reliable but it looks pointless. >>>>> Will make this change. >>>>> >>>>>> >>>>>> >>>>>> Why is this? Is this really a ShouldNotReachHere? should it be >>>>>> false? >>>>>> >>>>>> 215 assert(true, "not found a matching entry in pseudo-string map"); >>>>> >>>>> >>>>> A typo, must be false. >>>>> It is the last minute change. >>>>> Thanks for the catch! >>>>> >>>>> >>>>>> >>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/prims/jvmtiRedefineClasses.cpp.udiff.html >>>>>> >>>>>> Don't you have to move the value of the patched field from the >>>>>> old constant pool to the new one? I hate to ask but is there >>>>>> merging that needs to be done? I don't know how to write this >>>>>> test case though. Is it possible to redefine a class with a >>>>>> constant pool patches with another that has constant pool patches? >>>>> >>>>> Thank you for asking this question. >>>>> If I understand correctly, the patching comes from the compiler >>>>> side for anonymous classes only. >>>>> I saw it for LambdaForm's only. >>>>> I think, the patching can not be changed with a retransformation. >>>>> But I'm not sure if it can not be changed with a redefinition. >>>>> >>>>> But if it can - then it would be safe to merge the 'patched' >>>>> condition, i.e. make it patched >>>>> if either the_class or scratch class is patched. >>>>> >>>>>> >>>>>> Somehow I thought you'd have to save the value of the cp_patches >>>>>> oops passed in. >>>>>> >>>>>> So I was wondering why you can't change this instead: >>>>>> >>>>>> bool is_pseudo_string_at(int which) { >>>>>> // A pseudo string is a string that doesn't have a symbol in >>>>>> the cpSlot >>>>>> - return unresolved_string_at(which) == NULL; >>>>>> + return (strncmp(unresolved_string_at(which)->as_utf8(), >>>>>> "CONSTANT_PLACEHOLDER_" , strlen("CONSTANT_PLACEHOLDER")) == 0); >>>>>> } >>>>> >>>>> I was thinking about the same but was not sure if it would work >>>>> for the compiler team. >>>>> We have to ask John about this (added John and Christian to the >>>>> cc-list). >>>>> This question to John was in my plan! :) >>>>> >>>>> The beauty of the above approach is that there is no need to >>>>> create an intermediate >>>>> pseudo-string map and most of the code in from this webrev is not >>>>> needed. >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>>> >>>>>> And the asserts in the other functions below it. >>>>>> >>>>>> Coleen >>>>>> >>>>>> >>>>>> On 12/17/14, 12:26 PM, serguei.spitsyn at oracle.com wrote: >>>>>>> Please, review the second round fix for: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>>>> >>>>>>> Open webrev: >>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/ >>>>>>> >>>>>>> >>>>>>> Summary: >>>>>>> >>>>>>> This fix implements a footprint saving approach suggested by >>>>>>> Coleen. >>>>>>> To be able to reconstitute a class constant pool, an >>>>>>> intermediate pseudo-string map is used. >>>>>>> Now, this field is accounted optionally, only if the >>>>>>> 'cp_patches' is provided in the >>>>>>> ClassFileParser::parseClassFile() before ConstantPool is >>>>>>> allocated. >>>>>>> This fix is not elegant, even a little bit ugly, but it is the >>>>>>> only way I see so far. >>>>>>> >>>>>>> Unfortunately, this approach did not help much to make some >>>>>>> other fields (eg., 'operands') optional. >>>>>>> The problem is that we have to account optional fields before >>>>>>> parsing, at the CP allocation time. >>>>>>> It is possible to re-allocate the ConstantPool when any >>>>>>> InvokeDynamic bytecode is discovered, >>>>>>> but it looks too complicated. >>>>>>> >>>>>>> Testing: >>>>>>> - the unit test from bug report >>>>>>> - nsk.jvmti,testlist, nsk.jdi.testlist, JTREG java/lang/instrument >>>>>>> - vm.mlvm.testlist, vm.quick.testlist, >>>>>>> vm.parallel_class_loading.testlist (in progress) >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 11/26/14 11:53 AM, serguei.spitsyn at oracle.com wrote: >>>>>>>> Coleen, >>>>>>>> >>>>>>>> Thank you for looking at this! >>>>>>>> I'll check how this can be improved. >>>>>>>> It is my concern too. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> On 11/26/14 9:17 AM, Coleen Phillimore wrote: >>>>>>>>> >>>>>>>>> Serguei, >>>>>>>>> I had a quick look at this. I was wondering if we could make >>>>>>>>> the pseudo_string_map conditional in ConstantPool and not make >>>>>>>>> all classes pay in footprint for this field? The same thing >>>>>>>>> probably could be done for operands too. There are flags that >>>>>>>>> you can set to conditionally add a pointer to base() in this >>>>>>>>> function. >>>>>>>>> >>>>>>>>> Typical C++ would subclass ConstantPool to add >>>>>>>>> InvokeDynamicConstantPool fields, but this is not typical C++ >>>>>>>>> so the trick we use is like the one in ConstMethod. I think >>>>>>>>> it's worth doing in this case. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Coleen >>>>>>>>> >>>>>>>>> On 11/26/14, 3:59 AM, serguei.spitsyn at oracle.com wrote: >>>>>>>>>> Please, review the fix for: >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Open webrev: >>>>>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.1/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Summary: >>>>>>>>>> The pseudo-strings are currently not supported in >>>>>>>>>> reconstitution of constant pool. >>>>>>>>>> >>>>>>>>>> This is an explanation from John Rose about what the >>>>>>>>>> pseudo-strings are: >>>>>>>>>> >>>>>>>>>> "We still need "live" oop constants pre-linked into the >>>>>>>>>> constant pool of bytecodes which >>>>>>>>>> implement some method handles. We use the anonymous class >>>>>>>>>> pseudo-string feature for that. >>>>>>>>>> The relevant code is here: >>>>>>>>>> http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/tip/src/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java >>>>>>>>>> >>>>>>>>>> These oops are what "pseudo-strings" are. >>>>>>>>>> The odd name refers to the fact that, even though they >>>>>>>>>> are random oops, they appear in the constant pool >>>>>>>>>> where one would expect (because of class file syntax) to >>>>>>>>>> find a string." >>>>>>>>>> ... >>>>>>>>>> If you really wanted to reconstitute a class file for an >>>>>>>>>> anonymous class, and >>>>>>>>>> if that class has oop patching (pseudo-strings), you >>>>>>>>>> would need either to (a) reconstitute the patches array >>>>>>>>>> handed to Unsafe.defineAnonymousClass, or (b) accept >>>>>>>>>> whatever odd strings were there first, as an approximation. >>>>>>>>>> The "odd strings" are totally insignificant, and are >>>>>>>>>> typically something like "CONSTANT_PLACEHOLDER_42" >>>>>>>>>> (see java/lang/invoke/InvokerBytecodeGenerator.java)." >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Reconstitution of the ConstantPool is needed for both the >>>>>>>>>> JVMTI GetConstantPool() and RetransformClasses(). >>>>>>>>>> Finally, it goes to the ConstantPool::copy_cpool_bytes(). >>>>>>>>>> >>>>>>>>>> The problem is that a pseudo-string is a patched string >>>>>>>>>> that does not have >>>>>>>>>> a reference to the string symbol anymore: >>>>>>>>>> unresolved_string_at(idx) == NULL >>>>>>>>>> >>>>>>>>>> The fix is to create and fill in a map from >>>>>>>>>> JVM_CONSTANT_String cp index to the JVM_CONSTANT_Utf8 cp index >>>>>>>>>> to be able to restore this assotiation in the >>>>>>>>>> JvmtiConstantPoolReconstituter. >>>>>>>>>> >>>>>>>>>> Testing: >>>>>>>>>> Run: >>>>>>>>>> - java/lang/instrument tests >>>>>>>>>> - new jtreg test (see webrev) that was written by Filipp >>>>>>>>>> Zhinkin >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Serguei >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From john.r.rose at oracle.com Fri Jan 16 23:32:33 2015 From: john.r.rose at oracle.com (John Rose) Date: Fri, 16 Jan 2015 15:32:33 -0800 Subject: 3-rd round RFR (S) 8008678: JSR 292: constant pool reconstitution must support pseudo strings In-Reply-To: <54B998D7.1020208@oracle.com> References: <5475968C.40800@oracle.com> <54760B40.9040206@oracle.com> <54762FC9.2010800@oracle.com> <5491BCB9.10505@oracle.com> <5493299F.60005@oracle.com> <54934E73.4080808@oracle.com> <54B96F97.5070204@oracle.com> <54B99682.8070802@oracle.com> <54B99838.10601@oracle.com> <54B998D7.1020208@oracle.com> Message-ID: <82E12F9C-BE5E-4B66-B477-9F90956ECD6D@oracle.com> On Jan 16, 2015, at 3:03 PM, Coleen Phillimore wrote: > > I was sort of wondering about this along the same lines. You're setting the second bit, right? :) That sounds good, much better than the string prefix hack. Parsing the string would introduce too much coupling between the JVM and random details of the JDK. I also suggested to Serguei that redefinition of patched classes is not going to happen, since they are not user-visible. So all of the "EMCP" logic can punt on pseudo-strings, one way or another, if it needs to. BTW, since patched classes are always anonymous classes, you can't even get to them, unless you dig them out some private place like a lambda form. Reconstitution is also a lost cause, since the patches will be dropped, but I see how that might be something you might possibly run into, as a stress test or some sort of debugger display. ? John From serguei.spitsyn at oracle.com Fri Jan 16 23:35:19 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 16 Jan 2015 15:35:19 -0800 Subject: 3-rd round RFR (S) 8008678: JSR 292: constant pool reconstitution must support pseudo strings In-Reply-To: <54B99A55.6000002@oracle.com> References: <5475968C.40800@oracle.com> <54760B40.9040206@oracle.com> <54762FC9.2010800@oracle.com> <5491BCB9.10505@oracle.com> <5493299F.60005@oracle.com> <54934E73.4080808@oracle.com> <54B96F97.5070204@oracle.com> <54B99682.8070802@oracle.com> <54B99838.10601@oracle.com> <54B998D7.1020208@oracle.com> <54B99A55.6000002@oracle.com> Message-ID: <54B9A037.6040007@oracle.com> On 1/16/15 3:10 PM, serguei.spitsyn at oracle.com wrote: > On 1/16/15 3:03 PM, Coleen Phillimore wrote: >> >> On 1/16/15, 6:01 PM, serguei.spitsyn at oracle.com wrote: >>> John R. suggested to use the CPSlot(Symbol* ptr) to mark pseudo-strings. >> >> I was sort of wondering about this along the same lines. You're >> setting the second bit, right? :) > > I'm not sure yet. > I'll check if setting the first bit would not create an ambiguity. > Otherwise, will set the second one. In fact, the suggestion was to look at the PCSlot as an example. I'm not sure this class is good to extend or reuse for pseudo-strings as it was added for klass entries only. > > Thanks, > Serguei > >> Coleen >> >>> The updated webrev is going to be close to the .3 webrev. >>> I will send it soon. >>> >>> Thanks, >>> Serguei >>> >>> On 1/16/15 2:53 PM, Coleen Phillimore wrote: >>>> >>>> This change looks good to me also. >>>> Thanks, >>>> Coleen >>>> >>>> On 1/16/15, 3:07 PM, serguei.spitsyn at oracle.com wrote: >>>>> Please, review the fix for: >>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>> >>>>> >>>>> Open webrev: >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8008678-JVMTI-pseudo.3/ >>>>> >>>>> >>>>> Summary: >>>>> Currently, a JVM_CONSTANT_String CP entry having a NULL >>>>> reference to Symbol* >>>>> indicates that it is a pseudo-string (patched string). >>>>> This creates nasty issues for the constant pool reconstitution. >>>>> >>>>> Current suggestion is to avoid having a NULL reference and >>>>> retain the original >>>>> Symbol* reference for pseudo-strings. The pseudo-string >>>>> indication will be >>>>> if the Utf8 representation starts from "CONSTANT_PLACEHOLDER_". >>>>> This approach makes the fix much simpler. >>>>> >>>>> I need a confirmation from the Compiler team that this won't >>>>> break any assumptions or invariants. >>>>> Big thanks to Coleen for previous round reviews and good advices! >>>>> >>>>> >>>>> Testing: >>>>> Run: >>>>> - java/lang/instrument tests >>>>> - new jtreg test (see webrev) that was written by Filipp Zhinkin >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 12/18/14 2:00 PM, serguei.spitsyn at oracle.com wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> Thank you for reviewing! >>>>>> >>>>>> >>>>>> On 12/18/14 11:23 AM, Coleen Phillimore wrote: >>>>>>> >>>>>>> Hi Serguei, >>>>>>> >>>>>>> Thank you for making the patches an optional field. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/oops/constantPool.hpp.sdiff.html >>>>>>> 198 if (!patched()) { >>>>>>> 199 assert(false, "a pseudo-string map may exists for patched CP only"); >>>>>>> 200 return 0; >>>>>>> 201 } >>>>>>> Why not >>>>>>> assert(patched(), "a pseud-string map must exist >>>>>>> for patched CP only"); >>>>>> >>>>>> Wanted it to be more reliable but it looks pointless. >>>>>> Will make this change. >>>>>> >>>>>>> >>>>>>> >>>>>>> Why is this? Is this really a ShouldNotReachHere? should it be >>>>>>> false? >>>>>>> >>>>>>> 215 assert(true, "not found a matching entry in pseudo-string map"); >>>>>> >>>>>> >>>>>> A typo, must be false. >>>>>> It is the last minute change. >>>>>> Thanks for the catch! >>>>>> >>>>>> >>>>>>> >>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/prims/jvmtiRedefineClasses.cpp.udiff.html >>>>>>> >>>>>>> Don't you have to move the value of the patched field from the >>>>>>> old constant pool to the new one? I hate to ask but is there >>>>>>> merging that needs to be done? I don't know how to write this >>>>>>> test case though. Is it possible to redefine a class with a >>>>>>> constant pool patches with another that has constant pool patches? >>>>>> >>>>>> Thank you for asking this question. >>>>>> If I understand correctly, the patching comes from the compiler >>>>>> side for anonymous classes only. >>>>>> I saw it for LambdaForm's only. >>>>>> I think, the patching can not be changed with a retransformation. >>>>>> But I'm not sure if it can not be changed with a redefinition. >>>>>> >>>>>> But if it can - then it would be safe to merge the 'patched' >>>>>> condition, i.e. make it patched >>>>>> if either the_class or scratch class is patched. >>>>>> >>>>>>> >>>>>>> Somehow I thought you'd have to save the value of the cp_patches >>>>>>> oops passed in. >>>>>>> >>>>>>> So I was wondering why you can't change this instead: >>>>>>> >>>>>>> bool is_pseudo_string_at(int which) { >>>>>>> // A pseudo string is a string that doesn't have a symbol in >>>>>>> the cpSlot >>>>>>> - return unresolved_string_at(which) == NULL; >>>>>>> + return (strncmp(unresolved_string_at(which)->as_utf8(), >>>>>>> "CONSTANT_PLACEHOLDER_" , strlen("CONSTANT_PLACEHOLDER")) == 0); >>>>>>> } >>>>>> >>>>>> I was thinking about the same but was not sure if it would work >>>>>> for the compiler team. >>>>>> We have to ask John about this (added John and Christian to the >>>>>> cc-list). >>>>>> This question to John was in my plan! :) >>>>>> >>>>>> The beauty of the above approach is that there is no need to >>>>>> create an intermediate >>>>>> pseudo-string map and most of the code in from this webrev is not >>>>>> needed. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>>> >>>>>>> And the asserts in the other functions below it. >>>>>>> >>>>>>> Coleen >>>>>>> >>>>>>> >>>>>>> On 12/17/14, 12:26 PM, serguei.spitsyn at oracle.com wrote: >>>>>>>> Please, review the second round fix for: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>>>>> >>>>>>>> Open webrev: >>>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/ >>>>>>>> >>>>>>>> >>>>>>>> Summary: >>>>>>>> >>>>>>>> This fix implements a footprint saving approach suggested by >>>>>>>> Coleen. >>>>>>>> To be able to reconstitute a class constant pool, an >>>>>>>> intermediate pseudo-string map is used. >>>>>>>> Now, this field is accounted optionally, only if the >>>>>>>> 'cp_patches' is provided in the >>>>>>>> ClassFileParser::parseClassFile() before ConstantPool is >>>>>>>> allocated. >>>>>>>> This fix is not elegant, even a little bit ugly, but it is >>>>>>>> the only way I see so far. >>>>>>>> >>>>>>>> Unfortunately, this approach did not help much to make some >>>>>>>> other fields (eg., 'operands') optional. >>>>>>>> The problem is that we have to account optional fields before >>>>>>>> parsing, at the CP allocation time. >>>>>>>> It is possible to re-allocate the ConstantPool when any >>>>>>>> InvokeDynamic bytecode is discovered, >>>>>>>> but it looks too complicated. >>>>>>>> >>>>>>>> Testing: >>>>>>>> - the unit test from bug report >>>>>>>> - nsk.jvmti,testlist, nsk.jdi.testlist, JTREG >>>>>>>> java/lang/instrument >>>>>>>> - vm.mlvm.testlist, vm.quick.testlist, >>>>>>>> vm.parallel_class_loading.testlist (in progress) >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> >>>>>>>> On 11/26/14 11:53 AM, serguei.spitsyn at oracle.com wrote: >>>>>>>>> Coleen, >>>>>>>>> >>>>>>>>> Thank you for looking at this! >>>>>>>>> I'll check how this can be improved. >>>>>>>>> It is my concern too. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Serguei >>>>>>>>> >>>>>>>>> On 11/26/14 9:17 AM, Coleen Phillimore wrote: >>>>>>>>>> >>>>>>>>>> Serguei, >>>>>>>>>> I had a quick look at this. I was wondering if we could make >>>>>>>>>> the pseudo_string_map conditional in ConstantPool and not >>>>>>>>>> make all classes pay in footprint for this field? The same >>>>>>>>>> thing probably could be done for operands too. There are >>>>>>>>>> flags that you can set to conditionally add a pointer to >>>>>>>>>> base() in this function. >>>>>>>>>> >>>>>>>>>> Typical C++ would subclass ConstantPool to add >>>>>>>>>> InvokeDynamicConstantPool fields, but this is not typical C++ >>>>>>>>>> so the trick we use is like the one in ConstMethod. I think >>>>>>>>>> it's worth doing in this case. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Coleen >>>>>>>>>> >>>>>>>>>> On 11/26/14, 3:59 AM, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>> Please, review the fix for: >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Open webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.1/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Summary: >>>>>>>>>>> The pseudo-strings are currently not supported in >>>>>>>>>>> reconstitution of constant pool. >>>>>>>>>>> >>>>>>>>>>> This is an explanation from John Rose about what the >>>>>>>>>>> pseudo-strings are: >>>>>>>>>>> >>>>>>>>>>> "We still need "live" oop constants pre-linked into the >>>>>>>>>>> constant pool of bytecodes which >>>>>>>>>>> implement some method handles. We use the anonymous >>>>>>>>>>> class pseudo-string feature for that. >>>>>>>>>>> The relevant code is here: >>>>>>>>>>> http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/tip/src/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java >>>>>>>>>>> >>>>>>>>>>> These oops are what "pseudo-strings" are. >>>>>>>>>>> The odd name refers to the fact that, even though they >>>>>>>>>>> are random oops, they appear in the constant pool >>>>>>>>>>> where one would expect (because of class file syntax) to >>>>>>>>>>> find a string." >>>>>>>>>>> ... >>>>>>>>>>> If you really wanted to reconstitute a class file for an >>>>>>>>>>> anonymous class, and >>>>>>>>>>> if that class has oop patching (pseudo-strings), you >>>>>>>>>>> would need either to (a) reconstitute the patches array >>>>>>>>>>> handed to Unsafe.defineAnonymousClass, or (b) accept >>>>>>>>>>> whatever odd strings were there first, as an approximation. >>>>>>>>>>> The "odd strings" are totally insignificant, and are >>>>>>>>>>> typically something like "CONSTANT_PLACEHOLDER_42" >>>>>>>>>>> (see java/lang/invoke/InvokerBytecodeGenerator.java)." >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Reconstitution of the ConstantPool is needed for both the >>>>>>>>>>> JVMTI GetConstantPool() and RetransformClasses(). >>>>>>>>>>> Finally, it goes to the ConstantPool::copy_cpool_bytes(). >>>>>>>>>>> >>>>>>>>>>> The problem is that a pseudo-string is a patched string >>>>>>>>>>> that does not have >>>>>>>>>>> a reference to the string symbol anymore: >>>>>>>>>>> unresolved_string_at(idx) == NULL >>>>>>>>>>> >>>>>>>>>>> The fix is to create and fill in a map from >>>>>>>>>>> JVM_CONSTANT_String cp index to the JVM_CONSTANT_Utf8 cp index >>>>>>>>>>> to be able to restore this assotiation in the >>>>>>>>>>> JvmtiConstantPoolReconstituter. >>>>>>>>>>> >>>>>>>>>>> Testing: >>>>>>>>>>> Run: >>>>>>>>>>> - java/lang/instrument tests >>>>>>>>>>> - new jtreg test (see webrev) that was written by Filipp >>>>>>>>>>> Zhinkin >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Serguei >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From serguei.spitsyn at oracle.com Sat Jan 17 01:15:10 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 16 Jan 2015 17:15:10 -0800 Subject: 3-rd round RFR (S) 8008678: JSR 292: constant pool reconstitution must support pseudo strings In-Reply-To: <82E12F9C-BE5E-4B66-B477-9F90956ECD6D@oracle.com> References: <5475968C.40800@oracle.com> <54760B40.9040206@oracle.com> <54762FC9.2010800@oracle.com> <5491BCB9.10505@oracle.com> <5493299F.60005@oracle.com> <54934E73.4080808@oracle.com> <54B96F97.5070204@oracle.com> <54B99682.8070802@oracle.com> <54B99838.10601@oracle.com> <54B998D7.1020208@oracle.com> <82E12F9C-BE5E-4B66-B477-9F90956ECD6D@oracle.com> Message-ID: <54B9B79E.3060600@oracle.com> On 1/16/15 3:32 PM, John Rose wrote: > On Jan 16, 2015, at 3:03 PM, Coleen Phillimore > > > wrote: >> >> I was sort of wondering about this along the same lines. You're >> setting the second bit, right? :) > > That sounds good, much better than the string prefix hack. > > Parsing the string would introduce too much coupling between the JVM > and random details of the JDK. > > I also suggested to Serguei that redefinition of patched classes is > not going to happen, since they are not user-visible. So all of the > "EMCP" logic can punt on pseudo-strings, one way or another, if it > needs to. > > BTW, since patched classes are always anonymous classes, you can't > even get to them, unless you dig them out some private place like a > lambda form. Anonymous classes can be noticed by agents with the CL or CFLH events. Then nothing stops agents from redefining or re-transforming anonymous classes. > > Reconstitution is also a lost cause, since the patches will be > dropped, but I see how that might be something you might possibly run > into, as a stress test or some sort of debugger display. I've filed a new bug to track this issue as we early agreed with Coleen: https://bugs.openjdk.java.net/browse/JDK-8069233 Fill free to update it if necessary. Thanks, Serguei > > ? John From daniel.daugherty at oracle.com Sat Jan 17 02:24:11 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 16 Jan 2015 19:24:11 -0700 Subject: 4-th round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54B96301.40108@oracle.com> References: <54B60308.4000105@oracle.com> <54B96301.40108@oracle.com> Message-ID: <54B9C7CB.60103@oracle.com> > http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ src/share/vm/memory/universe.hpp No comments. src/share/vm/memory/universe.cpp No comments. src/share/vm/prims/jvmtiRedefineClasses.cpp So redefining the Unsafe class is now very expensive because we have to visit the i-table and v-table of every class (and maybe interface?)... Based on the bug report 'Unsafe::throw_illegal_access' is some magical method that can appear in any i-table or v-table entry. Maybe only as part of some default methods thing? That's not clear to me so I'm just guessing... Is there some way to limit this visit to classes where the magical method can appear? Or can it really appear anywhere? Dan On 1/16/15 12:14 PM, serguei.spitsyn at oracle.com wrote: > Dan, David H. or David C., > > May I ask one of you to look at the webrev below? > The issue itself is a little bit tricky, so it is not easy to review > despite the small size. > > Coleen, > > Does the webrev matches what we discussed with you? > Do you give me a thumbs up? > > Thanks, > Serguei > > May I ask > > On 1/13/15 9:47 PM, serguei.spitsyn at oracle.com wrote: >> Please, review the fix for: >> https://bugs.openjdk.java.net/browse/JDK-8068162 >> >> >> Open webrevs: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >> >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/jdk/8068162-Test-IsModifiableAgent/ >> >> >> >> Summary: >> >> The sun.misc.Unsafe:throwIllegalAccessError() method is used in >> place of a default >> interface method in the itable if a default method was not defined >> in the interface. >> In fact, it happens for two interfaces that purhaps are >> auto-generated: >> java/nio/CharBuffer >> java/nio/HeapCharBuffer >> >> This approach creates a problem when the class sun.misc.Unsafe is >> retransformed. >> The Method* pointer to the old (redefined) method in the itable >> triggers an assert >> (see the hs_err log in the bug report). >> Coleen told me that a similar approach is going to be implemented >> for some vtable entries. >> Coleen, thanks for suggesting a better fix for this issue! >> >> The fix is to replace the old Unsafe method in the itable/vtable >> with the latest method version. >> >> >> Testing: >> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >> >> >> Thanks, >> Serguei >> > From forax at univ-mlv.fr Sat Jan 17 11:12:31 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 17 Jan 2015 12:12:31 +0100 Subject: 3-rd round RFR (S) 8008678: JSR 292: constant pool reconstitution must support pseudo strings In-Reply-To: <82E12F9C-BE5E-4B66-B477-9F90956ECD6D@oracle.com> References: <5475968C.40800@oracle.com> <54760B40.9040206@oracle.com> <54762FC9.2010800@oracle.com> <5491BCB9.10505@oracle.com> <5493299F.60005@oracle.com> <54934E73.4080808@oracle.com> <54B96F97.5070204@oracle.com> <54B99682.8070802@oracle.com> <54B99838.10601@oracle.com> <54B998D7.1020208@oracle.com> <82E12F9C-BE5E-4B66-B477-9F90956ECD6D@oracle.com> Message-ID: <54BA439F.8070907@univ-mlv.fr> On 01/17/2015 12:32 AM, John Rose wrote: > On Jan 16, 2015, at 3:03 PM, Coleen Phillimore wrote: >> I was sort of wondering about this along the same lines. You're setting the second bit, right? :) > That sounds good, much better than the string prefix hack. > > Parsing the string would introduce too much coupling between the JVM and random details of the JDK. > > I also suggested to Serguei that redefinition of patched classes is not going to happen, since they are not user-visible. So all of the "EMCP" logic can punt on pseudo-strings, one way or another, if it needs to. It's doing to happen if some random guy simultaneously uses unsafe.defineAnonymousClass for implementing a dynamic language and use instrumentation to simulate a kind of debugger like API. > > BTW, since patched classes are always anonymous classes, you can't even get to them, unless you dig them out some private place like a lambda form. patched classes and anonymous classes are currently the same thing, but having the patched class mechanism with a classical classloader will be useful for every dynamic languages like Nashorn that needs to store deoptimization data along the generated bytecodes. I believe, but I've not spend much time to think on that, that such mechanism is not unsafe and can be added in java.lang.ClassLoader. > > Reconstitution is also a lost cause, since the patches will be dropped, but I see how that might be something you might possibly run into, as a stress test or some sort of debugger display. > > ? John R?mi From david.holmes at oracle.com Mon Jan 19 05:47:42 2015 From: david.holmes at oracle.com (David Holmes) Date: Mon, 19 Jan 2015 15:47:42 +1000 Subject: 4-th round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54B9C7CB.60103@oracle.com> References: <54B60308.4000105@oracle.com> <54B96301.40108@oracle.com> <54B9C7CB.60103@oracle.com> Message-ID: <54BC9A7E.5070607@oracle.com> On 17/01/2015 12:24 PM, Daniel D. Daugherty wrote: > > > http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ > > > src/share/vm/memory/universe.hpp > No comments. > > src/share/vm/memory/universe.cpp > No comments. > > src/share/vm/prims/jvmtiRedefineClasses.cpp > So redefining the Unsafe class is now very expensive because > we have to visit the i-table and v-table of every class (and > maybe interface?)... Sorry I didn't follow this through all the rounds, but the bug report states (and the code I looked at seems to be consistent with it) that the throw_IllegalAccessError call is only added to itables - so why are we also walking vtables? David H. ----- > Based on the bug report 'Unsafe::throw_illegal_access' is some > magical method that can appear in any i-table or v-table entry. > Maybe only as part of some default methods thing? That's not > clear to me so I'm just guessing... > > Is there some way to limit this visit to classes where the > magical method can appear? Or can it really appear anywhere? > > Dan > > > On 1/16/15 12:14 PM, serguei.spitsyn at oracle.com wrote: >> Dan, David H. or David C., >> >> May I ask one of you to look at the webrev below? >> The issue itself is a little bit tricky, so it is not easy to review >> despite the small size. >> >> Coleen, >> >> Does the webrev matches what we discussed with you? >> Do you give me a thumbs up? >> >> Thanks, >> Serguei >> >> May I ask >> >> On 1/13/15 9:47 PM, serguei.spitsyn at oracle.com wrote: >>> Please, review the fix for: >>> https://bugs.openjdk.java.net/browse/JDK-8068162 >>> >>> >>> Open webrevs: >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>> > >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/jdk/8068162-Test-IsModifiableAgent/ >>> >>> >>> >>> Summary: >>> >>> The sun.misc.Unsafe:throwIllegalAccessError() method is used in >>> place of a default >>> interface method in the itable if a default method was not defined >>> in the interface. >>> In fact, it happens for two interfaces that purhaps are >>> auto-generated: >>> java/nio/CharBuffer >>> java/nio/HeapCharBuffer >>> >>> This approach creates a problem when the class sun.misc.Unsafe is >>> retransformed. >>> The Method* pointer to the old (redefined) method in the itable >>> triggers an assert >>> (see the hs_err log in the bug report). >>> Coleen told me that a similar approach is going to be implemented >>> for some vtable entries. >>> Coleen, thanks for suggesting a better fix for this issue! >>> >>> The fix is to replace the old Unsafe method in the itable/vtable >>> with the latest method version. >>> >>> >>> Testing: >>> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >>> >>> >>> Thanks, >>> Serguei >>> >> > From erik.joelsson at oracle.com Mon Jan 19 08:29:57 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 19 Jan 2015 09:29:57 +0100 Subject: RFR: JDK-8067479: verify-modules fails in bootcycle build In-Reply-To: <54B33555.8030206@oracle.com> References: <54AFE70C.3070208@oracle.com> <54B33555.8030206@oracle.com> Message-ID: <54BCC085.50406@oracle.com> Hello, Any chance someone from serviceability could take a look at this? /Erik On 2015-01-12 03:45, David Holmes wrote: > Hi Erik, > > On 10/01/2015 12:34 AM, Erik Joelsson wrote: >> Hello, >> >> Please review this patch which fixes the verify-modules target when >> running bootcycle build, and also reenables verify-modules when running >> "make images". >> >> There were two problems: >> >> * The bootcycle build configuration was broken so that both the normal >> and the bootcycle build used the same HOTSPOT_DIST directory. The >> consequence of this was that verify-modules worked when run on its own, >> but not if bootcycle-images had been run before. This is fixed in >> bootcycle-spec.gmk.in. >> >> * Since javac in JDK 9 no longer emits classes for implicitly compiled >> sources, certain classes in sa-jdi.jar were not compiled during the >> bootcycle build. I fixed this by adding the missing classes to sa.files. >> Not having the classes there might have been intentional (in at least >> some cases), but since they were compiled anyway, I felt it safer to >> just add them to the list to fix this issue. If these classes shouldn't >> be included, then they need to be properly removed in a followup fix. > > SA is owned by serviceability - cc'd. Changes seem okay as a solution > to immediate problem, but I don't think anyone expects the IA64 stuff > to still be needed. It is on the todo list to eradicate IA64 IIRC. > > Looks like there is limited awareness of the need to keep sa.files up > to date. :( > > Thanks, > David > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8067479 >> Webrev: http://cr.openjdk.java.net/~erikj/8067479/webrev.01/ >> >> Since this is changing hotspot, I assume it will need to go in through a >> hotspot forest. Which one? >> >> /Erik From staffan.larsen at oracle.com Mon Jan 19 09:05:26 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 19 Jan 2015 10:05:26 +0100 Subject: RFR: JDK-8067479: verify-modules fails in bootcycle build In-Reply-To: <54BCC085.50406@oracle.com> References: <54AFE70C.3070208@oracle.com> <54B33555.8030206@oracle.com> <54BCC085.50406@oracle.com> Message-ID: <3A3C541D-E976-44EC-A2DC-B663D0049A4F@oracle.com> SA changes look ok - the IA64 stuff isn?t needed as we don?t support it and will remove it. /Staffan > On 19 jan 2015, at 09:29, Erik Joelsson wrote: > > Hello, > > Any chance someone from serviceability could take a look at this? > > /Erik > > On 2015-01-12 03:45, David Holmes wrote: >> Hi Erik, >> >> On 10/01/2015 12:34 AM, Erik Joelsson wrote: >>> Hello, >>> >>> Please review this patch which fixes the verify-modules target when >>> running bootcycle build, and also reenables verify-modules when running >>> "make images". >>> >>> There were two problems: >>> >>> * The bootcycle build configuration was broken so that both the normal >>> and the bootcycle build used the same HOTSPOT_DIST directory. The >>> consequence of this was that verify-modules worked when run on its own, >>> but not if bootcycle-images had been run before. This is fixed in >>> bootcycle-spec.gmk.in. >>> >>> * Since javac in JDK 9 no longer emits classes for implicitly compiled >>> sources, certain classes in sa-jdi.jar were not compiled during the >>> bootcycle build. I fixed this by adding the missing classes to sa.files. >>> Not having the classes there might have been intentional (in at least >>> some cases), but since they were compiled anyway, I felt it safer to >>> just add them to the list to fix this issue. If these classes shouldn't >>> be included, then they need to be properly removed in a followup fix. >> >> SA is owned by serviceability - cc'd. Changes seem okay as a solution to immediate problem, but I don't think anyone expects the IA64 stuff to still be needed. It is on the todo list to eradicate IA64 IIRC. >> >> Looks like there is limited awareness of the need to keep sa.files up to date. :( >> >> Thanks, >> David >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8067479 >>> Webrev: http://cr.openjdk.java.net/~erikj/8067479/webrev.01/ >>> >>> Since this is changing hotspot, I assume it will need to go in through a >>> hotspot forest. Which one? >>> >>> /Erik > From coleen.phillimore at oracle.com Mon Jan 19 15:51:07 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 19 Jan 2015 10:51:07 -0500 Subject: 4-th round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54BC9A7E.5070607@oracle.com> References: <54B60308.4000105@oracle.com> <54B96301.40108@oracle.com> <54B9C7CB.60103@oracle.com> <54BC9A7E.5070607@oracle.com> Message-ID: <54BD27EB.1030505@oracle.com> On 1/19/15, 12:47 AM, David Holmes wrote: > On 17/01/2015 12:24 PM, Daniel D. Daugherty wrote: >> > >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >> >> >> >> src/share/vm/memory/universe.hpp >> No comments. >> >> src/share/vm/memory/universe.cpp >> No comments. >> >> src/share/vm/prims/jvmtiRedefineClasses.cpp >> So redefining the Unsafe class is now very expensive because >> we have to visit the i-table and v-table of every class (and >> maybe interface?)... > > Sorry I didn't follow this through all the rounds, but the bug report > states (and the code I looked at seems to be consistent with it) that > the throw_IllegalAccessError call is only added to itables - so why > are we also walking vtables? We are thinking about using a similar mechanism to fill in exceptions in the vtable instead of overpass methods. Basically, a function in Unsafe can appear in both tables. Coleen > > David H. > ----- > >> Based on the bug report 'Unsafe::throw_illegal_access' is some >> magical method that can appear in any i-table or v-table entry. >> Maybe only as part of some default methods thing? That's not >> clear to me so I'm just guessing... >> >> Is there some way to limit this visit to classes where the >> magical method can appear? Or can it really appear anywhere? >> >> Dan >> >> >> On 1/16/15 12:14 PM, serguei.spitsyn at oracle.com wrote: >>> Dan, David H. or David C., >>> >>> May I ask one of you to look at the webrev below? >>> The issue itself is a little bit tricky, so it is not easy to review >>> despite the small size. >>> >>> Coleen, >>> >>> Does the webrev matches what we discussed with you? >>> Do you give me a thumbs up? >>> >>> Thanks, >>> Serguei >>> >>> May I ask >>> >>> On 1/13/15 9:47 PM, serguei.spitsyn at oracle.com wrote: >>>> Please, review the fix for: >>>> https://bugs.openjdk.java.net/browse/JDK-8068162 >>>> >>>> >>>> Open webrevs: >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>>> >>>> >> >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/jdk/8068162-Test-IsModifiableAgent/ >>>> >>>> >>>> >>>> >>>> Summary: >>>> >>>> The sun.misc.Unsafe:throwIllegalAccessError() method is used in >>>> place of a default >>>> interface method in the itable if a default method was not defined >>>> in the interface. >>>> In fact, it happens for two interfaces that purhaps are >>>> auto-generated: >>>> java/nio/CharBuffer >>>> java/nio/HeapCharBuffer >>>> >>>> This approach creates a problem when the class sun.misc.Unsafe is >>>> retransformed. >>>> The Method* pointer to the old (redefined) method in the itable >>>> triggers an assert >>>> (see the hs_err log in the bug report). >>>> Coleen told me that a similar approach is going to be implemented >>>> for some vtable entries. >>>> Coleen, thanks for suggesting a better fix for this issue! >>>> >>>> The fix is to replace the old Unsafe method in the itable/vtable >>>> with the latest method version. >>>> >>>> >>>> Testing: >>>> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >>> >> From coleen.phillimore at oracle.com Mon Jan 19 15:55:35 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 19 Jan 2015 10:55:35 -0500 Subject: 4-th round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54B9C7CB.60103@oracle.com> References: <54B60308.4000105@oracle.com> <54B96301.40108@oracle.com> <54B9C7CB.60103@oracle.com> Message-ID: <54BD28F7.7040905@oracle.com> On 1/16/15, 9:24 PM, Daniel D. Daugherty wrote: > > > http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ > > src/share/vm/memory/universe.hpp > No comments. > > src/share/vm/memory/universe.cpp > No comments. > > src/share/vm/prims/jvmtiRedefineClasses.cpp > So redefining the Unsafe class is now very expensive because > we have to visit the i-table and v-table of every class (and > maybe interface?)... > > Based on the bug report 'Unsafe::throw_illegal_access' is some > magical method that can appear in any i-table or v-table entry. > Maybe only as part of some default methods thing? That's not > clear to me so I'm just guessing... True. > > Is there some way to limit this visit to classes where the > magical method can appear? Or can it really appear anywhere? The Unsafe methods can appear in any itable now. I don't know of a way to limit this. Fortunately, redefining Unsafe seems to be an unusual thing to do, except for this stress test. Coleen > > Dan > > > On 1/16/15 12:14 PM, serguei.spitsyn at oracle.com wrote: >> Dan, David H. or David C., >> >> May I ask one of you to look at the webrev below? >> The issue itself is a little bit tricky, so it is not easy to review >> despite the small size. >> >> Coleen, >> >> Does the webrev matches what we discussed with you? >> Do you give me a thumbs up? >> >> Thanks, >> Serguei >> >> May I ask >> >> On 1/13/15 9:47 PM, serguei.spitsyn at oracle.com wrote: >>> Please, review the fix for: >>> https://bugs.openjdk.java.net/browse/JDK-8068162 >>> >>> >>> Open webrevs: >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>> > >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/jdk/8068162-Test-IsModifiableAgent/ >>> >>> >>> >>> Summary: >>> >>> The sun.misc.Unsafe:throwIllegalAccessError() method is used in >>> place of a default >>> interface method in the itable if a default method was not >>> defined in the interface. >>> In fact, it happens for two interfaces that purhaps are >>> auto-generated: >>> java/nio/CharBuffer >>> java/nio/HeapCharBuffer >>> >>> This approach creates a problem when the class sun.misc.Unsafe is >>> retransformed. >>> The Method* pointer to the old (redefined) method in the itable >>> triggers an assert >>> (see the hs_err log in the bug report). >>> Coleen told me that a similar approach is going to be implemented >>> for some vtable entries. >>> Coleen, thanks for suggesting a better fix for this issue! >>> >>> The fix is to replace the old Unsafe method in the itable/vtable >>> with the latest method version. >>> >>> >>> Testing: >>> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >>> >>> >>> Thanks, >>> Serguei >>> >> > From serguei.spitsyn at oracle.com Mon Jan 19 17:22:34 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 19 Jan 2015 09:22:34 -0800 Subject: 4-th round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54BD28F7.7040905@oracle.com> References: <54B60308.4000105@oracle.com> <54B96301.40108@oracle.com> <54B9C7CB.60103@oracle.com> <54BD28F7.7040905@oracle.com> Message-ID: <54BD3D5A.1030903@oracle.com> Coleen, Thank you for answering questions below! Thanks, Serguei On 1/19/15 7:55 AM, Coleen Phillimore wrote: > > On 1/16/15, 9:24 PM, Daniel D. Daugherty wrote: >> > >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >> >> src/share/vm/memory/universe.hpp >> No comments. >> >> src/share/vm/memory/universe.cpp >> No comments. >> >> src/share/vm/prims/jvmtiRedefineClasses.cpp >> So redefining the Unsafe class is now very expensive because >> we have to visit the i-table and v-table of every class (and >> maybe interface?)... >> >> Based on the bug report 'Unsafe::throw_illegal_access' is some >> magical method that can appear in any i-table or v-table entry. >> Maybe only as part of some default methods thing? That's not >> clear to me so I'm just guessing... > > True. >> >> Is there some way to limit this visit to classes where the >> magical method can appear? Or can it really appear anywhere? > > The Unsafe methods can appear in any itable now. I don't know of a > way to limit this. Fortunately, redefining Unsafe seems to be an > unusual thing to do, except for this stress test. > > Coleen > >> >> Dan >> >> >> On 1/16/15 12:14 PM, serguei.spitsyn at oracle.com wrote: >>> Dan, David H. or David C., >>> >>> May I ask one of you to look at the webrev below? >>> The issue itself is a little bit tricky, so it is not easy to review >>> despite the small size. >>> >>> Coleen, >>> >>> Does the webrev matches what we discussed with you? >>> Do you give me a thumbs up? >>> >>> Thanks, >>> Serguei >>> >>> May I ask >>> >>> On 1/13/15 9:47 PM, serguei.spitsyn at oracle.com wrote: >>>> Please, review the fix for: >>>> https://bugs.openjdk.java.net/browse/JDK-8068162 >>>> >>>> >>>> Open webrevs: >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>>> >> >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/jdk/8068162-Test-IsModifiableAgent/ >>>> >>>> >>>> >>>> Summary: >>>> >>>> The sun.misc.Unsafe:throwIllegalAccessError() method is used in >>>> place of a default >>>> interface method in the itable if a default method was not >>>> defined in the interface. >>>> In fact, it happens for two interfaces that purhaps are >>>> auto-generated: >>>> java/nio/CharBuffer >>>> java/nio/HeapCharBuffer >>>> >>>> This approach creates a problem when the class sun.misc.Unsafe >>>> is retransformed. >>>> The Method* pointer to the old (redefined) method in the itable >>>> triggers an assert >>>> (see the hs_err log in the bug report). >>>> Coleen told me that a similar approach is going to be >>>> implemented for some vtable entries. >>>> Coleen, thanks for suggesting a better fix for this issue! >>>> >>>> The fix is to replace the old Unsafe method in the itable/vtable >>>> with the latest method version. >>>> >>>> >>>> Testing: >>>> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >>> >> > From serguei.spitsyn at oracle.com Mon Jan 19 17:26:24 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 19 Jan 2015 09:26:24 -0800 Subject: 4-th round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54BC9A7E.5070607@oracle.com> References: <54B60308.4000105@oracle.com> <54B96301.40108@oracle.com> <54B9C7CB.60103@oracle.com> <54BC9A7E.5070607@oracle.com> Message-ID: <54BD3E40.5030400@oracle.com> On 1/18/15 9:47 PM, David Holmes wrote: > On 17/01/2015 12:24 PM, Daniel D. Daugherty wrote: >> > >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >> >> >> >> src/share/vm/memory/universe.hpp >> No comments. >> >> src/share/vm/memory/universe.cpp >> No comments. >> >> src/share/vm/prims/jvmtiRedefineClasses.cpp >> So redefining the Unsafe class is now very expensive because >> we have to visit the i-table and v-table of every class (and >> maybe interface?)... > > Sorry I didn't follow this through all the rounds, but the bug report > states (and the code I looked at seems to be consistent with it) that > the throw_IllegalAccessError call is only added to itables - so why > are we also walking vtables? The Runtime team is going to extend use of the throw_IllegalAccessError for vtables as well. Thank you for looking at the fix! Serguei > > David H. > ----- > >> Based on the bug report 'Unsafe::throw_illegal_access' is some >> magical method that can appear in any i-table or v-table entry. >> Maybe only as part of some default methods thing? That's not >> clear to me so I'm just guessing... >> >> Is there some way to limit this visit to classes where the >> magical method can appear? Or can it really appear anywhere? >> >> Dan >> >> >> On 1/16/15 12:14 PM, serguei.spitsyn at oracle.com wrote: >>> Dan, David H. or David C., >>> >>> May I ask one of you to look at the webrev below? >>> The issue itself is a little bit tricky, so it is not easy to review >>> despite the small size. >>> >>> Coleen, >>> >>> Does the webrev matches what we discussed with you? >>> Do you give me a thumbs up? >>> >>> Thanks, >>> Serguei >>> >>> May I ask >>> >>> On 1/13/15 9:47 PM, serguei.spitsyn at oracle.com wrote: >>>> Please, review the fix for: >>>> https://bugs.openjdk.java.net/browse/JDK-8068162 >>>> >>>> >>>> Open webrevs: >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>>> >>>> >> >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/jdk/8068162-Test-IsModifiableAgent/ >>>> >>>> >>>> >>>> >>>> Summary: >>>> >>>> The sun.misc.Unsafe:throwIllegalAccessError() method is used in >>>> place of a default >>>> interface method in the itable if a default method was not defined >>>> in the interface. >>>> In fact, it happens for two interfaces that purhaps are >>>> auto-generated: >>>> java/nio/CharBuffer >>>> java/nio/HeapCharBuffer >>>> >>>> This approach creates a problem when the class sun.misc.Unsafe is >>>> retransformed. >>>> The Method* pointer to the old (redefined) method in the itable >>>> triggers an assert >>>> (see the hs_err log in the bug report). >>>> Coleen told me that a similar approach is going to be implemented >>>> for some vtable entries. >>>> Coleen, thanks for suggesting a better fix for this issue! >>>> >>>> The fix is to replace the old Unsafe method in the itable/vtable >>>> with the latest method version. >>>> >>>> >>>> Testing: >>>> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >>> >> From david.holmes at oracle.com Tue Jan 20 01:40:31 2015 From: david.holmes at oracle.com (David Holmes) Date: Tue, 20 Jan 2015 11:40:31 +1000 Subject: 4-th round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54BD27EB.1030505@oracle.com> References: <54B60308.4000105@oracle.com> <54B96301.40108@oracle.com> <54B9C7CB.60103@oracle.com> <54BC9A7E.5070607@oracle.com> <54BD27EB.1030505@oracle.com> Message-ID: <54BDB20F.3070003@oracle.com> On 20/01/2015 1:51 AM, Coleen Phillimore wrote: > > On 1/19/15, 12:47 AM, David Holmes wrote: >> On 17/01/2015 12:24 PM, Daniel D. Daugherty wrote: >>> > >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>> >>> >>> >>> src/share/vm/memory/universe.hpp >>> No comments. >>> >>> src/share/vm/memory/universe.cpp >>> No comments. >>> >>> src/share/vm/prims/jvmtiRedefineClasses.cpp >>> So redefining the Unsafe class is now very expensive because >>> we have to visit the i-table and v-table of every class (and >>> maybe interface?)... >> >> Sorry I didn't follow this through all the rounds, but the bug report >> states (and the code I looked at seems to be consistent with it) that >> the throw_IllegalAccessError call is only added to itables - so why >> are we also walking vtables? > > We are thinking about using a similar mechanism to fill in exceptions in > the vtable instead of overpass methods. Basically, a function in > Unsafe can appear in both tables. Okay. But as we are in the process of trying to reduce Unsafe to its minimal form and potentially convert to supported API's where needed, perhaps this is not the best mechanism to be expanding. :) On the other hand ... if in a modular world Unsafe is truly inaccessible then perhaps we can prohibit it from being redefined in the first place. :) Cheers, David > Coleen >> >> David H. >> ----- >> >>> Based on the bug report 'Unsafe::throw_illegal_access' is some >>> magical method that can appear in any i-table or v-table entry. >>> Maybe only as part of some default methods thing? That's not >>> clear to me so I'm just guessing... >>> >>> Is there some way to limit this visit to classes where the >>> magical method can appear? Or can it really appear anywhere? >>> >>> Dan >>> >>> >>> On 1/16/15 12:14 PM, serguei.spitsyn at oracle.com wrote: >>>> Dan, David H. or David C., >>>> >>>> May I ask one of you to look at the webrev below? >>>> The issue itself is a little bit tricky, so it is not easy to review >>>> despite the small size. >>>> >>>> Coleen, >>>> >>>> Does the webrev matches what we discussed with you? >>>> Do you give me a thumbs up? >>>> >>>> Thanks, >>>> Serguei >>>> >>>> May I ask >>>> >>>> On 1/13/15 9:47 PM, serguei.spitsyn at oracle.com wrote: >>>>> Please, review the fix for: >>>>> https://bugs.openjdk.java.net/browse/JDK-8068162 >>>>> >>>>> >>>>> Open webrevs: >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>>>> >>>>> >>> >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/jdk/8068162-Test-IsModifiableAgent/ >>>>> >>>>> >>>>> >>>>> >>>>> Summary: >>>>> >>>>> The sun.misc.Unsafe:throwIllegalAccessError() method is used in >>>>> place of a default >>>>> interface method in the itable if a default method was not defined >>>>> in the interface. >>>>> In fact, it happens for two interfaces that purhaps are >>>>> auto-generated: >>>>> java/nio/CharBuffer >>>>> java/nio/HeapCharBuffer >>>>> >>>>> This approach creates a problem when the class sun.misc.Unsafe is >>>>> retransformed. >>>>> The Method* pointer to the old (redefined) method in the itable >>>>> triggers an assert >>>>> (see the hs_err log in the bug report). >>>>> Coleen told me that a similar approach is going to be implemented >>>>> for some vtable entries. >>>>> Coleen, thanks for suggesting a better fix for this issue! >>>>> >>>>> The fix is to replace the old Unsafe method in the itable/vtable >>>>> with the latest method version. >>>>> >>>>> >>>>> Testing: >>>>> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>> >>> > From serguei.spitsyn at oracle.com Tue Jan 20 01:56:54 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 19 Jan 2015 17:56:54 -0800 Subject: 4-th round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54BDB20F.3070003@oracle.com> References: <54B60308.4000105@oracle.com> <54B96301.40108@oracle.com> <54B9C7CB.60103@oracle.com> <54BC9A7E.5070607@oracle.com> <54BD27EB.1030505@oracle.com> <54BDB20F.3070003@oracle.com> Message-ID: <54BDB5E6.1080109@oracle.com> On 1/19/15 5:40 PM, David Holmes wrote: > On 20/01/2015 1:51 AM, Coleen Phillimore wrote: >> >> On 1/19/15, 12:47 AM, David Holmes wrote: >>> On 17/01/2015 12:24 PM, Daniel D. Daugherty wrote: >>>> > >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>>> >>>> >>>> >>>> >>>> src/share/vm/memory/universe.hpp >>>> No comments. >>>> >>>> src/share/vm/memory/universe.cpp >>>> No comments. >>>> >>>> src/share/vm/prims/jvmtiRedefineClasses.cpp >>>> So redefining the Unsafe class is now very expensive because >>>> we have to visit the i-table and v-table of every class (and >>>> maybe interface?)... >>> >>> Sorry I didn't follow this through all the rounds, but the bug report >>> states (and the code I looked at seems to be consistent with it) that >>> the throw_IllegalAccessError call is only added to itables - so why >>> are we also walking vtables? >> >> We are thinking about using a similar mechanism to fill in exceptions in >> the vtable instead of overpass methods. Basically, a function in >> Unsafe can appear in both tables. > > Okay. But as we are in the process of trying to reduce Unsafe to its > minimal form and potentially convert to supported API's where needed, > perhaps this is not the best mechanism to be expanding. :) > > On the other hand ... if in a modular world Unsafe is truly > inaccessible then perhaps we can prohibit it from being redefined in > the first place. :) Even if in a modular world the Unsafe is truly inaccessible, it should not be applied to the agents as they need redefineabilty, not accessibility. :) I do not see how we can prohibit any class from being redefined (instrumented). At least, this approach will impact the JVMTI spec. Thanks, Serguei > > Cheers, > David > >> Coleen >>> >>> David H. >>> ----- >>> >>>> Based on the bug report 'Unsafe::throw_illegal_access' is some >>>> magical method that can appear in any i-table or v-table entry. >>>> Maybe only as part of some default methods thing? That's not >>>> clear to me so I'm just guessing... >>>> >>>> Is there some way to limit this visit to classes where the >>>> magical method can appear? Or can it really appear anywhere? >>>> >>>> Dan >>>> >>>> >>>> On 1/16/15 12:14 PM, serguei.spitsyn at oracle.com wrote: >>>>> Dan, David H. or David C., >>>>> >>>>> May I ask one of you to look at the webrev below? >>>>> The issue itself is a little bit tricky, so it is not easy to review >>>>> despite the small size. >>>>> >>>>> Coleen, >>>>> >>>>> Does the webrev matches what we discussed with you? >>>>> Do you give me a thumbs up? >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> May I ask >>>>> >>>>> On 1/13/15 9:47 PM, serguei.spitsyn at oracle.com wrote: >>>>>> Please, review the fix for: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8068162 >>>>>> >>>>>> >>>>>> Open webrevs: >>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>>>>> >>>>>> >>>>>> >>>> >>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/jdk/8068162-Test-IsModifiableAgent/ >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Summary: >>>>>> >>>>>> The sun.misc.Unsafe:throwIllegalAccessError() method is used in >>>>>> place of a default >>>>>> interface method in the itable if a default method was not >>>>>> defined >>>>>> in the interface. >>>>>> In fact, it happens for two interfaces that purhaps are >>>>>> auto-generated: >>>>>> java/nio/CharBuffer >>>>>> java/nio/HeapCharBuffer >>>>>> >>>>>> This approach creates a problem when the class sun.misc.Unsafe is >>>>>> retransformed. >>>>>> The Method* pointer to the old (redefined) method in the itable >>>>>> triggers an assert >>>>>> (see the hs_err log in the bug report). >>>>>> Coleen told me that a similar approach is going to be implemented >>>>>> for some vtable entries. >>>>>> Coleen, thanks for suggesting a better fix for this issue! >>>>>> >>>>>> The fix is to replace the old Unsafe method in the itable/vtable >>>>>> with the latest method version. >>>>>> >>>>>> >>>>>> Testing: >>>>>> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>> >>>> >> From thomas.schatzl at oracle.com Tue Jan 20 12:44:53 2015 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 20 Jan 2015 13:44:53 +0100 Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF44B46@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF44B46@DEWDFEMB12A.global.corp.sap> Message-ID: <1421757893.3259.3.camel@oracle.com> Hi Goetz, On Mon, 2014-12-22 at 13:07 +0000, Lindenmaier, Goetz wrote: > Hi, > > I please need a review and a sponsor for this tiny change: > > http://cr.openjdk.java.net/~goetz/webrevs/8068026-jtregClient/webrev.01/ > The test did not check whether -client is supported. > > The patch should also go to 8u60, please. I think it would be cleaner to split the test into two tests, each @require'ing the particular vm flavor. And having a @run tag using -client/-server respectively. E.g. // client test: @... @requires vm.flavor == client @run main -client TestHumongousCodeCacheRootsClient class TestHumongousCodeCacheRootsClient { public static void main(String[] args) { TestHumongousCodeCacheRootsHelper.main(args); // put into an auxiliary class file... } } // server test: @... @requires vm.flavor == client @run main -client TestHumongousCodeCacheRootsServer class TestHumongousCodeCacheRootsServer { public static void main(String[] args) { TestHumongousCodeCacheRootsHelper.main(args); // put into an auxiliary class file... } } That *could* work, and avoid the dodgy exception catching. Thanks, Thomas From goetz.lindenmaier at sap.com Tue Jan 20 12:53:50 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 20 Jan 2015 12:53:50 +0000 Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java In-Reply-To: <1421757893.3259.3.camel@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF44B46@DEWDFEMB12A.global.corp.sap> <1421757893.3259.3.camel@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF70905@DEWDFEMB12A.global.corp.sap> Hi Thomas, yes, it would be better if the jtreg framework could sort it out right away. But in this case, you would get two very similar tests, one of which would be executed always, thus not saving anything. Except for having to adapt two tests if there is a change needed. Also, it's been pushed in the meantime. http://hg.openjdk.java.net/jdk9/hs-gc/hotspot/rev/8dfd8b00c7f1 Should I still improve the test? Thanks, Goetz. -----Original Message----- From: Thomas Schatzl [mailto:thomas.schatzl at oracle.com] Sent: Dienstag, 20. Januar 2015 13:45 To: Lindenmaier, Goetz Cc: 'hotspot-dev at openjdk.java.net' Subject: Re: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java Hi Goetz, On Mon, 2014-12-22 at 13:07 +0000, Lindenmaier, Goetz wrote: > Hi, > > I please need a review and a sponsor for this tiny change: > > http://cr.openjdk.java.net/~goetz/webrevs/8068026-jtregClient/webrev.01/ > The test did not check whether -client is supported. > > The patch should also go to 8u60, please. I think it would be cleaner to split the test into two tests, each @require'ing the particular vm flavor. And having a @run tag using -client/-server respectively. E.g. // client test: @... @requires vm.flavor == client @run main -client TestHumongousCodeCacheRootsClient class TestHumongousCodeCacheRootsClient { public static void main(String[] args) { TestHumongousCodeCacheRootsHelper.main(args); // put into an auxiliary class file... } } // server test: @... @requires vm.flavor == client @run main -client TestHumongousCodeCacheRootsServer class TestHumongousCodeCacheRootsServer { public static void main(String[] args) { TestHumongousCodeCacheRootsHelper.main(args); // put into an auxiliary class file... } } That *could* work, and avoid the dodgy exception catching. Thanks, Thomas From thomas.schatzl at oracle.com Tue Jan 20 13:13:01 2015 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 20 Jan 2015 14:13:01 +0100 Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF70905@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF44B46@DEWDFEMB12A.global.corp.sap> <1421757893.3259.3.camel@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF70905@DEWDFEMB12A.global.corp.sap> Message-ID: <1421759581.3259.5.camel@oracle.com> On Tue, 2015-01-20 at 12:53 +0000, Lindenmaier, Goetz wrote: > Hi Thomas, > > yes, it would be better if the jtreg framework could sort it out right away. > But in this case, you would get two very similar tests, one of which > would be executed always, thus not saving anything. Not sure what you mean here: currently, the test starts three VMs, ie. the main VM that executes two other VMs where one always fails if there is no client VM. With the proposed change, jtreg will only start the VMs that are supported, i.e. either one or two. > Except for having to adapt two tests if there is a change needed. Since the meat of the code (TestHumongousCodeCacheRootsHelper) would be in a shared class and the shells just forwarding to that, I do not see a problem with such changes. I would imagine that the code would get cleaner and smaller, because the setup to execute the TestHumongousCodeCacheRootsHelper would go away. > Also, it's been pushed in the meantime. > http://hg.openjdk.java.net/jdk9/hs-gc/hotspot/rev/8dfd8b00c7f1 I did not notice that, somehow overlooked that - it was still on my todo-list :) Sorry. > Should I still improve the test? It would be nice, but not that important I guess. I created an RFE at https://bugs.openjdk.java.net/browse/JDK-8069343 to work on if you want. Thanks, Thomas From aph at redhat.com Tue Jan 20 15:35:51 2015 From: aph at redhat.com (Andrew Haley) Date: Tue, 20 Jan 2015 15:35:51 +0000 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54B819BA.5040804@oracle.com> References: <54A6A933.8000204@redhat.com> <7C4F2EEA-04D9-4421-8231-23E0A47A6502@oracle.com> <54AD4A98.9080501@redhat.com> <54B809C7.2040006@redhat.com> <54B819BA.5040804@oracle.com> Message-ID: <54BE75D7.6020609@redhat.com> On 01/15/2015 07:49 PM, Vladimir Kozlov wrote: > Looks good to me. > > One thing caught my eye is relocInfo_aarch64.hpp: > > // FIXME for AARCH64 > > The values there are similar to relocInfo_x86.hpp. > > Do you still need to adjust them? The granularity of AArch64 relocs varies, but the smallest is one byte. We don't use format() for anything. (Perhaps we should, as it would make the gnarly code that handles relocs simpler and faster, but that's for another day.) So this is what I've ended up with: 29 // machine-dependent parts of class relocInfo 30 private: 31 enum { 32 // Relocations are byte-aligned. 33 offset_unit = 1, 34 // We don't use format(). 35 format_width = 0 36 }; Webrev at http://cr.openjdk.java.net/~aph/aarch64-8068054-5/ Thanks, Andrew. From vladimir.kozlov at oracle.com Tue Jan 20 17:42:56 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 20 Jan 2015 09:42:56 -0800 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54BE75D7.6020609@redhat.com> References: <54A6A933.8000204@redhat.com> <7C4F2EEA-04D9-4421-8231-23E0A47A6502@oracle.com> <54AD4A98.9080501@redhat.com> <54B809C7.2040006@redhat.com> <54B819BA.5040804@oracle.com> <54BE75D7.6020609@redhat.com> Message-ID: <54BE93A0.2090301@oracle.com> Fine with me if it works for you. I will push it today. Thanks, Vladimir On 1/20/15 7:35 AM, Andrew Haley wrote: > On 01/15/2015 07:49 PM, Vladimir Kozlov wrote: >> Looks good to me. >> >> One thing caught my eye is relocInfo_aarch64.hpp: >> >> // FIXME for AARCH64 >> >> The values there are similar to relocInfo_x86.hpp. >> >> Do you still need to adjust them? > > The granularity of AArch64 relocs varies, but the smallest is one > byte. We don't use format() for anything. (Perhaps we should, as it > would make the gnarly code that handles relocs simpler and faster, but > that's for another day.) > > So this is what I've ended up with: > > 29 // machine-dependent parts of class relocInfo > 30 private: > 31 enum { > 32 // Relocations are byte-aligned. > 33 offset_unit = 1, > 34 // We don't use format(). > 35 format_width = 0 > 36 }; > > Webrev at http://cr.openjdk.java.net/~aph/aarch64-8068054-5/ > > Thanks, > Andrew. > From aph at redhat.com Tue Jan 20 19:09:20 2015 From: aph at redhat.com (Andrew Haley) Date: Tue, 20 Jan 2015 19:09:20 +0000 Subject: Serviceability Agent support in JDK9 Message-ID: <54BEA7E0.9040603@redhat.com> I'm having trouble with compiler replay data in JDK 9: $ ./build/linux-aarch64-normal-server-slowdebug/jdk/bin/java -Xbootclasspath/p:boot.jar -cp app.jar -XX:ReplayDataFile=replay.foo -XX:+ReplayCompiles Error occurred during initialization of VM java/lang/ClassFormatError: Missing BootstrapMethods attribute in class file java/lang/CharSequence I guess this is because sun.jvm.hotspot.tools.jcore.ClassWriter doesn't handle the BootstrapMethods attribute. Is that right? It would be nice to have this working. Andrew. From vladimir.kozlov at oracle.com Tue Jan 20 19:46:30 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 20 Jan 2015 11:46:30 -0800 Subject: 8068053: AARCH64: C1 and C2 compilers In-Reply-To: <54B81459.8010809@oracle.com> References: <54A6A8F5.4040907@redhat.com> <928DC166-7256-4BCA-A9CA-C3ADB05834E5@oracle.com> <54AAD3FC.8000003@redhat.com> <54AAED1F.1000908@oracle.com> <54ABA2EA.4070802@redhat.com> <54B80921.50608@redhat.com> <54B81459.8010809@oracle.com> Message-ID: <54BEB096.2020009@oracle.com> Roland reviewed first version and I reviewed both. Roland's comments were addressed. So I am pushing these changes too. Thanks, Vladimir On 1/15/15 11:26 AM, Vladimir Kozlov wrote: > Looks good. > > Thanks, > Vladimir > > On 1/15/15 10:38 AM, Andrew Haley wrote: >> New webrev at http://cr.openjdk.java.net/~aph/aarch64-8068053-4/ >> >> Thanks, >> Andrew. >> From dean.long at oracle.com Tue Jan 20 19:51:08 2015 From: dean.long at oracle.com (Dean Long) Date: Tue, 20 Jan 2015 11:51:08 -0800 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54BE75D7.6020609@redhat.com> References: <54A6A933.8000204@redhat.com> <7C4F2EEA-04D9-4421-8231-23E0A47A6502@oracle.com> <54AD4A98.9080501@redhat.com> <54B809C7.2040006@redhat.com> <54B819BA.5040804@oracle.com> <54BE75D7.6020609@redhat.com> Message-ID: <54BEB1AC.7090508@oracle.com> I believe offset_unit refers to the location of the relocation (typically a 4-byte machine instruction) and not the size of the reloc data, so you may want to try changing to "4" in the future. dl On 1/20/2015 7:35 AM, Andrew Haley wrote: > On 01/15/2015 07:49 PM, Vladimir Kozlov wrote: >> Looks good to me. >> >> One thing caught my eye is relocInfo_aarch64.hpp: >> >> // FIXME for AARCH64 >> >> The values there are similar to relocInfo_x86.hpp. >> >> Do you still need to adjust them? > The granularity of AArch64 relocs varies, but the smallest is one > byte. We don't use format() for anything. (Perhaps we should, as it > would make the gnarly code that handles relocs simpler and faster, but > that's for another day.) > > So this is what I've ended up with: > > 29 // machine-dependent parts of class relocInfo > 30 private: > 31 enum { > 32 // Relocations are byte-aligned. > 33 offset_unit = 1, > 34 // We don't use format(). > 35 format_width = 0 > 36 }; > > Webrev at http://cr.openjdk.java.net/~aph/aarch64-8068054-5/ > > Thanks, > Andrew. From dean.long at oracle.com Tue Jan 20 19:52:35 2015 From: dean.long at oracle.com (Dean Long) Date: Tue, 20 Jan 2015 11:52:35 -0800 Subject: 8068053: AARCH64: C1 and C2 compilers In-Reply-To: <54BEB096.2020009@oracle.com> References: <54A6A8F5.4040907@redhat.com> <928DC166-7256-4BCA-A9CA-C3ADB05834E5@oracle.com> <54AAD3FC.8000003@redhat.com> <54AAED1F.1000908@oracle.com> <54ABA2EA.4070802@redhat.com> <54B80921.50608@redhat.com> <54B81459.8010809@oracle.com> <54BEB096.2020009@oracle.com> Message-ID: <54BEB203.4070006@oracle.com> Sounds good. What's next? dl On 1/20/2015 11:46 AM, Vladimir Kozlov wrote: > Roland reviewed first version and I reviewed both. Roland's comments > were addressed. So I am pushing these changes too. > > Thanks, > Vladimir > > On 1/15/15 11:26 AM, Vladimir Kozlov wrote: >> Looks good. >> >> Thanks, >> Vladimir >> >> On 1/15/15 10:38 AM, Andrew Haley wrote: >>> New webrev at http://cr.openjdk.java.net/~aph/aarch64-8068053-4/ >>> >>> Thanks, >>> Andrew. >>> From vladimir.kozlov at oracle.com Tue Jan 20 20:02:08 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 20 Jan 2015 12:02:08 -0800 Subject: [8u] backport RFR(XS) 8068909: SIGSEGV in c2 compiled code with OptimizeStringConcat Message-ID: <54BEB440.1090504@oracle.com> 8u60 backport request. Changes were pushed into jdk9 last week, no problems were found since then. Changes are applied to 8u cleanly. https://bugs.openjdk.java.net/browse/JDK-8068909 review: http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2015-January/016876.html webrev: http://cr.openjdk.java.net/~simonis/webrevs/2015/8068909.v2/ changeset http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/8596a20c8de1 Thanks, Vladimir From vladimir.kozlov at oracle.com Tue Jan 20 20:05:47 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 20 Jan 2015 12:05:47 -0800 Subject: 8068053: AARCH64: C1 and C2 compilers In-Reply-To: <54BEB203.4070006@oracle.com> References: <54A6A8F5.4040907@redhat.com> <928DC166-7256-4BCA-A9CA-C3ADB05834E5@oracle.com> <54AAD3FC.8000003@redhat.com> <54AAED1F.1000908@oracle.com> <54ABA2EA.4070802@redhat.com> <54B80921.50608@redhat.com> <54B81459.8010809@oracle.com> <54BEB096.2020009@oracle.com> <54BEB203.4070006@oracle.com> Message-ID: <54BEB51B.8070109@oracle.com> After 8068054 and 8068053 are pushed today I want Andrew to build and test sources from stage repo. If everything is fine I will sync it with latest jdk9/dev forest. Then we need to ask SQE to run PIT tests on it and we need to run parfait, findbugs. After that it should be ready to merge into jdk9 That is the plan. Regards, Vladimir On 1/20/15 11:52 AM, Dean Long wrote: > Sounds good. What's next? > > dl > > On 1/20/2015 11:46 AM, Vladimir Kozlov wrote: >> Roland reviewed first version and I reviewed both. Roland's comments >> were addressed. So I am pushing these changes too. >> >> Thanks, >> Vladimir >> >> On 1/15/15 11:26 AM, Vladimir Kozlov wrote: >>> Looks good. >>> >>> Thanks, >>> Vladimir >>> >>> On 1/15/15 10:38 AM, Andrew Haley wrote: >>>> New webrev at http://cr.openjdk.java.net/~aph/aarch64-8068053-4/ >>>> >>>> Thanks, >>>> Andrew. >>>> > From igor.veresov at oracle.com Tue Jan 20 20:06:58 2015 From: igor.veresov at oracle.com (Igor Veresov) Date: Tue, 20 Jan 2015 12:06:58 -0800 Subject: [8u] backport RFR(XS) 8068909: SIGSEGV in c2 compiled code with OptimizeStringConcat In-Reply-To: <54BEB440.1090504@oracle.com> References: <54BEB440.1090504@oracle.com> Message-ID: Good. igor > On Jan 20, 2015, at 12:02 PM, Vladimir Kozlov wrote: > > 8u60 backport request. Changes were pushed into jdk9 last week, no problems were found since then. Changes are applied to 8u cleanly. > > https://bugs.openjdk.java.net/browse/JDK-8068909 > > review: > http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2015-January/016876.html > webrev: > http://cr.openjdk.java.net/~simonis/webrevs/2015/8068909.v2/ > changeset > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/8596a20c8de1 > > Thanks, > Vladimir From vladimir.kozlov at oracle.com Tue Jan 20 20:08:10 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 20 Jan 2015 12:08:10 -0800 Subject: [8u] backport RFR(XS) 8068909: SIGSEGV in c2 compiled code with OptimizeStringConcat In-Reply-To: References: <54BEB440.1090504@oracle.com> Message-ID: <54BEB5AA.5030102@oracle.com> Thanks, Vladimir On 1/20/15 12:06 PM, Igor Veresov wrote: > Good. > > igor > >> On Jan 20, 2015, at 12:02 PM, Vladimir Kozlov wrote: >> >> 8u60 backport request. Changes were pushed into jdk9 last week, no problems were found since then. Changes are applied to 8u cleanly. >> >> https://bugs.openjdk.java.net/browse/JDK-8068909 >> >> review: >> http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2015-January/016876.html >> webrev: >> http://cr.openjdk.java.net/~simonis/webrevs/2015/8068909.v2/ >> changeset >> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/8596a20c8de1 >> >> Thanks, >> Vladimir > From igor.veresov at oracle.com Tue Jan 20 20:14:10 2015 From: igor.veresov at oracle.com (Igor Veresov) Date: Tue, 20 Jan 2015 12:14:10 -0800 Subject: [8u] backport of JDK-8068881 SIGBUS in C2 compiled method weblogic.wsee.jaxws.framework.jaxrpc.EnvironmentFactory$SimulatedWsdlDefinitions. Message-ID: Webrev: http://cr.openjdk.java.net/~iveresov/8068881/webrev.8u40/ The change is slightly different from the one in jdk9: - operator new needs Compile* when allocating a node - compile timers usage is different Here is the diff between the patches: http://cr.openjdk.java.net/~iveresov/8068881/8u40.patch.diff JDK9 changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/b0ce179e4a01 JDK9 webrev: http://cr.openjdk.java.net/~iveresov/8068881/webrev.02/ Nightlies seem fine. Thanks! igor From daniel.daugherty at oracle.com Tue Jan 20 20:25:29 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 20 Jan 2015 13:25:29 -0700 Subject: Serviceability Agent support in JDK9 In-Reply-To: <54BEA7E0.9040603@redhat.com> References: <54BEA7E0.9040603@redhat.com> Message-ID: <54BEB9B9.20409@oracle.com> Adding the Serviceability team since SA belongs to them... Dan On 1/20/15 12:09 PM, Andrew Haley wrote: > I'm having trouble with compiler replay data in JDK 9: > > $ ./build/linux-aarch64-normal-server-slowdebug/jdk/bin/java -Xbootclasspath/p:boot.jar -cp app.jar -XX:ReplayDataFile=replay.foo -XX:+ReplayCompiles > Error occurred during initialization of VM > java/lang/ClassFormatError: Missing BootstrapMethods attribute in class file java/lang/CharSequence > > I guess this is because sun.jvm.hotspot.tools.jcore.ClassWriter doesn't > handle the BootstrapMethods attribute. Is that right? It would be nice > to have this working. > > Andrew. From vladimir.kozlov at oracle.com Tue Jan 20 20:29:31 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 20 Jan 2015 12:29:31 -0800 Subject: [8u] backport of JDK-8068881 SIGBUS in C2 compiled method weblogic.wsee.jaxws.framework.jaxrpc.EnvironmentFactory$SimulatedWsdlDefinitions. In-Reply-To: References: Message-ID: <54BEBAAB.5010704@oracle.com> Looks good. Thanks, Vladimir On 1/20/15 12:14 PM, Igor Veresov wrote: > Webrev: http://cr.openjdk.java.net/~iveresov/8068881/webrev.8u40/ > > The change is slightly different from the one in jdk9: > - operator new needs Compile* when allocating a node > - compile timers usage is different > > Here is the diff between the patches: http://cr.openjdk.java.net/~iveresov/8068881/8u40.patch.diff > > JDK9 changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/b0ce179e4a01 > JDK9 webrev: http://cr.openjdk.java.net/~iveresov/8068881/webrev.02/ > > Nightlies seem fine. > > > Thanks! > igor > From igor.veresov at oracle.com Tue Jan 20 20:31:14 2015 From: igor.veresov at oracle.com (Igor Veresov) Date: Tue, 20 Jan 2015 12:31:14 -0800 Subject: [8u] backport of JDK-8068881 SIGBUS in C2 compiled method weblogic.wsee.jaxws.framework.jaxrpc.EnvironmentFactory$SimulatedWsdlDefinitions. In-Reply-To: <54BEBAAB.5010704@oracle.com> References: <54BEBAAB.5010704@oracle.com> Message-ID: <923930C9-5C44-40C6-88E5-1C9BF4335092@oracle.com> Thanks! igor > On Jan 20, 2015, at 12:29 PM, Vladimir Kozlov wrote: > > Looks good. > > Thanks, > Vladimir > > On 1/20/15 12:14 PM, Igor Veresov wrote: >> Webrev: http://cr.openjdk.java.net/~iveresov/8068881/webrev.8u40/ >> >> The change is slightly different from the one in jdk9: >> - operator new needs Compile* when allocating a node >> - compile timers usage is different >> >> Here is the diff between the patches: http://cr.openjdk.java.net/~iveresov/8068881/8u40.patch.diff >> >> JDK9 changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/b0ce179e4a01 >> JDK9 webrev: http://cr.openjdk.java.net/~iveresov/8068881/webrev.02/ >> >> Nightlies seem fine. >> >> >> Thanks! >> igor >> From dean.long at oracle.com Wed Jan 21 04:59:00 2015 From: dean.long at oracle.com (Dean Long) Date: Tue, 20 Jan 2015 20:59:00 -0800 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> <54B760DB.70200@oracle.com> <54B765DC.8050801@oracle.com> <54B845E3.7060000@oracle.com> <54B8EC3F.5030408@oracle.com> Message-ID: <54BF3214.2090903@oracle.com> Here's version 2, which does everything in vm.make and doesn't do anything that is shell-specific: http://cr.openjdk.java.net/~dlong/8031064//webrev.2/ thanks, dl From dmitry.samersoff at oracle.com Wed Jan 21 08:11:24 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Wed, 21 Jan 2015 11:11:24 +0300 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54BF3214.2090903@oracle.com> References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> <54B760DB.70200@oracle.com> <54B765DC.8050801@oracle.com> <54B845E3.7060000@oracle.com> <54B8EC3F.5030408@oracle.com> <54BF3214.2090903@oracle.com> Message-ID: <54BF5F2C.2020203@oracle.com> Dean, vm.make ll. 247 1. *.o should be $(Obj_Files) 2. $(NM) --defined-only *.o | sort -k3 -u | awk '/$(VMDEF_PAT)/{ print "\t" $$3 ";" }' should give you the same result with less efforts -Dmitry On 2015-01-21 07:59, Dean Long wrote: > Here's version 2, which does everything in vm.make and doesn't do > anything that is shell-specific: > > http://cr.openjdk.java.net/~dlong/8031064//webrev.2/ > > thanks, > > dl -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From dmitry.samersoff at oracle.com Wed Jan 21 08:16:40 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Wed, 21 Jan 2015 11:16:40 +0300 Subject: Serviceability Agent support in JDK9 In-Reply-To: <54BEA7E0.9040603@redhat.com> References: <54BEA7E0.9040603@redhat.com> Message-ID: <54BF6068.1060607@oracle.com> Andrew, You are correct. ClassWriter is missed couple of new classfile futures. Please, see https://bugs.openjdk.java.net/browse/JDK-8015848 -Dmitry On 2015-01-20 22:09, Andrew Haley wrote: > I'm having trouble with compiler replay data in JDK 9: > > $ ./build/linux-aarch64-normal-server-slowdebug/jdk/bin/java -Xbootclasspath/p:boot.jar -cp app.jar -XX:ReplayDataFile=replay.foo -XX:+ReplayCompiles > Error occurred during initialization of VM > java/lang/ClassFormatError: Missing BootstrapMethods attribute in class file java/lang/CharSequence > > I guess this is because sun.jvm.hotspot.tools.jcore.ClassWriter doesn't > handle the BootstrapMethods attribute. Is that right? It would be nice > to have this working. > > Andrew. > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From aph at redhat.com Wed Jan 21 09:15:00 2015 From: aph at redhat.com (Andrew Haley) Date: Wed, 21 Jan 2015 09:15:00 +0000 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54BEB1AC.7090508@oracle.com> References: <54A6A933.8000204@redhat.com> <7C4F2EEA-04D9-4421-8231-23E0A47A6502@oracle.com> <54AD4A98.9080501@redhat.com> <54B809C7.2040006@redhat.com> <54B819BA.5040804@oracle.com> <54BE75D7.6020609@redhat.com> <54BEB1AC.7090508@oracle.com> Message-ID: <54BF6E14.6080207@redhat.com> On 20/01/15 19:51, Dean Long wrote: > I believe offset_unit refers to the location of the relocation > (typically a 4-byte > machine instruction) and not the size of the reloc data, so you may want to > try changing to "4" in the future. Umm, really? I looked at the code and saw #ifdef ASSERT relocInfo::relocInfo(relocType t, int off, int f) { assert(t != data_prefix_tag, "cannot build a prefix this way"); assert((t & type_mask) == t, "wrong type"); assert((f & format_mask) == f, "wrong format"); assert(off >= 0 && off < offset_limit(), "offset out off bounds"); assert((off & (offset_unit-1)) == 0, "misaligned offset"); (*this) = relocInfo(t, RAW_BITS, off, f); } #endif which suggested to me that the offset must be aligned by offset_unit. Am I reading this wrongly? Andrew. From goetz.lindenmaier at sap.com Wed Jan 21 10:36:31 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 21 Jan 2015 10:36:31 +0000 Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java In-Reply-To: <1421759581.3259.5.camel@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF44B46@DEWDFEMB12A.global.corp.sap> <1421757893.3259.3.camel@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF70905@DEWDFEMB12A.global.corp.sap> <1421759581.3259.5.camel@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF70D12@DEWDFEMB12A.global.corp.sap> Hi Thomas, I also counted the VM used for compilation of the test. If I understood right, each test is first compiled, then executed. So for a VM with client and server it's 4 VM starts before (javac, test, 2xtestee), 6 after. And I have the feeling that the compilation eats a lot of test time. Setting up a shared helper class simplifies fixing the test. That's good. I've one change to go to fix the tests for ppc, then I'll address this one, ok? Best regards, Goetz. -----Original Message----- From: Thomas Schatzl [mailto:thomas.schatzl at oracle.com] Sent: Dienstag, 20. Januar 2015 14:13 To: Lindenmaier, Goetz Cc: 'hotspot-dev at openjdk.java.net' Subject: Re: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java On Tue, 2015-01-20 at 12:53 +0000, Lindenmaier, Goetz wrote: > Hi Thomas, > > yes, it would be better if the jtreg framework could sort it out right away. > But in this case, you would get two very similar tests, one of which > would be executed always, thus not saving anything. Not sure what you mean here: currently, the test starts three VMs, ie. the main VM that executes two other VMs where one always fails if there is no client VM. With the proposed change, jtreg will only start the VMs that are supported, i.e. either one or two. > Except for having to adapt two tests if there is a change needed. Since the meat of the code (TestHumongousCodeCacheRootsHelper) would be in a shared class and the shells just forwarding to that, I do not see a problem with such changes. I would imagine that the code would get cleaner and smaller, because the setup to execute the TestHumongousCodeCacheRootsHelper would go away. > Also, it's been pushed in the meantime. > http://hg.openjdk.java.net/jdk9/hs-gc/hotspot/rev/8dfd8b00c7f1 I did not notice that, somehow overlooked that - it was still on my todo-list :) Sorry. > Should I still improve the test? It would be nice, but not that important I guess. I created an RFE at https://bugs.openjdk.java.net/browse/JDK-8069343 to work on if you want. Thanks, Thomas From goetz.lindenmaier at sap.com Wed Jan 21 10:40:16 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 21 Jan 2015 10:40:16 +0000 Subject: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available In-Reply-To: <54B9970B.7040502@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF6DE04@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CF6FAFC@DEWDFEMB12A.global.corp.sap> <54B9970B.7040502@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF70D2F@DEWDFEMB12A.global.corp.sap> Hi, could please somebody from servicability sponsor this tiny change? Thanks, Goetz. -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore Sent: Freitag, 16. Januar 2015 23:56 To: hotspot-dev at openjdk.java.net Subject: Re: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available This seems fine. Someone from the serviceability group should probably sponsor. Thanks, Coleen On 1/16/15, 6:15 AM, Lindenmaier, Goetz wrote: > Hi, > > could I please get a review for this small fix? I also please need a sponsor. > > Thanks, > Goetz. > > From: goetz.lindenmaier at sap.com > Sent: Montag, 12. Januar 2015 09:23 > To: hotspot-dev at openjdk.java.net > Subject: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available > > Hi, > > This test uses SA, thus fails on platforms where that's not implemented. > I see problems on mac and aix. > Call Platform.shouldSAAttach() to check whether SA should work. > > Please review this small fix. I please need a sponsor. > http://cr.openjdk.java.net/~goetz/webrevs/8068778-jtregSA/webrev.01/ > > Best regards, > Goetz. From thomas.schatzl at oracle.com Wed Jan 21 11:06:04 2015 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 21 Jan 2015 12:06:04 +0100 Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF70D12@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF44B46@DEWDFEMB12A.global.corp.sap> <1421757893.3259.3.camel@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF70905@DEWDFEMB12A.global.corp.sap> <1421759581.3259.5.camel@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF70D12@DEWDFEMB12A.global.corp.sap> Message-ID: <1421838364.3298.7.camel@oracle.com> Hi Goetz, On Wed, 2015-01-21 at 10:36 +0000, Lindenmaier, Goetz wrote: > Hi Thomas, > > I also counted the VM used for compilation of the test. > If I understood right, each test is first compiled, then executed. > So for a VM with client and server it's 4 VM starts before (javac, > test, 2xtestee), 6 after. And I have the feeling that the compilation > eats a lot of test time. > > Setting up a shared helper class simplifies fixing the test. That's > good. > > I've one change to go to fix the tests for ppc, then I'll address this > one, ok? Dmitry Fazunenko mentioned that the proposed fix does not work due to shortcomings of jtreg. So there is no point in trying to improve this now. See the comment in the CR. Thanks for caring about this though. Thomas From goetz.lindenmaier at sap.com Wed Jan 21 11:16:56 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 21 Jan 2015 11:16:56 +0000 Subject: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java In-Reply-To: <1421838364.3298.7.camel@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF44B46@DEWDFEMB12A.global.corp.sap> <1421757893.3259.3.camel@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF70905@DEWDFEMB12A.global.corp.sap> <1421759581.3259.5.camel@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF70D12@DEWDFEMB12A.global.corp.sap> <1421838364.3298.7.camel@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF70E36@DEWDFEMB12A.global.corp.sap> OK! -----Original Message----- From: Thomas Schatzl [mailto:thomas.schatzl at oracle.com] Sent: Mittwoch, 21. Januar 2015 12:06 To: Lindenmaier, Goetz Cc: 'hotspot-dev at openjdk.java.net' Subject: Re: RFR (S): 8068026: [TESTBUG] Check for -client in gc/g1/TestHumongousCodeCacheRoots.java Hi Goetz, On Wed, 2015-01-21 at 10:36 +0000, Lindenmaier, Goetz wrote: > Hi Thomas, > > I also counted the VM used for compilation of the test. > If I understood right, each test is first compiled, then executed. > So for a VM with client and server it's 4 VM starts before (javac, > test, 2xtestee), 6 after. And I have the feeling that the compilation > eats a lot of test time. > > Setting up a shared helper class simplifies fixing the test. That's > good. > > I've one change to go to fix the tests for ppc, then I'll address this > one, ok? Dmitry Fazunenko mentioned that the proposed fix does not work due to shortcomings of jtreg. So there is no point in trying to improve this now. See the comment in the CR. Thanks for caring about this though. Thomas From dmitry.samersoff at oracle.com Wed Jan 21 13:23:08 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Wed, 21 Jan 2015 16:23:08 +0300 Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF69B65@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF69B49@DEWDFEMB12A.global.corp.sap> <54AD07C0.5030100@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF69B65@DEWDFEMB12A.global.corp.sap> Message-ID: <54BFA83C.8010200@oracle.com> Goetz, Looks good for me (not a reviewer). Platform.java:148 shouldSAAttach This function is not intended to control whether SA is implemented or not, it's just to check couple of well known conditions when attach fails unexpectedly. So it's better to return True (as it happens by default) for Aix and manage SA related problem on different layers (ProblemList.txt, etc). -Dmitry On 2015-01-07 13:52, Lindenmaier, Goetz wrote: > Thanks Fredrik! > > Best regards, > Goetz. > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Fredrik Arvidsson > Sent: Mittwoch, 7. Januar 2015 11:18 > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests > > Hi > > The changes to the DynLibDcmdTest look good to me. I am not a reviewer > though, but I wrote that test some time ago. > > /Fredrik > > On 2015-01-07 11:14, Lindenmaier, Goetz wrote: >> Hi, >> >> A row of hotspot jtreg tests fail because support to test for Aix is missing. >> To pass them, I would like to submit the following fixes: >> >> * runtime/6888954/vmerrors.sh: AIX can read from address '0', then throws SIGILL. >> * serviceability/dcmd/DynLibDcmdTest.java: Just add check for Aix. >> * test/test_env.sh: Add support for AIX. Sort OSes alphabetical. >> * test/testlibrary/com/oracle/java/testlibrary/Platform.java: add isAix(). Sort OSes alphabetical. >> >> Please review this change, and I please need a sponsor. >> http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev-01/ >> >> Thanks and best regards, >> Goetz. >> > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From goetz.lindenmaier at sap.com Wed Jan 21 14:10:50 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 21 Jan 2015 14:10:50 +0000 Subject: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests In-Reply-To: <54BFA83C.8010200@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF69B49@DEWDFEMB12A.global.corp.sap> <54AD07C0.5030100@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF69B65@DEWDFEMB12A.global.corp.sap> <54BFA83C.8010200@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF70ED6@DEWDFEMB12A.global.corp.sap> Hi Dmitry, I agree it would be better if the test would not be started at all. But I think this should be expressed in the tests. The ProblemList is not submitted to the repository, is it? So how should somebody, who just checks out the repositories, know the test is expected to fail? The change has been pushed in the meantime. Thanks, Goetz. -----Original Message----- From: Dmitry Samersoff [mailto:dmitry.samersoff at oracle.com] Sent: Mittwoch, 21. Januar 2015 14:23 To: Lindenmaier, Goetz; Fredrik Arvidsson; hotspot-dev at openjdk.java.net Subject: Re: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests Goetz, Looks good for me (not a reviewer). Platform.java:148 shouldSAAttach This function is not intended to control whether SA is implemented or not, it's just to check couple of well known conditions when attach fails unexpectedly. So it's better to return True (as it happens by default) for Aix and manage SA related problem on different layers (ProblemList.txt, etc). -Dmitry On 2015-01-07 13:52, Lindenmaier, Goetz wrote: > Thanks Fredrik! > > Best regards, > Goetz. > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Fredrik Arvidsson > Sent: Mittwoch, 7. Januar 2015 11:18 > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8068013: [TESTBUG] Aix support in hotspot jtreg tests > > Hi > > The changes to the DynLibDcmdTest look good to me. I am not a reviewer > though, but I wrote that test some time ago. > > /Fredrik > > On 2015-01-07 11:14, Lindenmaier, Goetz wrote: >> Hi, >> >> A row of hotspot jtreg tests fail because support to test for Aix is missing. >> To pass them, I would like to submit the following fixes: >> >> * runtime/6888954/vmerrors.sh: AIX can read from address '0', then throws SIGILL. >> * serviceability/dcmd/DynLibDcmdTest.java: Just add check for Aix. >> * test/test_env.sh: Add support for AIX. Sort OSes alphabetical. >> * test/testlibrary/com/oracle/java/testlibrary/Platform.java: add isAix(). Sort OSes alphabetical. >> >> Please review this change, and I please need a sponsor. >> http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev-01/ >> >> Thanks and best regards, >> Goetz. >> > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From john.r.rose at oracle.com Wed Jan 21 19:45:55 2015 From: john.r.rose at oracle.com (John Rose) Date: Wed, 21 Jan 2015 11:45:55 -0800 Subject: 8068054: AARCH64: Assembler interpreter, shared runtime In-Reply-To: <54BF6E14.6080207@redhat.com> References: <54A6A933.8000204@redhat.com> <7C4F2EEA-04D9-4421-8231-23E0A47A6502@oracle.com> <54AD4A98.9080501@redhat.com> <54B809C7.2040006@redhat.com> <54B819BA.5040804@oracle.com> <54BE75D7.6020609@redhat.com> <54BEB1AC.7090508@oracle.com> <54BF6E14.6080207@redhat.com> Message-ID: <7B56B96F-4063-4F8A-8FF5-22CA537F4845@oracle.com> On Jan 21, 2015, at 1:15 AM, Andrew Haley wrote: > > which suggested to me that the offset must be aligned by offset_unit. offset_unit is a platform-specific number which exposes instruction alignment for reloc offsets. It is 1 on x86 and 4 on SPARC. ? John From dean.long at oracle.com Wed Jan 21 22:39:31 2015 From: dean.long at oracle.com (Dean Long) Date: Wed, 21 Jan 2015 14:39:31 -0800 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54BF5F2C.2020203@oracle.com> References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> <54B760DB.70200@oracle.com> <54B765DC.8050801@oracle.com> <54B845E3.7060000@oracle.com> <54B8EC3F.5030408@oracle.com> <54BF3214.2090903@oracle.com> <54BF5F2C.2020203@oracle.com> Message-ID: <54C02AA3.6020503@oracle.com> Thanks Dmitry. The updated webrev is here: http://cr.openjdk.java.net/~dlong/8031064/webrev.3/ dl On 1/21/2015 12:11 AM, Dmitry Samersoff wrote: > Dean, > > vm.make ll. 247 > > 1. *.o should be $(Obj_Files) > > 2. > > $(NM) --defined-only *.o | sort -k3 -u | > awk '/$(VMDEF_PAT)/{ print "\t" $$3 ";" }' > > should give you the same result with less efforts > > -Dmitry > > On 2015-01-21 07:59, Dean Long wrote: >> Here's version 2, which does everything in vm.make and doesn't do >> anything that is shell-specific: >> >> http://cr.openjdk.java.net/~dlong/8031064//webrev.2/ >> >> thanks, >> >> dl > From dmitry.samersoff at oracle.com Thu Jan 22 08:32:06 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Thu, 22 Jan 2015 11:32:06 +0300 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54C02AA3.6020503@oracle.com> References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> <54B760DB.70200@oracle.com> <54B765DC.8050801@oracle.com> <54B845E3.7060000@oracle.com> <54B8EC3F.5030408@oracle.com> <54BF3214.2090903@oracle.com> <54BF5F2C.2020203@oracle.com> <54C02AA3.6020503@oracle.com> Message-ID: <54C0B586.3040004@oracle.com> Dean, Looks good for me. (not a reviewer) Thank you for fixing it. -Dmitry On 2015-01-22 01:39, Dean Long wrote: > Thanks Dmitry. The updated webrev is here: > > http://cr.openjdk.java.net/~dlong/8031064/webrev.3/ > > dl > > On 1/21/2015 12:11 AM, Dmitry Samersoff wrote: >> Dean, >> >> vm.make ll. 247 >> >> 1. *.o should be $(Obj_Files) >> >> 2. >> >> $(NM) --defined-only *.o | sort -k3 -u | >> awk '/$(VMDEF_PAT)/{ print "\t" $$3 ";" }' >> >> should give you the same result with less efforts >> >> -Dmitry >> >> On 2015-01-21 07:59, Dean Long wrote: >>> Here's version 2, which does everything in vm.make and doesn't do >>> anything that is shell-specific: >>> >>> http://cr.openjdk.java.net/~dlong/8031064//webrev.2/ >>> >>> thanks, >>> >>> dl >> > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From goetz.lindenmaier at sap.com Thu Jan 22 09:24:43 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 22 Jan 2015 09:24:43 +0000 Subject: [8u] backport 8068013: [TESTBUG] Aix support in hotspot jtreg tests Message-ID: <4295855A5C1DE049A61835A1887419CC2CF7125A@DEWDFEMB12A.global.corp.sap> Hi, I would like to backport this change. It basically makes the jtreg test infrastructure aware of aix. This fixes a lot of the jtreg tests. The change from 9 applied cleanly with some obvious exceptions: - The copyright adaption didn't merge in * test/runtime/6888954/vmerrors.sh * test/test_env.sh * test/testlibrary/com/oracle/java/testlibrary/Platform.java - Changes in files not available in 8: * test/serviceability/dcmd/DynLibDcmdTest.java: Test does not * test/testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java The webrev applying to 8: http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev.02-jdk8/ The original webrev http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev-02/ The incremental patch: http://cr.openjdk.java.net/~goetz/webrevs/8068013-jtregAix/webrev.02-jdk8/incremental_diff.patch I please need a sponsor. Best regards, Goetz. From david.holmes at oracle.com Thu Jan 22 10:19:40 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 22 Jan 2015 20:19:40 +1000 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54C02AA3.6020503@oracle.com> References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> <54B760DB.70200@oracle.com> <54B765DC.8050801@oracle.com> <54B845E3.7060000@oracle.com> <54B8EC3F.5030408@oracle.com> <54BF3214.2090903@oracle.com> <54BF5F2C.2020203@oracle.com> <54C02AA3.6020503@oracle.com> Message-ID: <54C0CEBC.4030807@oracle.com> On 22/01/2015 8:39 AM, Dean Long wrote: > Thanks Dmitry. The updated webrev is here: > > http://cr.openjdk.java.net/~dlong/8031064/webrev.3/ This looks weird: + VMDEF_PAT = ^_ZTV + VMDEF_PAT := ^gHotSpotVM|$(VMDEF_PAT) + VMDEF_PAT := ^UseSharedSpaces$$|$(VMDEF_PAT) + VMDEF_PAT := ^_ZN9Arguments17SharedArchivePathE$$|$(VMDEF_PAT) but I can sort of see why you wanted to do it that way. I assume you have verified the results are identical? I would be good to see this applied uniformly across all platforms as well (except windows). Thanks, David > dl > > On 1/21/2015 12:11 AM, Dmitry Samersoff wrote: >> Dean, >> >> vm.make ll. 247 >> >> 1. *.o should be $(Obj_Files) >> >> 2. >> >> $(NM) --defined-only *.o | sort -k3 -u | >> awk '/$(VMDEF_PAT)/{ print "\t" $$3 ";" }' >> >> should give you the same result with less efforts >> >> -Dmitry >> >> On 2015-01-21 07:59, Dean Long wrote: >>> Here's version 2, which does everything in vm.make and doesn't do >>> anything that is shell-specific: >>> >>> http://cr.openjdk.java.net/~dlong/8031064//webrev.2/ >>> >>> thanks, >>> >>> dl >> > From dean.long at oracle.com Thu Jan 22 18:01:07 2015 From: dean.long at oracle.com (Dean Long) Date: Thu, 22 Jan 2015 10:01:07 -0800 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54C0CEBC.4030807@oracle.com> References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> <54B760DB.70200@oracle.com> <54B765DC.8050801@oracle.com> <54B845E3.7060000@oracle.com> <54B8EC3F.5030408@oracle.com> <54BF3214.2090903@oracle.com> <54BF5F2C.2020203@oracle.com> <54C02AA3.6020503@oracle.com> <54C0CEBC.4030807@oracle.com> Message-ID: <54C13AE3.1020506@oracle.com> On 1/22/2015 2:19 AM, David Holmes wrote: > On 22/01/2015 8:39 AM, Dean Long wrote: >> Thanks Dmitry. The updated webrev is here: >> >> http://cr.openjdk.java.net/~dlong/8031064/webrev.3/ > > This looks weird: > > + VMDEF_PAT = ^_ZTV > + VMDEF_PAT := ^gHotSpotVM|$(VMDEF_PAT) > + VMDEF_PAT := ^UseSharedSpaces$$|$(VMDEF_PAT) > + VMDEF_PAT := ^_ZN9Arguments17SharedArchivePathE$$|$(VMDEF_PAT) > > but I can sort of see why you wanted to do it that way. > Do you have a suggestion for a less-weird-looking way to do it? > I assume you have verified the results are identical? > Yes. > I would be good to see this applied uniformly across all platforms as > well (except windows). > I suppose, but isn't linux the only platform where we might be cross-compiling? I'm not setup to build for aix, bsd, or solaris, and if I build in JPRT, I'm not sure it will save the vm.def or mapfile to make a comparison against. Can we make cleaning up the other platforms a separate RFE? dl > Thanks, > David > >> dl >> >> On 1/21/2015 12:11 AM, Dmitry Samersoff wrote: >>> Dean, >>> >>> vm.make ll. 247 >>> >>> 1. *.o should be $(Obj_Files) >>> >>> 2. >>> >>> $(NM) --defined-only *.o | sort -k3 -u | >>> awk '/$(VMDEF_PAT)/{ print "\t" $$3 ";" }' >>> >>> should give you the same result with less efforts >>> >>> -Dmitry >>> >>> On 2015-01-21 07:59, Dean Long wrote: >>>> Here's version 2, which does everything in vm.make and doesn't do >>>> anything that is shell-specific: >>>> >>>> http://cr.openjdk.java.net/~dlong/8031064//webrev.2/ >>>> >>>> thanks, >>>> >>>> dl >>> >> From erik.osterlund at lnu.se Fri Jan 23 01:19:12 2015 From: erik.osterlund at lnu.se (=?iso-8859-1?Q?Erik_=D6sterlund?=) Date: Fri, 23 Jan 2015 01:19:12 +0000 Subject: RFR: 7143664: Clean up OrderAccess implementations and usage Message-ID: Hi all, == Summary of Changes == This changeset fixes issues in OrderAccess on multiple levels from the memory model semantics to compiler reorderings, to addressing maintainability/portability issues which (almost) had to be fixed in order to fix the correctness issues. It is the result of discussions found in the previous "OrderAccess Refactoring" thread: http://openjdk.5641.n7.nabble.com/OrderAccess-Refactoring-td212050.html Bug ID: https://bugs.openjdk.java.net/browse/JDK-7143664 (updated to reflect these related changes) Webrev: http://cr.openjdk.java.net/~dholmes/7143664/webrev/ Before I describe more I would like to give special thanks to David Holmes for long discussions leading up to the currently proposed changes. I would also like to thank Jesper Wilhelmsson for helping me run my changes through JPRT. == Motivation == This change directly fixes a reported OrderAccess bug due to compiler reorderings which is still a vulnerability on almost all TSO platforms: https://bugs.openjdk.java.net/browse/JDK-806196 And directly fixes confusions like release_store() != release() store() due to memory model issues previously described in this bug ID. At the same time it provides clearer design with separation of concerns and generalization/specialization, removing a whole bunch of platform specific code which could be generalized. The platform specific files now only have a few LoC requirements (~7) to conform to the memory model by specifying what the stand alone barriers do. Then optionally optimizations to the general approach are possible if platforms support it. This also makes it much easier to port to new platforms. == Memory Model == The current definitions of acquire/release semantics are a bit fishy leading to problems previously described in the bug ID (release_store() != release() store()) and some other correctness issues. It has therefore been replaced with a new model. I would like to thank David Holmes for the long discussions leading up to the newly proposed model. The new model is formally defined like this: // T1: access_shared_data // T1: ]release // T1: (...) // T1: store(X) // // T2: load(X) // T2: (...) // T2: acquire[ // T2: access_shared_data // // It is guaranteed that if T2: load(X) synchronizes with (observes the // value written by) T1: store(X), then the memory accesses before the // T1: ]release happen before the memory accesses after the T2: acquire[. The orderAccess.hpp file and bug ID also has a few additional explanations making it more intuitive to the user when to use acquire/release and the resemblance to TSO abstract machines. Big thanks goes to David Holmes for discussing the memory model with me, which helped a lot in deriving it. Now it holds that release() store() == release_store(), and the other correctness issues are fixed as well. The new model is also closer to C++11 definitions which could give us more relaxed compiler reordering constraints in the future when compiler support for C++11 is there and ready. == Reliance on C++ Volatile Semantics == The C++ standard section 1.9 "Program Execution" is very vague about what the keyword volatile can actually do for us. It gives clear constraints in terms of volatile-volatile accesses but says little about nonvolatile-volatile accesses. Yet the current implementation heavily relies upon volatile to in terms of compiler reordering. But GCC explicitly declares that volatiles and non-volatiles may reorder freely ( https://gcc.gnu.org/onlinedocs/gcc/Volatiles.html ). The only compiler known to explicitly provide the wanted semantics with volatile is MSVC 2010 for windows ( https://msdn.microsoft.com/en-us/library/12a04hfd(v=vs.100).aspx ). Compilers not giving explicit guarantees, must be considered unsafe and revert to conservative behaviour. This was brought to attention after causing bugs, but was only fixed for x86 linux. This is a fundamental issue inherent to all TSO platforms except windows, and has to be fixed on all of them. Several barriers are unsafe to use because they lack compiler reordering constraints (e.g. fence and acquire on linux_SPARC). For TSO platforms they are typically implemented using dummy loads and stores. This seems to be another old volatile reliance that I fixed. These barriers sometimes have omitted compiler barriers (which is what we really want). This seems to be another example on incorrect reliance on the volatile semantics to help us. Therefore dummy loads/stores have been replaced with compiler barriers on TSO platforms. It is also worth noting that compilers like sun studio did previously not support inline asm syntax. Therefore, barriers were implemented in .il-files. However, using them does not give explicit compiler constraints for reordering AFAIK. Therefore, they have been reimplemented using inline asm with explicit compiler reordering constraints, as even sun (solaris?) studio now supports this. The windows variants have added a windows-style _ReadWriteBarrier() compiler barrier similarly. == Strange Hardware Reorderings == Fixed a weird inconsistency where acquire, loadstore and loadlaod would use isync instead of lwsync for PPC on linux_zero, but not in any other PPC platform in the repo. I assumed this is wrong and changed it to lwsync instead. == Code Redundancy and Refactoring == The OrderAccess code looks like it has been developed over a long period of time, with small incremental changes. This seems to have led to a lot of code duplication over time. For example, store_fence variants are not referenced from anywhere, yet contribute to a lot of the code base and a lot of awkwardness (such as being the one only exception not using volatiles for some reason). Moreover, store_fence is not used anywhere in hotspot because it is not a good fit for either the acquire/release semantics or the Java volatile semantics, leaving a big question mark on when it should ever be used. I took the liberty of removing it. Another redundancy issue is that most of the semantics is exactly the same for all platforms, yet all that default boilerplate such as how to make atomic accesses, where acquire/release is supposed to be placed w.r.t. the memory access, what the different barriers should do etc. is copied in redundantly for each os_cpu and each type of memory access for each os_cpu. This makes it extremely painful 1) to understand what actually defines a certain platform compared to the defaults and 2) to correct bugs like those discovered here 3) prevent silly mistakes and bugs, by simply having a lot less code defining the behaviour of OrderAccess that could go wrong. A new architecture/design for OrderAccess is proposed, using a generalization/specialization approach. A general implementation in /share/ defines how things work and splits into different concerns: 1) how to make an atomic memory access, 2) where to but barriers w.r.t. the memory access for things like load_acquire, release_store and release_store_fence, 3) how these barriers are defined. This allows a clear split between what is required for following the specifications, and optimizations, which become much more readable and only optimizations need to be reviewed in depth as the defaults can always be trusted given correct standalone barriers. The only thing a platform is required to specify, is what an implementation of acquire(), release() and fence() should do. If they are implemented properly, everything in OrderAccess is guaranteed to work according to specification using the generalized code. This makes it very easy to support new ports. ALL the other code in the os_cpu files is used /only/ for optimization purposes offered for specific configurations. However, it is highly customizable so that specific platform can perform any desired optimizations. For instance this load_acquire on PPC is optimized: template<> inline jbyte OrderAccess::specialized_load_acquire (volatile jbyte* p) { register jbyte t = load(p); inlasm_acquire_reg(t); return t; } This overrides the whole load_acquire implementation to do something custom. Platforms like x86 extensively use this for joined fencing variants to optimize. The default implementation of load_acquire() will make an atomic load() followed by acquire() since the semantics is generalized. The generalized semantics are defined using inlined postfix/prefix calls after/before the atomic access, as defined here: template<> inline void ScopedFenceGeneral::postfix() { OrderAccess::acquire(); } template<> inline void ScopedFenceGeneral::prefix() { OrderAccess::release(); } template<> inline void ScopedFenceGeneral::prefix() { OrderAccess::release(); } template<> inline void ScopedFenceGeneral::postfix() { OrderAccess::fence(); } For platforms that simply wish to override what e.g. acquire means for a joined ordered memory access in general, as different to calling stand alone acquire(), the semantics can be easily overridden for a platform such as windows like on windows: template<> inline void ScopedFence::postfix() { } template<> inline void ScopedFence::prefix() { } template<> inline void ScopedFence::prefix() { } template<> inline void ScopedFence::postfix() { OrderAccess::fence(); } In this example, since Windows (now) has a compiler barrier for acquire, but does not need it for joined accesses since volatile has stronger guarantees on windows, this is enough to specialize that for joined memory accesses, no extra protection is needed. == Backward Compatibility and Transitioning == Since the newly proposed code is structured differently to before, a #define was added for backward compatibility so that external repositories not adhering to this new architecture do not break. Furthermore, store_release was declared private and marked as deprecated. This allows for a smooth transition into the new style of OrderAccess. When the full transition is made in all known repos, the #define and store_fence may be safely removed, eventually. == Documentation == The documentation seems old and outdated, describing how it works on SPARC RMO and IA64, which are nowhere to be found in the repository. It also describes things about C++ volatiles which cannot be relied upon. The documentation has been cleaned up to match the current state of the implementation better, with architectures actually found in the repository. == Testing == JPRT. Big thanks to Jesper Wilhelmsson for helping me test these changes. Ran some DaCapo benchmarks (I know okay :p) for performance regression and there was no perceivable difference. Looking forward to feedback on this, and hope to get some reviews. :) Thanks, Erik From dean.long at oracle.com Fri Jan 23 06:36:03 2015 From: dean.long at oracle.com (Dean Long) Date: Thu, 22 Jan 2015 22:36:03 -0800 Subject: RFR: 7143664: Clean up OrderAccess implementations and usage In-Reply-To: References: Message-ID: <54C1EBD3.1000905@oracle.com> 147 // Ordering a load relative to preceding stores requires a fence, 148 // which implies a membar #StoreLoad between the store and load under 149 // sparc-TSO. A fence is required by x86. On x86, we use explicitly 150 // locked add. 151 // It sounds like the above is saying that fence is the same as StoreLoad ... 152 // 4. store, load <= is constrained by => store, fence, load 153 // 154 // Use store, fence to make sure all stores done in an 'interesting' 155 // region are made visible prior to both subsequent loads and stores. ... and this is like saying to use fence because StoreStore | StoreLoad isn't available. > Furthermore, store_release was declared private and marked as deprecated. I can't find where this was done. dl On 1/22/2015 5:19 PM, Erik ?sterlund wrote: > Hi all, > > == Summary of Changes == > > This changeset fixes issues in OrderAccess on multiple levels from the > memory model semantics to compiler reorderings, to addressing > maintainability/portability issues which (almost) had to be fixed in > order to fix the correctness issues. It is the result of discussions > found in the previous "OrderAccess Refactoring" thread: > http://openjdk.5641.n7.nabble.com/OrderAccess-Refactoring-td212050.html > > Bug ID: > https://bugs.openjdk.java.net/browse/JDK-7143664 > (updated to reflect these related changes) > > Webrev: > http://cr.openjdk.java.net/~dholmes/7143664/webrev/ > > Before I describe more I would like to give special thanks to David > Holmes for long discussions leading up to the currently proposed > changes. I would also like to thank Jesper Wilhelmsson for helping me > run my changes through JPRT. > > == Motivation == > > This change directly fixes a reported OrderAccess bug due to compiler > reorderings which is still a vulnerability on almost all TSO platforms: > https://bugs.openjdk.java.net/browse/JDK-806196 > > And directly fixes confusions like release_store() != release() store() > due to memory model issues previously described in this bug ID. > > At the same time it provides clearer design with separation of concerns > and generalization/specialization, removing a whole bunch of platform > specific code which could be generalized. The platform specific files > now only have a few LoC requirements (~7) to conform to the memory model > by specifying what the stand alone barriers do. Then optionally > optimizations to the general approach are possible if platforms support > it. This also makes it much easier to port to new platforms. > > == Memory Model == > > The current definitions of acquire/release semantics are a bit fishy > leading to problems previously described in the bug ID (release_store() > != release() store()) and some other correctness issues. It has > therefore been replaced with a new model. I would like to thank David > Holmes for the long discussions leading up to the newly proposed model. > > The new model is formally defined like this: > > // T1: access_shared_data > // T1: ]release > // T1: (...) > // T1: store(X) > // > // T2: load(X) > // T2: (...) > // T2: acquire[ > // T2: access_shared_data > // > // It is guaranteed that if T2: load(X) synchronizes with (observes the > // value written by) T1: store(X), then the memory accesses before the > // T1: ]release happen before the memory accesses after the T2: acquire[. > > The orderAccess.hpp file and bug ID also has a few additional > explanations making it more intuitive to the user when to use > acquire/release and the resemblance to TSO abstract machines. Big thanks > goes to David Holmes for discussing the memory model with me, which > helped a lot in deriving it. > > Now it holds that release() store() == release_store(), and the other > correctness issues are fixed as well. > > The new model is also closer to C++11 definitions which could give us > more relaxed compiler reordering constraints in the future when compiler > support for C++11 is there and ready. > > == Reliance on C++ Volatile Semantics == > > The C++ standard section 1.9 "Program Execution" is very vague about > what the keyword volatile can actually do for us. It gives clear > constraints in terms of volatile-volatile accesses but says little about > nonvolatile-volatile accesses. Yet the current implementation heavily > relies upon volatile to in terms of compiler reordering. But GCC > explicitly declares that volatiles and non-volatiles may reorder freely > ( https://gcc.gnu.org/onlinedocs/gcc/Volatiles.html ). The only compiler > known to explicitly provide the wanted semantics with volatile is MSVC > 2010 for windows ( > https://msdn.microsoft.com/en-us/library/12a04hfd(v=vs.100).aspx ). > Compilers not giving explicit guarantees, must be considered unsafe and > revert to conservative behaviour. > > This was brought to attention after causing bugs, but was only fixed for > x86 linux. This is a fundamental issue inherent to all TSO platforms > except windows, and has to be fixed on all of them. > > Several barriers are unsafe to use because they lack compiler reordering > constraints (e.g. fence and acquire on linux_SPARC). For TSO platforms > they are typically implemented using dummy loads and stores. This seems > to be another old volatile reliance that I fixed. These barriers > sometimes have omitted compiler barriers (which is what we really want). > This seems to be another example on incorrect reliance on the volatile > semantics to help us. Therefore dummy loads/stores have been replaced > with compiler barriers on TSO platforms. > > It is also worth noting that compilers like sun studio did previously > not support inline asm syntax. Therefore, barriers were implemented in > .il-files. However, using them does not give explicit compiler > constraints for reordering AFAIK. Therefore, they have been > reimplemented using inline asm with explicit compiler reordering > constraints, as even sun (solaris?) studio now supports this. > > The windows variants have added a windows-style _ReadWriteBarrier() > compiler barrier similarly. > > == Strange Hardware Reorderings == > > Fixed a weird inconsistency where acquire, loadstore and loadlaod would > use isync instead of lwsync for PPC on linux_zero, but not in any other > PPC platform in the repo. I assumed this is wrong and changed it to > lwsync instead. > > == Code Redundancy and Refactoring == > > The OrderAccess code looks like it has been developed over a long period > of time, with small incremental changes. This seems to have led to a lot > of code duplication over time. For example, store_fence variants are not > referenced from anywhere, yet contribute to a lot of the code base and a > lot of awkwardness (such as being the one only exception not using > volatiles for some reason). Moreover, store_fence is not used anywhere > in hotspot because it is not a good fit for either the acquire/release > semantics or the Java volatile semantics, leaving a big question mark on > when it should ever be used. I took the liberty of removing it. > > Another redundancy issue is that most of the semantics is exactly the > same for all platforms, yet all that default boilerplate such as how to > make atomic accesses, where acquire/release is supposed to be placed > w.r.t. the memory access, what the different barriers should do etc. is > copied in redundantly for each os_cpu and each type of memory access for > each os_cpu. This makes it extremely painful 1) to understand what > actually defines a certain platform compared to the defaults and 2) to > correct bugs like those discovered here 3) prevent silly mistakes and > bugs, by simply having a lot less code defining the behaviour of > OrderAccess that could go wrong. > > A new architecture/design for OrderAccess is proposed, using a > generalization/specialization approach. > > A general implementation in /share/ defines how things work and splits > into different concerns: 1) how to make an atomic memory access, 2) > where to but barriers w.r.t. the memory access for things like > load_acquire, release_store and release_store_fence, 3) how these > barriers are defined. > > This allows a clear split between what is required for following the > specifications, and optimizations, which become much more readable and > only optimizations need to be reviewed in depth as the defaults can > always be trusted given correct standalone barriers. > > The only thing a platform is required to specify, is what an > implementation of acquire(), release() and fence() should do. If they > are implemented properly, everything in OrderAccess is guaranteed to > work according to specification using the generalized code. This makes > it very easy to support new ports. ALL the other code in the os_cpu > files is used /only/ for optimization purposes offered for specific > configurations. > > However, it is highly customizable so that specific platform can perform > any desired optimizations. For instance this load_acquire on PPC is > optimized: > > template<> inline jbyte OrderAccess::specialized_load_acquire > (volatile jbyte* p) { register jbyte t = load(p); > inlasm_acquire_reg(t); return t; } > > This overrides the whole load_acquire implementation to do something > custom. Platforms like x86 extensively use this for joined fencing > variants to optimize. > > The default implementation of load_acquire() will make an atomic load() > followed by acquire() since the semantics is generalized. The > generalized semantics are defined using inlined postfix/prefix calls > after/before the atomic access, as defined here: > > template<> inline void ScopedFenceGeneral::postfix() { > OrderAccess::acquire(); } > template<> inline void ScopedFenceGeneral::prefix() { > OrderAccess::release(); } > template<> inline void ScopedFenceGeneral::prefix() { > OrderAccess::release(); } > template<> inline void ScopedFenceGeneral::postfix() { > OrderAccess::fence(); } > > For platforms that simply wish to override what e.g. acquire means for a > joined ordered memory access in general, as different to calling stand > alone acquire(), the semantics can be easily overridden for a platform > such as windows like on windows: > > template<> inline void ScopedFence::postfix() { } > template<> inline void ScopedFence::prefix() { } > template<> inline void ScopedFence::prefix() { } > template<> inline void ScopedFence::postfix() { > OrderAccess::fence(); } > > In this example, since Windows (now) has a compiler barrier for acquire, > but does not need it for joined accesses since volatile has stronger > guarantees on windows, this is enough to specialize that for joined > memory accesses, no extra protection is needed. > > == Backward Compatibility and Transitioning == > > Since the newly proposed code is structured differently to before, a > #define was added for backward compatibility so that external > repositories not adhering to this new architecture do not break. > Furthermore, store_release was declared private and marked as > deprecated. This allows for a smooth transition into the new style of > OrderAccess. When the full transition is made in all known repos, the > #define and store_fence may be safely removed, eventually. > > == Documentation == > > The documentation seems old and outdated, describing how it works on > SPARC RMO and IA64, which are nowhere to be found in the repository. It > also describes things about C++ volatiles which cannot be relied upon. > The documentation has been cleaned up to match the current state of the > implementation better, with architectures actually found in the repository. > > == Testing == > > JPRT. Big thanks to Jesper Wilhelmsson for helping me test these changes. > Ran some DaCapo benchmarks (I know okay :p) for performance regression > and there was no perceivable difference. > > Looking forward to feedback on this, and hope to get some reviews. :) > > Thanks, > Erik From dmitry.samersoff at oracle.com Fri Jan 23 06:45:27 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Fri, 23 Jan 2015 09:45:27 +0300 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54C13AE3.1020506@oracle.com> References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> <54B760DB.70200@oracle.com> <54B765DC.8050801@oracle.com> <54B845E3.7060000@oracle.com> <54B8EC3F.5030408@oracle.com> <54BF3214.2090903@oracle.com> <54BF5F2C.2020203@oracle.com> <54C02AA3.6020503@oracle.com> <54C0CEBC.4030807@oracle.com> <54C13AE3.1020506@oracle.com> Message-ID: <54C1EE07.6040308@oracle.com> Dean, > Can we make cleaning up the other platforms > a separate RFE? I think yes. So please file the RFE. -Dmitry On 2015-01-22 21:01, Dean Long wrote: > On 1/22/2015 2:19 AM, David Holmes wrote: >> On 22/01/2015 8:39 AM, Dean Long wrote: >>> Thanks Dmitry. The updated webrev is here: >>> >>> http://cr.openjdk.java.net/~dlong/8031064/webrev.3/ >> >> This looks weird: >> >> + VMDEF_PAT = ^_ZTV >> + VMDEF_PAT := ^gHotSpotVM|$(VMDEF_PAT) >> + VMDEF_PAT := ^UseSharedSpaces$$|$(VMDEF_PAT) >> + VMDEF_PAT := ^_ZN9Arguments17SharedArchivePathE$$|$(VMDEF_PAT) >> >> but I can sort of see why you wanted to do it that way. >> > > Do you have a suggestion for a less-weird-looking way to do it? > >> I assume you have verified the results are identical? >> > > Yes. > >> I would be good to see this applied uniformly across all platforms as >> well (except windows). >> > > I suppose, but isn't linux the only platform where we might be > cross-compiling? I'm not setup to > build for aix, bsd, or solaris, and if I build in JPRT, I'm not sure it > will save the vm.def or mapfile to > make a comparison against. Can we make cleaning up the other platforms > a separate RFE? > > dl > >> Thanks, >> David >> >>> dl >>> >>> On 1/21/2015 12:11 AM, Dmitry Samersoff wrote: >>>> Dean, >>>> >>>> vm.make ll. 247 >>>> >>>> 1. *.o should be $(Obj_Files) >>>> >>>> 2. >>>> >>>> $(NM) --defined-only *.o | sort -k3 -u | >>>> awk '/$(VMDEF_PAT)/{ print "\t" $$3 ";" }' >>>> >>>> should give you the same result with less efforts >>>> >>>> -Dmitry >>>> >>>> On 2015-01-21 07:59, Dean Long wrote: >>>>> Here's version 2, which does everything in vm.make and doesn't do >>>>> anything that is shell-specific: >>>>> >>>>> http://cr.openjdk.java.net/~dlong/8031064//webrev.2/ >>>>> >>>>> thanks, >>>>> >>>>> dl >>>> >>> > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From david.holmes at oracle.com Fri Jan 23 07:01:59 2015 From: david.holmes at oracle.com (David Holmes) Date: Fri, 23 Jan 2015 17:01:59 +1000 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54C13AE3.1020506@oracle.com> References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> <54B760DB.70200@oracle.com> <54B765DC.8050801@oracle.com> <54B845E3.7060000@oracle.com> <54B8EC3F.5030408@oracle.com> <54BF3214.2090903@oracle.com> <54BF5F2C.2020203@oracle.com> <54C02AA3.6020503@oracle.com> <54C0CEBC.4030807@oracle.com> <54C13AE3.1020506@oracle.com> Message-ID: <54C1F1E7.80201@oracle.com> On 23/01/2015 4:01 AM, Dean Long wrote: > On 1/22/2015 2:19 AM, David Holmes wrote: >> On 22/01/2015 8:39 AM, Dean Long wrote: >>> Thanks Dmitry. The updated webrev is here: >>> >>> http://cr.openjdk.java.net/~dlong/8031064/webrev.3/ >> >> This looks weird: >> >> + VMDEF_PAT = ^_ZTV >> + VMDEF_PAT := ^gHotSpotVM|$(VMDEF_PAT) >> + VMDEF_PAT := ^UseSharedSpaces$$|$(VMDEF_PAT) >> + VMDEF_PAT := ^_ZN9Arguments17SharedArchivePathE$$|$(VMDEF_PAT) >> >> but I can sort of see why you wanted to do it that way. >> > > Do you have a suggestion for a less-weird-looking way to do it? Only the obvious but hard to read: VMDEF_PAT := ^_ZN9Arguments17SharedArchivePathE$$|^UseSharedSpaces$$|^gHotSpotVM|^_ZTV >> I assume you have verified the results are identical? >> > > Yes. > >> I would be good to see this applied uniformly across all platforms as >> well (except windows). >> > > I suppose, but isn't linux the only platform where we might be > cross-compiling? I'm not setup to > build for aix, bsd, or solaris, and if I build in JPRT, I'm not sure it > will save the vm.def or mapfile to > make a comparison against. Can we make cleaning up the other platforms > a separate RFE? Getting rid of shell scripts from the build is a Good Thing(TM)! But yes we can make this a separate RFE. Thanks, David > > dl > >> Thanks, >> David >> >>> dl >>> >>> On 1/21/2015 12:11 AM, Dmitry Samersoff wrote: >>>> Dean, >>>> >>>> vm.make ll. 247 >>>> >>>> 1. *.o should be $(Obj_Files) >>>> >>>> 2. >>>> >>>> $(NM) --defined-only *.o | sort -k3 -u | >>>> awk '/$(VMDEF_PAT)/{ print "\t" $$3 ";" }' >>>> >>>> should give you the same result with less efforts >>>> >>>> -Dmitry >>>> >>>> On 2015-01-21 07:59, Dean Long wrote: >>>>> Here's version 2, which does everything in vm.make and doesn't do >>>>> anything that is shell-specific: >>>>> >>>>> http://cr.openjdk.java.net/~dlong/8031064//webrev.2/ >>>>> >>>>> thanks, >>>>> >>>>> dl >>>> >>> > From david.holmes at oracle.com Fri Jan 23 07:10:16 2015 From: david.holmes at oracle.com (David Holmes) Date: Fri, 23 Jan 2015 17:10:16 +1000 Subject: RFR: 7143664: Clean up OrderAccess implementations and usage In-Reply-To: <54C1EBD3.1000905@oracle.com> References: <54C1EBD3.1000905@oracle.com> Message-ID: <54C1F3D8.5040507@oracle.com> On 23/01/2015 4:36 PM, Dean Long wrote: > 147 // Ordering a load relative to preceding stores requires a fence, > 148 // which implies a membar #StoreLoad between the store and load under > 149 // sparc-TSO. A fence is required by x86. On x86, we use explicitly > 150 // locked add. > 151 // > > It sounds like the above is saying that fence is the same as StoreLoad ... That may be a s/store_fence/fence/ typo. The original text was: // Ordering a load relative to preceding stores requires a store_fence, // which implies a membar #StoreLoad between the store and load under // sparc-TSO. A fence is required by ia64. On x86, we use locked xchg. Actually seems like a couple of typos there: ia64 became x86, then we have x86 again. > 152 // 4. store, load <= is constrained by => store, fence, load > 153 // > 154 // Use store, fence to make sure all stores done in an 'interesting' > 155 // region are made visible prior to both subsequent loads and stores. > > ... and this is like saying to use fence because StoreStore | StoreLoad > isn't available. Again this seems an issue with store_fence being changed to "store, fence" which doesn't really make sense. David > >> Furthermore, store_release was declared private and marked as > deprecated. > > I can't find where this was done. > > dl > > On 1/22/2015 5:19 PM, Erik ?sterlund wrote: >> Hi all, >> >> == Summary of Changes == >> >> This changeset fixes issues in OrderAccess on multiple levels from the >> memory model semantics to compiler reorderings, to addressing >> maintainability/portability issues which (almost) had to be fixed in >> order to fix the correctness issues. It is the result of discussions >> found in the previous "OrderAccess Refactoring" thread: >> http://openjdk.5641.n7.nabble.com/OrderAccess-Refactoring-td212050.html >> >> Bug ID: >> https://bugs.openjdk.java.net/browse/JDK-7143664 >> (updated to reflect these related changes) >> >> Webrev: >> http://cr.openjdk.java.net/~dholmes/7143664/webrev/ >> >> Before I describe more I would like to give special thanks to David >> Holmes for long discussions leading up to the currently proposed >> changes. I would also like to thank Jesper Wilhelmsson for helping me >> run my changes through JPRT. >> >> == Motivation == >> >> This change directly fixes a reported OrderAccess bug due to compiler >> reorderings which is still a vulnerability on almost all TSO platforms: >> https://bugs.openjdk.java.net/browse/JDK-806196 >> >> And directly fixes confusions like release_store() != release() store() >> due to memory model issues previously described in this bug ID. >> >> At the same time it provides clearer design with separation of concerns >> and generalization/specialization, removing a whole bunch of platform >> specific code which could be generalized. The platform specific files >> now only have a few LoC requirements (~7) to conform to the memory model >> by specifying what the stand alone barriers do. Then optionally >> optimizations to the general approach are possible if platforms support >> it. This also makes it much easier to port to new platforms. >> >> == Memory Model == >> >> The current definitions of acquire/release semantics are a bit fishy >> leading to problems previously described in the bug ID (release_store() >> != release() store()) and some other correctness issues. It has >> therefore been replaced with a new model. I would like to thank David >> Holmes for the long discussions leading up to the newly proposed model. >> >> The new model is formally defined like this: >> >> // T1: access_shared_data >> // T1: ]release >> // T1: (...) >> // T1: store(X) >> // >> // T2: load(X) >> // T2: (...) >> // T2: acquire[ >> // T2: access_shared_data >> // >> // It is guaranteed that if T2: load(X) synchronizes with (observes the >> // value written by) T1: store(X), then the memory accesses before the >> // T1: ]release happen before the memory accesses after the T2: acquire[. >> >> The orderAccess.hpp file and bug ID also has a few additional >> explanations making it more intuitive to the user when to use >> acquire/release and the resemblance to TSO abstract machines. Big thanks >> goes to David Holmes for discussing the memory model with me, which >> helped a lot in deriving it. >> >> Now it holds that release() store() == release_store(), and the other >> correctness issues are fixed as well. >> >> The new model is also closer to C++11 definitions which could give us >> more relaxed compiler reordering constraints in the future when compiler >> support for C++11 is there and ready. >> >> == Reliance on C++ Volatile Semantics == >> >> The C++ standard section 1.9 "Program Execution" is very vague about >> what the keyword volatile can actually do for us. It gives clear >> constraints in terms of volatile-volatile accesses but says little about >> nonvolatile-volatile accesses. Yet the current implementation heavily >> relies upon volatile to in terms of compiler reordering. But GCC >> explicitly declares that volatiles and non-volatiles may reorder freely >> ( https://gcc.gnu.org/onlinedocs/gcc/Volatiles.html ). The only compiler >> known to explicitly provide the wanted semantics with volatile is MSVC >> 2010 for windows ( >> https://msdn.microsoft.com/en-us/library/12a04hfd(v=vs.100).aspx ). >> Compilers not giving explicit guarantees, must be considered unsafe and >> revert to conservative behaviour. >> >> This was brought to attention after causing bugs, but was only fixed for >> x86 linux. This is a fundamental issue inherent to all TSO platforms >> except windows, and has to be fixed on all of them. >> >> Several barriers are unsafe to use because they lack compiler reordering >> constraints (e.g. fence and acquire on linux_SPARC). For TSO platforms >> they are typically implemented using dummy loads and stores. This seems >> to be another old volatile reliance that I fixed. These barriers >> sometimes have omitted compiler barriers (which is what we really want). >> This seems to be another example on incorrect reliance on the volatile >> semantics to help us. Therefore dummy loads/stores have been replaced >> with compiler barriers on TSO platforms. >> >> It is also worth noting that compilers like sun studio did previously >> not support inline asm syntax. Therefore, barriers were implemented in >> .il-files. However, using them does not give explicit compiler >> constraints for reordering AFAIK. Therefore, they have been >> reimplemented using inline asm with explicit compiler reordering >> constraints, as even sun (solaris?) studio now supports this. >> >> The windows variants have added a windows-style _ReadWriteBarrier() >> compiler barrier similarly. >> >> == Strange Hardware Reorderings == >> >> Fixed a weird inconsistency where acquire, loadstore and loadlaod would >> use isync instead of lwsync for PPC on linux_zero, but not in any other >> PPC platform in the repo. I assumed this is wrong and changed it to >> lwsync instead. >> >> == Code Redundancy and Refactoring == >> >> The OrderAccess code looks like it has been developed over a long period >> of time, with small incremental changes. This seems to have led to a lot >> of code duplication over time. For example, store_fence variants are not >> referenced from anywhere, yet contribute to a lot of the code base and a >> lot of awkwardness (such as being the one only exception not using >> volatiles for some reason). Moreover, store_fence is not used anywhere >> in hotspot because it is not a good fit for either the acquire/release >> semantics or the Java volatile semantics, leaving a big question mark on >> when it should ever be used. I took the liberty of removing it. >> >> Another redundancy issue is that most of the semantics is exactly the >> same for all platforms, yet all that default boilerplate such as how to >> make atomic accesses, where acquire/release is supposed to be placed >> w.r.t. the memory access, what the different barriers should do etc. is >> copied in redundantly for each os_cpu and each type of memory access for >> each os_cpu. This makes it extremely painful 1) to understand what >> actually defines a certain platform compared to the defaults and 2) to >> correct bugs like those discovered here 3) prevent silly mistakes and >> bugs, by simply having a lot less code defining the behaviour of >> OrderAccess that could go wrong. >> >> A new architecture/design for OrderAccess is proposed, using a >> generalization/specialization approach. >> >> A general implementation in /share/ defines how things work and splits >> into different concerns: 1) how to make an atomic memory access, 2) >> where to but barriers w.r.t. the memory access for things like >> load_acquire, release_store and release_store_fence, 3) how these >> barriers are defined. >> >> This allows a clear split between what is required for following the >> specifications, and optimizations, which become much more readable and >> only optimizations need to be reviewed in depth as the defaults can >> always be trusted given correct standalone barriers. >> >> The only thing a platform is required to specify, is what an >> implementation of acquire(), release() and fence() should do. If they >> are implemented properly, everything in OrderAccess is guaranteed to >> work according to specification using the generalized code. This makes >> it very easy to support new ports. ALL the other code in the os_cpu >> files is used /only/ for optimization purposes offered for specific >> configurations. >> >> However, it is highly customizable so that specific platform can perform >> any desired optimizations. For instance this load_acquire on PPC is >> optimized: >> >> template<> inline jbyte OrderAccess::specialized_load_acquire >> (volatile jbyte* p) { register jbyte t = load(p); >> inlasm_acquire_reg(t); return t; } >> >> This overrides the whole load_acquire implementation to do something >> custom. Platforms like x86 extensively use this for joined fencing >> variants to optimize. >> >> The default implementation of load_acquire() will make an atomic load() >> followed by acquire() since the semantics is generalized. The >> generalized semantics are defined using inlined postfix/prefix calls >> after/before the atomic access, as defined here: >> >> template<> inline void ScopedFenceGeneral::postfix() { >> OrderAccess::acquire(); } >> template<> inline void ScopedFenceGeneral::prefix() { >> OrderAccess::release(); } >> template<> inline void ScopedFenceGeneral::prefix() { >> OrderAccess::release(); } >> template<> inline void ScopedFenceGeneral::postfix() { >> OrderAccess::fence(); } >> >> For platforms that simply wish to override what e.g. acquire means for a >> joined ordered memory access in general, as different to calling stand >> alone acquire(), the semantics can be easily overridden for a platform >> such as windows like on windows: >> >> template<> inline void ScopedFence::postfix() { } >> template<> inline void ScopedFence::prefix() { } >> template<> inline void ScopedFence::prefix() { } >> template<> inline void ScopedFence::postfix() { >> OrderAccess::fence(); } >> >> In this example, since Windows (now) has a compiler barrier for acquire, >> but does not need it for joined accesses since volatile has stronger >> guarantees on windows, this is enough to specialize that for joined >> memory accesses, no extra protection is needed. >> >> == Backward Compatibility and Transitioning == >> >> Since the newly proposed code is structured differently to before, a >> #define was added for backward compatibility so that external >> repositories not adhering to this new architecture do not break. >> Furthermore, store_release was declared private and marked as >> deprecated. This allows for a smooth transition into the new style of >> OrderAccess. When the full transition is made in all known repos, the >> #define and store_fence may be safely removed, eventually. >> >> == Documentation == >> >> The documentation seems old and outdated, describing how it works on >> SPARC RMO and IA64, which are nowhere to be found in the repository. It >> also describes things about C++ volatiles which cannot be relied upon. >> The documentation has been cleaned up to match the current state of the >> implementation better, with architectures actually found in the >> repository. >> >> == Testing == >> >> JPRT. Big thanks to Jesper Wilhelmsson for helping me test these changes. >> Ran some DaCapo benchmarks (I know okay :p) for performance regression >> and there was no perceivable difference. >> >> Looking forward to feedback on this, and hope to get some reviews. :) >> >> Thanks, >> Erik > From dean.long at oracle.com Fri Jan 23 07:29:52 2015 From: dean.long at oracle.com (Dean Long) Date: Thu, 22 Jan 2015 23:29:52 -0800 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54C1EE07.6040308@oracle.com> References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> <54B760DB.70200@oracle.com> <54B765DC.8050801@oracle.com> <54B845E3.7060000@oracle.com> <54B8EC3F.5030408@oracle.com> <54BF3214.2090903@oracle.com> <54BF5F2C.2020203@oracle.com> <54C02AA3.6020503@oracle.com> <54C0CEBC.4030807@oracle.com> <54C13AE3.1020506@oracle.com> <54C1EE07.6040308@oracle.com> Message-ID: <54C1F870.7050701@oracle.com> Done: https://bugs.openjdk.java.net/browse/JDK-8071436 dl On 1/22/2015 10:45 PM, Dmitry Samersoff wrote: > Dean, > >> Can we make cleaning up the other platforms >> a separate RFE? > I think yes. So please file the RFE. > > -Dmitry > > > On 2015-01-22 21:01, Dean Long wrote: >> On 1/22/2015 2:19 AM, David Holmes wrote: >>> On 22/01/2015 8:39 AM, Dean Long wrote: >>>> Thanks Dmitry. The updated webrev is here: >>>> >>>> http://cr.openjdk.java.net/~dlong/8031064/webrev.3/ >>> This looks weird: >>> >>> + VMDEF_PAT = ^_ZTV >>> + VMDEF_PAT := ^gHotSpotVM|$(VMDEF_PAT) >>> + VMDEF_PAT := ^UseSharedSpaces$$|$(VMDEF_PAT) >>> + VMDEF_PAT := ^_ZN9Arguments17SharedArchivePathE$$|$(VMDEF_PAT) >>> >>> but I can sort of see why you wanted to do it that way. >>> >> Do you have a suggestion for a less-weird-looking way to do it? >> >>> I assume you have verified the results are identical? >>> >> Yes. >> >>> I would be good to see this applied uniformly across all platforms as >>> well (except windows). >>> >> I suppose, but isn't linux the only platform where we might be >> cross-compiling? I'm not setup to >> build for aix, bsd, or solaris, and if I build in JPRT, I'm not sure it >> will save the vm.def or mapfile to >> make a comparison against. Can we make cleaning up the other platforms >> a separate RFE? >> >> dl >> >>> Thanks, >>> David >>> >>>> dl >>>> >>>> On 1/21/2015 12:11 AM, Dmitry Samersoff wrote: >>>>> Dean, >>>>> >>>>> vm.make ll. 247 >>>>> >>>>> 1. *.o should be $(Obj_Files) >>>>> >>>>> 2. >>>>> >>>>> $(NM) --defined-only *.o | sort -k3 -u | >>>>> awk '/$(VMDEF_PAT)/{ print "\t" $$3 ";" }' >>>>> >>>>> should give you the same result with less efforts >>>>> >>>>> -Dmitry >>>>> >>>>> On 2015-01-21 07:59, Dean Long wrote: >>>>>> Here's version 2, which does everything in vm.make and doesn't do >>>>>> anything that is shell-specific: >>>>>> >>>>>> http://cr.openjdk.java.net/~dlong/8031064//webrev.2/ >>>>>> >>>>>> thanks, >>>>>> >>>>>> dl > From dean.long at oracle.com Fri Jan 23 07:36:57 2015 From: dean.long at oracle.com (Dean Long) Date: Thu, 22 Jan 2015 23:36:57 -0800 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54C1F1E7.80201@oracle.com> References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> <54B760DB.70200@oracle.com> <54B765DC.8050801@oracle.com> <54B845E3.7060000@oracle.com> <54B8EC3F.5030408@oracle.com> <54BF3214.2090903@oracle.com> <54BF5F2C.2020203@oracle.com> <54C02AA3.6020503@oracle.com> <54C0CEBC.4030807@oracle.com> <54C13AE3.1020506@oracle.com> <54C1F1E7.80201@oracle.com> Message-ID: <54C1FA19.1010101@oracle.com> On 1/22/2015 11:01 PM, David Holmes wrote: > > > On 23/01/2015 4:01 AM, Dean Long wrote: >> On 1/22/2015 2:19 AM, David Holmes wrote: >>> On 22/01/2015 8:39 AM, Dean Long wrote: >>>> Thanks Dmitry. The updated webrev is here: >>>> >>>> http://cr.openjdk.java.net/~dlong/8031064/webrev.3/ >>> >>> This looks weird: >>> >>> + VMDEF_PAT = ^_ZTV >>> + VMDEF_PAT := ^gHotSpotVM|$(VMDEF_PAT) >>> + VMDEF_PAT := ^UseSharedSpaces$$|$(VMDEF_PAT) >>> + VMDEF_PAT := ^_ZN9Arguments17SharedArchivePathE$$|$(VMDEF_PAT) >>> >>> but I can sort of see why you wanted to do it that way. >>> >> >> Do you have a suggestion for a less-weird-looking way to do it? > > Only the obvious but hard to read: > > VMDEF_PAT := > ^_ZN9Arguments17SharedArchivePathE$$|^UseSharedSpaces$$|^gHotSpotVM|^_ZTV > Sure, I will do that. Can I count that as "reviewed"? And do I need another Reviewer for this change, or am I good to go? dl >>> I assume you have verified the results are identical? >>> >> >> Yes. >> >>> I would be good to see this applied uniformly across all platforms as >>> well (except windows). >>> >> >> I suppose, but isn't linux the only platform where we might be >> cross-compiling? I'm not setup to >> build for aix, bsd, or solaris, and if I build in JPRT, I'm not sure it >> will save the vm.def or mapfile to >> make a comparison against. Can we make cleaning up the other platforms >> a separate RFE? > > Getting rid of shell scripts from the build is a Good Thing(TM)! But > yes we can make this a separate RFE. > > Thanks, > David > >> >> dl >> >>> Thanks, >>> David >>> >>>> dl >>>> >>>> On 1/21/2015 12:11 AM, Dmitry Samersoff wrote: >>>>> Dean, >>>>> >>>>> vm.make ll. 247 >>>>> >>>>> 1. *.o should be $(Obj_Files) >>>>> >>>>> 2. >>>>> >>>>> $(NM) --defined-only *.o | sort -k3 -u | >>>>> awk '/$(VMDEF_PAT)/{ print "\t" $$3 ";" }' >>>>> >>>>> should give you the same result with less efforts >>>>> >>>>> -Dmitry >>>>> >>>>> On 2015-01-21 07:59, Dean Long wrote: >>>>>> Here's version 2, which does everything in vm.make and doesn't do >>>>>> anything that is shell-specific: >>>>>> >>>>>> http://cr.openjdk.java.net/~dlong/8031064//webrev.2/ >>>>>> >>>>>> thanks, >>>>>> >>>>>> dl >>>>> >>>> >> From david.holmes at oracle.com Fri Jan 23 07:41:07 2015 From: david.holmes at oracle.com (David Holmes) Date: Fri, 23 Jan 2015 17:41:07 +1000 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54C1FA19.1010101@oracle.com> References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> <54B760DB.70200@oracle.com> <54B765DC.8050801@oracle.com> <54B845E3.7060000@oracle.com> <54B8EC3F.5030408@oracle.com> <54BF3214.2090903@oracle.com> <54BF5F2C.2020203@oracle.com> <54C02AA3.6020503@oracle.com> <54C0CEBC.4030807@oracle.com> <54C13AE3.1020506@oracle.com> <54C1F1E7.80201@oracle.com> <54C1FA19.1010101@oracle.com> Message-ID: <54C1FB13.1080205@oracle.com> On 23/01/2015 5:36 PM, Dean Long wrote: > On 1/22/2015 11:01 PM, David Holmes wrote: >> >> >> On 23/01/2015 4:01 AM, Dean Long wrote: >>> On 1/22/2015 2:19 AM, David Holmes wrote: >>>> On 22/01/2015 8:39 AM, Dean Long wrote: >>>>> Thanks Dmitry. The updated webrev is here: >>>>> >>>>> http://cr.openjdk.java.net/~dlong/8031064/webrev.3/ >>>> >>>> This looks weird: >>>> >>>> + VMDEF_PAT = ^_ZTV >>>> + VMDEF_PAT := ^gHotSpotVM|$(VMDEF_PAT) >>>> + VMDEF_PAT := ^UseSharedSpaces$$|$(VMDEF_PAT) >>>> + VMDEF_PAT := ^_ZN9Arguments17SharedArchivePathE$$|$(VMDEF_PAT) >>>> >>>> but I can sort of see why you wanted to do it that way. >>>> >>> >>> Do you have a suggestion for a less-weird-looking way to do it? >> >> Only the obvious but hard to read: >> >> VMDEF_PAT := >> ^_ZN9Arguments17SharedArchivePathE$$|^UseSharedSpaces$$|^gHotSpotVM|^_ZTV >> > > Sure, I will do that. Can I count that as "reviewed"? And do I need > another Reviewer for this change, or am I good to go? If you change it you will have to redo all your testing. Just leave as-is - the readability wins here. Up to you though. Dmitry and I have reviewed this. So you have one "Reviewer" and one "reviewer" :) Thanks, David > dl > >>>> I assume you have verified the results are identical? >>>> >>> >>> Yes. >>> >>>> I would be good to see this applied uniformly across all platforms as >>>> well (except windows). >>>> >>> >>> I suppose, but isn't linux the only platform where we might be >>> cross-compiling? I'm not setup to >>> build for aix, bsd, or solaris, and if I build in JPRT, I'm not sure it >>> will save the vm.def or mapfile to >>> make a comparison against. Can we make cleaning up the other platforms >>> a separate RFE? >> >> Getting rid of shell scripts from the build is a Good Thing(TM)! But >> yes we can make this a separate RFE. >> >> Thanks, >> David >> >>> >>> dl >>> >>>> Thanks, >>>> David >>>> >>>>> dl >>>>> >>>>> On 1/21/2015 12:11 AM, Dmitry Samersoff wrote: >>>>>> Dean, >>>>>> >>>>>> vm.make ll. 247 >>>>>> >>>>>> 1. *.o should be $(Obj_Files) >>>>>> >>>>>> 2. >>>>>> >>>>>> $(NM) --defined-only *.o | sort -k3 -u | >>>>>> awk '/$(VMDEF_PAT)/{ print "\t" $$3 ";" }' >>>>>> >>>>>> should give you the same result with less efforts >>>>>> >>>>>> -Dmitry >>>>>> >>>>>> On 2015-01-21 07:59, Dean Long wrote: >>>>>>> Here's version 2, which does everything in vm.make and doesn't do >>>>>>> anything that is shell-specific: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dlong/8031064//webrev.2/ >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> dl >>>>>> >>>>> >>> > From dean.long at oracle.com Fri Jan 23 08:39:22 2015 From: dean.long at oracle.com (Dean Long) Date: Fri, 23 Jan 2015 00:39:22 -0800 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54C1FB13.1080205@oracle.com> References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> <54B760DB.70200@oracle.com> <54B765DC.8050801@oracle.com> <54B845E3.7060000@oracle.com> <54B8EC3F.5030408@oracle.com> <54BF3214.2090903@oracle.com> <54BF5F2C.2020203@oracle.com> <54C02AA3.6020503@oracle.com> <54C0CEBC.4030807@oracle.com> <54C13AE3.1020506@oracle.com> <54C1F1E7.80201@oracle.com> <54C1FA19.1010101@oracle.com> <54C1FB13.1080205@oracle.com> Message-ID: <54C208BA.40905@oracle.com> David and Dmitry, thanks for the reviews! dl On 1/22/2015 11:41 PM, David Holmes wrote: > On 23/01/2015 5:36 PM, Dean Long wrote: >> On 1/22/2015 11:01 PM, David Holmes wrote: >>> >>> >>> On 23/01/2015 4:01 AM, Dean Long wrote: >>>> On 1/22/2015 2:19 AM, David Holmes wrote: >>>>> On 22/01/2015 8:39 AM, Dean Long wrote: >>>>>> Thanks Dmitry. The updated webrev is here: >>>>>> >>>>>> http://cr.openjdk.java.net/~dlong/8031064/webrev.3/ >>>>> >>>>> This looks weird: >>>>> >>>>> + VMDEF_PAT = ^_ZTV >>>>> + VMDEF_PAT := ^gHotSpotVM|$(VMDEF_PAT) >>>>> + VMDEF_PAT := ^UseSharedSpaces$$|$(VMDEF_PAT) >>>>> + VMDEF_PAT := ^_ZN9Arguments17SharedArchivePathE$$|$(VMDEF_PAT) >>>>> >>>>> but I can sort of see why you wanted to do it that way. >>>>> >>>> >>>> Do you have a suggestion for a less-weird-looking way to do it? >>> >>> Only the obvious but hard to read: >>> >>> VMDEF_PAT := >>> ^_ZN9Arguments17SharedArchivePathE$$|^UseSharedSpaces$$|^gHotSpotVM|^_ZTV >>> >>> >> >> Sure, I will do that. Can I count that as "reviewed"? And do I need >> another Reviewer for this change, or am I good to go? > > If you change it you will have to redo all your testing. Just leave > as-is - the readability wins here. Up to you though. > > Dmitry and I have reviewed this. So you have one "Reviewer" and one > "reviewer" :) > > Thanks, > David > >> dl >> >>>>> I assume you have verified the results are identical? >>>>> >>>> >>>> Yes. >>>> >>>>> I would be good to see this applied uniformly across all platforms as >>>>> well (except windows). >>>>> >>>> >>>> I suppose, but isn't linux the only platform where we might be >>>> cross-compiling? I'm not setup to >>>> build for aix, bsd, or solaris, and if I build in JPRT, I'm not >>>> sure it >>>> will save the vm.def or mapfile to >>>> make a comparison against. Can we make cleaning up the other >>>> platforms >>>> a separate RFE? >>> >>> Getting rid of shell scripts from the build is a Good Thing(TM)! But >>> yes we can make this a separate RFE. >>> >>> Thanks, >>> David >>> >>>> >>>> dl >>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> dl >>>>>> >>>>>> On 1/21/2015 12:11 AM, Dmitry Samersoff wrote: >>>>>>> Dean, >>>>>>> >>>>>>> vm.make ll. 247 >>>>>>> >>>>>>> 1. *.o should be $(Obj_Files) >>>>>>> >>>>>>> 2. >>>>>>> >>>>>>> $(NM) --defined-only *.o | sort -k3 -u | >>>>>>> awk '/$(VMDEF_PAT)/{ print "\t" $$3 ";" }' >>>>>>> >>>>>>> should give you the same result with less efforts >>>>>>> >>>>>>> -Dmitry >>>>>>> >>>>>>> On 2015-01-21 07:59, Dean Long wrote: >>>>>>>> Here's version 2, which does everything in vm.make and doesn't do >>>>>>>> anything that is shell-specific: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dlong/8031064//webrev.2/ >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> dl >>>>>>> >>>>>> >>>> >> From daniel.daugherty at oracle.com Fri Jan 23 14:06:56 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 23 Jan 2015 07:06:56 -0700 Subject: 4-th round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54BD3D5A.1030903@oracle.com> References: <54B60308.4000105@oracle.com> <54B96301.40108@oracle.com> <54B9C7CB.60103@oracle.com> <54BD28F7.7040905@oracle.com> <54BD3D5A.1030903@oracle.com> Message-ID: <54C25580.8040404@oracle.com> Serguei, Sorry I forgot to close the loop on this review. I'm OK with the answers below. Thumbs up. Dan On 1/19/15 10:22 AM, serguei.spitsyn at oracle.com wrote: > Coleen, > > Thank you for answering questions below! > > Thanks, > Serguei > > > On 1/19/15 7:55 AM, Coleen Phillimore wrote: >> >> On 1/16/15, 9:24 PM, Daniel D. Daugherty wrote: >>> > >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>> >>> src/share/vm/memory/universe.hpp >>> No comments. >>> >>> src/share/vm/memory/universe.cpp >>> No comments. >>> >>> src/share/vm/prims/jvmtiRedefineClasses.cpp >>> So redefining the Unsafe class is now very expensive because >>> we have to visit the i-table and v-table of every class (and >>> maybe interface?)... >>> >>> Based on the bug report 'Unsafe::throw_illegal_access' is some >>> magical method that can appear in any i-table or v-table entry. >>> Maybe only as part of some default methods thing? That's not >>> clear to me so I'm just guessing... >> >> True. >>> >>> Is there some way to limit this visit to classes where the >>> magical method can appear? Or can it really appear anywhere? >> >> The Unsafe methods can appear in any itable now. I don't know of a >> way to limit this. Fortunately, redefining Unsafe seems to be an >> unusual thing to do, except for this stress test. >> >> Coleen >> >>> >>> Dan >>> >>> >>> On 1/16/15 12:14 PM, serguei.spitsyn at oracle.com wrote: >>>> Dan, David H. or David C., >>>> >>>> May I ask one of you to look at the webrev below? >>>> The issue itself is a little bit tricky, so it is not easy to >>>> review despite the small size. >>>> >>>> Coleen, >>>> >>>> Does the webrev matches what we discussed with you? >>>> Do you give me a thumbs up? >>>> >>>> Thanks, >>>> Serguei >>>> >>>> May I ask >>>> >>>> On 1/13/15 9:47 PM, serguei.spitsyn at oracle.com wrote: >>>>> Please, review the fix for: >>>>> https://bugs.openjdk.java.net/browse/JDK-8068162 >>>>> >>>>> >>>>> Open webrevs: >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>>>> >>> >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/jdk/8068162-Test-IsModifiableAgent/ >>>>> >>>>> >>>>> >>>>> Summary: >>>>> >>>>> The sun.misc.Unsafe:throwIllegalAccessError() method is used in >>>>> place of a default >>>>> interface method in the itable if a default method was not >>>>> defined in the interface. >>>>> In fact, it happens for two interfaces that purhaps are >>>>> auto-generated: >>>>> java/nio/CharBuffer >>>>> java/nio/HeapCharBuffer >>>>> >>>>> This approach creates a problem when the class sun.misc.Unsafe >>>>> is retransformed. >>>>> The Method* pointer to the old (redefined) method in the itable >>>>> triggers an assert >>>>> (see the hs_err log in the bug report). >>>>> Coleen told me that a similar approach is going to be >>>>> implemented for some vtable entries. >>>>> Coleen, thanks for suggesting a better fix for this issue! >>>>> >>>>> The fix is to replace the old Unsafe method in the >>>>> itable/vtable with the latest method version. >>>>> >>>>> >>>>> Testing: >>>>> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>> >>> >> > > From erik.osterlund at lnu.se Fri Jan 23 15:16:55 2015 From: erik.osterlund at lnu.se (=?iso-8859-1?Q?Erik_=D6sterlund?=) Date: Fri, 23 Jan 2015 15:16:55 +0000 Subject: RFR: 7143664: Clean up OrderAccess implementations and usage References: <54C1EBD3.1000905@oracle.com> <54C1F3D8.5040507@oracle.com> Message-ID: Hi Dean and David, Thanks for reviewing this change. On 23/01/15 08:12, David Holmes wrote: > On 23/01/2015 4:36 PM, Dean Long wrote: >> 147 // Ordering a load relative to preceding stores requires a fence, >> 148 // which implies a membar #StoreLoad between the store and load under >> 149 // sparc-TSO. A fence is required by x86. On x86, we use explicitly >> 150 // locked add. >> 151 // >> >> It sounds like the above is saying that fence is the same as StoreLoad ... > > That may be a s/store_fence/fence/ typo. The original text was: > > // Ordering a load relative to preceding stores requires a store_fence, > // which implies a membar #StoreLoad between the store and load under > // sparc-TSO. A fence is required by ia64. On x86, we use locked xchg. > > Actually seems like a couple of typos there: ia64 became x86, then we > have x86 again. I tried to remove ia64 from the comments as it is not in the repository, which in this case became a bit redundant. Are we okay if I remove the "A fence is required by x86" sentence? Oh and Dean about membar #StoreLoad on SPARC and locked add on x86... are you saying they are not equivalent? New version of that paragraph with that sentence removed: 147 // Ordering a load relative to preceding stores requires a fence, 148 // which implies a membar #StoreLoad between the store and load under 149 // sparc-TSO. On x86, we use explicitly locked add. >> 152 // 4. store, load <= is constrained by => store, fence, load >> 153 // >> 154 // Use store, fence to make sure all stores done in an 'interesting' >> 155 // region are made visible prior to both subsequent loads and stores. >> >> ... and this is like saying to use fence because StoreStore | StoreLoad >> isn't available. > > Again this seems an issue with store_fence being changed to "store, > fence" which doesn't really make sense. The thing is that I removed store_fence (not store_release as it says in the bug ID, sorry about that - there obviously is no store_release). So I wanted to remove references in the comments to store_fence as well. Therefore I changed the example to use store fence load instead of store_fence load. Perhaps it's better if I remove that sample all together instead? Thanks for reviewing my changes! /Erik > David > >> >>> Furthermore, store_release was declared private and marked as >> deprecated. >> >> I can't find where this was done. >> >> dl >> >> On 1/22/2015 5:19 PM, Erik ?sterlund wrote: >>> Hi all, >>> >>> == Summary of Changes == >>> >>> This changeset fixes issues in OrderAccess on multiple levels from the >>> memory model semantics to compiler reorderings, to addressing >>> maintainability/portability issues which (almost) had to be fixed in >>> order to fix the correctness issues. It is the result of discussions >>> found in the previous "OrderAccess Refactoring" thread: >>> http://openjdk.5641.n7.nabble.com/OrderAccess-Refactoring-td212050.html >>> >>> Bug ID: >>> https://bugs.openjdk.java.net/browse/JDK-7143664 >>> (updated to reflect these related changes) >>> >>> Webrev: >>> http://cr.openjdk.java.net/~dholmes/7143664/webrev/ >>> >>> Before I describe more I would like to give special thanks to David >>> Holmes for long discussions leading up to the currently proposed >>> changes. I would also like to thank Jesper Wilhelmsson for helping me >>> run my changes through JPRT. >>> >>> == Motivation == >>> >>> This change directly fixes a reported OrderAccess bug due to compiler >>> reorderings which is still a vulnerability on almost all TSO platforms: >>> https://bugs.openjdk.java.net/browse/JDK-806196 >>> >>> And directly fixes confusions like release_store() != release() store() >>> due to memory model issues previously described in this bug ID. >>> >>> At the same time it provides clearer design with separation of concerns >>> and generalization/specialization, removing a whole bunch of platform >>> specific code which could be generalized. The platform specific files >>> now only have a few LoC requirements (~7) to conform to the memory model >>> by specifying what the stand alone barriers do. Then optionally >>> optimizations to the general approach are possible if platforms support >>> it. This also makes it much easier to port to new platforms. >>> >>> == Memory Model == >>> >>> The current definitions of acquire/release semantics are a bit fishy >>> leading to problems previously described in the bug ID (release_store() >>> != release() store()) and some other correctness issues. It has >>> therefore been replaced with a new model. I would like to thank David >>> Holmes for the long discussions leading up to the newly proposed model. >>> >>> The new model is formally defined like this: >>> >>> // T1: access_shared_data >>> // T1: ]release >>> // T1: (...) >>> // T1: store(X) >>> // >>> // T2: load(X) >>> // T2: (...) >>> // T2: acquire[ >>> // T2: access_shared_data >>> // >>> // It is guaranteed that if T2: load(X) synchronizes with (observes the >>> // value written by) T1: store(X), then the memory accesses before the >>> // T1: ]release happen before the memory accesses after the T2: acquire[. >>> >>> The orderAccess.hpp file and bug ID also has a few additional >>> explanations making it more intuitive to the user when to use >>> acquire/release and the resemblance to TSO abstract machines. Big thanks >>> goes to David Holmes for discussing the memory model with me, which >>> helped a lot in deriving it. >>> >>> Now it holds that release() store() == release_store(), and the other >>> correctness issues are fixed as well. >>> >>> The new model is also closer to C++11 definitions which could give us >>> more relaxed compiler reordering constraints in the future when compiler >>> support for C++11 is there and ready. >>> >>> == Reliance on C++ Volatile Semantics == >>> >>> The C++ standard section 1.9 "Program Execution" is very vague about >>> what the keyword volatile can actually do for us. It gives clear >>> constraints in terms of volatile-volatile accesses but says little about >>> nonvolatile-volatile accesses. Yet the current implementation heavily >>> relies upon volatile to in terms of compiler reordering. But GCC >>> explicitly declares that volatiles and non-volatiles may reorder freely >>> ( https://gcc.gnu.org/onlinedocs/gcc/Volatiles.html ). The only compiler >>> known to explicitly provide the wanted semantics with volatile is MSVC >>> 2010 for windows ( >>> https://msdn.microsoft.com/en-us/library/12a04hfd(v=vs.100).aspx ). >>> Compilers not giving explicit guarantees, must be considered unsafe and >>> revert to conservative behaviour. >>> >>> This was brought to attention after causing bugs, but was only fixed for >>> x86 linux. This is a fundamental issue inherent to all TSO platforms >>> except windows, and has to be fixed on all of them. >>> >>> Several barriers are unsafe to use because they lack compiler reordering >>> constraints (e.g. fence and acquire on linux_SPARC). For TSO platforms >>> they are typically implemented using dummy loads and stores. This seems >>> to be another old volatile reliance that I fixed. These barriers >>> sometimes have omitted compiler barriers (which is what we really want). >>> This seems to be another example on incorrect reliance on the volatile >>> semantics to help us. Therefore dummy loads/stores have been replaced >>> with compiler barriers on TSO platforms. >>> >>> It is also worth noting that compilers like sun studio did previously >>> not support inline asm syntax. Therefore, barriers were implemented in >>> .il-files. However, using them does not give explicit compiler >>> constraints for reordering AFAIK. Therefore, they have been >>> reimplemented using inline asm with explicit compiler reordering >>> constraints, as even sun (solaris?) studio now supports this. >>> >>> The windows variants have added a windows-style _ReadWriteBarrier() >>> compiler barrier similarly. >>> >>> == Strange Hardware Reorderings == >>> >>> Fixed a weird inconsistency where acquire, loadstore and loadlaod would >>> use isync instead of lwsync for PPC on linux_zero, but not in any other >>> PPC platform in the repo. I assumed this is wrong and changed it to >>> lwsync instead. >>> >>> == Code Redundancy and Refactoring == >>> >>> The OrderAccess code looks like it has been developed over a long period >>> of time, with small incremental changes. This seems to have led to a lot >>> of code duplication over time. For example, store_fence variants are not >>> referenced from anywhere, yet contribute to a lot of the code base and a >>> lot of awkwardness (such as being the one only exception not using >>> volatiles for some reason). Moreover, store_fence is not used anywhere >>> in hotspot because it is not a good fit for either the acquire/release >>> semantics or the Java volatile semantics, leaving a big question mark on >>> when it should ever be used. I took the liberty of removing it. >>> >>> Another redundancy issue is that most of the semantics is exactly the >>> same for all platforms, yet all that default boilerplate such as how to >>> make atomic accesses, where acquire/release is supposed to be placed >>> w.r.t. the memory access, what the different barriers should do etc. is >>> copied in redundantly for each os_cpu and each type of memory access for >>> each os_cpu. This makes it extremely painful 1) to understand what >>> actually defines a certain platform compared to the defaults and 2) to >>> correct bugs like those discovered here 3) prevent silly mistakes and >>> bugs, by simply having a lot less code defining the behaviour of >>> OrderAccess that could go wrong. >>> >>> A new architecture/design for OrderAccess is proposed, using a >>> generalization/specialization approach. >>> >>> A general implementation in /share/ defines how things work and splits >>> into different concerns: 1) how to make an atomic memory access, 2) >>> where to but barriers w.r.t. the memory access for things like >>> load_acquire, release_store and release_store_fence, 3) how these >>> barriers are defined. >>> >>> This allows a clear split between what is required for following the >>> specifications, and optimizations, which become much more readable and >>> only optimizations need to be reviewed in depth as the defaults can >>> always be trusted given correct standalone barriers. >>> >>> The only thing a platform is required to specify, is what an >>> implementation of acquire(), release() and fence() should do. If they >>> are implemented properly, everything in OrderAccess is guaranteed to >>> work according to specification using the generalized code. This makes >>> it very easy to support new ports. ALL the other code in the os_cpu >>> files is used /only/ for optimization purposes offered for specific >>> configurations. >>> >>> However, it is highly customizable so that specific platform can perform >>> any desired optimizations. For instance this load_acquire on PPC is >>> optimized: >>> >>> template<> inline jbyte OrderAccess::specialized_load_acquire >>> (volatile jbyte* p) { register jbyte t = load(p); >>> inlasm_acquire_reg(t); return t; } >>> >>> This overrides the whole load_acquire implementation to do something >>> custom. Platforms like x86 extensively use this for joined fencing >>> variants to optimize. >>> >>> The default implementation of load_acquire() will make an atomic load() >>> followed by acquire() since the semantics is generalized. The >>> generalized semantics are defined using inlined postfix/prefix calls >>> after/before the atomic access, as defined here: >>> >>> template<> inline void ScopedFenceGeneral::postfix() { >>> OrderAccess::acquire(); } >>> template<> inline void ScopedFenceGeneral::prefix() { >>> OrderAccess::release(); } >>> template<> inline void ScopedFenceGeneral::prefix() { >>> OrderAccess::release(); } >>> template<> inline void ScopedFenceGeneral::postfix() { >>> OrderAccess::fence(); } >>> >>> For platforms that simply wish to override what e.g. acquire means for a >>> joined ordered memory access in general, as different to calling stand >>> alone acquire(), the semantics can be easily overridden for a platform >>> such as windows like on windows: >>> >>> template<> inline void ScopedFence::postfix() { } >>> template<> inline void ScopedFence::prefix() { } >>> template<> inline void ScopedFence::prefix() { } >>> template<> inline void ScopedFence::postfix() { >>> OrderAccess::fence(); } >>> >>> In this example, since Windows (now) has a compiler barrier for acquire, >>> but does not need it for joined accesses since volatile has stronger >>> guarantees on windows, this is enough to specialize that for joined >>> memory accesses, no extra protection is needed. >>> >>> == Backward Compatibility and Transitioning == >>> >>> Since the newly proposed code is structured differently to before, a >>> #define was added for backward compatibility so that external >>> repositories not adhering to this new architecture do not break. >>> Furthermore, store_release was declared private and marked as >>> deprecated. This allows for a smooth transition into the new style of >>> OrderAccess. When the full transition is made in all known repos, the >>> #define and store_fence may be safely removed, eventually. >>> >>> == Documentation == >>> >>> The documentation seems old and outdated, describing how it works on >>> SPARC RMO and IA64, which are nowhere to be found in the repository. It >>> also describes things about C++ volatiles which cannot be relied upon. >>> The documentation has been cleaned up to match the current state of the >>> implementation better, with architectures actually found in the >>> repository. >>> >>> == Testing == >>> >>> JPRT. Big thanks to Jesper Wilhelmsson for helping me test these changes. >>> Ran some DaCapo benchmarks (I know okay :p) for performance regression >>> and there was no perceivable difference. >>> >>> Looking forward to feedback on this, and hope to get some reviews. :) >>> >>> Thanks, >>> Erik >> > From aph at redhat.com Fri Jan 23 16:48:16 2015 From: aph at redhat.com (Andrew Haley) Date: Fri, 23 Jan 2015 16:48:16 +0000 Subject: narrow_klass_shift is nonzero on jdk9... Message-ID: <54C27B50.9020607@redhat.com> ... when it doesn't need to be. The reason for this is that we allocate the heap at the end of the lower 4*G region and we allocate the metaspace immediately after the heap, so we always need a nonzero narrow_klass_shift because narrow_klass_base is not less than 32 bits. If I set HeapBaseMinAddress=2g it's fine. The code which puts the heap right at the end of the lower 4*G region is quite deliberate: // Return specified base for the first request. if (!FLAG_IS_DEFAULT(HeapBaseMinAddress) && (mode == UnscaledNarrowOop)) { base = heap_base_min_address_aligned; // If the total size is small enough to allow UnscaledNarrowOop then // just use UnscaledNarrowOop. } else if ((total_size <= OopEncodingHeapMax) && (mode != HeapBasedNarrowOop)) { if ((total_size <= UnscaledOopHeapMax) && (mode == UnscaledNarrowOop) && (Universe::narrow_oop_shift() == 0)) { // Use 32-bits oops without encoding and // place heap's top on the 4Gb boundary base = (UnscaledOopHeapMax - heap_size); Why do we do this? I carefully set HeapBaseMinAddress in the AArch64 back end to a sensible value which would in many cases get me a zero narrow_klass_shift, but this code ignores the default. Thanks, Andrew. From coleen.phillimore at oracle.com Fri Jan 23 16:51:59 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 23 Jan 2015 11:51:59 -0500 Subject: 4-th round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54C25580.8040404@oracle.com> References: <54B60308.4000105@oracle.com> <54B96301.40108@oracle.com> <54B9C7CB.60103@oracle.com> <54BD28F7.7040905@oracle.com> <54BD3D5A.1030903@oracle.com> <54C25580.8040404@oracle.com> Message-ID: <54C27C2F.4020209@oracle.com> This still looks good to me. Coleen On 1/23/15, 9:06 AM, Daniel D. Daugherty wrote: > Serguei, > > Sorry I forgot to close the loop on this review. > > I'm OK with the answers below. Thumbs up. > > Dan > > > On 1/19/15 10:22 AM, serguei.spitsyn at oracle.com wrote: >> Coleen, >> >> Thank you for answering questions below! >> >> Thanks, >> Serguei >> >> >> On 1/19/15 7:55 AM, Coleen Phillimore wrote: >>> >>> On 1/16/15, 9:24 PM, Daniel D. Daugherty wrote: >>>> > >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>>> >>>> src/share/vm/memory/universe.hpp >>>> No comments. >>>> >>>> src/share/vm/memory/universe.cpp >>>> No comments. >>>> >>>> src/share/vm/prims/jvmtiRedefineClasses.cpp >>>> So redefining the Unsafe class is now very expensive because >>>> we have to visit the i-table and v-table of every class (and >>>> maybe interface?)... >>>> >>>> Based on the bug report 'Unsafe::throw_illegal_access' is some >>>> magical method that can appear in any i-table or v-table entry. >>>> Maybe only as part of some default methods thing? That's not >>>> clear to me so I'm just guessing... >>> >>> True. >>>> >>>> Is there some way to limit this visit to classes where the >>>> magical method can appear? Or can it really appear anywhere? >>> >>> The Unsafe methods can appear in any itable now. I don't know of a >>> way to limit this. Fortunately, redefining Unsafe seems to be an >>> unusual thing to do, except for this stress test. >>> >>> Coleen >>> >>>> >>>> Dan >>>> >>>> >>>> On 1/16/15 12:14 PM, serguei.spitsyn at oracle.com wrote: >>>>> Dan, David H. or David C., >>>>> >>>>> May I ask one of you to look at the webrev below? >>>>> The issue itself is a little bit tricky, so it is not easy to >>>>> review despite the small size. >>>>> >>>>> Coleen, >>>>> >>>>> Does the webrev matches what we discussed with you? >>>>> Do you give me a thumbs up? >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> May I ask >>>>> >>>>> On 1/13/15 9:47 PM, serguei.spitsyn at oracle.com wrote: >>>>>> Please, review the fix for: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8068162 >>>>>> >>>>>> >>>>>> Open webrevs: >>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>>>>> >>>> >>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/jdk/8068162-Test-IsModifiableAgent/ >>>>>> >>>>>> >>>>>> >>>>>> Summary: >>>>>> >>>>>> The sun.misc.Unsafe:throwIllegalAccessError() method is used >>>>>> in place of a default >>>>>> interface method in the itable if a default method was not >>>>>> defined in the interface. >>>>>> In fact, it happens for two interfaces that purhaps are >>>>>> auto-generated: >>>>>> java/nio/CharBuffer >>>>>> java/nio/HeapCharBuffer >>>>>> >>>>>> This approach creates a problem when the class sun.misc.Unsafe >>>>>> is retransformed. >>>>>> The Method* pointer to the old (redefined) method in the >>>>>> itable triggers an assert >>>>>> (see the hs_err log in the bug report). >>>>>> Coleen told me that a similar approach is going to be >>>>>> implemented for some vtable entries. >>>>>> Coleen, thanks for suggesting a better fix for this issue! >>>>>> >>>>>> The fix is to replace the old Unsafe method in the >>>>>> itable/vtable with the latest method version. >>>>>> >>>>>> >>>>>> Testing: >>>>>> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>> >>>> >>> >> >> > From coleen.phillimore at oracle.com Fri Jan 23 17:08:05 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 23 Jan 2015 12:08:05 -0500 Subject: narrow_klass_shift is nonzero on jdk9... In-Reply-To: <54C27B50.9020607@redhat.com> References: <54C27B50.9020607@redhat.com> Message-ID: <54C27FF5.9000808@oracle.com> Hi Andrew, Allocating this compressed class space is unfortunately complicated. I think you're looking at an old version of the code though, Goetz has changed this and moved it to virtualspace.cpp. The compressed class space is allocated above the heap because we want the java heap to have the most favorable compression algorithm. The code also has a special case so that for zero-based compressed oops; it saves space for the compressed class space to also get zero-based compressed class pointers. Decoding klasses without a base is better than decoding without a shift. The code doesn't try to fit both compressed oops and compressed class pointers in the lower 4G. It's generally improbable since the compressed class space is 1G size (because growing it is not supported atm) and doing so generally pushes the java heap over the 4*G size. Although I have to admit that HeapBaseMinAddress still mystifies me (for linux). Thanks, Coleen On 1/23/15, 11:48 AM, Andrew Haley wrote: > ... when it doesn't need to be. > > The reason for this is that we allocate the heap at the end of the > lower 4*G region and we allocate the metaspace immediately after the > heap, so we always need a nonzero narrow_klass_shift because > narrow_klass_base is not less than 32 bits. > > If I set HeapBaseMinAddress=2g it's fine. > > The code which puts the heap right at the end of the lower 4*G region > is quite deliberate: > > // Return specified base for the first request. > if (!FLAG_IS_DEFAULT(HeapBaseMinAddress) && (mode == UnscaledNarrowOop)) { > base = heap_base_min_address_aligned; > > // If the total size is small enough to allow UnscaledNarrowOop then > // just use UnscaledNarrowOop. > } else if ((total_size <= OopEncodingHeapMax) && (mode != HeapBasedNarrowOop)) { > if ((total_size <= UnscaledOopHeapMax) && (mode == UnscaledNarrowOop) && > (Universe::narrow_oop_shift() == 0)) { > // Use 32-bits oops without encoding and > // place heap's top on the 4Gb boundary > base = (UnscaledOopHeapMax - heap_size); > > Why do we do this? I carefully set HeapBaseMinAddress in the AArch64 > back end to a sensible value which would in many cases get me a zero > narrow_klass_shift, but this code ignores the default. > > Thanks, > Andrew. From aph at redhat.com Fri Jan 23 17:46:08 2015 From: aph at redhat.com (Andrew Haley) Date: Fri, 23 Jan 2015 17:46:08 +0000 Subject: narrow_klass_shift is nonzero on jdk9... In-Reply-To: <54C27FF5.9000808@oracle.com> References: <54C27B50.9020607@redhat.com> <54C27FF5.9000808@oracle.com> Message-ID: <54C288E0.6010101@redhat.com> Hi, On 01/23/2015 05:08 PM, Coleen Phillimore wrote: > > Allocating this compressed class space is unfortunately complicated. I > think you're looking at an old version of the code though, Goetz has > changed this and moved it to virtualspace.cpp. Okay. This is a fairly recent cut of JDK9, but we are a bit behind. > The compressed class space is allocated above the heap because we want > the java heap to have the most favorable compression algorithm. > > The code also has a special case so that for zero-based compressed oops; > it saves space for the compressed class space to also get zero-based > compressed class pointers. Decoding klasses without a base is better > than decoding without a shift. That's architecture-dependent, I would have thought. On the AArch64 port it can be somewhat better to have a nonzero base. (We just OR a bit or two into the register to decode.) > The code doesn't try to fit both compressed oops and compressed > class pointers in the lower 4G. It's generally improbable since the > compressed class space is 1G size (because growing it is not > supported atm) and doing so generally pushes the java heap over the > 4*G size. Although I have to admit that HeapBaseMinAddress still > mystifies me (for linux). Ah, okay, that makes sense. The application I'm running sets MaxHeapSize to 1*G. Wouldn't it at least be worth trying to allocate the metadata below the heap after the heap has been allocated? Thanks, Andrew. From goetz.lindenmaier at sap.com Fri Jan 23 17:52:27 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 23 Jan 2015 17:52:27 +0000 Subject: narrow_klass_shift is nonzero on jdk9... In-Reply-To: <54C27FF5.9000808@oracle.com> References: <54C27B50.9020607@redhat.com> <54C27FF5.9000808@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF71B5A@DEWDFEMB12A.global.corp.sap> Hi Andrew, that's right, I reworked finding a position for the heap with Coleens help: http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/fee07a43d038 https://bugs.openjdk.java.net/browse/JDK-8064457 To find the heap, we now - search from 4G down to HeapBaseMinAddress for an unscaled heap, then - search from 31G (1G for classes) down for a zerobased heap, then - try some addresses that are 32G aligned, called disjoint base (useful on PPC, you can do (base | (cOop<<3))), - then, a last, desperate and arbitrary try. See also the bug for some documentation. This new algorithm makes the following heap placements possible. - zerobased mode possible on solaris 11 - disjoint mode on PPC - zerobased and unscaled modes on AIX The change only indirectly affects the compressed class space placement. I didn't do any changes to that code. But heaps on x86 etc should be placed at the same location as before, except that heapbased can now be disjoint base. I proposed to place the compressed class space in the lower 4G for big heaps, but that violates other constraints to it's placement. (This should e.g. be possible for 8G heap which should be placed at 23G in general, etc) If I understand correctly, HeapBaseMinAddress is meant as a lower bound to protect the space needed for the system, not as the address where the heap should be placed. Actually, the code misuses it if it's set on the command line to enforce heapbased mode, e.g., for tests. Best regards, Goetz. -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore Sent: Friday, January 23, 2015 6:08 PM To: hotspot-dev at openjdk.java.net Subject: Re: narrow_klass_shift is nonzero on jdk9... Hi Andrew, Allocating this compressed class space is unfortunately complicated. I think you're looking at an old version of the code though, Goetz has changed this and moved it to virtualspace.cpp. The compressed class space is allocated above the heap because we want the java heap to have the most favorable compression algorithm. The code also has a special case so that for zero-based compressed oops; it saves space for the compressed class space to also get zero-based compressed class pointers. Decoding klasses without a base is better than decoding without a shift. The code doesn't try to fit both compressed oops and compressed class pointers in the lower 4G. It's generally improbable since the compressed class space is 1G size (because growing it is not supported atm) and doing so generally pushes the java heap over the 4*G size. Although I have to admit that HeapBaseMinAddress still mystifies me (for linux). Thanks, Coleen On 1/23/15, 11:48 AM, Andrew Haley wrote: > ... when it doesn't need to be. > > The reason for this is that we allocate the heap at the end of the > lower 4*G region and we allocate the metaspace immediately after the > heap, so we always need a nonzero narrow_klass_shift because > narrow_klass_base is not less than 32 bits. > > If I set HeapBaseMinAddress=2g it's fine. > > The code which puts the heap right at the end of the lower 4*G region > is quite deliberate: > > // Return specified base for the first request. > if (!FLAG_IS_DEFAULT(HeapBaseMinAddress) && (mode == UnscaledNarrowOop)) { > base = heap_base_min_address_aligned; > > // If the total size is small enough to allow UnscaledNarrowOop then > // just use UnscaledNarrowOop. > } else if ((total_size <= OopEncodingHeapMax) && (mode != HeapBasedNarrowOop)) { > if ((total_size <= UnscaledOopHeapMax) && (mode == UnscaledNarrowOop) && > (Universe::narrow_oop_shift() == 0)) { > // Use 32-bits oops without encoding and > // place heap's top on the 4Gb boundary > base = (UnscaledOopHeapMax - heap_size); > > Why do we do this? I carefully set HeapBaseMinAddress in the AArch64 > back end to a sensible value which would in many cases get me a zero > narrow_klass_shift, but this code ignores the default. > > Thanks, > Andrew. From aph at redhat.com Fri Jan 23 18:01:30 2015 From: aph at redhat.com (Andrew Haley) Date: Fri, 23 Jan 2015 18:01:30 +0000 Subject: narrow_klass_shift is nonzero on jdk9... In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF71B5A@DEWDFEMB12A.global.corp.sap> References: <54C27B50.9020607@redhat.com> <54C27FF5.9000808@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF71B5A@DEWDFEMB12A.global.corp.sap> Message-ID: <54C28C7A.20106@redhat.com> On 01/23/2015 05:52 PM, Lindenmaier, Goetz wrote: > If I understand correctly, HeapBaseMinAddress is meant as a lower > bound to protect the space needed for the system, not as the address where > the heap should be placed. Ah, right. Okay, now I get it! Thanks, Andrew. From coleen.phillimore at oracle.com Fri Jan 23 18:24:11 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 23 Jan 2015 13:24:11 -0500 Subject: narrow_klass_shift is nonzero on jdk9... In-Reply-To: <54C28C7A.20106@redhat.com> References: <54C27B50.9020607@redhat.com> <54C27FF5.9000808@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF71B5A@DEWDFEMB12A.global.corp.sap> <54C28C7A.20106@redhat.com> Message-ID: <54C291CB.8030508@oracle.com> On 1/23/15, 1:01 PM, Andrew Haley wrote: > On 01/23/2015 05:52 PM, Lindenmaier, Goetz wrote: >> If I understand correctly, HeapBaseMinAddress is meant as a lower >> bound to protect the space needed for the system, not as the address where >> the heap should be placed. > Ah, right. Okay, now I get it! Yes, and this further limits things because for most platforms it's 2*G. That's why we wanted to get the compressed class space out of the way. Coleen > > Thanks, > Andrew. > From vitalyd at gmail.com Fri Jan 23 18:32:02 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 23 Jan 2015 13:32:02 -0500 Subject: narrow_klass_shift is nonzero on jdk9... In-Reply-To: <54C28C7A.20106@redhat.com> References: <54C27B50.9020607@redhat.com> <54C27FF5.9000808@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF71B5A@DEWDFEMB12A.global.corp.sap> <54C28C7A.20106@redhat.com> Message-ID: I think this is for solaris in particular as malloc there likes low address space, if I recall the conversations correctly. sent from my phone On Jan 23, 2015 1:02 PM, "Andrew Haley" wrote: > On 01/23/2015 05:52 PM, Lindenmaier, Goetz wrote: > > If I understand correctly, HeapBaseMinAddress is meant as a lower > > bound to protect the space needed for the system, not as the address > where > > the heap should be placed. > > Ah, right. Okay, now I get it! > > Thanks, > Andrew. > > From serguei.spitsyn at oracle.com Fri Jan 23 18:50:05 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 23 Jan 2015 10:50:05 -0800 Subject: 4-th round RFR (XS) 8068162: jvmtiRedefineClasses.cpp: guarantee(false) failed: OLD and/or OBSOLETE method(s) found In-Reply-To: <54C27C2F.4020209@oracle.com> References: <54B60308.4000105@oracle.com> <54B96301.40108@oracle.com> <54B9C7CB.60103@oracle.com> <54BD28F7.7040905@oracle.com> <54BD3D5A.1030903@oracle.com> <54C25580.8040404@oracle.com> <54C27C2F.4020209@oracle.com> Message-ID: <54C297DD.7040906@oracle.com> Coleen and Dan, Thanks! Serguei On 1/23/15 8:51 AM, Coleen Phillimore wrote: > > This still looks good to me. > Coleen > > On 1/23/15, 9:06 AM, Daniel D. Daugherty wrote: >> Serguei, >> >> Sorry I forgot to close the loop on this review. >> >> I'm OK with the answers below. Thumbs up. >> >> Dan >> >> >> On 1/19/15 10:22 AM, serguei.spitsyn at oracle.com wrote: >>> Coleen, >>> >>> Thank you for answering questions below! >>> >>> Thanks, >>> Serguei >>> >>> >>> On 1/19/15 7:55 AM, Coleen Phillimore wrote: >>>> >>>> On 1/16/15, 9:24 PM, Daniel D. Daugherty wrote: >>>>> > >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>>>> >>>>> src/share/vm/memory/universe.hpp >>>>> No comments. >>>>> >>>>> src/share/vm/memory/universe.cpp >>>>> No comments. >>>>> >>>>> src/share/vm/prims/jvmtiRedefineClasses.cpp >>>>> So redefining the Unsafe class is now very expensive because >>>>> we have to visit the i-table and v-table of every class (and >>>>> maybe interface?)... >>>>> >>>>> Based on the bug report 'Unsafe::throw_illegal_access' is some >>>>> magical method that can appear in any i-table or v-table entry. >>>>> Maybe only as part of some default methods thing? That's not >>>>> clear to me so I'm just guessing... >>>> >>>> True. >>>>> >>>>> Is there some way to limit this visit to classes where the >>>>> magical method can appear? Or can it really appear anywhere? >>>> >>>> The Unsafe methods can appear in any itable now. I don't know of >>>> a way to limit this. Fortunately, redefining Unsafe seems to be an >>>> unusual thing to do, except for this stress test. >>>> >>>> Coleen >>>> >>>>> >>>>> Dan >>>>> >>>>> >>>>> On 1/16/15 12:14 PM, serguei.spitsyn at oracle.com wrote: >>>>>> Dan, David H. or David C., >>>>>> >>>>>> May I ask one of you to look at the webrev below? >>>>>> The issue itself is a little bit tricky, so it is not easy to >>>>>> review despite the small size. >>>>>> >>>>>> Coleen, >>>>>> >>>>>> Does the webrev matches what we discussed with you? >>>>>> Do you give me a thumbs up? >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> May I ask >>>>>> >>>>>> On 1/13/15 9:47 PM, serguei.spitsyn at oracle.com wrote: >>>>>>> Please, review the fix for: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8068162 >>>>>>> >>>>>>> >>>>>>> Open webrevs: >>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8068162-JVMTI-old.4/ >>>>>>> >>>>> >>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/jdk/8068162-Test-IsModifiableAgent/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> Summary: >>>>>>> >>>>>>> The sun.misc.Unsafe:throwIllegalAccessError() method is used >>>>>>> in place of a default >>>>>>> interface method in the itable if a default method was not >>>>>>> defined in the interface. >>>>>>> In fact, it happens for two interfaces that purhaps are >>>>>>> auto-generated: >>>>>>> java/nio/CharBuffer >>>>>>> java/nio/HeapCharBuffer >>>>>>> >>>>>>> This approach creates a problem when the class >>>>>>> sun.misc.Unsafe is retransformed. >>>>>>> The Method* pointer to the old (redefined) method in the >>>>>>> itable triggers an assert >>>>>>> (see the hs_err log in the bug report). >>>>>>> Coleen told me that a similar approach is going to be >>>>>>> implemented for some vtable entries. >>>>>>> Coleen, thanks for suggesting a better fix for this issue! >>>>>>> >>>>>>> The fix is to replace the old Unsafe method in the >>>>>>> itable/vtable with the latest method version. >>>>>>> >>>>>>> >>>>>>> Testing: >>>>>>> In progress: nsk.jdi.testlist, JTREG java/lang/instrument tests >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>> >>>>> >>>> >>> >>> >> > From dean.long at oracle.com Fri Jan 23 21:49:30 2015 From: dean.long at oracle.com (Dean Long) Date: Fri, 23 Jan 2015 13:49:30 -0800 Subject: RFR: 7143664: Clean up OrderAccess implementations and usage In-Reply-To: References: <54C1EBD3.1000905@oracle.com> <54C1F3D8.5040507@oracle.com> Message-ID: <54C2C1EA.5020507@oracle.com> On 1/23/2015 7:16 AM, Erik ?sterlund wrote: > Hi Dean and David, > > Thanks for reviewing this change. > > On 23/01/15 08:12, David Holmes wrote: >> On 23/01/2015 4:36 PM, Dean Long wrote: >>> 147 // Ordering a load relative to preceding stores requires a fence, >>> 148 // which implies a membar #StoreLoad between the store and load under >>> 149 // sparc-TSO. A fence is required by x86. On x86, we use explicitly >>> 150 // locked add. >>> 151 // >>> >>> It sounds like the above is saying that fence is the same as StoreLoad ... >> That may be a s/store_fence/fence/ typo. The original text was: >> >> // Ordering a load relative to preceding stores requires a store_fence, >> // which implies a membar #StoreLoad between the store and load under >> // sparc-TSO. A fence is required by ia64. On x86, we use locked xchg. >> >> Actually seems like a couple of typos there: ia64 became x86, then we >> have x86 again. > I tried to remove ia64 from the comments as it is not in the repository, > which in this case became a bit redundant. Are we okay if I remove the > "A fence is required by x86" sentence? > > Oh and Dean about membar #StoreLoad on SPARC and locked add on x86... > are you saying they are not equivalent? I wasn't trying to say anything abou SPARC or x86, only that "fence" != "StoreLoad". > New version of that paragraph with that sentence removed: > > 147 // Ordering a load relative to preceding stores requires a fence, > 148 // which implies a membar #StoreLoad between the store and load under > 149 // sparc-TSO. On x86, we use explicitly locked add. I would replace "fence" with "StoreLoad" above. >>> 152 // 4. store, load <= is constrained by => store, fence, load >>> 153 // >>> 154 // Use store, fence to make sure all stores done in an 'interesting' >>> 155 // region are made visible prior to both subsequent loads and stores. >>> >>> ... and this is like saying to use fence because StoreStore | StoreLoad >>> isn't available. >> Again this seems an issue with store_fence being changed to "store, >> fence" which doesn't really make sense. > The thing is that I removed store_fence (not store_release as it says in > the bug ID, sorry about that - there obviously is no store_release). So > I wanted to remove references in the comments to store_fence as well. > Therefore I changed the example to use store fence load instead of > store_fence load. Perhaps it's better if I remove that sample all > together instead? That works for me. dl > Thanks for reviewing my changes! > > /Erik > >> David >> >>>> Furthermore, store_release was declared private and marked as >>> deprecated. >>> >>> I can't find where this was done. >>> >>> dl >>> >>> On 1/22/2015 5:19 PM, Erik ?sterlund wrote: >>>> Hi all, >>>> >>>> == Summary of Changes == >>>> >>>> This changeset fixes issues in OrderAccess on multiple levels from the >>>> memory model semantics to compiler reorderings, to addressing >>>> maintainability/portability issues which (almost) had to be fixed in >>>> order to fix the correctness issues. It is the result of discussions >>>> found in the previous "OrderAccess Refactoring" thread: >>>> http://openjdk.5641.n7.nabble.com/OrderAccess-Refactoring-td212050.html >>>> >>>> Bug ID: >>>> https://bugs.openjdk.java.net/browse/JDK-7143664 >>>> (updated to reflect these related changes) >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~dholmes/7143664/webrev/ >>>> >>>> Before I describe more I would like to give special thanks to David >>>> Holmes for long discussions leading up to the currently proposed >>>> changes. I would also like to thank Jesper Wilhelmsson for helping me >>>> run my changes through JPRT. >>>> >>>> == Motivation == >>>> >>>> This change directly fixes a reported OrderAccess bug due to compiler >>>> reorderings which is still a vulnerability on almost all TSO platforms: >>>> https://bugs.openjdk.java.net/browse/JDK-806196 >>>> >>>> And directly fixes confusions like release_store() != release() store() >>>> due to memory model issues previously described in this bug ID. >>>> >>>> At the same time it provides clearer design with separation of concerns >>>> and generalization/specialization, removing a whole bunch of platform >>>> specific code which could be generalized. The platform specific files >>>> now only have a few LoC requirements (~7) to conform to the memory model >>>> by specifying what the stand alone barriers do. Then optionally >>>> optimizations to the general approach are possible if platforms support >>>> it. This also makes it much easier to port to new platforms. >>>> >>>> == Memory Model == >>>> >>>> The current definitions of acquire/release semantics are a bit fishy >>>> leading to problems previously described in the bug ID (release_store() >>>> != release() store()) and some other correctness issues. It has >>>> therefore been replaced with a new model. I would like to thank David >>>> Holmes for the long discussions leading up to the newly proposed model. >>>> >>>> The new model is formally defined like this: >>>> >>>> // T1: access_shared_data >>>> // T1: ]release >>>> // T1: (...) >>>> // T1: store(X) >>>> // >>>> // T2: load(X) >>>> // T2: (...) >>>> // T2: acquire[ >>>> // T2: access_shared_data >>>> // >>>> // It is guaranteed that if T2: load(X) synchronizes with (observes the >>>> // value written by) T1: store(X), then the memory accesses before the >>>> // T1: ]release happen before the memory accesses after the T2: acquire[. >>>> >>>> The orderAccess.hpp file and bug ID also has a few additional >>>> explanations making it more intuitive to the user when to use >>>> acquire/release and the resemblance to TSO abstract machines. Big thanks >>>> goes to David Holmes for discussing the memory model with me, which >>>> helped a lot in deriving it. >>>> >>>> Now it holds that release() store() == release_store(), and the other >>>> correctness issues are fixed as well. >>>> >>>> The new model is also closer to C++11 definitions which could give us >>>> more relaxed compiler reordering constraints in the future when compiler >>>> support for C++11 is there and ready. >>>> >>>> == Reliance on C++ Volatile Semantics == >>>> >>>> The C++ standard section 1.9 "Program Execution" is very vague about >>>> what the keyword volatile can actually do for us. It gives clear >>>> constraints in terms of volatile-volatile accesses but says little about >>>> nonvolatile-volatile accesses. Yet the current implementation heavily >>>> relies upon volatile to in terms of compiler reordering. But GCC >>>> explicitly declares that volatiles and non-volatiles may reorder freely >>>> ( https://gcc.gnu.org/onlinedocs/gcc/Volatiles.html ). The only compiler >>>> known to explicitly provide the wanted semantics with volatile is MSVC >>>> 2010 for windows ( >>>> https://msdn.microsoft.com/en-us/library/12a04hfd(v=vs.100).aspx ). >>>> Compilers not giving explicit guarantees, must be considered unsafe and >>>> revert to conservative behaviour. >>>> >>>> This was brought to attention after causing bugs, but was only fixed for >>>> x86 linux. This is a fundamental issue inherent to all TSO platforms >>>> except windows, and has to be fixed on all of them. >>>> >>>> Several barriers are unsafe to use because they lack compiler reordering >>>> constraints (e.g. fence and acquire on linux_SPARC). For TSO platforms >>>> they are typically implemented using dummy loads and stores. This seems >>>> to be another old volatile reliance that I fixed. These barriers >>>> sometimes have omitted compiler barriers (which is what we really want). >>>> This seems to be another example on incorrect reliance on the volatile >>>> semantics to help us. Therefore dummy loads/stores have been replaced >>>> with compiler barriers on TSO platforms. >>>> >>>> It is also worth noting that compilers like sun studio did previously >>>> not support inline asm syntax. Therefore, barriers were implemented in >>>> .il-files. However, using them does not give explicit compiler >>>> constraints for reordering AFAIK. Therefore, they have been >>>> reimplemented using inline asm with explicit compiler reordering >>>> constraints, as even sun (solaris?) studio now supports this. >>>> >>>> The windows variants have added a windows-style _ReadWriteBarrier() >>>> compiler barrier similarly. >>>> >>>> == Strange Hardware Reorderings == >>>> >>>> Fixed a weird inconsistency where acquire, loadstore and loadlaod would >>>> use isync instead of lwsync for PPC on linux_zero, but not in any other >>>> PPC platform in the repo. I assumed this is wrong and changed it to >>>> lwsync instead. >>>> >>>> == Code Redundancy and Refactoring == >>>> >>>> The OrderAccess code looks like it has been developed over a long period >>>> of time, with small incremental changes. This seems to have led to a lot >>>> of code duplication over time. For example, store_fence variants are not >>>> referenced from anywhere, yet contribute to a lot of the code base and a >>>> lot of awkwardness (such as being the one only exception not using >>>> volatiles for some reason). Moreover, store_fence is not used anywhere >>>> in hotspot because it is not a good fit for either the acquire/release >>>> semantics or the Java volatile semantics, leaving a big question mark on >>>> when it should ever be used. I took the liberty of removing it. >>>> >>>> Another redundancy issue is that most of the semantics is exactly the >>>> same for all platforms, yet all that default boilerplate such as how to >>>> make atomic accesses, where acquire/release is supposed to be placed >>>> w.r.t. the memory access, what the different barriers should do etc. is >>>> copied in redundantly for each os_cpu and each type of memory access for >>>> each os_cpu. This makes it extremely painful 1) to understand what >>>> actually defines a certain platform compared to the defaults and 2) to >>>> correct bugs like those discovered here 3) prevent silly mistakes and >>>> bugs, by simply having a lot less code defining the behaviour of >>>> OrderAccess that could go wrong. >>>> >>>> A new architecture/design for OrderAccess is proposed, using a >>>> generalization/specialization approach. >>>> >>>> A general implementation in /share/ defines how things work and splits >>>> into different concerns: 1) how to make an atomic memory access, 2) >>>> where to but barriers w.r.t. the memory access for things like >>>> load_acquire, release_store and release_store_fence, 3) how these >>>> barriers are defined. >>>> >>>> This allows a clear split between what is required for following the >>>> specifications, and optimizations, which become much more readable and >>>> only optimizations need to be reviewed in depth as the defaults can >>>> always be trusted given correct standalone barriers. >>>> >>>> The only thing a platform is required to specify, is what an >>>> implementation of acquire(), release() and fence() should do. If they >>>> are implemented properly, everything in OrderAccess is guaranteed to >>>> work according to specification using the generalized code. This makes >>>> it very easy to support new ports. ALL the other code in the os_cpu >>>> files is used /only/ for optimization purposes offered for specific >>>> configurations. >>>> >>>> However, it is highly customizable so that specific platform can perform >>>> any desired optimizations. For instance this load_acquire on PPC is >>>> optimized: >>>> >>>> template<> inline jbyte OrderAccess::specialized_load_acquire >>>> (volatile jbyte* p) { register jbyte t = load(p); >>>> inlasm_acquire_reg(t); return t; } >>>> >>>> This overrides the whole load_acquire implementation to do something >>>> custom. Platforms like x86 extensively use this for joined fencing >>>> variants to optimize. >>>> >>>> The default implementation of load_acquire() will make an atomic load() >>>> followed by acquire() since the semantics is generalized. The >>>> generalized semantics are defined using inlined postfix/prefix calls >>>> after/before the atomic access, as defined here: >>>> >>>> template<> inline void ScopedFenceGeneral::postfix() { >>>> OrderAccess::acquire(); } >>>> template<> inline void ScopedFenceGeneral::prefix() { >>>> OrderAccess::release(); } >>>> template<> inline void ScopedFenceGeneral::prefix() { >>>> OrderAccess::release(); } >>>> template<> inline void ScopedFenceGeneral::postfix() { >>>> OrderAccess::fence(); } >>>> >>>> For platforms that simply wish to override what e.g. acquire means for a >>>> joined ordered memory access in general, as different to calling stand >>>> alone acquire(), the semantics can be easily overridden for a platform >>>> such as windows like on windows: >>>> >>>> template<> inline void ScopedFence::postfix() { } >>>> template<> inline void ScopedFence::prefix() { } >>>> template<> inline void ScopedFence::prefix() { } >>>> template<> inline void ScopedFence::postfix() { >>>> OrderAccess::fence(); } >>>> >>>> In this example, since Windows (now) has a compiler barrier for acquire, >>>> but does not need it for joined accesses since volatile has stronger >>>> guarantees on windows, this is enough to specialize that for joined >>>> memory accesses, no extra protection is needed. >>>> >>>> == Backward Compatibility and Transitioning == >>>> >>>> Since the newly proposed code is structured differently to before, a >>>> #define was added for backward compatibility so that external >>>> repositories not adhering to this new architecture do not break. >>>> Furthermore, store_release was declared private and marked as >>>> deprecated. This allows for a smooth transition into the new style of >>>> OrderAccess. When the full transition is made in all known repos, the >>>> #define and store_fence may be safely removed, eventually. >>>> >>>> == Documentation == >>>> >>>> The documentation seems old and outdated, describing how it works on >>>> SPARC RMO and IA64, which are nowhere to be found in the repository. It >>>> also describes things about C++ volatiles which cannot be relied upon. >>>> The documentation has been cleaned up to match the current state of the >>>> implementation better, with architectures actually found in the >>>> repository. >>>> >>>> == Testing == >>>> >>>> JPRT. Big thanks to Jesper Wilhelmsson for helping me test these changes. >>>> Ran some DaCapo benchmarks (I know okay :p) for performance regression >>>> and there was no perceivable difference. >>>> >>>> Looking forward to feedback on this, and hope to get some reviews. :) >>>> >>>> Thanks, >>>> Erik From erik.osterlund at lnu.se Fri Jan 23 23:03:26 2015 From: erik.osterlund at lnu.se (=?iso-8859-1?Q?Erik_=D6sterlund?=) Date: Fri, 23 Jan 2015 23:03:26 +0000 Subject: RFR: 7143664: Clean up OrderAccess implementations and usage References: <54C1EBD3.1000905@oracle.com> <54C1F3D8.5040507@oracle.com> <54C2C1EA.5020507@oracle.com> Message-ID: Hi Dean, On 23/01/15 22:50, Dean Long wrote: > On 1/23/2015 7:16 AM, Erik ?sterlund wrote: >> Hi Dean and David, >> >> Thanks for reviewing this change. >> >> On 23/01/15 08:12, David Holmes wrote: >>> On 23/01/2015 4:36 PM, Dean Long wrote: >>>> 147 // Ordering a load relative to preceding stores requires a fence, >>>> 148 // which implies a membar #StoreLoad between the store and load under >>>> 149 // sparc-TSO. A fence is required by x86. On x86, we use explicitly >>>> 150 // locked add. >>>> 151 // >>>> >>>> It sounds like the above is saying that fence is the same as StoreLoad ... >>> That may be a s/store_fence/fence/ typo. The original text was: >>> >>> // Ordering a load relative to preceding stores requires a store_fence, >>> // which implies a membar #StoreLoad between the store and load under >>> // sparc-TSO. A fence is required by ia64. On x86, we use locked xchg. >>> >>> Actually seems like a couple of typos there: ia64 became x86, then we >>> have x86 again. >> I tried to remove ia64 from the comments as it is not in the repository, >> which in this case became a bit redundant. Are we okay if I remove the >> "A fence is required by x86" sentence? >> >> Oh and Dean about membar #StoreLoad on SPARC and locked add on x86... >> are you saying they are not equivalent? > > I wasn't trying to say anything abou SPARC or x86, only that "fence" != > "StoreLoad". Agreed. >> New version of that paragraph with that sentence removed: >> >> 147 // Ordering a load relative to preceding stores requires a fence, >> 148 // which implies a membar #StoreLoad between the store and load under >> 149 // sparc-TSO. On x86, we use explicitly locked add. > > I would replace "fence" with "StoreLoad" above. Again, I just changed any store_fence descriptions to use fence instead. But you are certainly right, it is more accurate to talk about StoreLoad here and not mention fence in the first place, I'll change this, thanks for the feedback. :) Changed to: 147 // Ordering a load relative to preceding stores requires a StoreLoad, 148 // which implies a membar #StoreLoad between the store and load under 149 // sparc-TSO. On x86, we use explicitly locked add. >>>> 152 // 4. store, load <= is constrained by => store, fence, load >>>> 153 // >>>> 154 // Use store, fence to make sure all stores done in an 'interesting' >>>> 155 // region are made visible prior to both subsequent loads and stores. >>>> >>>> ... and this is like saying to use fence because StoreStore | StoreLoad >>>> isn't available. >>> Again this seems an issue with store_fence being changed to "store, >>> fence" which doesn't really make sense. >> The thing is that I removed store_fence (not store_release as it says in >> the bug ID, sorry about that - there obviously is no store_release). So >> I wanted to remove references in the comments to store_fence as well. >> Therefore I changed the example to use store fence load instead of >> store_fence load. Perhaps it's better if I remove that sample all >> together instead? > > That works for me. Fixed. Thanks, /Erik > dl > >> Thanks for reviewing my changes! >> >> /Erik >> >>> David >>> >>>>> Furthermore, store_release was declared private and marked as >>>> deprecated. >>>> >>>> I can't find where this was done. >>>> >>>> dl >>>> >>>> On 1/22/2015 5:19 PM, Erik ?sterlund wrote: >>>>> Hi all, >>>>> >>>>> == Summary of Changes == >>>>> >>>>> This changeset fixes issues in OrderAccess on multiple levels from the >>>>> memory model semantics to compiler reorderings, to addressing >>>>> maintainability/portability issues which (almost) had to be fixed in >>>>> order to fix the correctness issues. It is the result of discussions >>>>> found in the previous "OrderAccess Refactoring" thread: >>>>> http://openjdk.5641.n7.nabble.com/OrderAccess-Refactoring-td212050.html >>>>> >>>>> Bug ID: >>>>> https://bugs.openjdk.java.net/browse/JDK-7143664 >>>>> (updated to reflect these related changes) >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~dholmes/7143664/webrev/ >>>>> >>>>> Before I describe more I would like to give special thanks to David >>>>> Holmes for long discussions leading up to the currently proposed >>>>> changes. I would also like to thank Jesper Wilhelmsson for helping me >>>>> run my changes through JPRT. >>>>> >>>>> == Motivation == >>>>> >>>>> This change directly fixes a reported OrderAccess bug due to compiler >>>>> reorderings which is still a vulnerability on almost all TSO platforms: >>>>> https://bugs.openjdk.java.net/browse/JDK-806196 >>>>> >>>>> And directly fixes confusions like release_store() != release() store() >>>>> due to memory model issues previously described in this bug ID. >>>>> >>>>> At the same time it provides clearer design with separation of concerns >>>>> and generalization/specialization, removing a whole bunch of platform >>>>> specific code which could be generalized. The platform specific files >>>>> now only have a few LoC requirements (~7) to conform to the memory model >>>>> by specifying what the stand alone barriers do. Then optionally >>>>> optimizations to the general approach are possible if platforms support >>>>> it. This also makes it much easier to port to new platforms. >>>>> >>>>> == Memory Model == >>>>> >>>>> The current definitions of acquire/release semantics are a bit fishy >>>>> leading to problems previously described in the bug ID (release_store() >>>>> != release() store()) and some other correctness issues. It has >>>>> therefore been replaced with a new model. I would like to thank David >>>>> Holmes for the long discussions leading up to the newly proposed model. >>>>> >>>>> The new model is formally defined like this: >>>>> >>>>> // T1: access_shared_data >>>>> // T1: ]release >>>>> // T1: (...) >>>>> // T1: store(X) >>>>> // >>>>> // T2: load(X) >>>>> // T2: (...) >>>>> // T2: acquire[ >>>>> // T2: access_shared_data >>>>> // >>>>> // It is guaranteed that if T2: load(X) synchronizes with (observes the >>>>> // value written by) T1: store(X), then the memory accesses before the >>>>> // T1: ]release happen before the memory accesses after the T2: acquire[. >>>>> >>>>> The orderAccess.hpp file and bug ID also has a few additional >>>>> explanations making it more intuitive to the user when to use >>>>> acquire/release and the resemblance to TSO abstract machines. Big thanks >>>>> goes to David Holmes for discussing the memory model with me, which >>>>> helped a lot in deriving it. >>>>> >>>>> Now it holds that release() store() == release_store(), and the other >>>>> correctness issues are fixed as well. >>>>> >>>>> The new model is also closer to C++11 definitions which could give us >>>>> more relaxed compiler reordering constraints in the future when compiler >>>>> support for C++11 is there and ready. >>>>> >>>>> == Reliance on C++ Volatile Semantics == >>>>> >>>>> The C++ standard section 1.9 "Program Execution" is very vague about >>>>> what the keyword volatile can actually do for us. It gives clear >>>>> constraints in terms of volatile-volatile accesses but says little about >>>>> nonvolatile-volatile accesses. Yet the current implementation heavily >>>>> relies upon volatile to in terms of compiler reordering. But GCC >>>>> explicitly declares that volatiles and non-volatiles may reorder freely >>>>> ( https://gcc.gnu.org/onlinedocs/gcc/Volatiles.html ). The only compiler >>>>> known to explicitly provide the wanted semantics with volatile is MSVC >>>>> 2010 for windows ( >>>>> https://msdn.microsoft.com/en-us/library/12a04hfd(v=vs.100).aspx ). >>>>> Compilers not giving explicit guarantees, must be considered unsafe and >>>>> revert to conservative behaviour. >>>>> >>>>> This was brought to attention after causing bugs, but was only fixed for >>>>> x86 linux. This is a fundamental issue inherent to all TSO platforms >>>>> except windows, and has to be fixed on all of them. >>>>> >>>>> Several barriers are unsafe to use because they lack compiler reordering >>>>> constraints (e.g. fence and acquire on linux_SPARC). For TSO platforms >>>>> they are typically implemented using dummy loads and stores. This seems >>>>> to be another old volatile reliance that I fixed. These barriers >>>>> sometimes have omitted compiler barriers (which is what we really want). >>>>> This seems to be another example on incorrect reliance on the volatile >>>>> semantics to help us. Therefore dummy loads/stores have been replaced >>>>> with compiler barriers on TSO platforms. >>>>> >>>>> It is also worth noting that compilers like sun studio did previously >>>>> not support inline asm syntax. Therefore, barriers were implemented in >>>>> .il-files. However, using them does not give explicit compiler >>>>> constraints for reordering AFAIK. Therefore, they have been >>>>> reimplemented using inline asm with explicit compiler reordering >>>>> constraints, as even sun (solaris?) studio now supports this. >>>>> >>>>> The windows variants have added a windows-style _ReadWriteBarrier() >>>>> compiler barrier similarly. >>>>> >>>>> == Strange Hardware Reorderings == >>>>> >>>>> Fixed a weird inconsistency where acquire, loadstore and loadlaod would >>>>> use isync instead of lwsync for PPC on linux_zero, but not in any other >>>>> PPC platform in the repo. I assumed this is wrong and changed it to >>>>> lwsync instead. >>>>> >>>>> == Code Redundancy and Refactoring == >>>>> >>>>> The OrderAccess code looks like it has been developed over a long period >>>>> of time, with small incremental changes. This seems to have led to a lot >>>>> of code duplication over time. For example, store_fence variants are not >>>>> referenced from anywhere, yet contribute to a lot of the code base and a >>>>> lot of awkwardness (such as being the one only exception not using >>>>> volatiles for some reason). Moreover, store_fence is not used anywhere >>>>> in hotspot because it is not a good fit for either the acquire/release >>>>> semantics or the Java volatile semantics, leaving a big question mark on >>>>> when it should ever be used. I took the liberty of removing it. >>>>> >>>>> Another redundancy issue is that most of the semantics is exactly the >>>>> same for all platforms, yet all that default boilerplate such as how to >>>>> make atomic accesses, where acquire/release is supposed to be placed >>>>> w.r.t. the memory access, what the different barriers should do etc. is >>>>> copied in redundantly for each os_cpu and each type of memory access for >>>>> each os_cpu. This makes it extremely painful 1) to understand what >>>>> actually defines a certain platform compared to the defaults and 2) to >>>>> correct bugs like those discovered here 3) prevent silly mistakes and >>>>> bugs, by simply having a lot less code defining the behaviour of >>>>> OrderAccess that could go wrong. >>>>> >>>>> A new architecture/design for OrderAccess is proposed, using a >>>>> generalization/specialization approach. >>>>> >>>>> A general implementation in /share/ defines how things work and splits >>>>> into different concerns: 1) how to make an atomic memory access, 2) >>>>> where to but barriers w.r.t. the memory access for things like >>>>> load_acquire, release_store and release_store_fence, 3) how these >>>>> barriers are defined. >>>>> >>>>> This allows a clear split between what is required for following the >>>>> specifications, and optimizations, which become much more readable and >>>>> only optimizations need to be reviewed in depth as the defaults can >>>>> always be trusted given correct standalone barriers. >>>>> >>>>> The only thing a platform is required to specify, is what an >>>>> implementation of acquire(), release() and fence() should do. If they >>>>> are implemented properly, everything in OrderAccess is guaranteed to >>>>> work according to specification using the generalized code. This makes >>>>> it very easy to support new ports. ALL the other code in the os_cpu >>>>> files is used /only/ for optimization purposes offered for specific >>>>> configurations. >>>>> >>>>> However, it is highly customizable so that specific platform can perform >>>>> any desired optimizations. For instance this load_acquire on PPC is >>>>> optimized: >>>>> >>>>> template<> inline jbyte OrderAccess::specialized_load_acquire >>>>> (volatile jbyte* p) { register jbyte t = load(p); >>>>> inlasm_acquire_reg(t); return t; } >>>>> >>>>> This overrides the whole load_acquire implementation to do something >>>>> custom. Platforms like x86 extensively use this for joined fencing >>>>> variants to optimize. >>>>> >>>>> The default implementation of load_acquire() will make an atomic load() >>>>> followed by acquire() since the semantics is generalized. The >>>>> generalized semantics are defined using inlined postfix/prefix calls >>>>> after/before the atomic access, as defined here: >>>>> >>>>> template<> inline void ScopedFenceGeneral::postfix() { >>>>> OrderAccess::acquire(); } >>>>> template<> inline void ScopedFenceGeneral::prefix() { >>>>> OrderAccess::release(); } >>>>> template<> inline void ScopedFenceGeneral::prefix() { >>>>> OrderAccess::release(); } >>>>> template<> inline void ScopedFenceGeneral::postfix() { >>>>> OrderAccess::fence(); } >>>>> >>>>> For platforms that simply wish to override what e.g. acquire means for a >>>>> joined ordered memory access in general, as different to calling stand >>>>> alone acquire(), the semantics can be easily overridden for a platform >>>>> such as windows like on windows: >>>>> >>>>> template<> inline void ScopedFence::postfix() { } >>>>> template<> inline void ScopedFence::prefix() { } >>>>> template<> inline void ScopedFence::prefix() { } >>>>> template<> inline void ScopedFence::postfix() { >>>>> OrderAccess::fence(); } >>>>> >>>>> In this example, since Windows (now) has a compiler barrier for acquire, >>>>> but does not need it for joined accesses since volatile has stronger >>>>> guarantees on windows, this is enough to specialize that for joined >>>>> memory accesses, no extra protection is needed. >>>>> >>>>> == Backward Compatibility and Transitioning == >>>>> >>>>> Since the newly proposed code is structured differently to before, a >>>>> #define was added for backward compatibility so that external >>>>> repositories not adhering to this new architecture do not break. >>>>> Furthermore, store_release was declared private and marked as >>>>> deprecated. This allows for a smooth transition into the new style of >>>>> OrderAccess. When the full transition is made in all known repos, the >>>>> #define and store_fence may be safely removed, eventually. >>>>> >>>>> == Documentation == >>>>> >>>>> The documentation seems old and outdated, describing how it works on >>>>> SPARC RMO and IA64, which are nowhere to be found in the repository. It >>>>> also describes things about C++ volatiles which cannot be relied upon. >>>>> The documentation has been cleaned up to match the current state of the >>>>> implementation better, with architectures actually found in the >>>>> repository. >>>>> >>>>> == Testing == >>>>> >>>>> JPRT. Big thanks to Jesper Wilhelmsson for helping me test these changes. >>>>> Ran some DaCapo benchmarks (I know okay :p) for performance regression >>>>> and there was no perceivable difference. >>>>> >>>>> Looking forward to feedback on this, and hope to get some reviews. :) >>>>> >>>>> Thanks, >>>>> Erik > > From igor.veresov at oracle.com Mon Jan 26 07:00:11 2015 From: igor.veresov at oracle.com (Igor Veresov) Date: Sun, 25 Jan 2015 23:00:11 -0800 Subject: [8u] RFR(XS): 8071302: assert(!_reg_node[reg_lo] || edge_from_to(_reg_node[reg_lo], def)) failed: after block local Message-ID: JDK9 webrev: http://cr.openjdk.java.net/~iveresov/8071302/webrev.00 JDK9 changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/b9b4b9394887 JBS: https://bugs.openjdk.java.net/browse/JDK-8071302 Thanks! igor From vladimir.kozlov at oracle.com Mon Jan 26 07:39:01 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Sun, 25 Jan 2015 23:39:01 -0800 Subject: [8u] RFR(XS): 8071302: assert(!_reg_node[reg_lo] || edge_from_to(_reg_node[reg_lo], def)) failed: after block local In-Reply-To: References: Message-ID: <54C5EF15.7060202@oracle.com> Okay. Thanks Vladimir On 1/25/15 11:00 PM, Igor Veresov wrote: > JDK9 webrev: http://cr.openjdk.java.net/~iveresov/8071302/webrev.00 > JDK9 changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/b9b4b9394887 > JBS: https://bugs.openjdk.java.net/browse/JDK-8071302 > > Thanks! > igor > From igor.veresov at oracle.com Mon Jan 26 08:07:47 2015 From: igor.veresov at oracle.com (Igor Veresov) Date: Mon, 26 Jan 2015 00:07:47 -0800 Subject: [8u] RFR(XS): 8071302: assert(!_reg_node[reg_lo] || edge_from_to(_reg_node[reg_lo], def)) failed: after block local In-Reply-To: <54C5EF15.7060202@oracle.com> References: <54C5EF15.7060202@oracle.com> Message-ID: <4F60685E-52A1-4013-AFA9-19127565B4E9@oracle.com> Thanks! igor > On Jan 25, 2015, at 11:39 PM, Vladimir Kozlov wrote: > > Okay. > > Thanks > Vladimir > > On 1/25/15 11:00 PM, Igor Veresov wrote: >> JDK9 webrev: http://cr.openjdk.java.net/~iveresov/8071302/webrev.00 >> JDK9 changeset: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/b9b4b9394887 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8071302 >> >> Thanks! >> igor >> From goetz.lindenmaier at sap.com Mon Jan 26 08:20:59 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 26 Jan 2015 08:20:59 +0000 Subject: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF70D40@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF6DE04@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CF6FAFC@DEWDFEMB12A.global.corp.sap> <54B9970B.7040502@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF70D40@DEWDFEMB12A.global.corp.sap> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF71FB7@DEWDFEMB12A.global.corp.sap> Hi, could please somebody from servicability sponsor this tiny change? (My first mail didn't go through, because I wasn't yet registered to servicability-dev). Thanks, Goetz. -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore Sent: Freitag, 16. Januar 2015 23:56 To: hotspot-dev at openjdk.java.net Subject: Re: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available This seems fine. Someone from the serviceability group should probably sponsor. Thanks, Coleen On 1/16/15, 6:15 AM, Lindenmaier, Goetz wrote: > Hi, > > could I please get a review for this small fix? I also please need a sponsor. > > Thanks, > Goetz. > > From: goetz.lindenmaier at sap.com > Sent: Montag, 12. Januar 2015 09:23 > To: hotspot-dev at openjdk.java.net > Subject: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available > > Hi, > > This test uses SA, thus fails on platforms where that's not implemented. > I see problems on mac and aix. > Call Platform.shouldSAAttach() to check whether SA should work. > > Please review this small fix. I please need a sponsor. > http://cr.openjdk.java.net/~goetz/webrevs/8068778-jtregSA/webrev.01/ > > Best regards, > Goetz. From staffan.larsen at oracle.com Mon Jan 26 08:42:59 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Mon, 26 Jan 2015 09:42:59 +0100 Subject: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF71FB7@DEWDFEMB12A.global.corp.sap> References: <4295855A5C1DE049A61835A1887419CC2CF6DE04@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CF6FAFC@DEWDFEMB12A.global.corp.sap> <54B9970B.7040502@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF70D40@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CF71FB7@DEWDFEMB12A.global.corp.sap> Message-ID: <763407CB-A3AE-44D9-A6CD-A864E3A62CBC@oracle.com> Change looks good to me - I?ll sponsor it! /Staffan > On 26 jan 2015, at 09:20, Lindenmaier, Goetz wrote: > > Hi, > > could please somebody from servicability sponsor this tiny change? > > (My first mail didn't go through, because I wasn't yet registered to servicability-dev). > > Thanks, > Goetz. > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore > Sent: Freitag, 16. Januar 2015 23:56 > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available > > > This seems fine. Someone from the serviceability group should probably > sponsor. > > Thanks, > Coleen > > On 1/16/15, 6:15 AM, Lindenmaier, Goetz wrote: >> Hi, >> >> could I please get a review for this small fix? I also please need a sponsor. >> >> Thanks, >> Goetz. >> >> From: goetz.lindenmaier at sap.com >> Sent: Montag, 12. Januar 2015 09:23 >> To: hotspot-dev at openjdk.java.net >> Subject: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available >> >> Hi, >> >> This test uses SA, thus fails on platforms where that's not implemented. >> I see problems on mac and aix. >> Call Platform.shouldSAAttach() to check whether SA should work. >> >> Please review this small fix. I please need a sponsor. >> http://cr.openjdk.java.net/~goetz/webrevs/8068778-jtregSA/webrev.01/ >> >> Best regards, >> Goetz. > From goetz.lindenmaier at sap.com Mon Jan 26 09:08:36 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 26 Jan 2015 09:08:36 +0000 Subject: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available In-Reply-To: <763407CB-A3AE-44D9-A6CD-A864E3A62CBC@oracle.com> References: <4295855A5C1DE049A61835A1887419CC2CF6DE04@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CF6FAFC@DEWDFEMB12A.global.corp.sap> <54B9970B.7040502@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF70D40@DEWDFEMB12A.global.corp.sap> <4295855A5C1DE049A61835A1887419CC2CF71FB7@DEWDFEMB12A.global.corp.sap> <763407CB-A3AE-44D9-A6CD-A864E3A62CBC@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF72005@DEWDFEMB12A.global.corp.sap> That's great, thanks a lot Staffan! Best regards, Goetz. -----Original Message----- From: Staffan Larsen [mailto:staffan.larsen at oracle.com] Sent: Montag, 26. Januar 2015 09:43 To: Lindenmaier, Goetz Cc: serviceability-dev (serviceability-dev at openjdk.java.net); hotspot-dev at openjdk.java.net Developers Subject: Re: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available Change looks good to me - I?ll sponsor it! /Staffan > On 26 jan 2015, at 09:20, Lindenmaier, Goetz wrote: > > Hi, > > could please somebody from servicability sponsor this tiny change? > > (My first mail didn't go through, because I wasn't yet registered to servicability-dev). > > Thanks, > Goetz. > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Coleen Phillimore > Sent: Freitag, 16. Januar 2015 23:56 > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available > > > This seems fine. Someone from the serviceability group should probably > sponsor. > > Thanks, > Coleen > > On 1/16/15, 6:15 AM, Lindenmaier, Goetz wrote: >> Hi, >> >> could I please get a review for this small fix? I also please need a sponsor. >> >> Thanks, >> Goetz. >> >> From: goetz.lindenmaier at sap.com >> Sent: Montag, 12. Januar 2015 09:23 >> To: hotspot-dev at openjdk.java.net >> Subject: RFR(XS): 8068778: [TESTBUG] CompressedClassSpaceSizeInJmapHeap.java fails if SA not available >> >> Hi, >> >> This test uses SA, thus fails on platforms where that's not implemented. >> I see problems on mac and aix. >> Call Platform.shouldSAAttach() to check whether SA should work. >> >> Please review this small fix. I please need a sponsor. >> http://cr.openjdk.java.net/~goetz/webrevs/8068778-jtregSA/webrev.01/ >> >> Best regards, >> Goetz. > From edward.nevill at linaro.org Mon Jan 26 14:12:27 2015 From: edward.nevill at linaro.org (Edward Nevill) Date: Mon, 26 Jan 2015 14:12:27 +0000 Subject: RFR: 8071563 AARCH64 Staging FTBFS Message-ID: <1422281547.1592.5.camel@mylittlepony.linaroharston> Hi, Could someone please review my change to fix issue https://bugs.openjdk.java.net/browse/JDK-8071563 and if approved push it to ssh://hg.openjdk.java.net/aarch64-port/stage/hotspot Here is the webrev http://cr.openjdk.java.net/~enevill/8071563/webrev.00/ Many thanks, Ed. From aph at redhat.com Mon Jan 26 15:20:19 2015 From: aph at redhat.com (Andrew Haley) Date: Mon, 26 Jan 2015 15:20:19 +0000 Subject: RFR: 8071563 AARCH64 Staging FTBFS In-Reply-To: <1422281547.1592.5.camel@mylittlepony.linaroharston> References: <1422281547.1592.5.camel@mylittlepony.linaroharston> Message-ID: <54C65B33.8090307@redhat.com> Hi, On 01/26/2015 02:12 PM, Edward Nevill wrote: > Could someone please review my change to fix issue > > https://bugs.openjdk.java.net/browse/JDK-8071563 > > and if approved push it to > > ssh://hg.openjdk.java.net/aarch64-port/stage/hotspot > > Here is the webrev > > http://cr.openjdk.java.net/~enevill/8071563/webrev.00/ I don't know why this patch is necessary. Builds for me. Andrew. From volker.simonis at gmail.com Mon Jan 26 15:21:40 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 26 Jan 2015 16:21:40 +0100 Subject: RFR(M): 8069590: AIX port of "8050807: Better performing performance data handling" Message-ID: Hi, this is a port of the security fix "8050807: Better performing performance data handling" which came from 8u31 to AIX. For some reason this AIX-version didn't made it right right in time into 8u31 so we have to manually port it now to jdk9-dev and jdk8u-dev. This change is AIX-only and already reviewed. I just post it here for your convenience: http://cr.openjdk.java.net/~simonis/webrevs/2015/8069590_jdk9/ https://bugs.openjdk.java.net/browse/JDK-8069590 Thank you and best regards, Volker From adinn at redhat.com Mon Jan 26 15:56:52 2015 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 26 Jan 2015 15:56:52 +0000 Subject: [aarch64-port-dev ] RFR: 8071563 AARCH64 Staging FTBFS In-Reply-To: <54C65B33.8090307@redhat.com> References: <1422281547.1592.5.camel@mylittlepony.linaroharston> <54C65B33.8090307@redhat.com> Message-ID: <54C663C4.4090609@redhat.com> On 26/01/15 15:20, Andrew Haley wrote: > Hi, > > On 01/26/2015 02:12 PM, Edward Nevill wrote: > >> Could someone please review my change to fix issue >> >> https://bugs.openjdk.java.net/browse/JDK-8071563 >> >> and if approved push it to >> >> ssh://hg.openjdk.java.net/aarch64-port/stage/hotspot >> >> Here is the webrev >> >> http://cr.openjdk.java.net/~enevill/8071563/webrev.00/ > > I don't know why this patch is necessary. Builds for me. Hmm, but not for me. I am still rebuilding with Ed's patch but I think it is the required fix . . . regards, Andrew Dinn ----------- From goetz.lindenmaier at sap.com Mon Jan 26 15:56:44 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 26 Jan 2015 15:56:44 +0000 Subject: RFR: 8071563 AARCH64 Staging FTBFS In-Reply-To: <54C65B33.8090307@redhat.com> References: <1422281547.1592.5.camel@mylittlepony.linaroharston> <54C65B33.8090307@redhat.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF721D0@DEWDFEMB12A.global.corp.sap> I guess this is the merge between your change to metaspace and mine removing the null argument: http://hg.openjdk.java.net/jdk9/hs/hotspot/rev/fee07a43d038 Best regards, Goetz. -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Andrew Haley Sent: Montag, 26. Januar 2015 16:20 To: edward.nevill at linaro.org; hotspot-dev Source Developers Cc: aarch64-port-dev at openjdk.java.net Subject: Re: RFR: 8071563 AARCH64 Staging FTBFS Hi, On 01/26/2015 02:12 PM, Edward Nevill wrote: > Could someone please review my change to fix issue > > https://bugs.openjdk.java.net/browse/JDK-8071563 > > and if approved push it to > > ssh://hg.openjdk.java.net/aarch64-port/stage/hotspot > > Here is the webrev > > http://cr.openjdk.java.net/~enevill/8071563/webrev.00/ I don't know why this patch is necessary. Builds for me. Andrew. From adinn at redhat.com Mon Jan 26 16:02:25 2015 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 26 Jan 2015 16:02:25 +0000 Subject: [aarch64-port-dev ] RFR: 8071563 AARCH64 Staging FTBFS In-Reply-To: <54C663C4.4090609@redhat.com> References: <1422281547.1592.5.camel@mylittlepony.linaroharston> <54C65B33.8090307@redhat.com> <54C663C4.4090609@redhat.com> Message-ID: <54C66511.2000903@redhat.com> On 26/01/15 15:56, Andrew Dinn wrote: > I am still rebuilding with Ed's patch but I think it is the required fix > . . . Just to confirm, Ed's patch fixes the build and the resulting image passes basic smoke tests (java Hello, javac Hello.java, netbeans). regards, Andrew Dinn ----------- From aph at redhat.com Mon Jan 26 16:09:36 2015 From: aph at redhat.com (Andrew Haley) Date: Mon, 26 Jan 2015 16:09:36 +0000 Subject: [aarch64-port-dev ] RFR: 8071563 AARCH64 Staging FTBFS In-Reply-To: <54C663C4.4090609@redhat.com> References: <1422281547.1592.5.camel@mylittlepony.linaroharston> <54C65B33.8090307@redhat.com> <54C663C4.4090609@redhat.com> Message-ID: <54C666C0.3060201@redhat.com> On 01/26/2015 03:56 PM, Andrew Dinn wrote: > Hmm, but not for me. > > I am still rebuilding with Ed's patch but I think it is the required fix Merge error at my end. You are right. Andrew. From volker.simonis at gmail.com Mon Jan 26 16:12:45 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 26 Jan 2015 17:12:45 +0100 Subject: [8u60] Request for approval: 8069590: AIX port of "8050807: Better performing performance data handling" Message-ID: Hi, can you please approve this security fix which is the AIX-port port of "8050807: Better performing performance data handling" from 8u31. For some reason this AIX-version didn't made it right right in time into 8u31 so we have to manually port it now to jdk8u-dev. This fix is AIX-only, it is reviewed and it cleanly applies to both, jdk8u-dev and jdk8u40 http://cr.openjdk.java.net/~simonis/webrevs/2015/8069590_jdk8u/ https://bugs.openjdk.java.net/browse/JDK-8069590 This fix should also be pushed to 8u40 if that is possible at all. Thank you and best regards, Volker From vladimir.kozlov at oracle.com Mon Jan 26 16:51:47 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 26 Jan 2015 08:51:47 -0800 Subject: RFR: 8071563 AARCH64 Staging FTBFS In-Reply-To: <1422281547.1592.5.camel@mylittlepony.linaroharston> References: <1422281547.1592.5.camel@mylittlepony.linaroharston> Message-ID: <54C670A3.4000703@oracle.com> Thank you, Edward Looks good. I will push it. Thanks, Vladimir On 1/26/15 6:12 AM, Edward Nevill wrote: > Hi, > > Could someone please review my change to fix issue > > https://bugs.openjdk.java.net/browse/JDK-8071563 > > and if approved push it to > > ssh://hg.openjdk.java.net/aarch64-port/stage/hotspot > > Here is the webrev > > http://cr.openjdk.java.net/~enevill/8071563/webrev.00/ > > Many thanks, > Ed. > > From dean.long at oracle.com Tue Jan 27 04:09:33 2015 From: dean.long at oracle.com (Dean Long) Date: Mon, 26 Jan 2015 20:09:33 -0800 Subject: RFR(XS) 8069030: support new PTRACE_GETREGSET Message-ID: <54C70F7D.3010003@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8069030 http://cr.openjdk.java.net/~dlong/8069030/webrev.00/ Testing on linux x86 by temporarily undefining PTRACE_GETREGS_REQ so that the new code would get triggered, then stepping through "jstack -m " with gdb to make sure the new code executed correctly. dl From staffan.larsen at oracle.com Tue Jan 27 07:02:50 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 27 Jan 2015 08:02:50 +0100 Subject: RFR(XS) 8069030: support new PTRACE_GETREGSET In-Reply-To: <54C70F7D.3010003@oracle.com> References: <54C70F7D.3010003@oracle.com> Message-ID: Looks good! Thanks, /Staffan > On 27 jan 2015, at 05:09, Dean Long wrote: > > https://bugs.openjdk.java.net/browse/JDK-8069030 > > http://cr.openjdk.java.net/~dlong/8069030/webrev.00/ > > Testing on linux x86 by temporarily undefining PTRACE_GETREGS_REQ so that the new code > would get triggered, then stepping through "jstack -m " with gdb to make sure the new > code executed correctly. > > dl > From volker.simonis at gmail.com Tue Jan 27 08:34:26 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 27 Jan 2015 09:34:26 +0100 Subject: [8u60] Request for approval: 8069590: AIX port of "8050807: Better performing performance data handling" In-Reply-To: References: Message-ID: Can I push this to 8u-dev now? Thanks, Volker On Mon, Jan 26, 2015 at 5:12 PM, Volker Simonis wrote: > Hi, > > can you please approve this security fix which is the AIX-port port of > "8050807: Better performing performance data handling" from 8u31. For > some reason this AIX-version didn't made it right right in time into > 8u31 so we have to manually port it now to jdk8u-dev. > > This fix is AIX-only, it is reviewed and it cleanly applies to both, > jdk8u-dev and jdk8u40 > > http://cr.openjdk.java.net/~simonis/webrevs/2015/8069590_jdk8u/ > https://bugs.openjdk.java.net/browse/JDK-8069590 > > This fix should also be pushed to 8u40 if that is possible at all. > > Thank you and best regards, > Volker From vladimir.kozlov at oracle.com Tue Jan 27 09:43:55 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 27 Jan 2015 01:43:55 -0800 Subject: [8u60] Request for approval: 8069590: AIX port of "8050807: Better performing performance data handling" In-Reply-To: References: Message-ID: <54C75DDB.6010409@oracle.com> Changes are fine and I think you can push it. Nothing in JPRT queue is pushing into 8u. Note, your changes are based on different repository from what we use to push Hotspot into 8u. We use: http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot/ Alejandro will promoted changes into /jdk8u/jdk8u-dev/hotspot after testing. But may be you have agreement with Alejandro where to push. Ignore me then. Thanks, Vladimir On 1/27/15 12:34 AM, Volker Simonis wrote: > Can I push this to 8u-dev now? > > Thanks, > Volker > > On Mon, Jan 26, 2015 at 5:12 PM, Volker Simonis > wrote: >> Hi, >> >> can you please approve this security fix which is the AIX-port port of >> "8050807: Better performing performance data handling" from 8u31. For >> some reason this AIX-version didn't made it right right in time into >> 8u31 so we have to manually port it now to jdk8u-dev. >> >> This fix is AIX-only, it is reviewed and it cleanly applies to both, >> jdk8u-dev and jdk8u40 >> >> http://cr.openjdk.java.net/~simonis/webrevs/2015/8069590_jdk8u/ >> https://bugs.openjdk.java.net/browse/JDK-8069590 >> >> This fix should also be pushed to 8u40 if that is possible at all. >> >> Thank you and best regards, >> Volker From volker.simonis at gmail.com Tue Jan 27 10:20:18 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 27 Jan 2015 11:20:18 +0100 Subject: [8u60] Request for approval: 8069590: AIX port of "8050807: Better performing performance data handling" In-Reply-To: <54C75DDB.6010409@oracle.com> References: <54C75DDB.6010409@oracle.com> Message-ID: My change is based on http://hg.openjdk.java.net/jdk8u/jdk8u-dev/hotspot and I'd like to push it there. Alejandro can not test this anyway as it is AIX-only. @Alejandro: can I push this directly to http://hg.openjdk.java.net/jdk8u/jdk8u-dev/hotspot now? If we go trough http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot/ we loose even more time and we'll never make it into 8u40. Thanks, Volker On Tue, Jan 27, 2015 at 10:43 AM, Vladimir Kozlov wrote: > Changes are fine and I think you can push it. Nothing in JPRT queue is > pushing into 8u. > Note, your changes are based on different repository from what we use to > push Hotspot into 8u. We use: > > http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot/ > > Alejandro will promoted changes into /jdk8u/jdk8u-dev/hotspot after testing. > > But may be you have agreement with Alejandro where to push. Ignore me then. > > Thanks, > Vladimir > > > On 1/27/15 12:34 AM, Volker Simonis wrote: >> >> Can I push this to 8u-dev now? >> >> Thanks, >> Volker >> >> On Mon, Jan 26, 2015 at 5:12 PM, Volker Simonis >> wrote: >>> >>> Hi, >>> >>> can you please approve this security fix which is the AIX-port port of >>> "8050807: Better performing performance data handling" from 8u31. For >>> some reason this AIX-version didn't made it right right in time into >>> 8u31 so we have to manually port it now to jdk8u-dev. >>> >>> This fix is AIX-only, it is reviewed and it cleanly applies to both, >>> jdk8u-dev and jdk8u40 >>> >>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8069590_jdk8u/ >>> https://bugs.openjdk.java.net/browse/JDK-8069590 >>> >>> This fix should also be pushed to 8u40 if that is possible at all. >>> >>> Thank you and best regards, >>> Volker From sean.coffey at oracle.com Tue Jan 27 11:58:55 2015 From: sean.coffey at oracle.com (=?UTF-8?B?U2XDoW4gQ29mZmV5?=) Date: Tue, 27 Jan 2015 11:58:55 +0000 Subject: [8u60] Request for approval: 8069590: AIX port of "8050807: Better performing performance data handling" In-Reply-To: References: <54C75DDB.6010409@oracle.com> Message-ID: <54C77D7F.7040707@oracle.com> Volker, I don't see any issues with this being pushed to the jdk8u-dev team forest. Consider it approved for there. Before any push, let's first see if Alejandro is ok with that. He should be online in a few hours. Once I see this in 8u60 team forest, I'll log a request to have this included in 8u40. regards, Sean. On 27/01/2015 10:20, Volker Simonis wrote: > My change is based on > http://hg.openjdk.java.net/jdk8u/jdk8u-dev/hotspot and I'd like to > push it there. > > Alejandro can not test this anyway as it is AIX-only. > > @Alejandro: can I push this directly to > http://hg.openjdk.java.net/jdk8u/jdk8u-dev/hotspot now? > > If we go trough http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot/ we > loose even more time and we'll never make it into 8u40. > > Thanks, > Volker > > > On Tue, Jan 27, 2015 at 10:43 AM, Vladimir Kozlov > wrote: >> Changes are fine and I think you can push it. Nothing in JPRT queue is >> pushing into 8u. >> Note, your changes are based on different repository from what we use to >> push Hotspot into 8u. We use: >> >> http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot/ >> >> Alejandro will promoted changes into /jdk8u/jdk8u-dev/hotspot after testing. >> >> But may be you have agreement with Alejandro where to push. Ignore me then. >> >> Thanks, >> Vladimir >> >> >> On 1/27/15 12:34 AM, Volker Simonis wrote: >>> Can I push this to 8u-dev now? >>> >>> Thanks, >>> Volker >>> >>> On Mon, Jan 26, 2015 at 5:12 PM, Volker Simonis >>> wrote: >>>> Hi, >>>> >>>> can you please approve this security fix which is the AIX-port port of >>>> "8050807: Better performing performance data handling" from 8u31. For >>>> some reason this AIX-version didn't made it right right in time into >>>> 8u31 so we have to manually port it now to jdk8u-dev. >>>> >>>> This fix is AIX-only, it is reviewed and it cleanly applies to both, >>>> jdk8u-dev and jdk8u40 >>>> >>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8069590_jdk8u/ >>>> https://bugs.openjdk.java.net/browse/JDK-8069590 >>>> >>>> This fix should also be pushed to 8u40 if that is possible at all. >>>> >>>> Thank you and best regards, >>>> Volker From sean.coffey at oracle.com Tue Jan 27 14:57:52 2015 From: sean.coffey at oracle.com (=?UTF-8?B?U2XDoW4gQ29mZmV5?=) Date: Tue, 27 Jan 2015 14:57:52 +0000 Subject: Useful message about NullPointerException In-Reply-To: References: <1421844308100-213240.post@n7.nabble.com> <54C78F44.1030104@redhat.com> Message-ID: <54C7A770.8080707@oracle.com> Adding hotspot-dev to this mail thread also as it's relevant to hotspot. (complete thread at http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-January/031015.html) As one who often has to dig through application logs and JDK issues, I think this would certainly be a useful addition to the NPE handling process. If the VM has extra info on the exact object that caused the NPE, it should be output in that exception. "a.getB().getC()" is a good example of the different code paths one has to go down to determine where the NPE could have arisen from. regards, Sean. On 27/01/2015 14:34, kedar mhaswade wrote: > When the JVM executes instructions like getfield > , > getstatic, invokevirtual etc. with *objref* on the operand stack and if > *objref* is null, an NPE is thrown. It appears that the JVM could tell us > more about which *objref* was null at run-time. Candidate for an RFE? > > That aside, (and Chris's trick is nice), but if you have no access to the > source for the offending code, life is hard in general, isn't it? Because > if you can't have control over the source, making that source run on a > platform where such an RFE would be perhaps fixed (a future release of the > JDK) would be even harder, no? > > On Tue, Jan 27, 2015 at 5:14 AM, Florian Weimer wrote: > >> On 01/21/2015 01:45 PM, pike wrote: >>> We frequently see NullPointerException in our logs. It's really a big >>> headache when we see a NullPointerException and it is encapsulated in >>> another exception as we don't know which object is null and it is >> throwing >>> an Exception. Is there any way we can get to know the object type or the >>> object variable name where the object is null and it is throwing a >>> NullPointerException? >> The line number gives you the position in the source code, and from >> that, you can usually figure out the static type. If this is not >> helpful in your case, you need to say why (no debugging information? >> multiple candidates per line?). >> >> The dynamic type is a different matter though, because null has no >> specific type at run time. It may be possible to provide type >> information in theory, at a cost, but this would best be prototyped >> through byte code rewriting. Nullable annotations would also help to >> pin-point location of the first leak, and you could record that >> (including a stack trace) if you want something really fancy. Whether >> it is helpful for legacy code, I don't know. There should be some >> research projects out there covering this area. >> >> -- >> Florian Weimer / Red Hat Product Security >> From fweimer at redhat.com Tue Jan 27 15:04:22 2015 From: fweimer at redhat.com (Florian Weimer) Date: Tue, 27 Jan 2015 16:04:22 +0100 Subject: Useful message about NullPointerException In-Reply-To: <54C7A770.8080707@oracle.com> References: <1421844308100-213240.post@n7.nabble.com> <54C78F44.1030104@redhat.com> <54C7A770.8080707@oracle.com> Message-ID: <54C7A8F6.5070606@redhat.com> On 01/27/2015 03:57 PM, Se?n Coffey wrote: > Adding hotspot-dev to this mail thread also as it's relevant to hotspot. > (complete thread at > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-January/031015.html) > > > As one who often has to dig through application logs and JDK issues, I > think this would certainly be a useful addition to the NPE handling > process. If the VM has extra info on the exact object that caused the > NPE, it should be output in that exception. "a.getB().getC()" is a good > example of the different code paths one has to go down to determine > where the NPE could have arisen from. Maybe it's possible to add a byte code office to the exception it self, or the stack trace? Especially the former wouldn't need much additional allocation. -- Florian Weimer / Red Hat Product Security From volker.simonis at gmail.com Tue Jan 27 17:29:28 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 27 Jan 2015 18:29:28 +0100 Subject: Request or comment: 8071690: Include local HotSpot headers before the system headers Message-ID: Hi, I've just opened "8071690: Include local HotSpot headers before the system headers" but before I start working on it I'd like to hear what others think about the problem. If there's a consensus that it will be worth while doing this change I'll be happy to start working on it. https://bugs.openjdk.java.net/browse/JDK-8071690 The following description is copied from the bug report for your convenience: There's no general rule in which order header files should be included but I think a good hint is to include local, project specific headers before system headers. This is mainly for two reasons: 1. It prevents that dependencies from local header files to system headers are hidden because a local header is included after a system header by chance. Instead, every local header should explicitly specify the system headers it depends on. 2. It enables the definition of local macros which control the behaviour of the system headers which are included later on. Point two may be considered bad style, but we actually use it for example in src/share/vm/utilities/globalDefinitions.hpp where we define "__STDC_FORMAT_MACROS" before we include "" in the compiler specific global definitions file. "__STDC_FORMAT_MACROS" controls the definition of the printf format macros in "" but this can only work if "" is really included AFTER the definition of "__STDC_FORMAT_MACROS". And this can only wok if every file includes the local HotSpot headers before any system headers, because otherwise "" may be indirectly included by a system header before we had a chance to define "__STDC_FORMAT_MACROS". This is exactly what happened after the integration of 8050807 which added the system include "" to vmError.cpp as follows: #include #include "precompiled.hpp" #include "code/codeCache.hpp" This change broke the build on AIX because "" indirectly included "" BEFORE we defined "__STDC_FORMAT_MACROS". To prevent such errors in the future I propose to change all HotSpot source files to always include the system headers AFTER the inclusion of the project specific HotSpot headers. I?ve attached a small Pythin script to this bug report which can be used as follows to detect the files which are currently violating this rule: find src/ \( -name "*.cpp" -o -name "*.hpp" \) -type f -exec python include.py {} \; src/os/solaris/dtrace/generateJvmOffsets.cpp: system header #include included before local header #include "code/codeBlob.hpp" src/os/windows/vm/decoder_windows.hpp: system header #include included before local header #include "utilities/decoder.hpp" src/os_cpu/bsd_zero/vm/os_bsd_zero.cpp: system header # include included before local header #include "assembler_zero.inline.hpp" src/share/vm/adlc/adlc.hpp: system header #include included before local header #include "string.h" src/share/vm/libadt/port.hpp: system header #include included before local header #include "port_tandem.hpp" src/share/vm/runtime/os.hpp: system header # include included before local header # include "jvm_solaris.h" src/share/vm/trace/traceDataTypes.hpp: system header #include included before local header #include "utilities/globalDefinitions.hpp" src/share/vm/utilities/dtrace.hpp: system header #include included before local header #include "dtracefiles/hotspot.h" src/share/vm/utilities/elfFile.cpp: system header #include included before local header #include "memory/allocation.inline.hpp" src/share/vm/utilities/elfFile.hpp: system header #include included before local header #include "globalDefinitions.hpp" src/share/vm/utilities/vmError.cpp: system header #include included before local header #include "precompiled.hpp" The script is written in Python intentionally such that it can be easily added as an automated hook to jcheck to prevent violations of this inclusion rule in the future. Of course we can also try to not rely on the inclusion order at all. IMHO it actually seems that this is the cleanest solution, but it may require moving some macro definitions from header files right into the command line (e.g. -D__STDC_FORMAT_MACROS for the example above). By the way, that's actually the way how I've fixed the above mentioned compile error on AIX without the need to touch any shared files. What do you think? Regards, Volker From alejandro.murillo at oracle.com Tue Jan 27 17:42:11 2015 From: alejandro.murillo at oracle.com (Alejandro E Murillo) Date: Tue, 27 Jan 2015 10:42:11 -0700 Subject: [8u60] Request for approval: 8069590: AIX port of "8050807: Better performing performance data handling" In-Reply-To: <54C77D7F.7040707@oracle.com> References: <54C75DDB.6010409@oracle.com> <54C77D7F.7040707@oracle.com> Message-ID: <54C7CDF3.1050106@oracle.com> yes, that's totally fine with me in fact, you should probably push that kind of changes to that repo going forward, and we will avoid possible conflicts with jprt. We might do the same with jdk9 changes Thanks Alejandro On 1/27/2015 4:58 AM, Se?n Coffey wrote: > Volker, > > I don't see any issues with this being pushed to the jdk8u-dev team > forest. Consider it approved for there. Before any push, let's first > see if Alejandro is ok with that. He should be online in a few hours. > > Once I see this in 8u60 team forest, I'll log a request to have this > included in 8u40. > > regards, > Sean. > > On 27/01/2015 10:20, Volker Simonis wrote: >> My change is based on >> http://hg.openjdk.java.net/jdk8u/jdk8u-dev/hotspot and I'd like to >> push it there. >> >> Alejandro can not test this anyway as it is AIX-only. >> >> @Alejandro: can I push this directly to >> http://hg.openjdk.java.net/jdk8u/jdk8u-dev/hotspot now? >> >> If we go trough http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot/ we >> loose even more time and we'll never make it into 8u40. >> >> Thanks, >> Volker >> >> >> On Tue, Jan 27, 2015 at 10:43 AM, Vladimir Kozlov >> wrote: >>> Changes are fine and I think you can push it. Nothing in JPRT queue is >>> pushing into 8u. >>> Note, your changes are based on different repository from what we >>> use to >>> push Hotspot into 8u. We use: >>> >>> http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot/ >>> >>> Alejandro will promoted changes into /jdk8u/jdk8u-dev/hotspot after >>> testing. >>> >>> But may be you have agreement with Alejandro where to push. Ignore >>> me then. >>> >>> Thanks, >>> Vladimir >>> >>> >>> On 1/27/15 12:34 AM, Volker Simonis wrote: >>>> Can I push this to 8u-dev now? >>>> >>>> Thanks, >>>> Volker >>>> >>>> On Mon, Jan 26, 2015 at 5:12 PM, Volker Simonis >>>> wrote: >>>>> Hi, >>>>> >>>>> can you please approve this security fix which is the AIX-port >>>>> port of >>>>> "8050807: Better performing performance data handling" from 8u31. For >>>>> some reason this AIX-version didn't made it right right in time into >>>>> 8u31 so we have to manually port it now to jdk8u-dev. >>>>> >>>>> This fix is AIX-only, it is reviewed and it cleanly applies to both, >>>>> jdk8u-dev and jdk8u40 >>>>> >>>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8069590_jdk8u/ >>>>> https://bugs.openjdk.java.net/browse/JDK-8069590 >>>>> >>>>> This fix should also be pushed to 8u40 if that is possible at all. >>>>> >>>>> Thank you and best regards, >>>>> Volker > -- Alejandro From volker.simonis at gmail.com Tue Jan 27 17:46:13 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 27 Jan 2015 18:46:13 +0100 Subject: [8u60] Request for approval: 8069590: AIX port of "8050807: Better performing performance data handling" In-Reply-To: <54C7CDF3.1050106@oracle.com> References: <54C75DDB.6010409@oracle.com> <54C77D7F.7040707@oracle.com> <54C7CDF3.1050106@oracle.com> Message-ID: Thanks Alejandro! Just pushed... Regards, Volker On Tue, Jan 27, 2015 at 6:42 PM, Alejandro E Murillo wrote: > > yes, that's totally fine with me > in fact, you should probably push that kind of changes > to that repo going forward, and we will avoid possible > conflicts with jprt. We might do the same with jdk9 changes > > Thanks > Alejandro > > > On 1/27/2015 4:58 AM, Se?n Coffey wrote: >> >> Volker, >> >> I don't see any issues with this being pushed to the jdk8u-dev team >> forest. Consider it approved for there. Before any push, let's first see if >> Alejandro is ok with that. He should be online in a few hours. >> >> Once I see this in 8u60 team forest, I'll log a request to have this >> included in 8u40. >> >> regards, >> Sean. >> >> On 27/01/2015 10:20, Volker Simonis wrote: >>> >>> My change is based on >>> http://hg.openjdk.java.net/jdk8u/jdk8u-dev/hotspot and I'd like to >>> push it there. >>> >>> Alejandro can not test this anyway as it is AIX-only. >>> >>> @Alejandro: can I push this directly to >>> http://hg.openjdk.java.net/jdk8u/jdk8u-dev/hotspot now? >>> >>> If we go trough http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot/ we >>> loose even more time and we'll never make it into 8u40. >>> >>> Thanks, >>> Volker >>> >>> >>> On Tue, Jan 27, 2015 at 10:43 AM, Vladimir Kozlov >>> wrote: >>>> >>>> Changes are fine and I think you can push it. Nothing in JPRT queue is >>>> pushing into 8u. >>>> Note, your changes are based on different repository from what we use to >>>> push Hotspot into 8u. We use: >>>> >>>> http://hg.openjdk.java.net/jdk8u/hs-dev/hotspot/ >>>> >>>> Alejandro will promoted changes into /jdk8u/jdk8u-dev/hotspot after >>>> testing. >>>> >>>> But may be you have agreement with Alejandro where to push. Ignore me >>>> then. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> >>>> On 1/27/15 12:34 AM, Volker Simonis wrote: >>>>> >>>>> Can I push this to 8u-dev now? >>>>> >>>>> Thanks, >>>>> Volker >>>>> >>>>> On Mon, Jan 26, 2015 at 5:12 PM, Volker Simonis >>>>> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> can you please approve this security fix which is the AIX-port port of >>>>>> "8050807: Better performing performance data handling" from 8u31. For >>>>>> some reason this AIX-version didn't made it right right in time into >>>>>> 8u31 so we have to manually port it now to jdk8u-dev. >>>>>> >>>>>> This fix is AIX-only, it is reviewed and it cleanly applies to both, >>>>>> jdk8u-dev and jdk8u40 >>>>>> >>>>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/8069590_jdk8u/ >>>>>> https://bugs.openjdk.java.net/browse/JDK-8069590 >>>>>> >>>>>> This fix should also be pushed to 8u40 if that is possible at all. >>>>>> >>>>>> Thank you and best regards, >>>>>> Volker >> >> > > -- > Alejandro > From aph at redhat.com Tue Jan 27 17:48:42 2015 From: aph at redhat.com (Andrew Haley) Date: Tue, 27 Jan 2015 17:48:42 +0000 Subject: RFR: 8071688: AARCH64: SIGSEGV in MethodData::next_data(ProfileData*) Message-ID: <54C7CF7A.7000502@redhat.com> This patch is already applied to the other HotSpot targets. It's 8039975. Webrev at http://cr.openjdk.java.net/~aph/aarch64-8071688/ Andrew. From dean.long at oracle.com Tue Jan 27 19:14:43 2015 From: dean.long at oracle.com (Dean Long) Date: Tue, 27 Jan 2015 11:14:43 -0800 Subject: RFR(XS) 8069030: support new PTRACE_GETREGSET In-Reply-To: References: <54C70F7D.3010003@oracle.com> Message-ID: <54C7E3A3.1090605@oracle.com> Thanks Staffan! Do I need another review for this small change? dl On 1/26/2015 11:02 PM, Staffan Larsen wrote: > Looks good! > > Thanks, > /Staffan > >> On 27 jan 2015, at 05:09, Dean Long wrote: >> >> https://bugs.openjdk.java.net/browse/JDK-8069030 >> >> http://cr.openjdk.java.net/~dlong/8069030/webrev.00/ >> >> Testing on linux x86 by temporarily undefining PTRACE_GETREGS_REQ so that the new code >> would get triggered, then stepping through "jstack -m " with gdb to make sure the new >> code executed correctly. >> >> dl >> From roland.westrelin at oracle.com Tue Jan 27 19:19:45 2015 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Tue, 27 Jan 2015 20:19:45 +0100 Subject: RFR: 8071688: AARCH64: SIGSEGV in MethodData::next_data(ProfileData*) In-Reply-To: <54C7CF7A.7000502@redhat.com> References: <54C7CF7A.7000502@redhat.com> Message-ID: <209E6B45-2D72-4AC5-91A4-10DD08159FAF@oracle.com> > This patch is already applied to the other HotSpot targets. It's 8039975. > > Webrev at http://cr.openjdk.java.net/~aph/aarch64-8071688/ It looks good to me. I will push it. Roland. From vladimir.kozlov at oracle.com Tue Jan 27 19:24:23 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 27 Jan 2015 11:24:23 -0800 Subject: RFR: 8071688: AARCH64: SIGSEGV in MethodData::next_data(ProfileData*) In-Reply-To: <209E6B45-2D72-4AC5-91A4-10DD08159FAF@oracle.com> References: <54C7CF7A.7000502@redhat.com> <209E6B45-2D72-4AC5-91A4-10DD08159FAF@oracle.com> Message-ID: <54C7E5E7.8020300@oracle.com> On 1/27/15 11:19 AM, Roland Westrelin wrote: >> This patch is already applied to the other HotSpot targets. It's 8039975. >> >> Webrev at http://cr.openjdk.java.net/~aph/aarch64-8071688/ > > It looks good to me. I will push it. Yes, it is good. Roland, thank you for sponsoring. Vladimir > > Roland. > From serguei.spitsyn at oracle.com Tue Jan 27 23:57:02 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 27 Jan 2015 15:57:02 -0800 Subject: 5-rd round RFR (S) 8008678: JSR 292: constant pool reconstitution must support pseudo strings In-Reply-To: <54B99838.10601@oracle.com> References: <5475968C.40800@oracle.com> <54760B40.9040206@oracle.com> <54762FC9.2010800@oracle.com> <5491BCB9.10505@oracle.com> <5493299F.60005@oracle.com> <54934E73.4080808@oracle.com> <54B96F97.5070204@oracle.com> <54B99682.8070802@oracle.com> <54B99838.10601@oracle.com> Message-ID: <54C825CE.1030105@oracle.com> This is 5-th round review of for: https://bugs.openjdk.java.net/browse/JDK-8008678 There was one (4-rd) private review discussion. The result is this webrev: http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8008678-JVMTI-pseudo.5/ Summary: Current fix is to use the 2-nd LSB bit in the CPSlot to mark pseudo-strings. A similar version of webrev was already reviewed by Coleen and John. The update is that it includes John's suggestion to introduce the CPSlot::TagBits to make code more clean. Also, I've discovered that there is no need to update the SymbolClosure::load_symbol() in iterator.hpp because the pseudo-string PCSlot is not used in the SymbolClosure. I need a final thumbs up for the push. Testing: Run: - nsk.jvmti.testlist, nsk.jdi.testlist, vm.mlvm.testlist - java/lang/instrument, com/sun/jdi and hotspot/test/runtime tests - new jtreg test (see webrev) that was written by Filipp Zhinkin Thanks, Serguei On 1/16/15 3:01 PM, serguei.spitsyn at oracle.com wrote: > John R. suggested to use the CPSlot(Symbol* ptr) to mark pseudo-strings. > The updated webrev is going to be close to the .3 webrev. > I will send it soon. > > Thanks, > Serguei > > On 1/16/15 2:53 PM, Coleen Phillimore wrote: >> >> This change looks good to me also. >> Thanks, >> Coleen >> >> On 1/16/15, 3:07 PM, serguei.spitsyn at oracle.com wrote: >>> Please, review the fix for: >>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>> >>> >>> Open webrev: >>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8008678-JVMTI-pseudo.3/ >>> >>> >>> Summary: >>> Currently, a JVM_CONSTANT_String CP entry having a NULL reference >>> to Symbol* >>> indicates that it is a pseudo-string (patched string). >>> This creates nasty issues for the constant pool reconstitution. >>> >>> Current suggestion is to avoid having a NULL reference and retain >>> the original >>> Symbol* reference for pseudo-strings. The pseudo-string >>> indication will be >>> if the Utf8 representation starts from "CONSTANT_PLACEHOLDER_". >>> This approach makes the fix much simpler. >>> >>> I need a confirmation from the Compiler team that this won't >>> break any assumptions or invariants. >>> Big thanks to Coleen for previous round reviews and good advices! >>> >>> >>> Testing: >>> Run: >>> - java/lang/instrument tests >>> - new jtreg test (see webrev) that was written by Filipp Zhinkin >>> >>> >>> Thanks, >>> Serguei >>> >>> >>> On 12/18/14 2:00 PM, serguei.spitsyn at oracle.com wrote: >>>> Hi Coleen, >>>> >>>> Thank you for reviewing! >>>> >>>> >>>> On 12/18/14 11:23 AM, Coleen Phillimore wrote: >>>>> >>>>> Hi Serguei, >>>>> >>>>> Thank you for making the patches an optional field. >>>>> >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/oops/constantPool.hpp.sdiff.html >>>>> 198 if (!patched()) { >>>>> 199 assert(false, "a pseudo-string map may exists for patched CP only"); >>>>> 200 return 0; >>>>> 201 } >>>>> Why not >>>>> assert(patched(), "a pseud-string map must exist >>>>> for patched CP only"); >>>> >>>> Wanted it to be more reliable but it looks pointless. >>>> Will make this change. >>>> >>>>> >>>>> >>>>> Why is this? Is this really a ShouldNotReachHere? should it be >>>>> false? >>>>> >>>>> 215 assert(true, "not found a matching entry in pseudo-string map"); >>>> >>>> >>>> A typo, must be false. >>>> It is the last minute change. >>>> Thanks for the catch! >>>> >>>> >>>>> >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/prims/jvmtiRedefineClasses.cpp.udiff.html >>>>> >>>>> Don't you have to move the value of the patched field from the old >>>>> constant pool to the new one? I hate to ask but is there merging >>>>> that needs to be done? I don't know how to write this test case >>>>> though. Is it possible to redefine a class with a constant pool >>>>> patches with another that has constant pool patches? >>>> >>>> Thank you for asking this question. >>>> If I understand correctly, the patching comes from the compiler >>>> side for anonymous classes only. >>>> I saw it for LambdaForm's only. >>>> I think, the patching can not be changed with a retransformation. >>>> But I'm not sure if it can not be changed with a redefinition. >>>> >>>> But if it can - then it would be safe to merge the 'patched' >>>> condition, i.e. make it patched >>>> if either the_class or scratch class is patched. >>>> >>>>> >>>>> Somehow I thought you'd have to save the value of the cp_patches >>>>> oops passed in. >>>>> >>>>> So I was wondering why you can't change this instead: >>>>> >>>>> bool is_pseudo_string_at(int which) { >>>>> // A pseudo string is a string that doesn't have a symbol in >>>>> the cpSlot >>>>> - return unresolved_string_at(which) == NULL; >>>>> + return (strncmp(unresolved_string_at(which)->as_utf8(), >>>>> "CONSTANT_PLACEHOLDER_" , strlen("CONSTANT_PLACEHOLDER")) == 0); >>>>> } >>>> >>>> I was thinking about the same but was not sure if it would work for >>>> the compiler team. >>>> We have to ask John about this (added John and Christian to the >>>> cc-list). >>>> This question to John was in my plan! :) >>>> >>>> The beauty of the above approach is that there is no need to create >>>> an intermediate >>>> pseudo-string map and most of the code in from this webrev is not >>>> needed. >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>>> >>>>> And the asserts in the other functions below it. >>>>> >>>>> Coleen >>>>> >>>>> >>>>> On 12/17/14, 12:26 PM, serguei.spitsyn at oracle.com wrote: >>>>>> Please, review the second round fix for: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>>> >>>>>> Open webrev: >>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/ >>>>>> >>>>>> >>>>>> Summary: >>>>>> >>>>>> This fix implements a footprint saving approach suggested by >>>>>> Coleen. >>>>>> To be able to reconstitute a class constant pool, an >>>>>> intermediate pseudo-string map is used. >>>>>> Now, this field is accounted optionally, only if the >>>>>> 'cp_patches' is provided in the >>>>>> ClassFileParser::parseClassFile() before ConstantPool is allocated. >>>>>> This fix is not elegant, even a little bit ugly, but it is the >>>>>> only way I see so far. >>>>>> >>>>>> Unfortunately, this approach did not help much to make some >>>>>> other fields (eg., 'operands') optional. >>>>>> The problem is that we have to account optional fields before >>>>>> parsing, at the CP allocation time. >>>>>> It is possible to re-allocate the ConstantPool when any >>>>>> InvokeDynamic bytecode is discovered, >>>>>> but it looks too complicated. >>>>>> >>>>>> Testing: >>>>>> - the unit test from bug report >>>>>> - nsk.jvmti,testlist, nsk.jdi.testlist, JTREG java/lang/instrument >>>>>> - vm.mlvm.testlist, vm.quick.testlist, >>>>>> vm.parallel_class_loading.testlist (in progress) >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 11/26/14 11:53 AM, serguei.spitsyn at oracle.com wrote: >>>>>>> Coleen, >>>>>>> >>>>>>> Thank you for looking at this! >>>>>>> I'll check how this can be improved. >>>>>>> It is my concern too. >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> On 11/26/14 9:17 AM, Coleen Phillimore wrote: >>>>>>>> >>>>>>>> Serguei, >>>>>>>> I had a quick look at this. I was wondering if we could make >>>>>>>> the pseudo_string_map conditional in ConstantPool and not make >>>>>>>> all classes pay in footprint for this field? The same thing >>>>>>>> probably could be done for operands too. There are flags that >>>>>>>> you can set to conditionally add a pointer to base() in this >>>>>>>> function. >>>>>>>> >>>>>>>> Typical C++ would subclass ConstantPool to add >>>>>>>> InvokeDynamicConstantPool fields, but this is not typical C++ >>>>>>>> so the trick we use is like the one in ConstMethod. I think >>>>>>>> it's worth doing in this case. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>>>>> >>>>>>>> On 11/26/14, 3:59 AM, serguei.spitsyn at oracle.com wrote: >>>>>>>>> Please, review the fix for: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>>>>>> >>>>>>>>> >>>>>>>>> Open webrev: >>>>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.1/ >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Summary: >>>>>>>>> The pseudo-strings are currently not supported in >>>>>>>>> reconstitution of constant pool. >>>>>>>>> >>>>>>>>> This is an explanation from John Rose about what the >>>>>>>>> pseudo-strings are: >>>>>>>>> >>>>>>>>> "We still need "live" oop constants pre-linked into the >>>>>>>>> constant pool of bytecodes which >>>>>>>>> implement some method handles. We use the anonymous class >>>>>>>>> pseudo-string feature for that. >>>>>>>>> The relevant code is here: >>>>>>>>> http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/tip/src/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java >>>>>>>>> >>>>>>>>> These oops are what "pseudo-strings" are. >>>>>>>>> The odd name refers to the fact that, even though they are >>>>>>>>> random oops, they appear in the constant pool >>>>>>>>> where one would expect (because of class file syntax) to >>>>>>>>> find a string." >>>>>>>>> ... >>>>>>>>> If you really wanted to reconstitute a class file for an >>>>>>>>> anonymous class, and >>>>>>>>> if that class has oop patching (pseudo-strings), you would >>>>>>>>> need either to (a) reconstitute the patches array >>>>>>>>> handed to Unsafe.defineAnonymousClass, or (b) accept >>>>>>>>> whatever odd strings were there first, as an approximation. >>>>>>>>> The "odd strings" are totally insignificant, and are >>>>>>>>> typically something like "CONSTANT_PLACEHOLDER_42" >>>>>>>>> (see java/lang/invoke/InvokerBytecodeGenerator.java)." >>>>>>>>> >>>>>>>>> >>>>>>>>> Reconstitution of the ConstantPool is needed for both the >>>>>>>>> JVMTI GetConstantPool() and RetransformClasses(). >>>>>>>>> Finally, it goes to the ConstantPool::copy_cpool_bytes(). >>>>>>>>> >>>>>>>>> The problem is that a pseudo-string is a patched string >>>>>>>>> that does not have >>>>>>>>> a reference to the string symbol anymore: >>>>>>>>> unresolved_string_at(idx) == NULL >>>>>>>>> >>>>>>>>> The fix is to create and fill in a map from >>>>>>>>> JVM_CONSTANT_String cp index to the JVM_CONSTANT_Utf8 cp index >>>>>>>>> to be able to restore this assotiation in the >>>>>>>>> JvmtiConstantPoolReconstituter. >>>>>>>>> >>>>>>>>> Testing: >>>>>>>>> Run: >>>>>>>>> - java/lang/instrument tests >>>>>>>>> - new jtreg test (see webrev) that was written by Filipp >>>>>>>>> Zhinkin >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Serguei >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From david.holmes at oracle.com Wed Jan 28 02:30:09 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 28 Jan 2015 12:30:09 +1000 Subject: Useful message about NullPointerException In-Reply-To: <54C7B653.1070509@gmail.com> References: <1421844308100-213240.post@n7.nabble.com> <54C78F44.1030104@redhat.com> <54C7B653.1070509@gmail.com> Message-ID: <54C849B1.80403@oracle.com> Adding back in hotspot-dev to this reply There have been previous RFEs for this dating back to 2006 - most closed as duplicates and the main one eventually closed as "will not fix" simply due to it being a low priority RFE for 8 years. Also see: https://bugs.openjdk.java.net/browse/JDK-6717558 On 28/01/2015 2:01 AM, Peter Levart wrote: > On 01/27/2015 03:34 PM, kedar mhaswade wrote: >> When the JVM executes instructions like getfield >> , >> >> getstatic, invokevirtual etc. with *objref* on the operand stack and if >> *objref* is null, an NPE is thrown. It appears that the JVM could tell us >> more about which *objref* was null at run-time. Candidate for an RFE? > > In general it is hard to deduce the meaningfull source of null *objref* > on the operand stack by analyzing the surrounding bytecodes. It could be > a result of complex logic executed by bytecodes. Imagine the following: > > int length(boolean first, String s1, String s2) { > return (first ? s1 : s2).length(); > } > > ...the analysis would have to trace the live execution so that it could > be rolled-back to the meaningful source of null *objref*. Exactly right - all the VM knows at that exact point is that the thing on the stack is an object reference - if its value is null that doesn't tell you where that came from. You have to look around to see what pushed that objRef onto the stack. Might be doable from the interpreter, but from compiled code? > All VM might semi-realistically do is report the action VM was trying to > perform when it dereferenced a null *objref*. Like > "NullPointerException: while invoking method Xxxx.yyyy on a null > target", or "NullPointerException: while de-referencing instance field > Xxxx.yyyy of a null reference" > > But what does this help if you don't have access to sources? Might be a > hint, but not much. As I said there may be some context to assist with how the null was introduced, but in general it isn't something that is readily apparent - especially in compiled code. > If you have access to sources, then perhaps an easier solution would be > for stack traces to include column number in addition to line number of > location in source that resulted in bytecodes that include the one that > triggered the NPE. > > There is already a RFE for that: > > https://bugs.openjdk.java.net/browse/JDK-8020204 Once past suggestion has been to include the ByteCode Index (BCI) as part of the exception stacktrace information: https://bugs.openjdk.java.net/browse/JDK-4185378 David ----- > It seems that javac part is already there. The VM part and public API > part (StackTraceElement) is not, though. > > Regards, Peter > >> >> That aside, (and Chris's trick is nice), but if you have no access to the >> source for the offending code, life is hard in general, isn't it? Because >> if you can't have control over the source, making that source run on a >> platform where such an RFE would be perhaps fixed (a future release of >> the >> JDK) would be even harder, no? >> >> On Tue, Jan 27, 2015 at 5:14 AM, Florian Weimer >> wrote: >> >>> On 01/21/2015 01:45 PM, pike wrote: >>>> We frequently see NullPointerException in our logs. It's really a big >>>> headache when we see a NullPointerException and it is encapsulated in >>>> another exception as we don't know which object is null and it is >>> throwing >>>> an Exception. Is there any way we can get to know the object type or >>>> the >>>> object variable name where the object is null and it is throwing a >>>> NullPointerException? >>> The line number gives you the position in the source code, and from >>> that, you can usually figure out the static type. If this is not >>> helpful in your case, you need to say why (no debugging information? >>> multiple candidates per line?). >>> >>> The dynamic type is a different matter though, because null has no >>> specific type at run time. It may be possible to provide type >>> information in theory, at a cost, but this would best be prototyped >>> through byte code rewriting. Nullable annotations would also help to >>> pin-point location of the first leak, and you could record that >>> (including a stack trace) if you want something really fancy. Whether >>> it is helpful for legacy code, I don't know. There should be some >>> research projects out there covering this area. >>> >>> -- >>> Florian Weimer / Red Hat Product Security >>> > From david.holmes at oracle.com Wed Jan 28 04:21:43 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 28 Jan 2015 14:21:43 +1000 Subject: Request or comment: 8071690: Include local HotSpot headers before the system headers In-Reply-To: References: Message-ID: <54C863D7.7000808@oracle.com> Hi Volker, On 28/01/2015 3:29 AM, Volker Simonis wrote: > Hi, > > I've just opened "8071690: Include local HotSpot headers before the > system headers" but before I start working on it I'd like to hear what > others think about the problem. If there's a consensus that it will be > worth while doing this change I'll be happy to start working on it. As I wrote in the bug report: I don't see how you can apply this as a general rule - or at least not without some further rules. If local foo.h depends on a system header then it will #include it, so a file that #includes foo.h can't control the order of that system header include. We need to recognize (and detect!) where we have implicit ordering constraints but I don't think a general rule actually helps with that. And there may be cases where we rely on a system include coming first eg: #ifndef SOME_SYSTEM_THING #define SOME_SYSTEM_THING xxx #endif David ----- > https://bugs.openjdk.java.net/browse/JDK-8071690 > > The following description is copied from the bug report for your convenience: > > There's no general rule in which order header files should be included > but I think a good hint is to include local, project specific headers > before system headers. This is mainly for two reasons: > > 1. It prevents that dependencies from local header files to system > headers are hidden because a local header is included after a system > header by chance. Instead, every local header should explicitly > specify the system headers it depends on. > > 2. It enables the definition of local macros which control the > behaviour of the system headers which are included later on. > > Point two may be considered bad style, but we actually use it for > example in src/share/vm/utilities/globalDefinitions.hpp where we > define "__STDC_FORMAT_MACROS" before we include "" in the > compiler specific global definitions file. > > "__STDC_FORMAT_MACROS" controls the definition of the printf format > macros in "" but this can only work if "" is > really included AFTER the definition of "__STDC_FORMAT_MACROS". And > this can only wok if every file includes the local HotSpot headers > before any system headers, because otherwise "" may be > indirectly included by a system header before we had a chance to > define "__STDC_FORMAT_MACROS". > > This is exactly what happened after the integration of 8050807 which > added the system include "" to vmError.cpp as follows: > > #include > #include "precompiled.hpp" > #include "code/codeCache.hpp" > > This change broke the build on AIX because "" indirectly > included "" BEFORE we defined "__STDC_FORMAT_MACROS". > > To prevent such errors in the future I propose to change all HotSpot > source files to always include the system headers AFTER the inclusion > of the project specific HotSpot headers. > > I?ve attached a small Pythin script to this bug report which can be > used as follows to detect the files which are currently violating this > rule: > > find src/ \( -name "*.cpp" -o -name "*.hpp" \) -type f -exec python > include.py {} \; > > src/os/solaris/dtrace/generateJvmOffsets.cpp: system header #include > included before local header #include > "code/codeBlob.hpp" > src/os/windows/vm/decoder_windows.hpp: system header #include > included before local header #include > "utilities/decoder.hpp" > src/os_cpu/bsd_zero/vm/os_bsd_zero.cpp: system header # include > included before local header #include > "assembler_zero.inline.hpp" > src/share/vm/adlc/adlc.hpp: system header #include included > before local header #include "string.h" > src/share/vm/libadt/port.hpp: system header #include > included before local header #include "port_tandem.hpp" > src/share/vm/runtime/os.hpp: system header # include > included before local header # include "jvm_solaris.h" > src/share/vm/trace/traceDataTypes.hpp: system header #include > included before local header #include > "utilities/globalDefinitions.hpp" > src/share/vm/utilities/dtrace.hpp: system header #include > included before local header #include > "dtracefiles/hotspot.h" > src/share/vm/utilities/elfFile.cpp: system header #include > included before local header #include "memory/allocation.inline.hpp" > src/share/vm/utilities/elfFile.hpp: system header #include > included before local header #include "globalDefinitions.hpp" > src/share/vm/utilities/vmError.cpp: system header #include > included before local header #include "precompiled.hpp" > > The script is written in Python intentionally such that it can be > easily added as an automated hook to jcheck to prevent violations of > this inclusion rule in the future. > > Of course we can also try to not rely on the inclusion order at all. > IMHO it actually seems that this is the cleanest solution, but it may > require moving some macro definitions from header files right into the > command line (e.g. -D__STDC_FORMAT_MACROS for the example above). By > the way, that's actually the way how I've fixed the above mentioned > compile error on AIX without the need to touch any shared files. > > What do you think? > > Regards, > Volker > From david.holmes at oracle.com Wed Jan 28 06:09:26 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 28 Jan 2015 16:09:26 +1000 Subject: RFR: 7143664: Clean up OrderAccess implementations and usage In-Reply-To: References: <54C1EBD3.1000905@oracle.com> <54C1F3D8.5040507@oracle.com> <54C2C1EA.5020507@oracle.com> Message-ID: <54C87D16.40601@oracle.com> Hi Erik, Sorry for the delay, it was a long weekend here and I'm playing catch up. :) On 24/01/2015 9:03 AM, Erik ?sterlund wrote: > Hi Dean, > > On 23/01/15 22:50, Dean Long wrote: >> On 1/23/2015 7:16 AM, Erik ?sterlund wrote: >>> Hi Dean and David, >>> >>> Thanks for reviewing this change. >>> >>> On 23/01/15 08:12, David Holmes wrote: >>>> On 23/01/2015 4:36 PM, Dean Long wrote: >>>>> 147 // Ordering a load relative to preceding stores requires a fence, >>>>> 148 // which implies a membar #StoreLoad between the store and load under >>>>> 149 // sparc-TSO. A fence is required by x86. On x86, we use explicitly >>>>> 150 // locked add. >>>>> 151 // >>>>> >>>>> It sounds like the above is saying that fence is the same as StoreLoad ... >>>> That may be a s/store_fence/fence/ typo. The original text was: >>>> >>>> // Ordering a load relative to preceding stores requires a store_fence, >>>> // which implies a membar #StoreLoad between the store and load under >>>> // sparc-TSO. A fence is required by ia64. On x86, we use locked xchg. >>>> >>>> Actually seems like a couple of typos there: ia64 became x86, then we >>>> have x86 again. >>> I tried to remove ia64 from the comments as it is not in the repository, >>> which in this case became a bit redundant. Are we okay if I remove the >>> "A fence is required by x86" sentence? >>> >>> Oh and Dean about membar #StoreLoad on SPARC and locked add on x86... >>> are you saying they are not equivalent? >> >> I wasn't trying to say anything abou SPARC or x86, only that "fence" != >> "StoreLoad". > > Agreed. > >>> New version of that paragraph with that sentence removed: >>> >>> 147 // Ordering a load relative to preceding stores requires a fence, >>> 148 // which implies a membar #StoreLoad between the store and load under >>> 149 // sparc-TSO. On x86, we use explicitly locked add. >> >> I would replace "fence" with "StoreLoad" above. > > Again, I just changed any store_fence descriptions to use fence instead. > But you are certainly right, it is more accurate to talk about StoreLoad > here and not mention fence in the first place, I'll change this, thanks > for the feedback. :) > > Changed to: > > 147 // Ordering a load relative to preceding stores requires a StoreLoad, > 148 // which implies a membar #StoreLoad between the store and load under > 149 // sparc-TSO. On x86, we use explicitly locked add. The confusion here is what "store-fence" was intended to mean. In this part we're implying it is a storeLoad, but in the discussion below it indicates store-fence orders subsequent loads and stores - which makes it a storeStore|storeLoad barrier. Of course storeLoad subsumes storeStore so the above is accurate - though I think the original text would have been more clearly stated as "Ordering a load relative to preceding stores _can be achieved using a_ store-fence, ..." So in summary: update above is okay. >>>>> 152 // 4. store, load <= is constrained by => store, fence, load >>>>> 153 // >>>>> 154 // Use store, fence to make sure all stores done in an 'interesting' >>>>> 155 // region are made visible prior to both subsequent loads and stores. >>>>> >>>>> ... and this is like saying to use fence because StoreStore | StoreLoad >>>>> isn't available. >>>> Again this seems an issue with store_fence being changed to "store, >>>> fence" which doesn't really make sense. >>> The thing is that I removed store_fence (not store_release as it says in >>> the bug ID, sorry about that - there obviously is no store_release). So >>> I wanted to remove references in the comments to store_fence as well. >>> Therefore I changed the example to use store fence load instead of >>> store_fence load. Perhaps it's better if I remove that sample all >>> together instead? >> >> That works for me. > > Fixed. Removal is an option but again replacing store-fence with what it really means "store; membar" would also suffice. But if we don't need the combined membar such that abstracting it into something like store_fence() makes sense, then getting rid of everything pertaining to store-fence makes sense. I see there are no uses of store_fence() other than a commented out reference in ./cpu/x86/vm/c1_LIRAssembler_x86.cpp, which also contains a similar commented out reference to the non-existent load_fence() - both of which should be removed. If you'd like to send me an updated patch I will generate a new webrev. Then I can get on with the functional part of the review. Thanks, David > Thanks, > /Erik > >> dl >> >>> Thanks for reviewing my changes! >>> >>> /Erik >>> >>>> David >>>> >>>>>> Furthermore, store_release was declared private and marked as >>>>> deprecated. >>>>> >>>>> I can't find where this was done. >>>>> >>>>> dl >>>>> >>>>> On 1/22/2015 5:19 PM, Erik ?sterlund wrote: >>>>>> Hi all, >>>>>> >>>>>> == Summary of Changes == >>>>>> >>>>>> This changeset fixes issues in OrderAccess on multiple levels from the >>>>>> memory model semantics to compiler reorderings, to addressing >>>>>> maintainability/portability issues which (almost) had to be fixed in >>>>>> order to fix the correctness issues. It is the result of discussions >>>>>> found in the previous "OrderAccess Refactoring" thread: >>>>>> http://openjdk.5641.n7.nabble.com/OrderAccess-Refactoring-td212050.html >>>>>> >>>>>> Bug ID: >>>>>> https://bugs.openjdk.java.net/browse/JDK-7143664 >>>>>> (updated to reflect these related changes) >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~dholmes/7143664/webrev/ >>>>>> >>>>>> Before I describe more I would like to give special thanks to David >>>>>> Holmes for long discussions leading up to the currently proposed >>>>>> changes. I would also like to thank Jesper Wilhelmsson for helping me >>>>>> run my changes through JPRT. >>>>>> >>>>>> == Motivation == >>>>>> >>>>>> This change directly fixes a reported OrderAccess bug due to compiler >>>>>> reorderings which is still a vulnerability on almost all TSO platforms: >>>>>> https://bugs.openjdk.java.net/browse/JDK-806196 >>>>>> >>>>>> And directly fixes confusions like release_store() != release() store() >>>>>> due to memory model issues previously described in this bug ID. >>>>>> >>>>>> At the same time it provides clearer design with separation of concerns >>>>>> and generalization/specialization, removing a whole bunch of platform >>>>>> specific code which could be generalized. The platform specific files >>>>>> now only have a few LoC requirements (~7) to conform to the memory model >>>>>> by specifying what the stand alone barriers do. Then optionally >>>>>> optimizations to the general approach are possible if platforms support >>>>>> it. This also makes it much easier to port to new platforms. >>>>>> >>>>>> == Memory Model == >>>>>> >>>>>> The current definitions of acquire/release semantics are a bit fishy >>>>>> leading to problems previously described in the bug ID (release_store() >>>>>> != release() store()) and some other correctness issues. It has >>>>>> therefore been replaced with a new model. I would like to thank David >>>>>> Holmes for the long discussions leading up to the newly proposed model. >>>>>> >>>>>> The new model is formally defined like this: >>>>>> >>>>>> // T1: access_shared_data >>>>>> // T1: ]release >>>>>> // T1: (...) >>>>>> // T1: store(X) >>>>>> // >>>>>> // T2: load(X) >>>>>> // T2: (...) >>>>>> // T2: acquire[ >>>>>> // T2: access_shared_data >>>>>> // >>>>>> // It is guaranteed that if T2: load(X) synchronizes with (observes the >>>>>> // value written by) T1: store(X), then the memory accesses before the >>>>>> // T1: ]release happen before the memory accesses after the T2: acquire[. >>>>>> >>>>>> The orderAccess.hpp file and bug ID also has a few additional >>>>>> explanations making it more intuitive to the user when to use >>>>>> acquire/release and the resemblance to TSO abstract machines. Big thanks >>>>>> goes to David Holmes for discussing the memory model with me, which >>>>>> helped a lot in deriving it. >>>>>> >>>>>> Now it holds that release() store() == release_store(), and the other >>>>>> correctness issues are fixed as well. >>>>>> >>>>>> The new model is also closer to C++11 definitions which could give us >>>>>> more relaxed compiler reordering constraints in the future when compiler >>>>>> support for C++11 is there and ready. >>>>>> >>>>>> == Reliance on C++ Volatile Semantics == >>>>>> >>>>>> The C++ standard section 1.9 "Program Execution" is very vague about >>>>>> what the keyword volatile can actually do for us. It gives clear >>>>>> constraints in terms of volatile-volatile accesses but says little about >>>>>> nonvolatile-volatile accesses. Yet the current implementation heavily >>>>>> relies upon volatile to in terms of compiler reordering. But GCC >>>>>> explicitly declares that volatiles and non-volatiles may reorder freely >>>>>> ( https://gcc.gnu.org/onlinedocs/gcc/Volatiles.html ). The only compiler >>>>>> known to explicitly provide the wanted semantics with volatile is MSVC >>>>>> 2010 for windows ( >>>>>> https://msdn.microsoft.com/en-us/library/12a04hfd(v=vs.100).aspx ). >>>>>> Compilers not giving explicit guarantees, must be considered unsafe and >>>>>> revert to conservative behaviour. >>>>>> >>>>>> This was brought to attention after causing bugs, but was only fixed for >>>>>> x86 linux. This is a fundamental issue inherent to all TSO platforms >>>>>> except windows, and has to be fixed on all of them. >>>>>> >>>>>> Several barriers are unsafe to use because they lack compiler reordering >>>>>> constraints (e.g. fence and acquire on linux_SPARC). For TSO platforms >>>>>> they are typically implemented using dummy loads and stores. This seems >>>>>> to be another old volatile reliance that I fixed. These barriers >>>>>> sometimes have omitted compiler barriers (which is what we really want). >>>>>> This seems to be another example on incorrect reliance on the volatile >>>>>> semantics to help us. Therefore dummy loads/stores have been replaced >>>>>> with compiler barriers on TSO platforms. >>>>>> >>>>>> It is also worth noting that compilers like sun studio did previously >>>>>> not support inline asm syntax. Therefore, barriers were implemented in >>>>>> .il-files. However, using them does not give explicit compiler >>>>>> constraints for reordering AFAIK. Therefore, they have been >>>>>> reimplemented using inline asm with explicit compiler reordering >>>>>> constraints, as even sun (solaris?) studio now supports this. >>>>>> >>>>>> The windows variants have added a windows-style _ReadWriteBarrier() >>>>>> compiler barrier similarly. >>>>>> >>>>>> == Strange Hardware Reorderings == >>>>>> >>>>>> Fixed a weird inconsistency where acquire, loadstore and loadlaod would >>>>>> use isync instead of lwsync for PPC on linux_zero, but not in any other >>>>>> PPC platform in the repo. I assumed this is wrong and changed it to >>>>>> lwsync instead. >>>>>> >>>>>> == Code Redundancy and Refactoring == >>>>>> >>>>>> The OrderAccess code looks like it has been developed over a long period >>>>>> of time, with small incremental changes. This seems to have led to a lot >>>>>> of code duplication over time. For example, store_fence variants are not >>>>>> referenced from anywhere, yet contribute to a lot of the code base and a >>>>>> lot of awkwardness (such as being the one only exception not using >>>>>> volatiles for some reason). Moreover, store_fence is not used anywhere >>>>>> in hotspot because it is not a good fit for either the acquire/release >>>>>> semantics or the Java volatile semantics, leaving a big question mark on >>>>>> when it should ever be used. I took the liberty of removing it. >>>>>> >>>>>> Another redundancy issue is that most of the semantics is exactly the >>>>>> same for all platforms, yet all that default boilerplate such as how to >>>>>> make atomic accesses, where acquire/release is supposed to be placed >>>>>> w.r.t. the memory access, what the different barriers should do etc. is >>>>>> copied in redundantly for each os_cpu and each type of memory access for >>>>>> each os_cpu. This makes it extremely painful 1) to understand what >>>>>> actually defines a certain platform compared to the defaults and 2) to >>>>>> correct bugs like those discovered here 3) prevent silly mistakes and >>>>>> bugs, by simply having a lot less code defining the behaviour of >>>>>> OrderAccess that could go wrong. >>>>>> >>>>>> A new architecture/design for OrderAccess is proposed, using a >>>>>> generalization/specialization approach. >>>>>> >>>>>> A general implementation in /share/ defines how things work and splits >>>>>> into different concerns: 1) how to make an atomic memory access, 2) >>>>>> where to but barriers w.r.t. the memory access for things like >>>>>> load_acquire, release_store and release_store_fence, 3) how these >>>>>> barriers are defined. >>>>>> >>>>>> This allows a clear split between what is required for following the >>>>>> specifications, and optimizations, which become much more readable and >>>>>> only optimizations need to be reviewed in depth as the defaults can >>>>>> always be trusted given correct standalone barriers. >>>>>> >>>>>> The only thing a platform is required to specify, is what an >>>>>> implementation of acquire(), release() and fence() should do. If they >>>>>> are implemented properly, everything in OrderAccess is guaranteed to >>>>>> work according to specification using the generalized code. This makes >>>>>> it very easy to support new ports. ALL the other code in the os_cpu >>>>>> files is used /only/ for optimization purposes offered for specific >>>>>> configurations. >>>>>> >>>>>> However, it is highly customizable so that specific platform can perform >>>>>> any desired optimizations. For instance this load_acquire on PPC is >>>>>> optimized: >>>>>> >>>>>> template<> inline jbyte OrderAccess::specialized_load_acquire >>>>>> (volatile jbyte* p) { register jbyte t = load(p); >>>>>> inlasm_acquire_reg(t); return t; } >>>>>> >>>>>> This overrides the whole load_acquire implementation to do something >>>>>> custom. Platforms like x86 extensively use this for joined fencing >>>>>> variants to optimize. >>>>>> >>>>>> The default implementation of load_acquire() will make an atomic load() >>>>>> followed by acquire() since the semantics is generalized. The >>>>>> generalized semantics are defined using inlined postfix/prefix calls >>>>>> after/before the atomic access, as defined here: >>>>>> >>>>>> template<> inline void ScopedFenceGeneral::postfix() { >>>>>> OrderAccess::acquire(); } >>>>>> template<> inline void ScopedFenceGeneral::prefix() { >>>>>> OrderAccess::release(); } >>>>>> template<> inline void ScopedFenceGeneral::prefix() { >>>>>> OrderAccess::release(); } >>>>>> template<> inline void ScopedFenceGeneral::postfix() { >>>>>> OrderAccess::fence(); } >>>>>> >>>>>> For platforms that simply wish to override what e.g. acquire means for a >>>>>> joined ordered memory access in general, as different to calling stand >>>>>> alone acquire(), the semantics can be easily overridden for a platform >>>>>> such as windows like on windows: >>>>>> >>>>>> template<> inline void ScopedFence::postfix() { } >>>>>> template<> inline void ScopedFence::prefix() { } >>>>>> template<> inline void ScopedFence::prefix() { } >>>>>> template<> inline void ScopedFence::postfix() { >>>>>> OrderAccess::fence(); } >>>>>> >>>>>> In this example, since Windows (now) has a compiler barrier for acquire, >>>>>> but does not need it for joined accesses since volatile has stronger >>>>>> guarantees on windows, this is enough to specialize that for joined >>>>>> memory accesses, no extra protection is needed. >>>>>> >>>>>> == Backward Compatibility and Transitioning == >>>>>> >>>>>> Since the newly proposed code is structured differently to before, a >>>>>> #define was added for backward compatibility so that external >>>>>> repositories not adhering to this new architecture do not break. >>>>>> Furthermore, store_release was declared private and marked as >>>>>> deprecated. This allows for a smooth transition into the new style of >>>>>> OrderAccess. When the full transition is made in all known repos, the >>>>>> #define and store_fence may be safely removed, eventually. >>>>>> >>>>>> == Documentation == >>>>>> >>>>>> The documentation seems old and outdated, describing how it works on >>>>>> SPARC RMO and IA64, which are nowhere to be found in the repository. It >>>>>> also describes things about C++ volatiles which cannot be relied upon. >>>>>> The documentation has been cleaned up to match the current state of the >>>>>> implementation better, with architectures actually found in the >>>>>> repository. >>>>>> >>>>>> == Testing == >>>>>> >>>>>> JPRT. Big thanks to Jesper Wilhelmsson for helping me test these changes. >>>>>> Ran some DaCapo benchmarks (I know okay :p) for performance regression >>>>>> and there was no perceivable difference. >>>>>> >>>>>> Looking forward to feedback on this, and hope to get some reviews. :) >>>>>> >>>>>> Thanks, >>>>>> Erik >> >> From goetz.lindenmaier at sap.com Wed Jan 28 08:26:20 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 28 Jan 2015 08:26:20 +0000 Subject: Request or comment: 8071690: Include local HotSpot headers before the system headers In-Reply-To: <54C863D7.7000808@oracle.com> References: <54C863D7.7000808@oracle.com> Message-ID: <4295855A5C1DE049A61835A1887419CC2CF726BF@DEWDFEMB12A.global.corp.sap> Hi David, but by enforcing that rule, the effect is limited to the header that includes other system headers. You can fix the problem by adding globalDefinitions.hpp in that one header foo.h, and whoever includes foo.h get's it right. If fcntl.h had been added to foo.h in first place, all files that include foo.h must include globalDefinitions.hpp before it ... not very easy to catch. Best regards, Goetz. -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of David Holmes Sent: Wednesday, January 28, 2015 5:22 AM To: Volker Simonis; HotSpot Open Source Developers Subject: Re: Request or comment: 8071690: Include local HotSpot headers before the system headers Hi Volker, On 28/01/2015 3:29 AM, Volker Simonis wrote: > Hi, > > I've just opened "8071690: Include local HotSpot headers before the > system headers" but before I start working on it I'd like to hear what > others think about the problem. If there's a consensus that it will be > worth while doing this change I'll be happy to start working on it. As I wrote in the bug report: I don't see how you can apply this as a general rule - or at least not without some further rules. If local foo.h depends on a system header then it will #include it, so a file that #includes foo.h can't control the order of that system header include. We need to recognize (and detect!) where we have implicit ordering constraints but I don't think a general rule actually helps with that. And there may be cases where we rely on a system include coming first eg: #ifndef SOME_SYSTEM_THING #define SOME_SYSTEM_THING xxx #endif David ----- > https://bugs.openjdk.java.net/browse/JDK-8071690 > > The following description is copied from the bug report for your convenience: > > There's no general rule in which order header files should be included > but I think a good hint is to include local, project specific headers > before system headers. This is mainly for two reasons: > > 1. It prevents that dependencies from local header files to system > headers are hidden because a local header is included after a system > header by chance. Instead, every local header should explicitly > specify the system headers it depends on. > > 2. It enables the definition of local macros which control the > behaviour of the system headers which are included later on. > > Point two may be considered bad style, but we actually use it for > example in src/share/vm/utilities/globalDefinitions.hpp where we > define "__STDC_FORMAT_MACROS" before we include "" in the > compiler specific global definitions file. > > "__STDC_FORMAT_MACROS" controls the definition of the printf format > macros in "" but this can only work if "" is > really included AFTER the definition of "__STDC_FORMAT_MACROS". And > this can only wok if every file includes the local HotSpot headers > before any system headers, because otherwise "" may be > indirectly included by a system header before we had a chance to > define "__STDC_FORMAT_MACROS". > > This is exactly what happened after the integration of 8050807 which > added the system include "" to vmError.cpp as follows: > > #include > #include "precompiled.hpp" > #include "code/codeCache.hpp" > > This change broke the build on AIX because "" indirectly > included "" BEFORE we defined "__STDC_FORMAT_MACROS". > > To prevent such errors in the future I propose to change all HotSpot > source files to always include the system headers AFTER the inclusion > of the project specific HotSpot headers. > > I?ve attached a small Pythin script to this bug report which can be > used as follows to detect the files which are currently violating this > rule: > > find src/ \( -name "*.cpp" -o -name "*.hpp" \) -type f -exec python > include.py {} \; > > src/os/solaris/dtrace/generateJvmOffsets.cpp: system header #include > included before local header #include > "code/codeBlob.hpp" > src/os/windows/vm/decoder_windows.hpp: system header #include > included before local header #include > "utilities/decoder.hpp" > src/os_cpu/bsd_zero/vm/os_bsd_zero.cpp: system header # include > included before local header #include > "assembler_zero.inline.hpp" > src/share/vm/adlc/adlc.hpp: system header #include included > before local header #include "string.h" > src/share/vm/libadt/port.hpp: system header #include > included before local header #include "port_tandem.hpp" > src/share/vm/runtime/os.hpp: system header # include > included before local header # include "jvm_solaris.h" > src/share/vm/trace/traceDataTypes.hpp: system header #include > included before local header #include > "utilities/globalDefinitions.hpp" > src/share/vm/utilities/dtrace.hpp: system header #include > included before local header #include > "dtracefiles/hotspot.h" > src/share/vm/utilities/elfFile.cpp: system header #include > included before local header #include "memory/allocation.inline.hpp" > src/share/vm/utilities/elfFile.hpp: system header #include > included before local header #include "globalDefinitions.hpp" > src/share/vm/utilities/vmError.cpp: system header #include > included before local header #include "precompiled.hpp" > > The script is written in Python intentionally such that it can be > easily added as an automated hook to jcheck to prevent violations of > this inclusion rule in the future. > > Of course we can also try to not rely on the inclusion order at all. > IMHO it actually seems that this is the cleanest solution, but it may > require moving some macro definitions from header files right into the > command line (e.g. -D__STDC_FORMAT_MACROS for the example above). By > the way, that's actually the way how I've fixed the above mentioned > compile error on AIX without the need to touch any shared files. > > What do you think? > > Regards, > Volker > From aph at redhat.com Wed Jan 28 09:00:47 2015 From: aph at redhat.com (Andrew Haley) Date: Wed, 28 Jan 2015 09:00:47 +0000 Subject: RFR: 8071688: AARCH64: SIGSEGV in MethodData::next_data(ProfileData*) In-Reply-To: <54C7E5E7.8020300@oracle.com> References: <54C7CF7A.7000502@redhat.com> <209E6B45-2D72-4AC5-91A4-10DD08159FAF@oracle.com> <54C7E5E7.8020300@oracle.com> Message-ID: <54C8A53F.80109@redhat.com> On 27/01/15 19:24, Vladimir Kozlov wrote: > On 1/27/15 11:19 AM, Roland Westrelin wrote: >>> This patch is already applied to the other HotSpot targets. It's 8039975. >>> >>> Webrev at http://cr.openjdk.java.net/~aph/aarch64-8071688/ >> >> It looks good to me. I will push it. > > Yes, it is good. Roland, thank you for sponsoring. Boy, that was fast. Thanks! Andrew. From david.holmes at oracle.com Wed Jan 28 09:29:50 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 28 Jan 2015 19:29:50 +1000 Subject: Request or comment: 8071690: Include local HotSpot headers before the system headers In-Reply-To: <4295855A5C1DE049A61835A1887419CC2CF726BF@DEWDFEMB12A.global.corp.sap> References: <54C863D7.7000808@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF726BF@DEWDFEMB12A.global.corp.sap> Message-ID: <54C8AC0E.2040500@oracle.com> Hi Goetz, On 28/01/2015 6:26 PM, Lindenmaier, Goetz wrote: > Hi David, > > but by enforcing that rule, the effect is limited to the header > that includes other system headers. You can fix the problem > by adding globalDefinitions.hpp in that one header foo.h, and whoever > includes foo.h get's it right. Maybe we need to back up a step - what is the rule about header files including other header files? David > If fcntl.h had been added to foo.h in first place, all files that include > foo.h must include globalDefinitions.hpp before it ... not very easy to > catch. > > Best regards, > Goetz. > > > > > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of David Holmes > Sent: Wednesday, January 28, 2015 5:22 AM > To: Volker Simonis; HotSpot Open Source Developers > Subject: Re: Request or comment: 8071690: Include local HotSpot headers before the system headers > > Hi Volker, > > On 28/01/2015 3:29 AM, Volker Simonis wrote: >> Hi, >> >> I've just opened "8071690: Include local HotSpot headers before the >> system headers" but before I start working on it I'd like to hear what >> others think about the problem. If there's a consensus that it will be >> worth while doing this change I'll be happy to start working on it. > > As I wrote in the bug report: > > I don't see how you can apply this as a general rule - or at least not > without some further rules. If local foo.h depends on a system header > then it will #include it, so a file that #includes foo.h can't control > the order of that system header include. > > We need to recognize (and detect!) where we have implicit ordering > constraints but I don't think a general rule actually helps with that. > And there may be cases where we rely on a system include coming first eg: > > #ifndef SOME_SYSTEM_THING > #define SOME_SYSTEM_THING xxx > #endif > > David > ----- > >> https://bugs.openjdk.java.net/browse/JDK-8071690 >> >> The following description is copied from the bug report for your convenience: >> >> There's no general rule in which order header files should be included >> but I think a good hint is to include local, project specific headers >> before system headers. This is mainly for two reasons: >> >> 1. It prevents that dependencies from local header files to system >> headers are hidden because a local header is included after a system >> header by chance. Instead, every local header should explicitly >> specify the system headers it depends on. >> >> 2. It enables the definition of local macros which control the >> behaviour of the system headers which are included later on. >> >> Point two may be considered bad style, but we actually use it for >> example in src/share/vm/utilities/globalDefinitions.hpp where we >> define "__STDC_FORMAT_MACROS" before we include "" in the >> compiler specific global definitions file. >> >> "__STDC_FORMAT_MACROS" controls the definition of the printf format >> macros in "" but this can only work if "" is >> really included AFTER the definition of "__STDC_FORMAT_MACROS". And >> this can only wok if every file includes the local HotSpot headers >> before any system headers, because otherwise "" may be >> indirectly included by a system header before we had a chance to >> define "__STDC_FORMAT_MACROS". >> >> This is exactly what happened after the integration of 8050807 which >> added the system include "" to vmError.cpp as follows: >> >> #include >> #include "precompiled.hpp" >> #include "code/codeCache.hpp" >> >> This change broke the build on AIX because "" indirectly >> included "" BEFORE we defined "__STDC_FORMAT_MACROS". >> >> To prevent such errors in the future I propose to change all HotSpot >> source files to always include the system headers AFTER the inclusion >> of the project specific HotSpot headers. >> >> I?ve attached a small Pythin script to this bug report which can be >> used as follows to detect the files which are currently violating this >> rule: >> >> find src/ \( -name "*.cpp" -o -name "*.hpp" \) -type f -exec python >> include.py {} \; >> >> src/os/solaris/dtrace/generateJvmOffsets.cpp: system header #include >> included before local header #include >> "code/codeBlob.hpp" >> src/os/windows/vm/decoder_windows.hpp: system header #include >> included before local header #include >> "utilities/decoder.hpp" >> src/os_cpu/bsd_zero/vm/os_bsd_zero.cpp: system header # include >> included before local header #include >> "assembler_zero.inline.hpp" >> src/share/vm/adlc/adlc.hpp: system header #include included >> before local header #include "string.h" >> src/share/vm/libadt/port.hpp: system header #include >> included before local header #include "port_tandem.hpp" >> src/share/vm/runtime/os.hpp: system header # include >> included before local header # include "jvm_solaris.h" >> src/share/vm/trace/traceDataTypes.hpp: system header #include >> included before local header #include >> "utilities/globalDefinitions.hpp" >> src/share/vm/utilities/dtrace.hpp: system header #include >> included before local header #include >> "dtracefiles/hotspot.h" >> src/share/vm/utilities/elfFile.cpp: system header #include >> included before local header #include "memory/allocation.inline.hpp" >> src/share/vm/utilities/elfFile.hpp: system header #include >> included before local header #include "globalDefinitions.hpp" >> src/share/vm/utilities/vmError.cpp: system header #include >> included before local header #include "precompiled.hpp" >> >> The script is written in Python intentionally such that it can be >> easily added as an automated hook to jcheck to prevent violations of >> this inclusion rule in the future. >> >> Of course we can also try to not rely on the inclusion order at all. >> IMHO it actually seems that this is the cleanest solution, but it may >> require moving some macro definitions from header files right into the >> command line (e.g. -D__STDC_FORMAT_MACROS for the example above). By >> the way, that's actually the way how I've fixed the above mentioned >> compile error on AIX without the need to touch any shared files. >> >> What do you think? >> >> Regards, >> Volker >> From volker.simonis at gmail.com Wed Jan 28 10:26:46 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 28 Jan 2015 11:26:46 +0100 Subject: Request or comment: 8071690: Include local HotSpot headers before the system headers In-Reply-To: <54C8AC0E.2040500@oracle.com> References: <54C863D7.7000808@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF726BF@DEWDFEMB12A.global.corp.sap> <54C8AC0E.2040500@oracle.com> Message-ID: On Wed, Jan 28, 2015 at 10:29 AM, David Holmes wrote: > Hi Goetz, > > On 28/01/2015 6:26 PM, Lindenmaier, Goetz wrote: >> >> Hi David, >> >> but by enforcing that rule, the effect is limited to the header >> that includes other system headers. You can fix the problem >> by adding globalDefinitions.hpp in that one header foo.h, and whoever >> includes foo.h get's it right. > > > Maybe we need to back up a step - what is the rule about header files > including other header files? > I think the problem is that there is no general rule. There are just hints like "every header file should be self-contained", i.e. it should include all the other headers it depends on. Following this rule should allow any arbitrary inclusion order. But in our case, we rely on a specific macro ("__STDC_FORMAT_MACROS") being defined before we include a system header. So we have three possibilities: 1. define the macro at the top of every file before including anything. 2. define the macro on the command line (actually a more elegant version of 1.) 3. define the macro once in a local header and make sure the local header is always included before any system header. Solution 1 and 2 should still allow an arbitrary inclusion order if all headers are self-contained. The more I think about the problem the more I come to the conclusion that for this specific issue solution 2 (defining the macro on the command line) is the right solution. We already do this in other cases in order to control how and what will be included from the system headers (e.g. -D_GNU_SOURCE, -D_REENTRANT on Linux) and -D__STDC_FORMAT_MACROS would perfectly fit into this list. It is unclear to me why the latter was defined in a local header file. By the way, the example you mentioned, where we rely on a system include coming first: #ifndef SOME_SYSTEM_THING #define SOME_SYSTEM_THING xxx #endif can be easily solved by having a local wrapper which only includes the system header and afterwards performs the required configuration steps. Other files of course will have to include the local wrapper instead of the system header. Regards, Volker > David > > >> If fcntl.h had been added to foo.h in first place, all files that include >> foo.h must include globalDefinitions.hpp before it ... not very easy to >> catch. >> >> Best regards, >> Goetz. >> >> >> >> >> >> -----Original Message----- >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf >> Of David Holmes >> Sent: Wednesday, January 28, 2015 5:22 AM >> To: Volker Simonis; HotSpot Open Source Developers >> Subject: Re: Request or comment: 8071690: Include local HotSpot headers >> before the system headers >> >> Hi Volker, >> >> On 28/01/2015 3:29 AM, Volker Simonis wrote: >>> >>> Hi, >>> >>> I've just opened "8071690: Include local HotSpot headers before the >>> system headers" but before I start working on it I'd like to hear what >>> others think about the problem. If there's a consensus that it will be >>> worth while doing this change I'll be happy to start working on it. >> >> >> As I wrote in the bug report: >> >> I don't see how you can apply this as a general rule - or at least not >> without some further rules. If local foo.h depends on a system header >> then it will #include it, so a file that #includes foo.h can't control >> the order of that system header include. >> >> We need to recognize (and detect!) where we have implicit ordering >> constraints but I don't think a general rule actually helps with that. >> And there may be cases where we rely on a system include coming first eg: >> >> #ifndef SOME_SYSTEM_THING >> #define SOME_SYSTEM_THING xxx >> #endif >> >> David >> ----- >> >>> https://bugs.openjdk.java.net/browse/JDK-8071690 >>> >>> The following description is copied from the bug report for your >>> convenience: >>> >>> There's no general rule in which order header files should be included >>> but I think a good hint is to include local, project specific headers >>> before system headers. This is mainly for two reasons: >>> >>> 1. It prevents that dependencies from local header files to system >>> headers are hidden because a local header is included after a system >>> header by chance. Instead, every local header should explicitly >>> specify the system headers it depends on. >>> >>> 2. It enables the definition of local macros which control the >>> behaviour of the system headers which are included later on. >>> >>> Point two may be considered bad style, but we actually use it for >>> example in src/share/vm/utilities/globalDefinitions.hpp where we >>> define "__STDC_FORMAT_MACROS" before we include "" in the >>> compiler specific global definitions file. >>> >>> "__STDC_FORMAT_MACROS" controls the definition of the printf format >>> macros in "" but this can only work if "" is >>> really included AFTER the definition of "__STDC_FORMAT_MACROS". And >>> this can only wok if every file includes the local HotSpot headers >>> before any system headers, because otherwise "" may be >>> indirectly included by a system header before we had a chance to >>> define "__STDC_FORMAT_MACROS". >>> >>> This is exactly what happened after the integration of 8050807 which >>> added the system include "" to vmError.cpp as follows: >>> >>> #include >>> #include "precompiled.hpp" >>> #include "code/codeCache.hpp" >>> >>> This change broke the build on AIX because "" indirectly >>> included "" BEFORE we defined "__STDC_FORMAT_MACROS". >>> >>> To prevent such errors in the future I propose to change all HotSpot >>> source files to always include the system headers AFTER the inclusion >>> of the project specific HotSpot headers. >>> >>> I?ve attached a small Pythin script to this bug report which can be >>> used as follows to detect the files which are currently violating this >>> rule: >>> >>> find src/ \( -name "*.cpp" -o -name "*.hpp" \) -type f -exec python >>> include.py {} \; >>> >>> src/os/solaris/dtrace/generateJvmOffsets.cpp: system header #include >>> included before local header #include >>> "code/codeBlob.hpp" >>> src/os/windows/vm/decoder_windows.hpp: system header #include >>> included before local header #include >>> "utilities/decoder.hpp" >>> src/os_cpu/bsd_zero/vm/os_bsd_zero.cpp: system header # include >>> included before local header #include >>> "assembler_zero.inline.hpp" >>> src/share/vm/adlc/adlc.hpp: system header #include included >>> before local header #include "string.h" >>> src/share/vm/libadt/port.hpp: system header #include >>> included before local header #include "port_tandem.hpp" >>> src/share/vm/runtime/os.hpp: system header # include >>> included before local header # include "jvm_solaris.h" >>> src/share/vm/trace/traceDataTypes.hpp: system header #include >>> included before local header #include >>> "utilities/globalDefinitions.hpp" >>> src/share/vm/utilities/dtrace.hpp: system header #include >>> included before local header #include >>> "dtracefiles/hotspot.h" >>> src/share/vm/utilities/elfFile.cpp: system header #include >>> included before local header #include "memory/allocation.inline.hpp" >>> src/share/vm/utilities/elfFile.hpp: system header #include >>> included before local header #include "globalDefinitions.hpp" >>> src/share/vm/utilities/vmError.cpp: system header #include >>> included before local header #include "precompiled.hpp" >>> >>> The script is written in Python intentionally such that it can be >>> easily added as an automated hook to jcheck to prevent violations of >>> this inclusion rule in the future. >>> >>> Of course we can also try to not rely on the inclusion order at all. >>> IMHO it actually seems that this is the cleanest solution, but it may >>> require moving some macro definitions from header files right into the >>> command line (e.g. -D__STDC_FORMAT_MACROS for the example above). By >>> the way, that's actually the way how I've fixed the above mentioned >>> compile error on AIX without the need to touch any shared files. >>> >>> What do you think? >>> >>> Regards, >>> Volker >>> > From david.holmes at oracle.com Wed Jan 28 10:50:17 2015 From: david.holmes at oracle.com (David Holmes) Date: Wed, 28 Jan 2015 20:50:17 +1000 Subject: Request or comment: 8071690: Include local HotSpot headers before the system headers In-Reply-To: References: <54C863D7.7000808@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF726BF@DEWDFEMB12A.global.corp.sap> <54C8AC0E.2040500@oracle.com> Message-ID: <54C8BEE9.50607@oracle.com> On 28/01/2015 8:26 PM, Volker Simonis wrote: > On Wed, Jan 28, 2015 at 10:29 AM, David Holmes wrote: >> Hi Goetz, >> >> On 28/01/2015 6:26 PM, Lindenmaier, Goetz wrote: >>> >>> Hi David, >>> >>> but by enforcing that rule, the effect is limited to the header >>> that includes other system headers. You can fix the problem >>> by adding globalDefinitions.hpp in that one header foo.h, and whoever >>> includes foo.h get's it right. >> >> >> Maybe we need to back up a step - what is the rule about header files >> including other header files? >> > > I think the problem is that there is no general rule. There are just > hints like "every header file should be self-contained", i.e. it > should include all the other headers it depends on. Following this > rule should allow any arbitrary inclusion order. > > But in our case, we rely on a specific macro ("__STDC_FORMAT_MACROS") > being defined before we include a system header. So we have three > possibilities: > 1. define the macro at the top of every file before including anything. > 2. define the macro on the command line (actually a more elegant version of 1.) > 3. define the macro once in a local header and make sure the local > header is always included before any system header. > > Solution 1 and 2 should still allow an arbitrary inclusion order if > all headers are self-contained. > > The more I think about the problem the more I come to the conclusion > that for this specific issue solution 2 (defining the macro on the > command line) is the right solution. We already do this in other cases > in order to control how and what will be included from the system > headers (e.g. -D_GNU_SOURCE, -D_REENTRANT on Linux) and > -D__STDC_FORMAT_MACROS would perfectly fit into this list. It is > unclear to me why the latter was defined in a local header file. I'm a bit confused about this whole STDC_FORMAT_MACRO business. Here is what I see: ./share/vm/utilities/globalDefinitions.hpp:#ifndef __STDC_FORMAT_MACROS ./share/vm/utilities/globalDefinitions.hpp:#define __STDC_FORMAT_MACROS ./os/aix/vm/loadlib_aix.cpp:#ifndef __STDC_FORMAT_MACROS ./os/aix/vm/loadlib_aix.cpp:#define __STDC_FORMAT_MACROS ./os/aix/vm/loadlib_aix.cpp:// are only defined if '__STDC_FORMAT_MACROS' is defined! ./os/aix/vm/jvm_aix.h:#if !defined(__STDC_FORMAT_MACROS) ./os/aix/vm/jvm_aix.h:# define __STDC_FORMAT_MACROS 1 ./os/bsd/vm/jvm_bsd.h:# if !defined(__STDC_FORMAT_MACROS) ./os/bsd/vm/jvm_bsd.h:# define __STDC_FORMAT_MACROS 1 I thought the double underscore macros were only for compiler use - so what normally sets it and what does it get used for? David ----- > By the way, the example you mentioned, where we rely on a system > include coming first: > > #ifndef SOME_SYSTEM_THING > #define SOME_SYSTEM_THING xxx > #endif > > can be easily solved by having a local wrapper which only includes the > system header and afterwards performs the required configuration > steps. Other files of course will have to include the local wrapper > instead of the system header. > > Regards, > Volker > >> David >> >> >>> If fcntl.h had been added to foo.h in first place, all files that include >>> foo.h must include globalDefinitions.hpp before it ... not very easy to >>> catch. >>> >>> Best regards, >>> Goetz. >>> >>> >>> >>> >>> >>> -----Original Message----- >>> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf >>> Of David Holmes >>> Sent: Wednesday, January 28, 2015 5:22 AM >>> To: Volker Simonis; HotSpot Open Source Developers >>> Subject: Re: Request or comment: 8071690: Include local HotSpot headers >>> before the system headers >>> >>> Hi Volker, >>> >>> On 28/01/2015 3:29 AM, Volker Simonis wrote: >>>> >>>> Hi, >>>> >>>> I've just opened "8071690: Include local HotSpot headers before the >>>> system headers" but before I start working on it I'd like to hear what >>>> others think about the problem. If there's a consensus that it will be >>>> worth while doing this change I'll be happy to start working on it. >>> >>> >>> As I wrote in the bug report: >>> >>> I don't see how you can apply this as a general rule - or at least not >>> without some further rules. If local foo.h depends on a system header >>> then it will #include it, so a file that #includes foo.h can't control >>> the order of that system header include. >>> >>> We need to recognize (and detect!) where we have implicit ordering >>> constraints but I don't think a general rule actually helps with that. >>> And there may be cases where we rely on a system include coming first eg: >>> >>> #ifndef SOME_SYSTEM_THING >>> #define SOME_SYSTEM_THING xxx >>> #endif >>> >>> David >>> ----- >>> >>>> https://bugs.openjdk.java.net/browse/JDK-8071690 >>>> >>>> The following description is copied from the bug report for your >>>> convenience: >>>> >>>> There's no general rule in which order header files should be included >>>> but I think a good hint is to include local, project specific headers >>>> before system headers. This is mainly for two reasons: >>>> >>>> 1. It prevents that dependencies from local header files to system >>>> headers are hidden because a local header is included after a system >>>> header by chance. Instead, every local header should explicitly >>>> specify the system headers it depends on. >>>> >>>> 2. It enables the definition of local macros which control the >>>> behaviour of the system headers which are included later on. >>>> >>>> Point two may be considered bad style, but we actually use it for >>>> example in src/share/vm/utilities/globalDefinitions.hpp where we >>>> define "__STDC_FORMAT_MACROS" before we include "" in the >>>> compiler specific global definitions file. >>>> >>>> "__STDC_FORMAT_MACROS" controls the definition of the printf format >>>> macros in "" but this can only work if "" is >>>> really included AFTER the definition of "__STDC_FORMAT_MACROS". And >>>> this can only wok if every file includes the local HotSpot headers >>>> before any system headers, because otherwise "" may be >>>> indirectly included by a system header before we had a chance to >>>> define "__STDC_FORMAT_MACROS". >>>> >>>> This is exactly what happened after the integration of 8050807 which >>>> added the system include "" to vmError.cpp as follows: >>>> >>>> #include >>>> #include "precompiled.hpp" >>>> #include "code/codeCache.hpp" >>>> >>>> This change broke the build on AIX because "" indirectly >>>> included "" BEFORE we defined "__STDC_FORMAT_MACROS". >>>> >>>> To prevent such errors in the future I propose to change all HotSpot >>>> source files to always include the system headers AFTER the inclusion >>>> of the project specific HotSpot headers. >>>> >>>> I?ve attached a small Pythin script to this bug report which can be >>>> used as follows to detect the files which are currently violating this >>>> rule: >>>> >>>> find src/ \( -name "*.cpp" -o -name "*.hpp" \) -type f -exec python >>>> include.py {} \; >>>> >>>> src/os/solaris/dtrace/generateJvmOffsets.cpp: system header #include >>>> included before local header #include >>>> "code/codeBlob.hpp" >>>> src/os/windows/vm/decoder_windows.hpp: system header #include >>>> included before local header #include >>>> "utilities/decoder.hpp" >>>> src/os_cpu/bsd_zero/vm/os_bsd_zero.cpp: system header # include >>>> included before local header #include >>>> "assembler_zero.inline.hpp" >>>> src/share/vm/adlc/adlc.hpp: system header #include included >>>> before local header #include "string.h" >>>> src/share/vm/libadt/port.hpp: system header #include >>>> included before local header #include "port_tandem.hpp" >>>> src/share/vm/runtime/os.hpp: system header # include >>>> included before local header # include "jvm_solaris.h" >>>> src/share/vm/trace/traceDataTypes.hpp: system header #include >>>> included before local header #include >>>> "utilities/globalDefinitions.hpp" >>>> src/share/vm/utilities/dtrace.hpp: system header #include >>>> included before local header #include >>>> "dtracefiles/hotspot.h" >>>> src/share/vm/utilities/elfFile.cpp: system header #include >>>> included before local header #include "memory/allocation.inline.hpp" >>>> src/share/vm/utilities/elfFile.hpp: system header #include >>>> included before local header #include "globalDefinitions.hpp" >>>> src/share/vm/utilities/vmError.cpp: system header #include >>>> included before local header #include "precompiled.hpp" >>>> >>>> The script is written in Python intentionally such that it can be >>>> easily added as an automated hook to jcheck to prevent violations of >>>> this inclusion rule in the future. >>>> >>>> Of course we can also try to not rely on the inclusion order at all. >>>> IMHO it actually seems that this is the cleanest solution, but it may >>>> require moving some macro definitions from header files right into the >>>> command line (e.g. -D__STDC_FORMAT_MACROS for the example above). By >>>> the way, that's actually the way how I've fixed the above mentioned >>>> compile error on AIX without the need to touch any shared files. >>>> >>>> What do you think? >>>> >>>> Regards, >>>> Volker >>>> >> From volker.simonis at gmail.com Wed Jan 28 13:41:25 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 28 Jan 2015 14:41:25 +0100 Subject: Request or comment: 8071690: Include local HotSpot headers before the system headers In-Reply-To: <54C8BEE9.50607@oracle.com> References: <54C863D7.7000808@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF726BF@DEWDFEMB12A.global.corp.sap> <54C8AC0E.2040500@oracle.com> <54C8BEE9.50607@oracle.com> Message-ID: The standard format macros like PRId32, PRIi64, etc. are specified by C99 in but C99 states the these macros should only be defined in C++ if __STDC_FORMAT_MACROS is set before the inclusion of (see footnote 185 on page 198 in http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf) On Wed, Jan 28, 2015 at 11:50 AM, David Holmes wrote: > On 28/01/2015 8:26 PM, Volker Simonis wrote: >> >> On Wed, Jan 28, 2015 at 10:29 AM, David Holmes >> wrote: >>> >>> Hi Goetz, >>> >>> On 28/01/2015 6:26 PM, Lindenmaier, Goetz wrote: >>>> >>>> >>>> Hi David, >>>> >>>> but by enforcing that rule, the effect is limited to the header >>>> that includes other system headers. You can fix the problem >>>> by adding globalDefinitions.hpp in that one header foo.h, and whoever >>>> includes foo.h get's it right. >>> >>> >>> >>> Maybe we need to back up a step - what is the rule about header files >>> including other header files? >>> >> >> I think the problem is that there is no general rule. There are just >> hints like "every header file should be self-contained", i.e. it >> should include all the other headers it depends on. Following this >> rule should allow any arbitrary inclusion order. >> >> But in our case, we rely on a specific macro ("__STDC_FORMAT_MACROS") >> being defined before we include a system header. So we have three >> possibilities: >> 1. define the macro at the top of every file before including anything. >> 2. define the macro on the command line (actually a more elegant version >> of 1.) >> 3. define the macro once in a local header and make sure the local >> header is always included before any system header. >> >> Solution 1 and 2 should still allow an arbitrary inclusion order if >> all headers are self-contained. >> >> The more I think about the problem the more I come to the conclusion >> that for this specific issue solution 2 (defining the macro on the >> command line) is the right solution. We already do this in other cases >> in order to control how and what will be included from the system >> headers (e.g. -D_GNU_SOURCE, -D_REENTRANT on Linux) and >> -D__STDC_FORMAT_MACROS would perfectly fit into this list. It is >> unclear to me why the latter was defined in a local header file. > > > I'm a bit confused about this whole STDC_FORMAT_MACRO business. Here is what > I see: > > ./share/vm/utilities/globalDefinitions.hpp:#ifndef __STDC_FORMAT_MACROS > ./share/vm/utilities/globalDefinitions.hpp:#define __STDC_FORMAT_MACROS > ./os/aix/vm/loadlib_aix.cpp:#ifndef __STDC_FORMAT_MACROS > ./os/aix/vm/loadlib_aix.cpp:#define __STDC_FORMAT_MACROS > ./os/aix/vm/loadlib_aix.cpp:// are only defined if '__STDC_FORMAT_MACROS' is > defined! > ./os/aix/vm/jvm_aix.h:#if !defined(__STDC_FORMAT_MACROS) > ./os/aix/vm/jvm_aix.h:# define __STDC_FORMAT_MACROS 1 > ./os/bsd/vm/jvm_bsd.h:# if !defined(__STDC_FORMAT_MACROS) > ./os/bsd/vm/jvm_bsd.h:# define __STDC_FORMAT_MACROS 1 > > I thought the double underscore macros were only for compiler use - so what > normally sets it and what does it get used for? > > David > ----- > > >> By the way, the example you mentioned, where we rely on a system >> include coming first: >> >> #ifndef SOME_SYSTEM_THING >> #define SOME_SYSTEM_THING xxx >> #endif >> >> can be easily solved by having a local wrapper which only includes the >> system header and afterwards performs the required configuration >> steps. Other files of course will have to include the local wrapper >> instead of the system header. >> >> Regards, >> Volker >> >>> David >>> >>> >>>> If fcntl.h had been added to foo.h in first place, all files that >>>> include >>>> foo.h must include globalDefinitions.hpp before it ... not very easy to >>>> catch. >>>> >>>> Best regards, >>>> Goetz. >>>> >>>> >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On >>>> Behalf >>>> Of David Holmes >>>> Sent: Wednesday, January 28, 2015 5:22 AM >>>> To: Volker Simonis; HotSpot Open Source Developers >>>> Subject: Re: Request or comment: 8071690: Include local HotSpot headers >>>> before the system headers >>>> >>>> Hi Volker, >>>> >>>> On 28/01/2015 3:29 AM, Volker Simonis wrote: >>>>> >>>>> >>>>> Hi, >>>>> >>>>> I've just opened "8071690: Include local HotSpot headers before the >>>>> system headers" but before I start working on it I'd like to hear what >>>>> others think about the problem. If there's a consensus that it will be >>>>> worth while doing this change I'll be happy to start working on it. >>>> >>>> >>>> >>>> As I wrote in the bug report: >>>> >>>> I don't see how you can apply this as a general rule - or at least not >>>> without some further rules. If local foo.h depends on a system header >>>> then it will #include it, so a file that #includes foo.h can't control >>>> the order of that system header include. >>>> >>>> We need to recognize (and detect!) where we have implicit ordering >>>> constraints but I don't think a general rule actually helps with that. >>>> And there may be cases where we rely on a system include coming first >>>> eg: >>>> >>>> #ifndef SOME_SYSTEM_THING >>>> #define SOME_SYSTEM_THING xxx >>>> #endif >>>> >>>> David >>>> ----- >>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8071690 >>>>> >>>>> The following description is copied from the bug report for your >>>>> convenience: >>>>> >>>>> There's no general rule in which order header files should be included >>>>> but I think a good hint is to include local, project specific headers >>>>> before system headers. This is mainly for two reasons: >>>>> >>>>> 1. It prevents that dependencies from local header files to system >>>>> headers are hidden because a local header is included after a system >>>>> header by chance. Instead, every local header should explicitly >>>>> specify the system headers it depends on. >>>>> >>>>> 2. It enables the definition of local macros which control the >>>>> behaviour of the system headers which are included later on. >>>>> >>>>> Point two may be considered bad style, but we actually use it for >>>>> example in src/share/vm/utilities/globalDefinitions.hpp where we >>>>> define "__STDC_FORMAT_MACROS" before we include "" in the >>>>> compiler specific global definitions file. >>>>> >>>>> "__STDC_FORMAT_MACROS" controls the definition of the printf format >>>>> macros in "" but this can only work if "" is >>>>> really included AFTER the definition of "__STDC_FORMAT_MACROS". And >>>>> this can only wok if every file includes the local HotSpot headers >>>>> before any system headers, because otherwise "" may be >>>>> indirectly included by a system header before we had a chance to >>>>> define "__STDC_FORMAT_MACROS". >>>>> >>>>> This is exactly what happened after the integration of 8050807 which >>>>> added the system include "" to vmError.cpp as follows: >>>>> >>>>> #include >>>>> #include "precompiled.hpp" >>>>> #include "code/codeCache.hpp" >>>>> >>>>> This change broke the build on AIX because "" indirectly >>>>> included "" BEFORE we defined "__STDC_FORMAT_MACROS". >>>>> >>>>> To prevent such errors in the future I propose to change all HotSpot >>>>> source files to always include the system headers AFTER the inclusion >>>>> of the project specific HotSpot headers. >>>>> >>>>> I?ve attached a small Pythin script to this bug report which can be >>>>> used as follows to detect the files which are currently violating this >>>>> rule: >>>>> >>>>> find src/ \( -name "*.cpp" -o -name "*.hpp" \) -type f -exec python >>>>> include.py {} \; >>>>> >>>>> src/os/solaris/dtrace/generateJvmOffsets.cpp: system header #include >>>>> included before local header #include >>>>> "code/codeBlob.hpp" >>>>> src/os/windows/vm/decoder_windows.hpp: system header #include >>>>> included before local header #include >>>>> "utilities/decoder.hpp" >>>>> src/os_cpu/bsd_zero/vm/os_bsd_zero.cpp: system header # include >>>>> included before local header #include >>>>> "assembler_zero.inline.hpp" >>>>> src/share/vm/adlc/adlc.hpp: system header #include included >>>>> before local header #include "string.h" >>>>> src/share/vm/libadt/port.hpp: system header #include >>>>> included before local header #include "port_tandem.hpp" >>>>> src/share/vm/runtime/os.hpp: system header # include >>>>> included before local header # include "jvm_solaris.h" >>>>> src/share/vm/trace/traceDataTypes.hpp: system header #include >>>>> included before local header #include >>>>> "utilities/globalDefinitions.hpp" >>>>> src/share/vm/utilities/dtrace.hpp: system header #include >>>>> included before local header #include >>>>> "dtracefiles/hotspot.h" >>>>> src/share/vm/utilities/elfFile.cpp: system header #include >>>>> included before local header #include "memory/allocation.inline.hpp" >>>>> src/share/vm/utilities/elfFile.hpp: system header #include >>>>> included before local header #include "globalDefinitions.hpp" >>>>> src/share/vm/utilities/vmError.cpp: system header #include >>>>> included before local header #include "precompiled.hpp" >>>>> >>>>> The script is written in Python intentionally such that it can be >>>>> easily added as an automated hook to jcheck to prevent violations of >>>>> this inclusion rule in the future. >>>>> >>>>> Of course we can also try to not rely on the inclusion order at all. >>>>> IMHO it actually seems that this is the cleanest solution, but it may >>>>> require moving some macro definitions from header files right into the >>>>> command line (e.g. -D__STDC_FORMAT_MACROS for the example above). By >>>>> the way, that's actually the way how I've fixed the above mentioned >>>>> compile error on AIX without the need to touch any shared files. >>>>> >>>>> What do you think? >>>>> >>>>> Regards, >>>>> Volker >>>>> >>> > From coleen.phillimore at oracle.com Wed Jan 28 14:28:15 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 28 Jan 2015 09:28:15 -0500 Subject: 5-rd round RFR (S) 8008678: JSR 292: constant pool reconstitution must support pseudo strings In-Reply-To: <54C825CE.1030105@oracle.com> References: <5475968C.40800@oracle.com> <54760B40.9040206@oracle.com> <54762FC9.2010800@oracle.com> <5491BCB9.10505@oracle.com> <5493299F.60005@oracle.com> <54934E73.4080808@oracle.com> <54B96F97.5070204@oracle.com> <54B99682.8070802@oracle.com> <54B99838.10601@oracle.com> <54C825CE.1030105@oracle.com> Message-ID: <54C8F1FF.3050301@oracle.com> Yes, this looks great. Ship it! Coleen On 1/27/15, 6:57 PM, serguei.spitsyn at oracle.com wrote: > This is 5-th round review of for: > https://bugs.openjdk.java.net/browse/JDK-8008678 > > There was one (4-rd) private review discussion. The result is this webrev: > http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8008678-JVMTI-pseudo.5/ > > > Summary: > Current fix is to use the 2-nd LSB bit in the CPSlot to mark > pseudo-strings. > A similar version of webrev was already reviewed by Coleen and John. > The update is that it includes John's suggestion to introduce the > CPSlot::TagBits to make code more clean. > Also, I've discovered that there is no need to update the > SymbolClosure::load_symbol() in iterator.hpp > because the pseudo-string PCSlot is not used in the SymbolClosure. > I need a final thumbs up for the push. > > Testing: > Run: > - nsk.jvmti.testlist, nsk.jdi.testlist, vm.mlvm.testlist > - java/lang/instrument, com/sun/jdi and hotspot/test/runtime tests > - new jtreg test (see webrev) that was written by Filipp Zhinkin > > > Thanks, > Serguei > > On 1/16/15 3:01 PM, serguei.spitsyn at oracle.com wrote: >> John R. suggested to use the CPSlot(Symbol* ptr) to mark pseudo-strings. >> The updated webrev is going to be close to the .3 webrev. >> I will send it soon. >> >> Thanks, >> Serguei >> >> On 1/16/15 2:53 PM, Coleen Phillimore wrote: >>> >>> This change looks good to me also. >>> Thanks, >>> Coleen >>> >>> On 1/16/15, 3:07 PM, serguei.spitsyn at oracle.com wrote: >>>> Please, review the fix for: >>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>> >>>> >>>> Open webrev: >>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8008678-JVMTI-pseudo.3/ >>>> >>>> >>>> Summary: >>>> Currently, a JVM_CONSTANT_String CP entry having a NULL >>>> reference to Symbol* >>>> indicates that it is a pseudo-string (patched string). >>>> This creates nasty issues for the constant pool reconstitution. >>>> >>>> Current suggestion is to avoid having a NULL reference and >>>> retain the original >>>> Symbol* reference for pseudo-strings. The pseudo-string >>>> indication will be >>>> if the Utf8 representation starts from "CONSTANT_PLACEHOLDER_". >>>> This approach makes the fix much simpler. >>>> >>>> I need a confirmation from the Compiler team that this won't >>>> break any assumptions or invariants. >>>> Big thanks to Coleen for previous round reviews and good advices! >>>> >>>> >>>> Testing: >>>> Run: >>>> - java/lang/instrument tests >>>> - new jtreg test (see webrev) that was written by Filipp Zhinkin >>>> >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 12/18/14 2:00 PM, serguei.spitsyn at oracle.com wrote: >>>>> Hi Coleen, >>>>> >>>>> Thank you for reviewing! >>>>> >>>>> >>>>> On 12/18/14 11:23 AM, Coleen Phillimore wrote: >>>>>> >>>>>> Hi Serguei, >>>>>> >>>>>> Thank you for making the patches an optional field. >>>>>> >>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/oops/constantPool.hpp.sdiff.html >>>>>> 198 if (!patched()) { >>>>>> 199 assert(false, "a pseudo-string map may exists for patched CP only"); >>>>>> 200 return 0; >>>>>> 201 } >>>>>> Why not >>>>>> assert(patched(), "a pseud-string map must exist >>>>>> for patched CP only"); >>>>> >>>>> Wanted it to be more reliable but it looks pointless. >>>>> Will make this change. >>>>> >>>>>> >>>>>> >>>>>> Why is this? Is this really a ShouldNotReachHere? should it be >>>>>> false? >>>>>> >>>>>> 215 assert(true, "not found a matching entry in pseudo-string map"); >>>>> >>>>> >>>>> A typo, must be false. >>>>> It is the last minute change. >>>>> Thanks for the catch! >>>>> >>>>> >>>>>> >>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/prims/jvmtiRedefineClasses.cpp.udiff.html >>>>>> >>>>>> Don't you have to move the value of the patched field from the >>>>>> old constant pool to the new one? I hate to ask but is there >>>>>> merging that needs to be done? I don't know how to write this >>>>>> test case though. Is it possible to redefine a class with a >>>>>> constant pool patches with another that has constant pool patches? >>>>> >>>>> Thank you for asking this question. >>>>> If I understand correctly, the patching comes from the compiler >>>>> side for anonymous classes only. >>>>> I saw it for LambdaForm's only. >>>>> I think, the patching can not be changed with a retransformation. >>>>> But I'm not sure if it can not be changed with a redefinition. >>>>> >>>>> But if it can - then it would be safe to merge the 'patched' >>>>> condition, i.e. make it patched >>>>> if either the_class or scratch class is patched. >>>>> >>>>>> >>>>>> Somehow I thought you'd have to save the value of the cp_patches >>>>>> oops passed in. >>>>>> >>>>>> So I was wondering why you can't change this instead: >>>>>> >>>>>> bool is_pseudo_string_at(int which) { >>>>>> // A pseudo string is a string that doesn't have a symbol in >>>>>> the cpSlot >>>>>> - return unresolved_string_at(which) == NULL; >>>>>> + return (strncmp(unresolved_string_at(which)->as_utf8(), >>>>>> "CONSTANT_PLACEHOLDER_" , strlen("CONSTANT_PLACEHOLDER")) == 0); >>>>>> } >>>>> >>>>> I was thinking about the same but was not sure if it would work >>>>> for the compiler team. >>>>> We have to ask John about this (added John and Christian to the >>>>> cc-list). >>>>> This question to John was in my plan! :) >>>>> >>>>> The beauty of the above approach is that there is no need to >>>>> create an intermediate >>>>> pseudo-string map and most of the code in from this webrev is not >>>>> needed. >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>>> >>>>>> And the asserts in the other functions below it. >>>>>> >>>>>> Coleen >>>>>> >>>>>> >>>>>> On 12/17/14, 12:26 PM, serguei.spitsyn at oracle.com wrote: >>>>>>> Please, review the second round fix for: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>>>> >>>>>>> Open webrev: >>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/ >>>>>>> >>>>>>> >>>>>>> Summary: >>>>>>> >>>>>>> This fix implements a footprint saving approach suggested by >>>>>>> Coleen. >>>>>>> To be able to reconstitute a class constant pool, an >>>>>>> intermediate pseudo-string map is used. >>>>>>> Now, this field is accounted optionally, only if the >>>>>>> 'cp_patches' is provided in the >>>>>>> ClassFileParser::parseClassFile() before ConstantPool is >>>>>>> allocated. >>>>>>> This fix is not elegant, even a little bit ugly, but it is the >>>>>>> only way I see so far. >>>>>>> >>>>>>> Unfortunately, this approach did not help much to make some >>>>>>> other fields (eg., 'operands') optional. >>>>>>> The problem is that we have to account optional fields before >>>>>>> parsing, at the CP allocation time. >>>>>>> It is possible to re-allocate the ConstantPool when any >>>>>>> InvokeDynamic bytecode is discovered, >>>>>>> but it looks too complicated. >>>>>>> >>>>>>> Testing: >>>>>>> - the unit test from bug report >>>>>>> - nsk.jvmti,testlist, nsk.jdi.testlist, JTREG java/lang/instrument >>>>>>> - vm.mlvm.testlist, vm.quick.testlist, >>>>>>> vm.parallel_class_loading.testlist (in progress) >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 11/26/14 11:53 AM, serguei.spitsyn at oracle.com wrote: >>>>>>>> Coleen, >>>>>>>> >>>>>>>> Thank you for looking at this! >>>>>>>> I'll check how this can be improved. >>>>>>>> It is my concern too. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> On 11/26/14 9:17 AM, Coleen Phillimore wrote: >>>>>>>>> >>>>>>>>> Serguei, >>>>>>>>> I had a quick look at this. I was wondering if we could make >>>>>>>>> the pseudo_string_map conditional in ConstantPool and not make >>>>>>>>> all classes pay in footprint for this field? The same thing >>>>>>>>> probably could be done for operands too. There are flags that >>>>>>>>> you can set to conditionally add a pointer to base() in this >>>>>>>>> function. >>>>>>>>> >>>>>>>>> Typical C++ would subclass ConstantPool to add >>>>>>>>> InvokeDynamicConstantPool fields, but this is not typical C++ >>>>>>>>> so the trick we use is like the one in ConstMethod. I think >>>>>>>>> it's worth doing in this case. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Coleen >>>>>>>>> >>>>>>>>> On 11/26/14, 3:59 AM, serguei.spitsyn at oracle.com wrote: >>>>>>>>>> Please, review the fix for: >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Open webrev: >>>>>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.1/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Summary: >>>>>>>>>> The pseudo-strings are currently not supported in >>>>>>>>>> reconstitution of constant pool. >>>>>>>>>> >>>>>>>>>> This is an explanation from John Rose about what the >>>>>>>>>> pseudo-strings are: >>>>>>>>>> >>>>>>>>>> "We still need "live" oop constants pre-linked into the >>>>>>>>>> constant pool of bytecodes which >>>>>>>>>> implement some method handles. We use the anonymous class >>>>>>>>>> pseudo-string feature for that. >>>>>>>>>> The relevant code is here: >>>>>>>>>> http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/tip/src/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java >>>>>>>>>> >>>>>>>>>> These oops are what "pseudo-strings" are. >>>>>>>>>> The odd name refers to the fact that, even though they >>>>>>>>>> are random oops, they appear in the constant pool >>>>>>>>>> where one would expect (because of class file syntax) to >>>>>>>>>> find a string." >>>>>>>>>> ... >>>>>>>>>> If you really wanted to reconstitute a class file for an >>>>>>>>>> anonymous class, and >>>>>>>>>> if that class has oop patching (pseudo-strings), you >>>>>>>>>> would need either to (a) reconstitute the patches array >>>>>>>>>> handed to Unsafe.defineAnonymousClass, or (b) accept >>>>>>>>>> whatever odd strings were there first, as an approximation. >>>>>>>>>> The "odd strings" are totally insignificant, and are >>>>>>>>>> typically something like "CONSTANT_PLACEHOLDER_42" >>>>>>>>>> (see java/lang/invoke/InvokerBytecodeGenerator.java)." >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Reconstitution of the ConstantPool is needed for both the >>>>>>>>>> JVMTI GetConstantPool() and RetransformClasses(). >>>>>>>>>> Finally, it goes to the ConstantPool::copy_cpool_bytes(). >>>>>>>>>> >>>>>>>>>> The problem is that a pseudo-string is a patched string >>>>>>>>>> that does not have >>>>>>>>>> a reference to the string symbol anymore: >>>>>>>>>> unresolved_string_at(idx) == NULL >>>>>>>>>> >>>>>>>>>> The fix is to create and fill in a map from >>>>>>>>>> JVM_CONSTANT_String cp index to the JVM_CONSTANT_Utf8 cp index >>>>>>>>>> to be able to restore this assotiation in the >>>>>>>>>> JvmtiConstantPoolReconstituter. >>>>>>>>>> >>>>>>>>>> Testing: >>>>>>>>>> Run: >>>>>>>>>> - java/lang/instrument tests >>>>>>>>>> - new jtreg test (see webrev) that was written by Filipp >>>>>>>>>> Zhinkin >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Serguei >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From n210241048576 at gmail.com Tue Jan 13 06:27:26 2015 From: n210241048576 at gmail.com (Robert Grosse) Date: Tue, 13 Jan 2015 06:27:26 -0000 Subject: Regarding the ability to catch exceptions on final instruction with exclusive end_pc Message-ID: The JVM specification says The fact that end_pc is exclusive is a historical mistake in the design of the Java Virtual Machine: if the Java Virtual Machine code for a method is exactly 65535 bytes long and ends with an instruction that is 1 byte long, then that instruction cannot be protected by an exception handler. A compiler writer can work around this bug by limiting the maximum size of the generated Java Virtual Machine code for any method, instance initialization method, or static initializer (the size of any code array) to 65534 bytes. However, code_length is actually limited to 65535, which means the final instruction is at index 65534 and can be covered by an exception handler with no problems. To confirm this, I tested it myself. A method with 65536 bytes of bytecode is rejected. A method with exactly 65535 bytes of bytecode ending in a one byte athrow instruction still successfully catches the thrown exception with an exception handler where end_pc=65535. Does anyone know why the specification says this? What is going on? From peter.levart at gmail.com Wed Jan 28 15:59:38 2015 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 28 Jan 2015 16:59:38 +0100 Subject: Useful message about NullPointerException In-Reply-To: <54C849B1.80403@oracle.com> References: <1421844308100-213240.post@n7.nabble.com> <54C78F44.1030104@redhat.com> <54C7B653.1070509@gmail.com> <54C849B1.80403@oracle.com> Message-ID: <54C9076A.2060903@gmail.com> On 01/28/2015 03:30 AM, David Holmes wrote: >> If you have access to sources, then perhaps an easier solution would be >> for stack traces to include column number in addition to line number of >> location in source that resulted in bytecodes that include the one that >> triggered the NPE. >> >> There is already a RFE for that: >> >> https://bugs.openjdk.java.net/browse/JDK-8020204 > > Once past suggestion has been to include the ByteCode Index (BCI) as > part of the exception stacktrace information: > > https://bugs.openjdk.java.net/browse/JDK-4185378 > > David Right, I checked per-method CharacterRangeTable that gets emitted by javac -Xjcov option, and unfortunately it is of limited use. The CharacterRangeTable contains mappings from byte code index ranges (start-bci, end-bci) -> character ranges (start-line, start-column, end-line, end-column) of adequate code in source file. The ranges I have observed have 2 granularities: "statement" and "block". For example, the following program: public class CharRangeTest { String str() { return "ABC"; } public static void main(String[] args) { int i = new CharRangeTest().str().substring(1).length(); System.out.println(i); } } Compiles to the following bytecodes for main method: public static void main(java.lang.String[]); descriptor: ([Ljava/lang/String;)V flags: ACC_PUBLIC, ACC_STATIC Code: stack=2, locals=2, args_size=1 0: new #3 // class CharRangeTest 3: dup 4: invokespecial #4 // Method "":()V 7: invokevirtual #5 // Method str:()Ljava/lang/String; 10: iconst_1 11: invokevirtual #6 // Method java/lang/String.substring:(I)Ljava/lang/String; 14: invokevirtual #7 // Method java/lang/String.length:()I 17: istore_1 18: getstatic #8 // Field java/lang/System.out:Ljava/io/PrintStream; 21: iload_1 22: invokevirtual #9 // Method java/io/PrintStream.println:(I)V 25: return LineNumberTable: line 11: 0 line 12: 18 line 13: 25 CharacterRangeTable: 0, 17, 2c09, 2c41, 1 // 0, 17, 11:09, 11:65, statement 18, 24, 3009, 301f, 1 // 18, 24, 12:09, 12:31, statement 0, 25, 282c, 3406, 2 // 0, 25, 10:44, 13:06, block CharacterRangeTable says that bytecodes at indexes 0 to 17 map to source code from (line 11, column 9) to (line 11, column 64) and that this range is a "statement". Unfortunately, the assignment to i of the result of the whole chain of invocations is a single Java "statement": int i = new CharRangeTest().str().substring(1).length(); So if NPE happens anywhere in this statement, we could only pin-point the statement and not a particular null dereference. The only time this would give us some additional info is when there are more statements in the line, like: x.doSomething(); y.doSomeMore(); ...but such code formatting is very rare if non-existent. If adding column number to StackTraceElement (JDK-8020204) is currently not an easy thing to do, since it would require javac changes, adding byte code index (JDK-4185378 ) is trivial: http://cr.openjdk.java.net/~plevart/jdk9-dev/StackTraceElement.byteCodeIndex/jdk.webrev.01/ http://cr.openjdk.java.net/~plevart/jdk9-dev/StackTraceElement.byteCodeIndex/hotspot.webrev.01/ With this patch, I get the following style of stack-traces: Exception in thread "main" java.io.FileNotFoundException: /tmp/x (No such file or directory) at java.io.FileInputStream.open0(Native Method) at java.io.FileInputStream.open[2](FileInputStream.java:195) at java.io.FileInputStream.[97](FileInputStream.java:138) at java.io.FileInputStream.[17](FileInputStream.java:93) at Test.dump[9](Test.java:12) at Test.main[3](Test.java:19) The numbers in square brackets are byte code indexes that pin-point the location in byte code. With the help of javap, one can use this info to find out the cause of exception even without having access to sources. Regards, Peter From pointo1d at gmail.com Mon Jan 19 11:00:11 2015 From: pointo1d at gmail.com (Dave Pointon) Date: Mon, 19 Jan 2015 11:00:11 +0000 Subject: RFR (XS) 8031064: build_vm_def.sh not working correctly for new build cross compile In-Reply-To: <54B8EC3F.5030408@oracle.com> References: <54B6FA6C.40202@oracle.com> <54B75EB5.3020407@oracle.com> <54B760DB.70200@oracle.com> <54B765DC.8050801@oracle.com> <54B845E3.7060000@oracle.com> <54B8EC3F.5030408@oracle.com> Message-ID: Hi all , IMO, we should be fine as long as we continue to restrict ourselves to the use of Bourne syntax i.e. avoid wandering into the realms of the quirks provided by bash, in both make recipes and the utility & tool scripts - I say this only because shell shock affected certain platforms such that bash is not available on those platform(s) e.g. AIX. Just my 10c FWIW .... -- Dave Pointon FIAP MBCS - ? Contractor engaged by IBM ?? Now I saw, tho' too late, the folly of beginning a work before we count the cost and before we we judge rightly of our strength to go thro' with it - Robinson Crusoe On 16 January 2015 at 10:47, Dean Long wrote: > On 1/15/2015 2:57 PM, Magnus Ihse Bursie wrote: > >> On 2015-01-15 08:01, David Holmes wrote: >> >>> >>> If the build-dev guys confirm we already assume bash that is fine. >>> >> >> For the rest of the world, we only use bash. For hotspot, we will use >> bash if called from the top-level Makefile. I can't say anything about what >> the convention have been for calling the hotspot makefiles directly, though. >> >> /Magnus >> > > "sh" or "bash", it shouldn't matter. Our makefiles are already using > Bourne shell syntax, > so if someone tries to use "make SHELL=csh", it will fail almost > immediately. Therefore, > I think "NM=$(NM) sh ..." should be safe. > > dl > > https://www.gnu.org/software/make/manual/make.html#Choosing-the-Shell > From pslack at wavedna.com Thu Jan 15 15:15:37 2015 From: pslack at wavedna.com (Peter J Slack) Date: Thu, 15 Jan 2015 10:15:37 -0500 Subject: De-universalizing hotspot in jdk8u In-Reply-To: <54B47791.2040301@oracle.com> References: <54B300FE.2000804@oracle.com> <54B3A4D0.3080604@oracle.com> <54B3AF6D.20607@oracle.com> <9B1E95A7-8496-4E59-9B00-FCC1F4B780FA@apple.com> <54B47791.2040301@oracle.com> Message-ID: Apologies in advance, I felt necessary to send up some smoke signals on this 32 bit feature decision. I am grateful for this hard working and diligent Java community and for Java itself, I'm a staunch Java supporter and long time Java programmer. If one considers marketing and product placement of Java, the removal of 32 bit Java for Mac OSX creates a perceived limitation of a platform that is supposed to be write once, run anywhere, and a real limitation for developers who rely on the Java platform to integrate with libraries that are used in other applications. The removal of a 32 bit option seems to be counter culture to the Java community and "write once, run anywhere" brand. Java 1.9 is the only version that works properly for us on Mac OSX at the moment, the migration of 32 bit Java to legacy has been a sales killer for our little company, so these types of decisions are never without any consequences. Instead of designing killer apps Mac OSX using Java, I have to write installer code and spend time helping upset customers navigate through these moving goal posts. Whew!! .. OK rant complete the smoke is starting to clear and we will make it through this ;-) PJ On Mon, Jan 12, 2015 at 8:40 PM, David Holmes wrote: > On 13/01/2015 5:20 AM, Alex Strange wrote: > >> >> On Jan 12, 2015, at 3:26 AM, David Holmes >>> wrote: >>> >>> cc'ing macosx-port-dev and hotspot-dev >>> >>> On 12/01/2015 8:41 PM, Erik Joelsson wrote: >>> >>>> >>>> On 2015-01-12 00:02, David Holmes wrote: >>>> >>>>> Hi David, >>>>> >>>>> On 10/01/2015 2:00 AM, David DeHaven wrote: >>>>> >>>>>> >>>>>> We have this nice little comment in common/autoconf/jdk-options.m4: >>>>>> # On Macosx universal binaries are produced, but they only contain >>>>>> # 64 bit intel. This invalidates control of which jvms are built >>>>>> # from configure, but only server is valid anyway. Fix this >>>>>> # when hotspot makefiles are rewritten. >>>>>> if test "x$MACOSX_UNIVERSAL" = xtrue; then >>>>>> HOTSPOT_TARGET=universal_${HOTSPOT_EXPORT} >>>>>> fi >>>>>> >>>>>> >>>>>> So.. I turned it off: >>>>>> if test "x$OPENJDK_TARGET_OS" = "xmacosx"; then >>>>>> # MACOSX_UNIVERSAL="true" >>>>>> MACOSX_UNIVERSAL="false" >>>>>> fi >>>>>> >>>>>> And in hotspot/make/bsd/makefiles/defs.make: >>>>>> # Universal build settings >>>>>> ifeq ($(OS_VENDOR), Darwin) >>>>>> # Build universal binaries by default on Mac OS X >>>>>> # MACOSX_UNIVERSAL = true >>>>>> MACOSX_UNIVERSAL = false >>>>>> ifneq ($(ALT_MACOSX_UNIVERSAL),) >>>>>> >>>>>> hotspot still seems to build happily (I'm kicking off tests now). Is >>>>>> there are particular reason this hasn't been done yet? I'd like to >>>>>> know before I pursue this as a means of eliminating our dependency on >>>>>> lipo which is causing an inordinate amount of grief when trying to >>>>>> build on 10.9 and later when Xcode 5+ is coinstalled. I have some >>>>>> logic to work around using the broken /usr/bin/lipo, but it doesn't >>>>>> help later in some "other" build target that ends up using '-arch >>>>>> i386 -arch x86_64'. It does not explicitly run lipo, it relies on gcc >>>>>> to do the fattening itself and so it fails even though I've gone to >>>>>> the effort of working around the broken lipo (and it isn't necessary >>>>>> anyways because it doesn't even build the i386 binaries even though >>>>>> it's supposed to be "universal"). >>>>>> >>>>> >>>>> The hotspot makefiles still haven't been rewritten (though Magnus had >>>>> been working on it). >>>>> >>>>> Also I was under the impression that we used the universal stuff >>>>> because we had to deliver in a particular way on OSX. But I don't know >>>>> the details and the people who dealt with the original OSX port effort >>>>> are no longer here. >>>>> >>>>> I don't know why Hotspot is packaged as a universal binary. In the >>>> jdk, >>>> only libJObjC was built as a universal binary and that lib was removed >>>> before JDK 8 shipped. What part of Macosx would need to interact >>>> directly with libjvm.dylib in a way that required a (fake) universal >>>> format, but no other libs in the jdk? I would say go ahead and change >>>> it. >>>> >>> >>> In the spirit of "never take down a fence until you know why it was put >>> up in the first place" I've cc'd macosx-port-dev to see if there is anyone >>> around who recalls why hotspot is using the universal binary feature. >>> >>> David H. >>> ------- >>> >> >> I did the original universal binary work for hotspot when bringing up >> macosx-port. >> >> At the time we were building and testing JDK 7 as universal (i386+x86-64) >> since e.g. some apps had JNI code that was only built 32-bit. >> The other jdk components didn't need any makefile work, because the >> compiler can build them universal by itself, but hotspot autogenerates a >> lot of arch-specific code and it was easier to build it twice and glue them >> together in the makefile. >> >> As long as Java is only been shipped 64-bit these days, I personally >> don't see a reason to keep it. >> > > Thanks for the info Alex. Based on that and Dan's response this proposed > change seems good to go ahead. > > David H. > > > >> /Erik >>> >>>> >>>> David H. >>>>> >>>>> >>>>>> Quite frankly I'd rather just remove all the universal binary stuff >>>>>> from hotspot, but I'm trying my best to minimize the changes there... >>>>>> the "other" problem I can handle easily. >>>>>> >>>>>> -DrD- >>>>>> >>>>>> >>>> >> -- Senior Software Developer / IT Administrator Work: (416) 466-9283 Fax : (866) 855-2605 From serguei.spitsyn at oracle.com Wed Jan 28 18:57:23 2015 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 28 Jan 2015 10:57:23 -0800 Subject: 5-rd round RFR (S) 8008678: JSR 292: constant pool reconstitution must support pseudo strings In-Reply-To: <54C8F1FF.3050301@oracle.com> References: <5475968C.40800@oracle.com> <54760B40.9040206@oracle.com> <54762FC9.2010800@oracle.com> <5491BCB9.10505@oracle.com> <5493299F.60005@oracle.com> <54934E73.4080808@oracle.com> <54B96F97.5070204@oracle.com> <54B99682.8070802@oracle.com> <54B99838.10601@oracle.com> <54C825CE.1030105@oracle.com> <54C8F1FF.3050301@oracle.com> Message-ID: <54C93113.7000308@oracle.com> Thank you, Coleen! Serguei On 1/28/15 6:28 AM, Coleen Phillimore wrote: > > Yes, this looks great. Ship it! > Coleen > > On 1/27/15, 6:57 PM, serguei.spitsyn at oracle.com wrote: >> This is 5-th round review of for: >> https://bugs.openjdk.java.net/browse/JDK-8008678 >> >> There was one (4-rd) private review discussion. The result is this >> webrev: >> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8008678-JVMTI-pseudo.5/ >> >> >> Summary: >> Current fix is to use the 2-nd LSB bit in the CPSlot to mark >> pseudo-strings. >> A similar version of webrev was already reviewed by Coleen and John. >> The update is that it includes John's suggestion to introduce the >> CPSlot::TagBits to make code more clean. >> Also, I've discovered that there is no need to update the >> SymbolClosure::load_symbol() in iterator.hpp >> because the pseudo-string PCSlot is not used in the SymbolClosure. >> I need a final thumbs up for the push. >> >> Testing: >> Run: >> - nsk.jvmti.testlist, nsk.jdi.testlist, vm.mlvm.testlist >> - java/lang/instrument, com/sun/jdi and hotspot/test/runtime tests >> - new jtreg test (see webrev) that was written by Filipp Zhinkin >> >> >> Thanks, >> Serguei >> >> On 1/16/15 3:01 PM, serguei.spitsyn at oracle.com wrote: >>> John R. suggested to use the CPSlot(Symbol* ptr) to mark pseudo-strings. >>> The updated webrev is going to be close to the .3 webrev. >>> I will send it soon. >>> >>> Thanks, >>> Serguei >>> >>> On 1/16/15 2:53 PM, Coleen Phillimore wrote: >>>> >>>> This change looks good to me also. >>>> Thanks, >>>> Coleen >>>> >>>> On 1/16/15, 3:07 PM, serguei.spitsyn at oracle.com wrote: >>>>> Please, review the fix for: >>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>> >>>>> >>>>> Open webrev: >>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2015/hotspot/8008678-JVMTI-pseudo.3/ >>>>> >>>>> >>>>> Summary: >>>>> Currently, a JVM_CONSTANT_String CP entry having a NULL >>>>> reference to Symbol* >>>>> indicates that it is a pseudo-string (patched string). >>>>> This creates nasty issues for the constant pool reconstitution. >>>>> >>>>> Current suggestion is to avoid having a NULL reference and >>>>> retain the original >>>>> Symbol* reference for pseudo-strings. The pseudo-string >>>>> indication will be >>>>> if the Utf8 representation starts from "CONSTANT_PLACEHOLDER_". >>>>> This approach makes the fix much simpler. >>>>> >>>>> I need a confirmation from the Compiler team that this won't >>>>> break any assumptions or invariants. >>>>> Big thanks to Coleen for previous round reviews and good advices! >>>>> >>>>> >>>>> Testing: >>>>> Run: >>>>> - java/lang/instrument tests >>>>> - new jtreg test (see webrev) that was written by Filipp Zhinkin >>>>> >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 12/18/14 2:00 PM, serguei.spitsyn at oracle.com wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> Thank you for reviewing! >>>>>> >>>>>> >>>>>> On 12/18/14 11:23 AM, Coleen Phillimore wrote: >>>>>>> >>>>>>> Hi Serguei, >>>>>>> >>>>>>> Thank you for making the patches an optional field. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/oops/constantPool.hpp.sdiff.html >>>>>>> 198 if (!patched()) { >>>>>>> 199 assert(false, "a pseudo-string map may exists for patched CP only"); >>>>>>> 200 return 0; >>>>>>> 201 } >>>>>>> Why not >>>>>>> assert(patched(), "a pseud-string map must exist >>>>>>> for patched CP only"); >>>>>> >>>>>> Wanted it to be more reliable but it looks pointless. >>>>>> Will make this change. >>>>>> >>>>>>> >>>>>>> >>>>>>> Why is this? Is this really a ShouldNotReachHere? should it be >>>>>>> false? >>>>>>> >>>>>>> 215 assert(true, "not found a matching entry in pseudo-string map"); >>>>>> >>>>>> >>>>>> A typo, must be false. >>>>>> It is the last minute change. >>>>>> Thanks for the catch! >>>>>> >>>>>> >>>>>>> >>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/src/share/vm/prims/jvmtiRedefineClasses.cpp.udiff.html >>>>>>> >>>>>>> Don't you have to move the value of the patched field from the >>>>>>> old constant pool to the new one? I hate to ask but is there >>>>>>> merging that needs to be done? I don't know how to write this >>>>>>> test case though. Is it possible to redefine a class with a >>>>>>> constant pool patches with another that has constant pool patches? >>>>>> >>>>>> Thank you for asking this question. >>>>>> If I understand correctly, the patching comes from the compiler >>>>>> side for anonymous classes only. >>>>>> I saw it for LambdaForm's only. >>>>>> I think, the patching can not be changed with a retransformation. >>>>>> But I'm not sure if it can not be changed with a redefinition. >>>>>> >>>>>> But if it can - then it would be safe to merge the 'patched' >>>>>> condition, i.e. make it patched >>>>>> if either the_class or scratch class is patched. >>>>>> >>>>>>> >>>>>>> Somehow I thought you'd have to save the value of the cp_patches >>>>>>> oops passed in. >>>>>>> >>>>>>> So I was wondering why you can't change this instead: >>>>>>> >>>>>>> bool is_pseudo_string_at(int which) { >>>>>>> // A pseudo string is a string that doesn't have a symbol in >>>>>>> the cpSlot >>>>>>> - return unresolved_string_at(which) == NULL; >>>>>>> + return (strncmp(unresolved_string_at(which)->as_utf8(), >>>>>>> "CONSTANT_PLACEHOLDER_" , strlen("CONSTANT_PLACEHOLDER")) == 0); >>>>>>> } >>>>>> >>>>>> I was thinking about the same but was not sure if it would work >>>>>> for the compiler team. >>>>>> We have to ask John about this (added John and Christian to the >>>>>> cc-list). >>>>>> This question to John was in my plan! :) >>>>>> >>>>>> The beauty of the above approach is that there is no need to >>>>>> create an intermediate >>>>>> pseudo-string map and most of the code in from this webrev is not >>>>>> needed. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>>> >>>>>>> And the asserts in the other functions below it. >>>>>>> >>>>>>> Coleen >>>>>>> >>>>>>> >>>>>>> On 12/17/14, 12:26 PM, serguei.spitsyn at oracle.com wrote: >>>>>>>> Please, review the second round fix for: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>>>>> >>>>>>>> Open webrev: >>>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.2/ >>>>>>>> >>>>>>>> >>>>>>>> Summary: >>>>>>>> >>>>>>>> This fix implements a footprint saving approach suggested by >>>>>>>> Coleen. >>>>>>>> To be able to reconstitute a class constant pool, an >>>>>>>> intermediate pseudo-string map is used. >>>>>>>> Now, this field is accounted optionally, only if the >>>>>>>> 'cp_patches' is provided in the >>>>>>>> ClassFileParser::parseClassFile() before ConstantPool is >>>>>>>> allocated. >>>>>>>> This fix is not elegant, even a little bit ugly, but it is >>>>>>>> the only way I see so far. >>>>>>>> >>>>>>>> Unfortunately, this approach did not help much to make some >>>>>>>> other fields (eg., 'operands') optional. >>>>>>>> The problem is that we have to account optional fields before >>>>>>>> parsing, at the CP allocation time. >>>>>>>> It is possible to re-allocate the ConstantPool when any >>>>>>>> InvokeDynamic bytecode is discovered, >>>>>>>> but it looks too complicated. >>>>>>>> >>>>>>>> Testing: >>>>>>>> - the unit test from bug report >>>>>>>> - nsk.jvmti,testlist, nsk.jdi.testlist, JTREG >>>>>>>> java/lang/instrument >>>>>>>> - vm.mlvm.testlist, vm.quick.testlist, >>>>>>>> vm.parallel_class_loading.testlist (in progress) >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> >>>>>>>> On 11/26/14 11:53 AM, serguei.spitsyn at oracle.com wrote: >>>>>>>>> Coleen, >>>>>>>>> >>>>>>>>> Thank you for looking at this! >>>>>>>>> I'll check how this can be improved. >>>>>>>>> It is my concern too. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Serguei >>>>>>>>> >>>>>>>>> On 11/26/14 9:17 AM, Coleen Phillimore wrote: >>>>>>>>>> >>>>>>>>>> Serguei, >>>>>>>>>> I had a quick look at this. I was wondering if we could make >>>>>>>>>> the pseudo_string_map conditional in ConstantPool and not >>>>>>>>>> make all classes pay in footprint for this field? The same >>>>>>>>>> thing probably could be done for operands too. There are >>>>>>>>>> flags that you can set to conditionally add a pointer to >>>>>>>>>> base() in this function. >>>>>>>>>> >>>>>>>>>> Typical C++ would subclass ConstantPool to add >>>>>>>>>> InvokeDynamicConstantPool fields, but this is not typical C++ >>>>>>>>>> so the trick we use is like the one in ConstMethod. I think >>>>>>>>>> it's worth doing in this case. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Coleen >>>>>>>>>> >>>>>>>>>> On 11/26/14, 3:59 AM, serguei.spitsyn at oracle.com wrote: >>>>>>>>>>> Please, review the fix for: >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8008678 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Open webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2014/hotspot/8008678-JVMTI-pseudo.1/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Summary: >>>>>>>>>>> The pseudo-strings are currently not supported in >>>>>>>>>>> reconstitution of constant pool. >>>>>>>>>>> >>>>>>>>>>> This is an explanation from John Rose about what the >>>>>>>>>>> pseudo-strings are: >>>>>>>>>>> >>>>>>>>>>> "We still need "live" oop constants pre-linked into the >>>>>>>>>>> constant pool of bytecodes which >>>>>>>>>>> implement some method handles. We use the anonymous >>>>>>>>>>> class pseudo-string feature for that. >>>>>>>>>>> The relevant code is here: >>>>>>>>>>> http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/tip/src/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java >>>>>>>>>>> >>>>>>>>>>> These oops are what "pseudo-strings" are. >>>>>>>>>>> The odd name refers to the fact that, even though they >>>>>>>>>>> are random oops, they appear in the constant pool >>>>>>>>>>> where one would expect (because of class file syntax) to >>>>>>>>>>> find a string." >>>>>>>>>>> ... >>>>>>>>>>> If you really wanted to reconstitute a class file for an >>>>>>>>>>> anonymous class, and >>>>>>>>>>> if that class has oop patching (pseudo-strings), you >>>>>>>>>>> would need either to (a) reconstitute the patches array >>>>>>>>>>> handed to Unsafe.defineAnonymousClass, or (b) accept >>>>>>>>>>> whatever odd strings were there first, as an approximation. >>>>>>>>>>> The "odd strings" are totally insignificant, and are >>>>>>>>>>> typically something like "CONSTANT_PLACEHOLDER_42" >>>>>>>>>>> (see java/lang/invoke/InvokerBytecodeGenerator.java)." >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Reconstitution of the ConstantPool is needed for both the >>>>>>>>>>> JVMTI GetConstantPool() and RetransformClasses(). >>>>>>>>>>> Finally, it goes to the ConstantPool::copy_cpool_bytes(). >>>>>>>>>>> >>>>>>>>>>> The problem is that a pseudo-string is a patched string >>>>>>>>>>> that does not have >>>>>>>>>>> a reference to the string symbol anymore: >>>>>>>>>>> unresolved_string_at(idx) == NULL >>>>>>>>>>> >>>>>>>>>>> The fix is to create and fill in a map from >>>>>>>>>>> JVM_CONSTANT_String cp index to the JVM_CONSTANT_Utf8 cp index >>>>>>>>>>> to be able to restore this assotiation in the >>>>>>>>>>> JvmtiConstantPoolReconstituter. >>>>>>>>>>> >>>>>>>>>>> Testing: >>>>>>>>>>> Run: >>>>>>>>>>> - java/lang/instrument tests >>>>>>>>>>> - new jtreg test (see webrev) that was written by Filipp >>>>>>>>>>> Zhinkin >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Serguei >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From david.holmes at oracle.com Wed Jan 28 22:48:44 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 29 Jan 2015 08:48:44 +1000 Subject: RFR: 7143664: Clean up OrderAccess implementations and usage In-Reply-To: <54C87D16.40601@oracle.com> References: <54C1EBD3.1000905@oracle.com> <54C1F3D8.5040507@oracle.com> <54C2C1EA.5020507@oracle.com> <54C87D16.40601@oracle.com> Message-ID: <54C9674C.2080600@oracle.com> Updated webrev: http://cr.openjdk.java.net/~dholmes/7143664/webrev.v2/ Thanks, David On 28/01/2015 4:09 PM, David Holmes wrote: > Hi Erik, > > Sorry for the delay, it was a long weekend here and I'm playing catch > up. :) > > On 24/01/2015 9:03 AM, Erik ?sterlund wrote: >> Hi Dean, >> >> On 23/01/15 22:50, Dean Long wrote: >>> On 1/23/2015 7:16 AM, Erik ?sterlund wrote: >>>> Hi Dean and David, >>>> >>>> Thanks for reviewing this change. >>>> >>>> On 23/01/15 08:12, David Holmes wrote: >>>>> On 23/01/2015 4:36 PM, Dean Long wrote: >>>>>> 147 // Ordering a load relative to preceding stores requires >>>>>> a fence, >>>>>> 148 // which implies a membar #StoreLoad between the store >>>>>> and load under >>>>>> 149 // sparc-TSO. A fence is required by x86. On x86, we >>>>>> use explicitly >>>>>> 150 // locked add. >>>>>> 151 // >>>>>> >>>>>> It sounds like the above is saying that fence is the same as >>>>>> StoreLoad ... >>>>> That may be a s/store_fence/fence/ typo. The original text was: >>>>> >>>>> // Ordering a load relative to preceding stores requires a >>>>> store_fence, >>>>> // which implies a membar #StoreLoad between the store and load under >>>>> // sparc-TSO. A fence is required by ia64. On x86, we use locked >>>>> xchg. >>>>> >>>>> Actually seems like a couple of typos there: ia64 became x86, then we >>>>> have x86 again. >>>> I tried to remove ia64 from the comments as it is not in the >>>> repository, >>>> which in this case became a bit redundant. Are we okay if I remove the >>>> "A fence is required by x86" sentence? >>>> >>>> Oh and Dean about membar #StoreLoad on SPARC and locked add on x86... >>>> are you saying they are not equivalent? >>> >>> I wasn't trying to say anything abou SPARC or x86, only that "fence" != >>> "StoreLoad". >> >> Agreed. >> >>>> New version of that paragraph with that sentence removed: >>>> >>>> 147 // Ordering a load relative to preceding stores requires a fence, >>>> 148 // which implies a membar #StoreLoad between the store and load >>>> under >>>> 149 // sparc-TSO. On x86, we use explicitly locked add. >>> >>> I would replace "fence" with "StoreLoad" above. >> >> Again, I just changed any store_fence descriptions to use fence instead. >> But you are certainly right, it is more accurate to talk about StoreLoad >> here and not mention fence in the first place, I'll change this, thanks >> for the feedback. :) >> >> Changed to: >> >> 147 // Ordering a load relative to preceding stores requires a StoreLoad, >> 148 // which implies a membar #StoreLoad between the store and load under >> 149 // sparc-TSO. On x86, we use explicitly locked add. > > The confusion here is what "store-fence" was intended to mean. In this > part we're implying it is a storeLoad, but in the discussion below it > indicates store-fence orders subsequent loads and stores - which makes > it a storeStore|storeLoad barrier. Of course storeLoad subsumes > storeStore so the above is accurate - though I think the original text > would have been more clearly stated as "Ordering a load relative to > preceding stores _can be achieved using a_ store-fence, ..." > > So in summary: update above is okay. > >>>>>> 152 // 4. store, load <= is constrained by => store, fence, >>>>>> load >>>>>> 153 // >>>>>> 154 // Use store, fence to make sure all stores done in an >>>>>> 'interesting' >>>>>> 155 // region are made visible prior to both subsequent >>>>>> loads and stores. >>>>>> >>>>>> ... and this is like saying to use fence because StoreStore | >>>>>> StoreLoad >>>>>> isn't available. >>>>> Again this seems an issue with store_fence being changed to "store, >>>>> fence" which doesn't really make sense. >>>> The thing is that I removed store_fence (not store_release as it >>>> says in >>>> the bug ID, sorry about that - there obviously is no store_release). So >>>> I wanted to remove references in the comments to store_fence as well. >>>> Therefore I changed the example to use store fence load instead of >>>> store_fence load. Perhaps it's better if I remove that sample all >>>> together instead? >>> >>> That works for me. >> >> Fixed. > > Removal is an option but again replacing store-fence with what it really > means "store; membar" would also suffice. But if > we don't need the combined membar such that > abstracting it into something like store_fence() makes sense, then > getting rid of everything pertaining to store-fence makes sense. I see > there are no uses of store_fence() other than a commented out reference > in ./cpu/x86/vm/c1_LIRAssembler_x86.cpp, which also contains a similar > commented out reference to the non-existent load_fence() - both of which > should be removed. > > If you'd like to send me an updated patch I will generate a new webrev. > Then I can get on with the functional part of the review. > > Thanks, > David > >> Thanks, >> /Erik >> >>> dl >>> >>>> Thanks for reviewing my changes! >>>> >>>> /Erik >>>> >>>>> David >>>>> >>>>>>> Furthermore, store_release was declared private and marked as >>>>>> deprecated. >>>>>> >>>>>> I can't find where this was done. >>>>>> >>>>>> dl >>>>>> >>>>>> On 1/22/2015 5:19 PM, Erik ?sterlund wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> == Summary of Changes == >>>>>>> >>>>>>> This changeset fixes issues in OrderAccess on multiple levels >>>>>>> from the >>>>>>> memory model semantics to compiler reorderings, to addressing >>>>>>> maintainability/portability issues which (almost) had to be fixed in >>>>>>> order to fix the correctness issues. It is the result of discussions >>>>>>> found in the previous "OrderAccess Refactoring" thread: >>>>>>> http://openjdk.5641.n7.nabble.com/OrderAccess-Refactoring-td212050.html >>>>>>> >>>>>>> >>>>>>> Bug ID: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-7143664 >>>>>>> (updated to reflect these related changes) >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~dholmes/7143664/webrev/ >>>>>>> >>>>>>> Before I describe more I would like to give special thanks to David >>>>>>> Holmes for long discussions leading up to the currently proposed >>>>>>> changes. I would also like to thank Jesper Wilhelmsson for >>>>>>> helping me >>>>>>> run my changes through JPRT. >>>>>>> >>>>>>> == Motivation == >>>>>>> >>>>>>> This change directly fixes a reported OrderAccess bug due to >>>>>>> compiler >>>>>>> reorderings which is still a vulnerability on almost all TSO >>>>>>> platforms: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-806196 >>>>>>> >>>>>>> And directly fixes confusions like release_store() != release() >>>>>>> store() >>>>>>> due to memory model issues previously described in this bug ID. >>>>>>> >>>>>>> At the same time it provides clearer design with separation of >>>>>>> concerns >>>>>>> and generalization/specialization, removing a whole bunch of >>>>>>> platform >>>>>>> specific code which could be generalized. The platform specific >>>>>>> files >>>>>>> now only have a few LoC requirements (~7) to conform to the >>>>>>> memory model >>>>>>> by specifying what the stand alone barriers do. Then optionally >>>>>>> optimizations to the general approach are possible if platforms >>>>>>> support >>>>>>> it. This also makes it much easier to port to new platforms. >>>>>>> >>>>>>> == Memory Model == >>>>>>> >>>>>>> The current definitions of acquire/release semantics are a bit fishy >>>>>>> leading to problems previously described in the bug ID >>>>>>> (release_store() >>>>>>> != release() store()) and some other correctness issues. It has >>>>>>> therefore been replaced with a new model. I would like to thank >>>>>>> David >>>>>>> Holmes for the long discussions leading up to the newly proposed >>>>>>> model. >>>>>>> >>>>>>> The new model is formally defined like this: >>>>>>> >>>>>>> // T1: access_shared_data >>>>>>> // T1: ]release >>>>>>> // T1: (...) >>>>>>> // T1: store(X) >>>>>>> // >>>>>>> // T2: load(X) >>>>>>> // T2: (...) >>>>>>> // T2: acquire[ >>>>>>> // T2: access_shared_data >>>>>>> // >>>>>>> // It is guaranteed that if T2: load(X) synchronizes with >>>>>>> (observes the >>>>>>> // value written by) T1: store(X), then the memory accesses >>>>>>> before the >>>>>>> // T1: ]release happen before the memory accesses after the T2: >>>>>>> acquire[. >>>>>>> >>>>>>> The orderAccess.hpp file and bug ID also has a few additional >>>>>>> explanations making it more intuitive to the user when to use >>>>>>> acquire/release and the resemblance to TSO abstract machines. Big >>>>>>> thanks >>>>>>> goes to David Holmes for discussing the memory model with me, which >>>>>>> helped a lot in deriving it. >>>>>>> >>>>>>> Now it holds that release() store() == release_store(), and the >>>>>>> other >>>>>>> correctness issues are fixed as well. >>>>>>> >>>>>>> The new model is also closer to C++11 definitions which could >>>>>>> give us >>>>>>> more relaxed compiler reordering constraints in the future when >>>>>>> compiler >>>>>>> support for C++11 is there and ready. >>>>>>> >>>>>>> == Reliance on C++ Volatile Semantics == >>>>>>> >>>>>>> The C++ standard section 1.9 "Program Execution" is very vague about >>>>>>> what the keyword volatile can actually do for us. It gives clear >>>>>>> constraints in terms of volatile-volatile accesses but says >>>>>>> little about >>>>>>> nonvolatile-volatile accesses. Yet the current implementation >>>>>>> heavily >>>>>>> relies upon volatile to in terms of compiler reordering. But GCC >>>>>>> explicitly declares that volatiles and non-volatiles may reorder >>>>>>> freely >>>>>>> ( https://gcc.gnu.org/onlinedocs/gcc/Volatiles.html ). The only >>>>>>> compiler >>>>>>> known to explicitly provide the wanted semantics with volatile is >>>>>>> MSVC >>>>>>> 2010 for windows ( >>>>>>> https://msdn.microsoft.com/en-us/library/12a04hfd(v=vs.100).aspx ). >>>>>>> Compilers not giving explicit guarantees, must be considered >>>>>>> unsafe and >>>>>>> revert to conservative behaviour. >>>>>>> >>>>>>> This was brought to attention after causing bugs, but was only >>>>>>> fixed for >>>>>>> x86 linux. This is a fundamental issue inherent to all TSO platforms >>>>>>> except windows, and has to be fixed on all of them. >>>>>>> >>>>>>> Several barriers are unsafe to use because they lack compiler >>>>>>> reordering >>>>>>> constraints (e.g. fence and acquire on linux_SPARC). For TSO >>>>>>> platforms >>>>>>> they are typically implemented using dummy loads and stores. This >>>>>>> seems >>>>>>> to be another old volatile reliance that I fixed. These barriers >>>>>>> sometimes have omitted compiler barriers (which is what we really >>>>>>> want). >>>>>>> This seems to be another example on incorrect reliance on the >>>>>>> volatile >>>>>>> semantics to help us. Therefore dummy loads/stores have been >>>>>>> replaced >>>>>>> with compiler barriers on TSO platforms. >>>>>>> >>>>>>> It is also worth noting that compilers like sun studio did >>>>>>> previously >>>>>>> not support inline asm syntax. Therefore, barriers were >>>>>>> implemented in >>>>>>> .il-files. However, using them does not give explicit compiler >>>>>>> constraints for reordering AFAIK. Therefore, they have been >>>>>>> reimplemented using inline asm with explicit compiler reordering >>>>>>> constraints, as even sun (solaris?) studio now supports this. >>>>>>> >>>>>>> The windows variants have added a windows-style _ReadWriteBarrier() >>>>>>> compiler barrier similarly. >>>>>>> >>>>>>> == Strange Hardware Reorderings == >>>>>>> >>>>>>> Fixed a weird inconsistency where acquire, loadstore and loadlaod >>>>>>> would >>>>>>> use isync instead of lwsync for PPC on linux_zero, but not in any >>>>>>> other >>>>>>> PPC platform in the repo. I assumed this is wrong and changed it to >>>>>>> lwsync instead. >>>>>>> >>>>>>> == Code Redundancy and Refactoring == >>>>>>> >>>>>>> The OrderAccess code looks like it has been developed over a long >>>>>>> period >>>>>>> of time, with small incremental changes. This seems to have led >>>>>>> to a lot >>>>>>> of code duplication over time. For example, store_fence variants >>>>>>> are not >>>>>>> referenced from anywhere, yet contribute to a lot of the code >>>>>>> base and a >>>>>>> lot of awkwardness (such as being the one only exception not using >>>>>>> volatiles for some reason). Moreover, store_fence is not used >>>>>>> anywhere >>>>>>> in hotspot because it is not a good fit for either the >>>>>>> acquire/release >>>>>>> semantics or the Java volatile semantics, leaving a big question >>>>>>> mark on >>>>>>> when it should ever be used. I took the liberty of removing it. >>>>>>> >>>>>>> Another redundancy issue is that most of the semantics is exactly >>>>>>> the >>>>>>> same for all platforms, yet all that default boilerplate such as >>>>>>> how to >>>>>>> make atomic accesses, where acquire/release is supposed to be placed >>>>>>> w.r.t. the memory access, what the different barriers should do >>>>>>> etc. is >>>>>>> copied in redundantly for each os_cpu and each type of memory >>>>>>> access for >>>>>>> each os_cpu. This makes it extremely painful 1) to understand what >>>>>>> actually defines a certain platform compared to the defaults and >>>>>>> 2) to >>>>>>> correct bugs like those discovered here 3) prevent silly mistakes >>>>>>> and >>>>>>> bugs, by simply having a lot less code defining the behaviour of >>>>>>> OrderAccess that could go wrong. >>>>>>> >>>>>>> A new architecture/design for OrderAccess is proposed, using a >>>>>>> generalization/specialization approach. >>>>>>> >>>>>>> A general implementation in /share/ defines how things work and >>>>>>> splits >>>>>>> into different concerns: 1) how to make an atomic memory access, 2) >>>>>>> where to but barriers w.r.t. the memory access for things like >>>>>>> load_acquire, release_store and release_store_fence, 3) how these >>>>>>> barriers are defined. >>>>>>> >>>>>>> This allows a clear split between what is required for following the >>>>>>> specifications, and optimizations, which become much more >>>>>>> readable and >>>>>>> only optimizations need to be reviewed in depth as the defaults can >>>>>>> always be trusted given correct standalone barriers. >>>>>>> >>>>>>> The only thing a platform is required to specify, is what an >>>>>>> implementation of acquire(), release() and fence() should do. If >>>>>>> they >>>>>>> are implemented properly, everything in OrderAccess is guaranteed to >>>>>>> work according to specification using the generalized code. This >>>>>>> makes >>>>>>> it very easy to support new ports. ALL the other code in the os_cpu >>>>>>> files is used /only/ for optimization purposes offered for specific >>>>>>> configurations. >>>>>>> >>>>>>> However, it is highly customizable so that specific platform can >>>>>>> perform >>>>>>> any desired optimizations. For instance this load_acquire on PPC is >>>>>>> optimized: >>>>>>> >>>>>>> template<> inline jbyte >>>>>>> OrderAccess::specialized_load_acquire >>>>>>> (volatile jbyte* p) { register jbyte t = load(p); >>>>>>> inlasm_acquire_reg(t); return t; } >>>>>>> >>>>>>> This overrides the whole load_acquire implementation to do something >>>>>>> custom. Platforms like x86 extensively use this for joined fencing >>>>>>> variants to optimize. >>>>>>> >>>>>>> The default implementation of load_acquire() will make an atomic >>>>>>> load() >>>>>>> followed by acquire() since the semantics is generalized. The >>>>>>> generalized semantics are defined using inlined postfix/prefix calls >>>>>>> after/before the atomic access, as defined here: >>>>>>> >>>>>>> template<> inline void >>>>>>> ScopedFenceGeneral::postfix() { >>>>>>> OrderAccess::acquire(); } >>>>>>> template<> inline void >>>>>>> ScopedFenceGeneral::prefix() { >>>>>>> OrderAccess::release(); } >>>>>>> template<> inline void >>>>>>> ScopedFenceGeneral::prefix() { >>>>>>> OrderAccess::release(); } >>>>>>> template<> inline void >>>>>>> ScopedFenceGeneral::postfix() { >>>>>>> OrderAccess::fence(); } >>>>>>> >>>>>>> For platforms that simply wish to override what e.g. acquire >>>>>>> means for a >>>>>>> joined ordered memory access in general, as different to calling >>>>>>> stand >>>>>>> alone acquire(), the semantics can be easily overridden for a >>>>>>> platform >>>>>>> such as windows like on windows: >>>>>>> >>>>>>> template<> inline void ScopedFence::postfix() { } >>>>>>> template<> inline void ScopedFence::prefix() { } >>>>>>> template<> inline void ScopedFence::prefix() { } >>>>>>> template<> inline void ScopedFence::postfix() { >>>>>>> OrderAccess::fence(); } >>>>>>> >>>>>>> In this example, since Windows (now) has a compiler barrier for >>>>>>> acquire, >>>>>>> but does not need it for joined accesses since volatile has stronger >>>>>>> guarantees on windows, this is enough to specialize that for joined >>>>>>> memory accesses, no extra protection is needed. >>>>>>> >>>>>>> == Backward Compatibility and Transitioning == >>>>>>> >>>>>>> Since the newly proposed code is structured differently to before, a >>>>>>> #define was added for backward compatibility so that external >>>>>>> repositories not adhering to this new architecture do not break. >>>>>>> Furthermore, store_release was declared private and marked as >>>>>>> deprecated. This allows for a smooth transition into the new >>>>>>> style of >>>>>>> OrderAccess. When the full transition is made in all known repos, >>>>>>> the >>>>>>> #define and store_fence may be safely removed, eventually. >>>>>>> >>>>>>> == Documentation == >>>>>>> >>>>>>> The documentation seems old and outdated, describing how it works on >>>>>>> SPARC RMO and IA64, which are nowhere to be found in the >>>>>>> repository. It >>>>>>> also describes things about C++ volatiles which cannot be relied >>>>>>> upon. >>>>>>> The documentation has been cleaned up to match the current state >>>>>>> of the >>>>>>> implementation better, with architectures actually found in the >>>>>>> repository. >>>>>>> >>>>>>> == Testing == >>>>>>> >>>>>>> JPRT. Big thanks to Jesper Wilhelmsson for helping me test these >>>>>>> changes. >>>>>>> Ran some DaCapo benchmarks (I know okay :p) for performance >>>>>>> regression >>>>>>> and there was no perceivable difference. >>>>>>> >>>>>>> Looking forward to feedback on this, and hope to get some >>>>>>> reviews. :) >>>>>>> >>>>>>> Thanks, >>>>>>> Erik >>> >>> From mikael.vidstedt at oracle.com Wed Jan 28 22:58:08 2015 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 28 Jan 2015 14:58:08 -0800 Subject: Request or comment: 8071690: Include local HotSpot headers before the system headers In-Reply-To: References: <54C863D7.7000808@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF726BF@DEWDFEMB12A.global.corp.sap> <54C8AC0E.2040500@oracle.com> Message-ID: <54C96980.8010304@oracle.com> On 2015-01-28 02:26, Volker Simonis wrote: > On Wed, Jan 28, 2015 at 10:29 AM, David Holmes wrote: >> Hi Goetz, >> >> On 28/01/2015 6:26 PM, Lindenmaier, Goetz wrote: >>> Hi David, >>> >>> but by enforcing that rule, the effect is limited to the header >>> that includes other system headers. You can fix the problem >>> by adding globalDefinitions.hpp in that one header foo.h, and whoever >>> includes foo.h get's it right. >> >> Maybe we need to back up a step - what is the rule about header files >> including other header files? >> > I think the problem is that there is no general rule. There are just > hints like "every header file should be self-contained", i.e. it > should include all the other headers it depends on. Following this > rule should allow any arbitrary inclusion order. > > But in our case, we rely on a specific macro ("__STDC_FORMAT_MACROS") > being defined before we include a system header. So we have three > possibilities: > 1. define the macro at the top of every file before including anything. > 2. define the macro on the command line (actually a more elegant version of 1.) > 3. define the macro once in a local header and make sure the local > header is always included before any system header. I don't think that what I'm about to say can be relied on for all the platforms (compilers) we support, but just mentioning it anyway: gcc provides an option which is a bit like a combination of all the options above - the "-include " option will have the effect of including the specified header file as if "#include " appeared as the first line in the primary source file. Again, not sure there are similar options for the other compilers, but perhaps interesting to keep in mind. Cheers, Mikael > > Solution 1 and 2 should still allow an arbitrary inclusion order if > all headers are self-contained. > > The more I think about the problem the more I come to the conclusion > that for this specific issue solution 2 (defining the macro on the > command line) is the right solution. We already do this in other cases > in order to control how and what will be included from the system > headers (e.g. -D_GNU_SOURCE, -D_REENTRANT on Linux) and > -D__STDC_FORMAT_MACROS would perfectly fit into this list. It is > unclear to me why the latter was defined in a local header file. > > By the way, the example you mentioned, where we rely on a system > include coming first: > > #ifndef SOME_SYSTEM_THING > #define SOME_SYSTEM_THING xxx > #endif > > can be easily solved by having a local wrapper which only includes the > system header and afterwards performs the required configuration > steps. Other files of course will have to include the local wrapper > instead of the system header. > > Regards, > Volker > >> David >> >> >>> If fcntl.h had been added to foo.h in first place, all files that include >>> foo.h must include globalDefinitions.hpp before it ... not very easy to >>> catch. >>> >>> Best regards, >>> Goetz. >>> >>> >>> >>> >>> >>> -----Original Message----- >>> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf >>> Of David Holmes >>> Sent: Wednesday, January 28, 2015 5:22 AM >>> To: Volker Simonis; HotSpot Open Source Developers >>> Subject: Re: Request or comment: 8071690: Include local HotSpot headers >>> before the system headers >>> >>> Hi Volker, >>> >>> On 28/01/2015 3:29 AM, Volker Simonis wrote: >>>> Hi, >>>> >>>> I've just opened "8071690: Include local HotSpot headers before the >>>> system headers" but before I start working on it I'd like to hear what >>>> others think about the problem. If there's a consensus that it will be >>>> worth while doing this change I'll be happy to start working on it. >>> >>> As I wrote in the bug report: >>> >>> I don't see how you can apply this as a general rule - or at least not >>> without some further rules. If local foo.h depends on a system header >>> then it will #include it, so a file that #includes foo.h can't control >>> the order of that system header include. >>> >>> We need to recognize (and detect!) where we have implicit ordering >>> constraints but I don't think a general rule actually helps with that. >>> And there may be cases where we rely on a system include coming first eg: >>> >>> #ifndef SOME_SYSTEM_THING >>> #define SOME_SYSTEM_THING xxx >>> #endif >>> >>> David >>> ----- >>> >>>> https://bugs.openjdk.java.net/browse/JDK-8071690 >>>> >>>> The following description is copied from the bug report for your >>>> convenience: >>>> >>>> There's no general rule in which order header files should be included >>>> but I think a good hint is to include local, project specific headers >>>> before system headers. This is mainly for two reasons: >>>> >>>> 1. It prevents that dependencies from local header files to system >>>> headers are hidden because a local header is included after a system >>>> header by chance. Instead, every local header should explicitly >>>> specify the system headers it depends on. >>>> >>>> 2. It enables the definition of local macros which control the >>>> behaviour of the system headers which are included later on. >>>> >>>> Point two may be considered bad style, but we actually use it for >>>> example in src/share/vm/utilities/globalDefinitions.hpp where we >>>> define "__STDC_FORMAT_MACROS" before we include "" in the >>>> compiler specific global definitions file. >>>> >>>> "__STDC_FORMAT_MACROS" controls the definition of the printf format >>>> macros in "" but this can only work if "" is >>>> really included AFTER the definition of "__STDC_FORMAT_MACROS". And >>>> this can only wok if every file includes the local HotSpot headers >>>> before any system headers, because otherwise "" may be >>>> indirectly included by a system header before we had a chance to >>>> define "__STDC_FORMAT_MACROS". >>>> >>>> This is exactly what happened after the integration of 8050807 which >>>> added the system include "" to vmError.cpp as follows: >>>> >>>> #include >>>> #include "precompiled.hpp" >>>> #include "code/codeCache.hpp" >>>> >>>> This change broke the build on AIX because "" indirectly >>>> included "" BEFORE we defined "__STDC_FORMAT_MACROS". >>>> >>>> To prevent such errors in the future I propose to change all HotSpot >>>> source files to always include the system headers AFTER the inclusion >>>> of the project specific HotSpot headers. >>>> >>>> I?ve attached a small Pythin script to this bug report which can be >>>> used as follows to detect the files which are currently violating this >>>> rule: >>>> >>>> find src/ \( -name "*.cpp" -o -name "*.hpp" \) -type f -exec python >>>> include.py {} \; >>>> >>>> src/os/solaris/dtrace/generateJvmOffsets.cpp: system header #include >>>> included before local header #include >>>> "code/codeBlob.hpp" >>>> src/os/windows/vm/decoder_windows.hpp: system header #include >>>> included before local header #include >>>> "utilities/decoder.hpp" >>>> src/os_cpu/bsd_zero/vm/os_bsd_zero.cpp: system header # include >>>> included before local header #include >>>> "assembler_zero.inline.hpp" >>>> src/share/vm/adlc/adlc.hpp: system header #include included >>>> before local header #include "string.h" >>>> src/share/vm/libadt/port.hpp: system header #include >>>> included before local header #include "port_tandem.hpp" >>>> src/share/vm/runtime/os.hpp: system header # include >>>> included before local header # include "jvm_solaris.h" >>>> src/share/vm/trace/traceDataTypes.hpp: system header #include >>>> included before local header #include >>>> "utilities/globalDefinitions.hpp" >>>> src/share/vm/utilities/dtrace.hpp: system header #include >>>> included before local header #include >>>> "dtracefiles/hotspot.h" >>>> src/share/vm/utilities/elfFile.cpp: system header #include >>>> included before local header #include "memory/allocation.inline.hpp" >>>> src/share/vm/utilities/elfFile.hpp: system header #include >>>> included before local header #include "globalDefinitions.hpp" >>>> src/share/vm/utilities/vmError.cpp: system header #include >>>> included before local header #include "precompiled.hpp" >>>> >>>> The script is written in Python intentionally such that it can be >>>> easily added as an automated hook to jcheck to prevent violations of >>>> this inclusion rule in the future. >>>> >>>> Of course we can also try to not rely on the inclusion order at all. >>>> IMHO it actually seems that this is the cleanest solution, but it may >>>> require moving some macro definitions from header files right into the >>>> command line (e.g. -D__STDC_FORMAT_MACROS for the example above). By >>>> the way, that's actually the way how I've fixed the above mentioned >>>> compile error on AIX without the need to touch any shared files. >>>> >>>> What do you think? >>>> >>>> Regards, >>>> Volker >>>> From dean.long at oracle.com Thu Jan 29 01:15:06 2015 From: dean.long at oracle.com (Dean Long) Date: Wed, 28 Jan 2015 17:15:06 -0800 Subject: RFR(XS) 8069030: support new PTRACE_GETREGSET In-Reply-To: References: <54C70F7D.3010003@oracle.com> Message-ID: <54C9899A.2070206@oracle.com> Adding serviceability-dev at openjdk.java.net. Can I get another review for this please? thanks, dl On 1/26/2015 11:02 PM, Staffan Larsen wrote: > Looks good! > > Thanks, > /Staffan > >> On 27 jan 2015, at 05:09, Dean Long wrote: >> >> https://bugs.openjdk.java.net/browse/JDK-8069030 >> >> http://cr.openjdk.java.net/~dlong/8069030/webrev.00/ >> >> Testing on linux x86 by temporarily undefining PTRACE_GETREGS_REQ so that the new code >> would get triggered, then stepping through "jstack -m " with gdb to make sure the new >> code executed correctly. >> >> dl >> From erik.osterlund at lnu.se Thu Jan 29 02:16:01 2015 From: erik.osterlund at lnu.se (=?iso-8859-1?Q?Erik_=D6sterlund?=) Date: Thu, 29 Jan 2015 02:16:01 +0000 Subject: RFR: 7143664: Clean up OrderAccess implementations and usage References: <54C1EBD3.1000905@oracle.com> <54C1F3D8.5040507@oracle.com> <54C2C1EA.5020507@oracle.com> <54C87D16.40601@oracle.com> <54C9674C.2080600@oracle.com> Message-ID: Hi David, Thank you for uploading the latest webrev. :) /Erik On 28/01/15 23:48, David Holmes wrote: > Updated webrev: > > http://cr.openjdk.java.net/~dholmes/7143664/webrev.v2/ > > Thanks, > David > > On 28/01/2015 4:09 PM, David Holmes wrote: >> Hi Erik, >> >> Sorry for the delay, it was a long weekend here and I'm playing catch >> up. :) >> >> On 24/01/2015 9:03 AM, Erik ?sterlund wrote: >>> Hi Dean, >>> >>> On 23/01/15 22:50, Dean Long wrote: >>>> On 1/23/2015 7:16 AM, Erik ?sterlund wrote: >>>>> Hi Dean and David, >>>>> >>>>> Thanks for reviewing this change. >>>>> >>>>> On 23/01/15 08:12, David Holmes wrote: >>>>>> On 23/01/2015 4:36 PM, Dean Long wrote: >>>>>>> 147 // Ordering a load relative to preceding stores requires >>>>>>> a fence, >>>>>>> 148 // which implies a membar #StoreLoad between the store >>>>>>> and load under >>>>>>> 149 // sparc-TSO. A fence is required by x86. On x86, we >>>>>>> use explicitly >>>>>>> 150 // locked add. >>>>>>> 151 // >>>>>>> >>>>>>> It sounds like the above is saying that fence is the same as >>>>>>> StoreLoad ... >>>>>> That may be a s/store_fence/fence/ typo. The original text was: >>>>>> >>>>>> // Ordering a load relative to preceding stores requires a >>>>>> store_fence, >>>>>> // which implies a membar #StoreLoad between the store and load under >>>>>> // sparc-TSO. A fence is required by ia64. On x86, we use locked >>>>>> xchg. >>>>>> >>>>>> Actually seems like a couple of typos there: ia64 became x86, then we >>>>>> have x86 again. >>>>> I tried to remove ia64 from the comments as it is not in the >>>>> repository, >>>>> which in this case became a bit redundant. Are we okay if I remove the >>>>> "A fence is required by x86" sentence? >>>>> >>>>> Oh and Dean about membar #StoreLoad on SPARC and locked add on x86... >>>>> are you saying they are not equivalent? >>>> >>>> I wasn't trying to say anything abou SPARC or x86, only that "fence" != >>>> "StoreLoad". >>> >>> Agreed. >>> >>>>> New version of that paragraph with that sentence removed: >>>>> >>>>> 147 // Ordering a load relative to preceding stores requires a fence, >>>>> 148 // which implies a membar #StoreLoad between the store and load >>>>> under >>>>> 149 // sparc-TSO. On x86, we use explicitly locked add. >>>> >>>> I would replace "fence" with "StoreLoad" above. >>> >>> Again, I just changed any store_fence descriptions to use fence instead. >>> But you are certainly right, it is more accurate to talk about StoreLoad >>> here and not mention fence in the first place, I'll change this, thanks >>> for the feedback. :) >>> >>> Changed to: >>> >>> 147 // Ordering a load relative to preceding stores requires a StoreLoad, >>> 148 // which implies a membar #StoreLoad between the store and load under >>> 149 // sparc-TSO. On x86, we use explicitly locked add. >> >> The confusion here is what "store-fence" was intended to mean. In this >> part we're implying it is a storeLoad, but in the discussion below it >> indicates store-fence orders subsequent loads and stores - which makes >> it a storeStore|storeLoad barrier. Of course storeLoad subsumes >> storeStore so the above is accurate - though I think the original text >> would have been more clearly stated as "Ordering a load relative to >> preceding stores _can be achieved using a_ store-fence, ..." >> >> So in summary: update above is okay. >> >>>>>>> 152 // 4. store, load <= is constrained by => store, fence, >>>>>>> load >>>>>>> 153 // >>>>>>> 154 // Use store, fence to make sure all stores done in an >>>>>>> 'interesting' >>>>>>> 155 // region are made visible prior to both subsequent >>>>>>> loads and stores. >>>>>>> >>>>>>> ... and this is like saying to use fence because StoreStore | >>>>>>> StoreLoad >>>>>>> isn't available. >>>>>> Again this seems an issue with store_fence being changed to "store, >>>>>> fence" which doesn't really make sense. >>>>> The thing is that I removed store_fence (not store_release as it >>>>> says in >>>>> the bug ID, sorry about that - there obviously is no store_release). So >>>>> I wanted to remove references in the comments to store_fence as well. >>>>> Therefore I changed the example to use store fence load instead of >>>>> store_fence load. Perhaps it's better if I remove that sample all >>>>> together instead? >>>> >>>> That works for me. >>> >>> Fixed. >> >> Removal is an option but again replacing store-fence with what it really >> means "store; membar" would also suffice. But if >> we don't need the combined membar such that >> abstracting it into something like store_fence() makes sense, then >> getting rid of everything pertaining to store-fence makes sense. I see >> there are no uses of store_fence() other than a commented out reference >> in ./cpu/x86/vm/c1_LIRAssembler_x86.cpp, which also contains a similar >> commented out reference to the non-existent load_fence() - both of which >> should be removed. >> >> If you'd like to send me an updated patch I will generate a new webrev. >> Then I can get on with the functional part of the review. >> >> Thanks, >> David >> >>> Thanks, >>> /Erik >>> >>>> dl >>>> >>>>> Thanks for reviewing my changes! >>>>> >>>>> /Erik >>>>> >>>>>> David >>>>>> >>>>>>>> Furthermore, store_release was declared private and marked as >>>>>>> deprecated. >>>>>>> >>>>>>> I can't find where this was done. >>>>>>> >>>>>>> dl >>>>>>> >>>>>>> On 1/22/2015 5:19 PM, Erik ?sterlund wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> == Summary of Changes == >>>>>>>> >>>>>>>> This changeset fixes issues in OrderAccess on multiple levels >>>>>>>> from the >>>>>>>> memory model semantics to compiler reorderings, to addressing >>>>>>>> maintainability/portability issues which (almost) had to be fixed in >>>>>>>> order to fix the correctness issues. It is the result of discussions >>>>>>>> found in the previous "OrderAccess Refactoring" thread: >>>>>>>> http://openjdk.5641.n7.nabble.com/OrderAccess-Refactoring-td212050.html >>>>>>>> >>>>>>>> >>>>>>>> Bug ID: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-7143664 >>>>>>>> (updated to reflect these related changes) >>>>>>>> >>>>>>>> Webrev: >>>>>>>> http://cr.openjdk.java.net/~dholmes/7143664/webrev/ >>>>>>>> >>>>>>>> Before I describe more I would like to give special thanks to David >>>>>>>> Holmes for long discussions leading up to the currently proposed >>>>>>>> changes. I would also like to thank Jesper Wilhelmsson for >>>>>>>> helping me >>>>>>>> run my changes through JPRT. >>>>>>>> >>>>>>>> == Motivation == >>>>>>>> >>>>>>>> This change directly fixes a reported OrderAccess bug due to >>>>>>>> compiler >>>>>>>> reorderings which is still a vulnerability on almost all TSO >>>>>>>> platforms: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-806196 >>>>>>>> >>>>>>>> And directly fixes confusions like release_store() != release() >>>>>>>> store() >>>>>>>> due to memory model issues previously described in this bug ID. >>>>>>>> >>>>>>>> At the same time it provides clearer design with separation of >>>>>>>> concerns >>>>>>>> and generalization/specialization, removing a whole bunch of >>>>>>>> platform >>>>>>>> specific code which could be generalized. The platform specific >>>>>>>> files >>>>>>>> now only have a few LoC requirements (~7) to conform to the >>>>>>>> memory model >>>>>>>> by specifying what the stand alone barriers do. Then optionally >>>>>>>> optimizations to the general approach are possible if platforms >>>>>>>> support >>>>>>>> it. This also makes it much easier to port to new platforms. >>>>>>>> >>>>>>>> == Memory Model == >>>>>>>> >>>>>>>> The current definitions of acquire/release semantics are a bit fishy >>>>>>>> leading to problems previously described in the bug ID >>>>>>>> (release_store() >>>>>>>> != release() store()) and some other correctness issues. It has >>>>>>>> therefore been replaced with a new model. I would like to thank >>>>>>>> David >>>>>>>> Holmes for the long discussions leading up to the newly proposed >>>>>>>> model. >>>>>>>> >>>>>>>> The new model is formally defined like this: >>>>>>>> >>>>>>>> // T1: access_shared_data >>>>>>>> // T1: ]release >>>>>>>> // T1: (...) >>>>>>>> // T1: store(X) >>>>>>>> // >>>>>>>> // T2: load(X) >>>>>>>> // T2: (...) >>>>>>>> // T2: acquire[ >>>>>>>> // T2: access_shared_data >>>>>>>> // >>>>>>>> // It is guaranteed that if T2: load(X) synchronizes with >>>>>>>> (observes the >>>>>>>> // value written by) T1: store(X), then the memory accesses >>>>>>>> before the >>>>>>>> // T1: ]release happen before the memory accesses after the T2: >>>>>>>> acquire[. >>>>>>>> >>>>>>>> The orderAccess.hpp file and bug ID also has a few additional >>>>>>>> explanations making it more intuitive to the user when to use >>>>>>>> acquire/release and the resemblance to TSO abstract machines. Big >>>>>>>> thanks >>>>>>>> goes to David Holmes for discussing the memory model with me, which >>>>>>>> helped a lot in deriving it. >>>>>>>> >>>>>>>> Now it holds that release() store() == release_store(), and the >>>>>>>> other >>>>>>>> correctness issues are fixed as well. >>>>>>>> >>>>>>>> The new model is also closer to C++11 definitions which could >>>>>>>> give us >>>>>>>> more relaxed compiler reordering constraints in the future when >>>>>>>> compiler >>>>>>>> support for C++11 is there and ready. >>>>>>>> >>>>>>>> == Reliance on C++ Volatile Semantics == >>>>>>>> >>>>>>>> The C++ standard section 1.9 "Program Execution" is very vague about >>>>>>>> what the keyword volatile can actually do for us. It gives clear >>>>>>>> constraints in terms of volatile-volatile accesses but says >>>>>>>> little about >>>>>>>> nonvolatile-volatile accesses. Yet the current implementation >>>>>>>> heavily >>>>>>>> relies upon volatile to in terms of compiler reordering. But GCC >>>>>>>> explicitly declares that volatiles and non-volatiles may reorder >>>>>>>> freely >>>>>>>> ( https://gcc.gnu.org/onlinedocs/gcc/Volatiles.html ). The only >>>>>>>> compiler >>>>>>>> known to explicitly provide the wanted semantics with volatile is >>>>>>>> MSVC >>>>>>>> 2010 for windows ( >>>>>>>> https://msdn.microsoft.com/en-us/library/12a04hfd(v=vs.100).aspx ). >>>>>>>> Compilers not giving explicit guarantees, must be considered >>>>>>>> unsafe and >>>>>>>> revert to conservative behaviour. >>>>>>>> >>>>>>>> This was brought to attention after causing bugs, but was only >>>>>>>> fixed for >>>>>>>> x86 linux. This is a fundamental issue inherent to all TSO platforms >>>>>>>> except windows, and has to be fixed on all of them. >>>>>>>> >>>>>>>> Several barriers are unsafe to use because they lack compiler >>>>>>>> reordering >>>>>>>> constraints (e.g. fence and acquire on linux_SPARC). For TSO >>>>>>>> platforms >>>>>>>> they are typically implemented using dummy loads and stores. This >>>>>>>> seems >>>>>>>> to be another old volatile reliance that I fixed. These barriers >>>>>>>> sometimes have omitted compiler barriers (which is what we really >>>>>>>> want). >>>>>>>> This seems to be another example on incorrect reliance on the >>>>>>>> volatile >>>>>>>> semantics to help us. Therefore dummy loads/stores have been >>>>>>>> replaced >>>>>>>> with compiler barriers on TSO platforms. >>>>>>>> >>>>>>>> It is also worth noting that compilers like sun studio did >>>>>>>> previously >>>>>>>> not support inline asm syntax. Therefore, barriers were >>>>>>>> implemented in >>>>>>>> .il-files. However, using them does not give explicit compiler >>>>>>>> constraints for reordering AFAIK. Therefore, they have been >>>>>>>> reimplemented using inline asm with explicit compiler reordering >>>>>>>> constraints, as even sun (solaris?) studio now supports this. >>>>>>>> >>>>>>>> The windows variants have added a windows-style _ReadWriteBarrier() >>>>>>>> compiler barrier similarly. >>>>>>>> >>>>>>>> == Strange Hardware Reorderings == >>>>>>>> >>>>>>>> Fixed a weird inconsistency where acquire, loadstore and loadlaod >>>>>>>> would >>>>>>>> use isync instead of lwsync for PPC on linux_zero, but not in any >>>>>>>> other >>>>>>>> PPC platform in the repo. I assumed this is wrong and changed it to >>>>>>>> lwsync instead. >>>>>>>> >>>>>>>> == Code Redundancy and Refactoring == >>>>>>>> >>>>>>>> The OrderAccess code looks like it has been developed over a long >>>>>>>> period >>>>>>>> of time, with small incremental changes. This seems to have led >>>>>>>> to a lot >>>>>>>> of code duplication over time. For example, store_fence variants >>>>>>>> are not >>>>>>>> referenced from anywhere, yet contribute to a lot of the code >>>>>>>> base and a >>>>>>>> lot of awkwardness (such as being the one only exception not using >>>>>>>> volatiles for some reason). Moreover, store_fence is not used >>>>>>>> anywhere >>>>>>>> in hotspot because it is not a good fit for either the >>>>>>>> acquire/release >>>>>>>> semantics or the Java volatile semantics, leaving a big question >>>>>>>> mark on >>>>>>>> when it should ever be used. I took the liberty of removing it. >>>>>>>> >>>>>>>> Another redundancy issue is that most of the semantics is exactly >>>>>>>> the >>>>>>>> same for all platforms, yet all that default boilerplate such as >>>>>>>> how to >>>>>>>> make atomic accesses, where acquire/release is supposed to be placed >>>>>>>> w.r.t. the memory access, what the different barriers should do >>>>>>>> etc. is >>>>>>>> copied in redundantly for each os_cpu and each type of memory >>>>>>>> access for >>>>>>>> each os_cpu. This makes it extremely painful 1) to understand what >>>>>>>> actually defines a certain platform compared to the defaults and >>>>>>>> 2) to >>>>>>>> correct bugs like those discovered here 3) prevent silly mistakes >>>>>>>> and >>>>>>>> bugs, by simply having a lot less code defining the behaviour of >>>>>>>> OrderAccess that could go wrong. >>>>>>>> >>>>>>>> A new architecture/design for OrderAccess is proposed, using a >>>>>>>> generalization/specialization approach. >>>>>>>> >>>>>>>> A general implementation in /share/ defines how things work and >>>>>>>> splits >>>>>>>> into different concerns: 1) how to make an atomic memory access, 2) >>>>>>>> where to but barriers w.r.t. the memory access for things like >>>>>>>> load_acquire, release_store and release_store_fence, 3) how these >>>>>>>> barriers are defined. >>>>>>>> >>>>>>>> This allows a clear split between what is required for following the >>>>>>>> specifications, and optimizations, which become much more >>>>>>>> readable and >>>>>>>> only optimizations need to be reviewed in depth as the defaults can >>>>>>>> always be trusted given correct standalone barriers. >>>>>>>> >>>>>>>> The only thing a platform is required to specify, is what an >>>>>>>> implementation of acquire(), release() and fence() should do. If >>>>>>>> they >>>>>>>> are implemented properly, everything in OrderAccess is guaranteed to >>>>>>>> work according to specification using the generalized code. This >>>>>>>> makes >>>>>>>> it very easy to support new ports. ALL the other code in the os_cpu >>>>>>>> files is used /only/ for optimization purposes offered for specific >>>>>>>> configurations. >>>>>>>> >>>>>>>> However, it is highly customizable so that specific platform can >>>>>>>> perform >>>>>>>> any desired optimizations. For instance this load_acquire on PPC is >>>>>>>> optimized: >>>>>>>> >>>>>>>> template<> inline jbyte >>>>>>>> OrderAccess::specialized_load_acquire >>>>>>>> (volatile jbyte* p) { register jbyte t = load(p); >>>>>>>> inlasm_acquire_reg(t); return t; } >>>>>>>> >>>>>>>> This overrides the whole load_acquire implementation to do something >>>>>>>> custom. Platforms like x86 extensively use this for joined fencing >>>>>>>> variants to optimize. >>>>>>>> >>>>>>>> The default implementation of load_acquire() will make an atomic >>>>>>>> load() >>>>>>>> followed by acquire() since the semantics is generalized. The >>>>>>>> generalized semantics are defined using inlined postfix/prefix calls >>>>>>>> after/before the atomic access, as defined here: >>>>>>>> >>>>>>>> template<> inline void >>>>>>>> ScopedFenceGeneral::postfix() { >>>>>>>> OrderAccess::acquire(); } >>>>>>>> template<> inline void >>>>>>>> ScopedFenceGeneral::prefix() { >>>>>>>> OrderAccess::release(); } >>>>>>>> template<> inline void >>>>>>>> ScopedFenceGeneral::prefix() { >>>>>>>> OrderAccess::release(); } >>>>>>>> template<> inline void >>>>>>>> ScopedFenceGeneral::postfix() { >>>>>>>> OrderAccess::fence(); } >>>>>>>> >>>>>>>> For platforms that simply wish to override what e.g. acquire >>>>>>>> means for a >>>>>>>> joined ordered memory access in general, as different to calling >>>>>>>> stand >>>>>>>> alone acquire(), the semantics can be easily overridden for a >>>>>>>> platform >>>>>>>> such as windows like on windows: >>>>>>>> >>>>>>>> template<> inline void ScopedFence::postfix() { } >>>>>>>> template<> inline void ScopedFence::prefix() { } >>>>>>>> template<> inline void ScopedFence::prefix() { } >>>>>>>> template<> inline void ScopedFence::postfix() { >>>>>>>> OrderAccess::fence(); } >>>>>>>> >>>>>>>> In this example, since Windows (now) has a compiler barrier for >>>>>>>> acquire, >>>>>>>> but does not need it for joined accesses since volatile has stronger >>>>>>>> guarantees on windows, this is enough to specialize that for joined >>>>>>>> memory accesses, no extra protection is needed. >>>>>>>> >>>>>>>> == Backward Compatibility and Transitioning == >>>>>>>> >>>>>>>> Since the newly proposed code is structured differently to before, a >>>>>>>> #define was added for backward compatibility so that external >>>>>>>> repositories not adhering to this new architecture do not break. >>>>>>>> Furthermore, store_release was declared private and marked as >>>>>>>> deprecated. This allows for a smooth transition into the new >>>>>>>> style of >>>>>>>> OrderAccess. When the full transition is made in all known repos, >>>>>>>> the >>>>>>>> #define and store_fence may be safely removed, eventually. >>>>>>>> >>>>>>>> == Documentation == >>>>>>>> >>>>>>>> The documentation seems old and outdated, describing how it works on >>>>>>>> SPARC RMO and IA64, which are nowhere to be found in the >>>>>>>> repository. It >>>>>>>> also describes things about C++ volatiles which cannot be relied >>>>>>>> upon. >>>>>>>> The documentation has been cleaned up to match the current state >>>>>>>> of the >>>>>>>> implementation better, with architectures actually found in the >>>>>>>> repository. >>>>>>>> >>>>>>>> == Testing == >>>>>>>> >>>>>>>> JPRT. Big thanks to Jesper Wilhelmsson for helping me test these >>>>>>>> changes. >>>>>>>> Ran some DaCapo benchmarks (I know okay :p) for performance >>>>>>>> regression >>>>>>>> and there was no perceivable difference. >>>>>>>> >>>>>>>> Looking forward to feedback on this, and hope to get some >>>>>>>> reviews. :) >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Erik >>>> >>>> > From stefan.karlsson at oracle.com Thu Jan 29 09:19:52 2015 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 29 Jan 2015 10:19:52 +0100 Subject: Request or comment: 8071690: Include local HotSpot headers before the system headers In-Reply-To: <54C96980.8010304@oracle.com> References: <54C863D7.7000808@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF726BF@DEWDFEMB12A.global.corp.sap> <54C8AC0E.2040500@oracle.com> <54C96980.8010304@oracle.com> Message-ID: <54C9FB38.4090705@oracle.com> On 2015-01-28 23:58, Mikael Vidstedt wrote: > > On 2015-01-28 02:26, Volker Simonis wrote: >> On Wed, Jan 28, 2015 at 10:29 AM, David Holmes >> wrote: >>> Hi Goetz, >>> >>> On 28/01/2015 6:26 PM, Lindenmaier, Goetz wrote: >>>> Hi David, >>>> >>>> but by enforcing that rule, the effect is limited to the header >>>> that includes other system headers. You can fix the problem >>>> by adding globalDefinitions.hpp in that one header foo.h, and whoever >>>> includes foo.h get's it right. >>> >>> Maybe we need to back up a step - what is the rule about header files >>> including other header files? >>> >> I think the problem is that there is no general rule. There are just >> hints like "every header file should be self-contained", i.e. it >> should include all the other headers it depends on. Following this >> rule should allow any arbitrary inclusion order. >> >> But in our case, we rely on a specific macro ("__STDC_FORMAT_MACROS") >> being defined before we include a system header. So we have three >> possibilities: >> 1. define the macro at the top of every file before including >> anything. >> 2. define the macro on the command line (actually a more elegant >> version of 1.) >> 3. define the macro once in a local header and make sure the local >> header is always included before any system header. > > I don't think that what I'm about to say can be relied on for all the > platforms (compilers) we support, but just mentioning it anyway: > > gcc provides an option which is a bit like a combination of all the > options above - the "-include " option will have the effect of > including the specified header file as if "#include " appeared > as the first line in the primary source file. Again, not sure there > are similar options for the other compilers, but perhaps interesting > to keep in mind. It might also be worth considering having a header file that is always included at the top of all files, and let that file contain a bare minimum set of defines. A good starting point for such a file would be the macros.hpp and selected parts of globalDefinitions.hpp. We would have to do something about precompiled.hpp, since no includes are allowed to be put before the precompiled.hpp include line. A slightly different topic, but maybe it's time to get rid of precompiled.hpp? StefanK > > Cheers, > Mikael > >> >> Solution 1 and 2 should still allow an arbitrary inclusion order if >> all headers are self-contained. >> >> The more I think about the problem the more I come to the conclusion >> that for this specific issue solution 2 (defining the macro on the >> command line) is the right solution. We already do this in other cases >> in order to control how and what will be included from the system >> headers (e.g. -D_GNU_SOURCE, -D_REENTRANT on Linux) and >> -D__STDC_FORMAT_MACROS would perfectly fit into this list. It is >> unclear to me why the latter was defined in a local header file. >> >> By the way, the example you mentioned, where we rely on a system >> include coming first: >> >> #ifndef SOME_SYSTEM_THING >> #define SOME_SYSTEM_THING xxx >> #endif >> >> can be easily solved by having a local wrapper which only includes the >> system header and afterwards performs the required configuration >> steps. Other files of course will have to include the local wrapper >> instead of the system header. >> >> Regards, >> Volker >> >>> David >>> >>> >>>> If fcntl.h had been added to foo.h in first place, all files that >>>> include >>>> foo.h must include globalDefinitions.hpp before it ... not very >>>> easy to >>>> catch. >>>> >>>> Best regards, >>>> Goetz. >>>> >>>> >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On >>>> Behalf >>>> Of David Holmes >>>> Sent: Wednesday, January 28, 2015 5:22 AM >>>> To: Volker Simonis; HotSpot Open Source Developers >>>> Subject: Re: Request or comment: 8071690: Include local HotSpot >>>> headers >>>> before the system headers >>>> >>>> Hi Volker, >>>> >>>> On 28/01/2015 3:29 AM, Volker Simonis wrote: >>>>> Hi, >>>>> >>>>> I've just opened "8071690: Include local HotSpot headers before the >>>>> system headers" but before I start working on it I'd like to hear >>>>> what >>>>> others think about the problem. If there's a consensus that it >>>>> will be >>>>> worth while doing this change I'll be happy to start working on it. >>>> >>>> As I wrote in the bug report: >>>> >>>> I don't see how you can apply this as a general rule - or at least not >>>> without some further rules. If local foo.h depends on a system header >>>> then it will #include it, so a file that #includes foo.h can't control >>>> the order of that system header include. >>>> >>>> We need to recognize (and detect!) where we have implicit ordering >>>> constraints but I don't think a general rule actually helps with that. >>>> And there may be cases where we rely on a system include coming >>>> first eg: >>>> >>>> #ifndef SOME_SYSTEM_THING >>>> #define SOME_SYSTEM_THING xxx >>>> #endif >>>> >>>> David >>>> ----- >>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8071690 >>>>> >>>>> The following description is copied from the bug report for your >>>>> convenience: >>>>> >>>>> There's no general rule in which order header files should be >>>>> included >>>>> but I think a good hint is to include local, project specific headers >>>>> before system headers. This is mainly for two reasons: >>>>> >>>>> 1. It prevents that dependencies from local header files to system >>>>> headers are hidden because a local header is included after a system >>>>> header by chance. Instead, every local header should explicitly >>>>> specify the system headers it depends on. >>>>> >>>>> 2. It enables the definition of local macros which control the >>>>> behaviour of the system headers which are included later on. >>>>> >>>>> Point two may be considered bad style, but we actually use it for >>>>> example in src/share/vm/utilities/globalDefinitions.hpp where we >>>>> define "__STDC_FORMAT_MACROS" before we include "" in the >>>>> compiler specific global definitions file. >>>>> >>>>> "__STDC_FORMAT_MACROS" controls the definition of the printf format >>>>> macros in "" but this can only work if "" is >>>>> really included AFTER the definition of "__STDC_FORMAT_MACROS". And >>>>> this can only wok if every file includes the local HotSpot headers >>>>> before any system headers, because otherwise "" may be >>>>> indirectly included by a system header before we had a chance to >>>>> define "__STDC_FORMAT_MACROS". >>>>> >>>>> This is exactly what happened after the integration of 8050807 which >>>>> added the system include "" to vmError.cpp as follows: >>>>> >>>>> #include >>>>> #include "precompiled.hpp" >>>>> #include "code/codeCache.hpp" >>>>> >>>>> This change broke the build on AIX because "" indirectly >>>>> included "" BEFORE we defined "__STDC_FORMAT_MACROS". >>>>> >>>>> To prevent such errors in the future I propose to change all HotSpot >>>>> source files to always include the system headers AFTER the inclusion >>>>> of the project specific HotSpot headers. >>>>> >>>>> I?ve attached a small Pythin script to this bug report which can be >>>>> used as follows to detect the files which are currently violating >>>>> this >>>>> rule: >>>>> >>>>> find src/ \( -name "*.cpp" -o -name "*.hpp" \) -type f -exec python >>>>> include.py {} \; >>>>> >>>>> src/os/solaris/dtrace/generateJvmOffsets.cpp: system header #include >>>>> included before local header #include >>>>> "code/codeBlob.hpp" >>>>> src/os/windows/vm/decoder_windows.hpp: system header #include >>>>> included before local header #include >>>>> "utilities/decoder.hpp" >>>>> src/os_cpu/bsd_zero/vm/os_bsd_zero.cpp: system header # include >>>>> included before local header #include >>>>> "assembler_zero.inline.hpp" >>>>> src/share/vm/adlc/adlc.hpp: system header #include >>>>> included >>>>> before local header #include "string.h" >>>>> src/share/vm/libadt/port.hpp: system header #include >>>>> included before local header #include "port_tandem.hpp" >>>>> src/share/vm/runtime/os.hpp: system header # include >>>>> included before local header # include "jvm_solaris.h" >>>>> src/share/vm/trace/traceDataTypes.hpp: system header #include >>>>> included before local header #include >>>>> "utilities/globalDefinitions.hpp" >>>>> src/share/vm/utilities/dtrace.hpp: system header #include >>>>> included before local header #include >>>>> "dtracefiles/hotspot.h" >>>>> src/share/vm/utilities/elfFile.cpp: system header #include >>>>> included before local header #include "memory/allocation.inline.hpp" >>>>> src/share/vm/utilities/elfFile.hpp: system header #include >>>>> included before local header #include "globalDefinitions.hpp" >>>>> src/share/vm/utilities/vmError.cpp: system header #include >>>>> included before local header #include "precompiled.hpp" >>>>> >>>>> The script is written in Python intentionally such that it can be >>>>> easily added as an automated hook to jcheck to prevent violations of >>>>> this inclusion rule in the future. >>>>> >>>>> Of course we can also try to not rely on the inclusion order at all. >>>>> IMHO it actually seems that this is the cleanest solution, but it may >>>>> require moving some macro definitions from header files right into >>>>> the >>>>> command line (e.g. -D__STDC_FORMAT_MACROS for the example above). By >>>>> the way, that's actually the way how I've fixed the above mentioned >>>>> compile error on AIX without the need to touch any shared files. >>>>> >>>>> What do you think? >>>>> >>>>> Regards, >>>>> Volker >>>>> > From volker.simonis at gmail.com Thu Jan 29 10:47:36 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 29 Jan 2015 11:47:36 +0100 Subject: Request or comment: 8071690: Include local HotSpot headers before the system headers In-Reply-To: <54C9FB38.4090705@oracle.com> References: <54C863D7.7000808@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF726BF@DEWDFEMB12A.global.corp.sap> <54C8AC0E.2040500@oracle.com> <54C96980.8010304@oracle.com> <54C9FB38.4090705@oracle.com> Message-ID: On Thu, Jan 29, 2015 at 10:19 AM, Stefan Karlsson wrote: > On 2015-01-28 23:58, Mikael Vidstedt wrote: >> >> >> On 2015-01-28 02:26, Volker Simonis wrote: >>> >>> On Wed, Jan 28, 2015 at 10:29 AM, David Holmes >>> wrote: >>>> >>>> Hi Goetz, >>>> >>>> On 28/01/2015 6:26 PM, Lindenmaier, Goetz wrote: >>>>> >>>>> Hi David, >>>>> >>>>> but by enforcing that rule, the effect is limited to the header >>>>> that includes other system headers. You can fix the problem >>>>> by adding globalDefinitions.hpp in that one header foo.h, and whoever >>>>> includes foo.h get's it right. >>>> >>>> >>>> Maybe we need to back up a step - what is the rule about header files >>>> including other header files? >>>> >>> I think the problem is that there is no general rule. There are just >>> hints like "every header file should be self-contained", i.e. it >>> should include all the other headers it depends on. Following this >>> rule should allow any arbitrary inclusion order. >>> >>> But in our case, we rely on a specific macro ("__STDC_FORMAT_MACROS") >>> being defined before we include a system header. So we have three >>> possibilities: >>> 1. define the macro at the top of every file before including anything. >>> 2. define the macro on the command line (actually a more elegant >>> version of 1.) >>> 3. define the macro once in a local header and make sure the local >>> header is always included before any system header. >> >> >> I don't think that what I'm about to say can be relied on for all the >> platforms (compilers) we support, but just mentioning it anyway: >> >> gcc provides an option which is a bit like a combination of all the >> options above - the "-include " option will have the effect of >> including the specified header file as if "#include " appeared as the >> first line in the primary source file. Again, not sure there are similar >> options for the other compilers, but perhaps interesting to keep in mind. > Interesting option I didn't knew which further complicates things:) Just saw that there's also a "-imacros " option which behaves like "-include" but throws away all of the content with the exception of macro definitions. But I really think we should strive for a generic solution here which is as much as possible "standard" conformant. > > It might also be worth considering having a header file that is always > included at the top of all files, and let that file contain a bare minimum > set of defines. A good starting point for such a file would be the > macros.hpp and selected parts of globalDefinitions.hpp. > > We would have to do something about precompiled.hpp, since no includes are > allowed to be put before the precompiled.hpp include line. Good point. In that case, precompiled.hpp would have to first include that new header as well. > A slightly > different topic, but maybe it's time to get rid of precompiled.hpp? > What do you mean? How could we do that without giving up the precompiled header build? Volker > StefanK > > >> >> Cheers, >> Mikael >> >>> >>> Solution 1 and 2 should still allow an arbitrary inclusion order if >>> all headers are self-contained. >>> >>> The more I think about the problem the more I come to the conclusion >>> that for this specific issue solution 2 (defining the macro on the >>> command line) is the right solution. We already do this in other cases >>> in order to control how and what will be included from the system >>> headers (e.g. -D_GNU_SOURCE, -D_REENTRANT on Linux) and >>> -D__STDC_FORMAT_MACROS would perfectly fit into this list. It is >>> unclear to me why the latter was defined in a local header file. >>> >>> By the way, the example you mentioned, where we rely on a system >>> include coming first: >>> >>> #ifndef SOME_SYSTEM_THING >>> #define SOME_SYSTEM_THING xxx >>> #endif >>> >>> can be easily solved by having a local wrapper which only includes the >>> system header and afterwards performs the required configuration >>> steps. Other files of course will have to include the local wrapper >>> instead of the system header. >>> >>> Regards, >>> Volker >>> >>>> David >>>> >>>> >>>>> If fcntl.h had been added to foo.h in first place, all files that >>>>> include >>>>> foo.h must include globalDefinitions.hpp before it ... not very easy to >>>>> catch. >>>>> >>>>> Best regards, >>>>> Goetz. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On >>>>> Behalf >>>>> Of David Holmes >>>>> Sent: Wednesday, January 28, 2015 5:22 AM >>>>> To: Volker Simonis; HotSpot Open Source Developers >>>>> Subject: Re: Request or comment: 8071690: Include local HotSpot headers >>>>> before the system headers >>>>> >>>>> Hi Volker, >>>>> >>>>> On 28/01/2015 3:29 AM, Volker Simonis wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> I've just opened "8071690: Include local HotSpot headers before the >>>>>> system headers" but before I start working on it I'd like to hear what >>>>>> others think about the problem. If there's a consensus that it will be >>>>>> worth while doing this change I'll be happy to start working on it. >>>>> >>>>> >>>>> As I wrote in the bug report: >>>>> >>>>> I don't see how you can apply this as a general rule - or at least not >>>>> without some further rules. If local foo.h depends on a system header >>>>> then it will #include it, so a file that #includes foo.h can't control >>>>> the order of that system header include. >>>>> >>>>> We need to recognize (and detect!) where we have implicit ordering >>>>> constraints but I don't think a general rule actually helps with that. >>>>> And there may be cases where we rely on a system include coming first >>>>> eg: >>>>> >>>>> #ifndef SOME_SYSTEM_THING >>>>> #define SOME_SYSTEM_THING xxx >>>>> #endif >>>>> >>>>> David >>>>> ----- >>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8071690 >>>>>> >>>>>> The following description is copied from the bug report for your >>>>>> convenience: >>>>>> >>>>>> There's no general rule in which order header files should be included >>>>>> but I think a good hint is to include local, project specific headers >>>>>> before system headers. This is mainly for two reasons: >>>>>> >>>>>> 1. It prevents that dependencies from local header files to system >>>>>> headers are hidden because a local header is included after a system >>>>>> header by chance. Instead, every local header should explicitly >>>>>> specify the system headers it depends on. >>>>>> >>>>>> 2. It enables the definition of local macros which control the >>>>>> behaviour of the system headers which are included later on. >>>>>> >>>>>> Point two may be considered bad style, but we actually use it for >>>>>> example in src/share/vm/utilities/globalDefinitions.hpp where we >>>>>> define "__STDC_FORMAT_MACROS" before we include "" in the >>>>>> compiler specific global definitions file. >>>>>> >>>>>> "__STDC_FORMAT_MACROS" controls the definition of the printf format >>>>>> macros in "" but this can only work if "" is >>>>>> really included AFTER the definition of "__STDC_FORMAT_MACROS". And >>>>>> this can only wok if every file includes the local HotSpot headers >>>>>> before any system headers, because otherwise "" may be >>>>>> indirectly included by a system header before we had a chance to >>>>>> define "__STDC_FORMAT_MACROS". >>>>>> >>>>>> This is exactly what happened after the integration of 8050807 which >>>>>> added the system include "" to vmError.cpp as follows: >>>>>> >>>>>> #include >>>>>> #include "precompiled.hpp" >>>>>> #include "code/codeCache.hpp" >>>>>> >>>>>> This change broke the build on AIX because "" indirectly >>>>>> included "" BEFORE we defined "__STDC_FORMAT_MACROS". >>>>>> >>>>>> To prevent such errors in the future I propose to change all HotSpot >>>>>> source files to always include the system headers AFTER the inclusion >>>>>> of the project specific HotSpot headers. >>>>>> >>>>>> I?ve attached a small Pythin script to this bug report which can be >>>>>> used as follows to detect the files which are currently violating this >>>>>> rule: >>>>>> >>>>>> find src/ \( -name "*.cpp" -o -name "*.hpp" \) -type f -exec python >>>>>> include.py {} \; >>>>>> >>>>>> src/os/solaris/dtrace/generateJvmOffsets.cpp: system header #include >>>>>> included before local header #include >>>>>> "code/codeBlob.hpp" >>>>>> src/os/windows/vm/decoder_windows.hpp: system header #include >>>>>> included before local header #include >>>>>> "utilities/decoder.hpp" >>>>>> src/os_cpu/bsd_zero/vm/os_bsd_zero.cpp: system header # include >>>>>> included before local header #include >>>>>> "assembler_zero.inline.hpp" >>>>>> src/share/vm/adlc/adlc.hpp: system header #include included >>>>>> before local header #include "string.h" >>>>>> src/share/vm/libadt/port.hpp: system header #include >>>>>> included before local header #include "port_tandem.hpp" >>>>>> src/share/vm/runtime/os.hpp: system header # include >>>>>> included before local header # include "jvm_solaris.h" >>>>>> src/share/vm/trace/traceDataTypes.hpp: system header #include >>>>>> included before local header #include >>>>>> "utilities/globalDefinitions.hpp" >>>>>> src/share/vm/utilities/dtrace.hpp: system header #include >>>>>> included before local header #include >>>>>> "dtracefiles/hotspot.h" >>>>>> src/share/vm/utilities/elfFile.cpp: system header #include >>>>>> included before local header #include "memory/allocation.inline.hpp" >>>>>> src/share/vm/utilities/elfFile.hpp: system header #include >>>>>> included before local header #include "globalDefinitions.hpp" >>>>>> src/share/vm/utilities/vmError.cpp: system header #include >>>>>> included before local header #include "precompiled.hpp" >>>>>> >>>>>> The script is written in Python intentionally such that it can be >>>>>> easily added as an automated hook to jcheck to prevent violations of >>>>>> this inclusion rule in the future. >>>>>> >>>>>> Of course we can also try to not rely on the inclusion order at all. >>>>>> IMHO it actually seems that this is the cleanest solution, but it may >>>>>> require moving some macro definitions from header files right into the >>>>>> command line (e.g. -D__STDC_FORMAT_MACROS for the example above). By >>>>>> the way, that's actually the way how I've fixed the above mentioned >>>>>> compile error on AIX without the need to touch any shared files. >>>>>> >>>>>> What do you think? >>>>>> >>>>>> Regards, >>>>>> Volker >>>>>> >> > From stefan.karlsson at oracle.com Thu Jan 29 12:13:31 2015 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 29 Jan 2015 13:13:31 +0100 Subject: Request or comment: 8071690: Include local HotSpot headers before the system headers In-Reply-To: References: <54C863D7.7000808@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF726BF@DEWDFEMB12A.global.corp.sap> <54C8AC0E.2040500@oracle.com> <54C96980.8010304@oracle.com> <54C9FB38.4090705@oracle.com> Message-ID: <54CA23EB.4080000@oracle.com> On 2015-01-29 11:47, Volker Simonis wrote: > On Thu, Jan 29, 2015 at 10:19 AM, Stefan Karlsson > wrote: >> On 2015-01-28 23:58, Mikael Vidstedt wrote: >>> >>> On 2015-01-28 02:26, Volker Simonis wrote: >>>> On Wed, Jan 28, 2015 at 10:29 AM, David Holmes >>>> wrote: >>>>> Hi Goetz, >>>>> >>>>> On 28/01/2015 6:26 PM, Lindenmaier, Goetz wrote: >>>>>> Hi David, >>>>>> >>>>>> but by enforcing that rule, the effect is limited to the header >>>>>> that includes other system headers. You can fix the problem >>>>>> by adding globalDefinitions.hpp in that one header foo.h, and whoever >>>>>> includes foo.h get's it right. >>>>> >>>>> Maybe we need to back up a step - what is the rule about header files >>>>> including other header files? >>>>> >>>> I think the problem is that there is no general rule. There are just >>>> hints like "every header file should be self-contained", i.e. it >>>> should include all the other headers it depends on. Following this >>>> rule should allow any arbitrary inclusion order. >>>> >>>> But in our case, we rely on a specific macro ("__STDC_FORMAT_MACROS") >>>> being defined before we include a system header. So we have three >>>> possibilities: >>>> 1. define the macro at the top of every file before including anything. >>>> 2. define the macro on the command line (actually a more elegant >>>> version of 1.) >>>> 3. define the macro once in a local header and make sure the local >>>> header is always included before any system header. >>> >>> I don't think that what I'm about to say can be relied on for all the >>> platforms (compilers) we support, but just mentioning it anyway: >>> >>> gcc provides an option which is a bit like a combination of all the >>> options above - the "-include " option will have the effect of >>> including the specified header file as if "#include " appeared as the >>> first line in the primary source file. Again, not sure there are similar >>> options for the other compilers, but perhaps interesting to keep in mind. > Interesting option I didn't knew which further complicates things:) > > Just saw that there's also a "-imacros " option which behaves > like "-include" but throws away all of the content with the > exception of macro definitions. > > But I really think we should strive for a generic solution here which > is as much as possible "standard" conformant. > >> It might also be worth considering having a header file that is always >> included at the top of all files, and let that file contain a bare minimum >> set of defines. A good starting point for such a file would be the >> macros.hpp and selected parts of globalDefinitions.hpp. >> >> We would have to do something about precompiled.hpp, since no includes are >> allowed to be put before the precompiled.hpp include line. > Good point. In that case, precompiled.hpp would have to first include > that new header as well. > >> A slightly >> different topic, but maybe it's time to get rid of precompiled.hpp? >> > What do you mean? How could we do that without giving up the > precompiled header build? I actually meant that we could consider to stop building with precompiled headers, if it helps resolving some of the include file problems that we are often seeing. Removing the precompiled headers and changing our includes to be more precise might even lower the compile times of incremental builds. StefanK > > Volker > >> StefanK >> >> >>> Cheers, >>> Mikael >>> >>>> Solution 1 and 2 should still allow an arbitrary inclusion order if >>>> all headers are self-contained. >>>> >>>> The more I think about the problem the more I come to the conclusion >>>> that for this specific issue solution 2 (defining the macro on the >>>> command line) is the right solution. We already do this in other cases >>>> in order to control how and what will be included from the system >>>> headers (e.g. -D_GNU_SOURCE, -D_REENTRANT on Linux) and >>>> -D__STDC_FORMAT_MACROS would perfectly fit into this list. It is >>>> unclear to me why the latter was defined in a local header file. >>>> >>>> By the way, the example you mentioned, where we rely on a system >>>> include coming first: >>>> >>>> #ifndef SOME_SYSTEM_THING >>>> #define SOME_SYSTEM_THING xxx >>>> #endif >>>> >>>> can be easily solved by having a local wrapper which only includes the >>>> system header and afterwards performs the required configuration >>>> steps. Other files of course will have to include the local wrapper >>>> instead of the system header. >>>> >>>> Regards, >>>> Volker >>>> >>>>> David >>>>> >>>>> >>>>>> If fcntl.h had been added to foo.h in first place, all files that >>>>>> include >>>>>> foo.h must include globalDefinitions.hpp before it ... not very easy to >>>>>> catch. >>>>>> >>>>>> Best regards, >>>>>> Goetz. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -----Original Message----- >>>>>> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On >>>>>> Behalf >>>>>> Of David Holmes >>>>>> Sent: Wednesday, January 28, 2015 5:22 AM >>>>>> To: Volker Simonis; HotSpot Open Source Developers >>>>>> Subject: Re: Request or comment: 8071690: Include local HotSpot headers >>>>>> before the system headers >>>>>> >>>>>> Hi Volker, >>>>>> >>>>>> On 28/01/2015 3:29 AM, Volker Simonis wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I've just opened "8071690: Include local HotSpot headers before the >>>>>>> system headers" but before I start working on it I'd like to hear what >>>>>>> others think about the problem. If there's a consensus that it will be >>>>>>> worth while doing this change I'll be happy to start working on it. >>>>>> >>>>>> As I wrote in the bug report: >>>>>> >>>>>> I don't see how you can apply this as a general rule - or at least not >>>>>> without some further rules. If local foo.h depends on a system header >>>>>> then it will #include it, so a file that #includes foo.h can't control >>>>>> the order of that system header include. >>>>>> >>>>>> We need to recognize (and detect!) where we have implicit ordering >>>>>> constraints but I don't think a general rule actually helps with that. >>>>>> And there may be cases where we rely on a system include coming first >>>>>> eg: >>>>>> >>>>>> #ifndef SOME_SYSTEM_THING >>>>>> #define SOME_SYSTEM_THING xxx >>>>>> #endif >>>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8071690 >>>>>>> >>>>>>> The following description is copied from the bug report for your >>>>>>> convenience: >>>>>>> >>>>>>> There's no general rule in which order header files should be included >>>>>>> but I think a good hint is to include local, project specific headers >>>>>>> before system headers. This is mainly for two reasons: >>>>>>> >>>>>>> 1. It prevents that dependencies from local header files to system >>>>>>> headers are hidden because a local header is included after a system >>>>>>> header by chance. Instead, every local header should explicitly >>>>>>> specify the system headers it depends on. >>>>>>> >>>>>>> 2. It enables the definition of local macros which control the >>>>>>> behaviour of the system headers which are included later on. >>>>>>> >>>>>>> Point two may be considered bad style, but we actually use it for >>>>>>> example in src/share/vm/utilities/globalDefinitions.hpp where we >>>>>>> define "__STDC_FORMAT_MACROS" before we include "" in the >>>>>>> compiler specific global definitions file. >>>>>>> >>>>>>> "__STDC_FORMAT_MACROS" controls the definition of the printf format >>>>>>> macros in "" but this can only work if "" is >>>>>>> really included AFTER the definition of "__STDC_FORMAT_MACROS". And >>>>>>> this can only wok if every file includes the local HotSpot headers >>>>>>> before any system headers, because otherwise "" may be >>>>>>> indirectly included by a system header before we had a chance to >>>>>>> define "__STDC_FORMAT_MACROS". >>>>>>> >>>>>>> This is exactly what happened after the integration of 8050807 which >>>>>>> added the system include "" to vmError.cpp as follows: >>>>>>> >>>>>>> #include >>>>>>> #include "precompiled.hpp" >>>>>>> #include "code/codeCache.hpp" >>>>>>> >>>>>>> This change broke the build on AIX because "" indirectly >>>>>>> included "" BEFORE we defined "__STDC_FORMAT_MACROS". >>>>>>> >>>>>>> To prevent such errors in the future I propose to change all HotSpot >>>>>>> source files to always include the system headers AFTER the inclusion >>>>>>> of the project specific HotSpot headers. >>>>>>> >>>>>>> I?ve attached a small Pythin script to this bug report which can be >>>>>>> used as follows to detect the files which are currently violating this >>>>>>> rule: >>>>>>> >>>>>>> find src/ \( -name "*.cpp" -o -name "*.hpp" \) -type f -exec python >>>>>>> include.py {} \; >>>>>>> >>>>>>> src/os/solaris/dtrace/generateJvmOffsets.cpp: system header #include >>>>>>> included before local header #include >>>>>>> "code/codeBlob.hpp" >>>>>>> src/os/windows/vm/decoder_windows.hpp: system header #include >>>>>>> included before local header #include >>>>>>> "utilities/decoder.hpp" >>>>>>> src/os_cpu/bsd_zero/vm/os_bsd_zero.cpp: system header # include >>>>>>> included before local header #include >>>>>>> "assembler_zero.inline.hpp" >>>>>>> src/share/vm/adlc/adlc.hpp: system header #include included >>>>>>> before local header #include "string.h" >>>>>>> src/share/vm/libadt/port.hpp: system header #include >>>>>>> included before local header #include "port_tandem.hpp" >>>>>>> src/share/vm/runtime/os.hpp: system header # include >>>>>>> included before local header # include "jvm_solaris.h" >>>>>>> src/share/vm/trace/traceDataTypes.hpp: system header #include >>>>>>> included before local header #include >>>>>>> "utilities/globalDefinitions.hpp" >>>>>>> src/share/vm/utilities/dtrace.hpp: system header #include >>>>>>> included before local header #include >>>>>>> "dtracefiles/hotspot.h" >>>>>>> src/share/vm/utilities/elfFile.cpp: system header #include >>>>>>> included before local header #include "memory/allocation.inline.hpp" >>>>>>> src/share/vm/utilities/elfFile.hpp: system header #include >>>>>>> included before local header #include "globalDefinitions.hpp" >>>>>>> src/share/vm/utilities/vmError.cpp: system header #include >>>>>>> included before local header #include "precompiled.hpp" >>>>>>> >>>>>>> The script is written in Python intentionally such that it can be >>>>>>> easily added as an automated hook to jcheck to prevent violations of >>>>>>> this inclusion rule in the future. >>>>>>> >>>>>>> Of course we can also try to not rely on the inclusion order at all. >>>>>>> IMHO it actually seems that this is the cleanest solution, but it may >>>>>>> require moving some macro definitions from header files right into the >>>>>>> command line (e.g. -D__STDC_FORMAT_MACROS for the example above). By >>>>>>> the way, that's actually the way how I've fixed the above mentioned >>>>>>> compile error on AIX without the need to touch any shared files. >>>>>>> >>>>>>> What do you think? >>>>>>> >>>>>>> Regards, >>>>>>> Volker >>>>>>> From staffan.larsen at oracle.com Thu Jan 29 12:27:34 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Thu, 29 Jan 2015 13:27:34 +0100 Subject: RFR 8068976: Remove JSDT implementation In-Reply-To: <54B677B5.3080809@oracle.com> References: <54B677B5.3080809@oracle.com> Message-ID: <762CF5C2-A799-46D3-A6E9-84FDB07F89CC@oracle.com> Looks good to me. Thanks, /Staffan > On 14 jan 2015, at 15:05, Jaroslav Bachorik wrote: > > Please, review the following removal of functionality. > > Issue : https://bugs.openjdk.java.net/browse/JDK-8068976 > Webrev: http://cr.openjdk.java.net/~jbachorik/8068976/webrev.00 > > JDK-8062303 [1] requests the removal of com.sun.tracing API. This API was added as experimental in JDK7 [2][3] and haven't seen any significant uptake since then. > > JSDT builds on top of the com.sun.tracing API and provides the bridge to DTrace. It allows Java programs to fire custom DTrace probes. As an implementation of com.sun.tracing API it needs to be removed before proceeding with the removal of that API. > > This change is basically an anti-delta to [4] - minus the changes not related to JSDT and/or DTrace. > > This change passes all the regression and JCK tests. > > Thanks, > > -JB- > > > [1] https://bugs.openjdk.java.net/browse/JDK-8062303 > [2] https://bugs.openjdk.java.net/browse/JDK-6537506 > [3] https://bugs.openjdk.java.net/browse/JDK-7037081 > [4] http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/018d5b58dd4f From thomas.stuefe at gmail.com Thu Jan 29 13:53:07 2015 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 29 Jan 2015 14:53:07 +0100 Subject: Request or comment: 8071690: Include local HotSpot headers before the system headers In-Reply-To: <54CA23EB.4080000@oracle.com> References: <54C863D7.7000808@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF726BF@DEWDFEMB12A.global.corp.sap> <54C8AC0E.2040500@oracle.com> <54C96980.8010304@oracle.com> <54C9FB38.4090705@oracle.com> <54CA23EB.4080000@oracle.com> Message-ID: > > > I actually meant that we could consider to stop building with precompiled > headers, if it helps resolving some of the include file problems that we > are often seeing. Removing the precompiled headers and changing our > includes to be more precise might even lower the compile times of > incremental builds. > > StefanK > > Removing unnecessary dependencies from headers would be very nice. That has always been a pain point at least in our code, because developers tend to forget to remove headers when the code using them is removed. Would be even nicer to catch unnecessary includes automatically. Thomas From mandy.chung at oracle.com Thu Jan 29 21:52:44 2015 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 29 Jan 2015 13:52:44 -0800 Subject: RFR 8068976: Remove JSDT implementation In-Reply-To: <54B677B5.3080809@oracle.com> References: <54B677B5.3080809@oracle.com> Message-ID: <54CAABAC.8090905@oracle.com> Looks good to me. Do you plan to push the jdk side change to remove com.sun.tracing together with the hotspot change? I notice that you have a separate bug JDK-8062303 for jdk change. It's fine to choose to do. It'd also be fine to push the jdk change together with this hotspot change. Mandy On 1/14/15 6:05 AM, Jaroslav Bachorik wrote: > Please, review the following removal of functionality. > > Issue : https://bugs.openjdk.java.net/browse/JDK-8068976 > Webrev: http://cr.openjdk.java.net/~jbachorik/8068976/webrev.00 > > JDK-8062303 [1] requests the removal of com.sun.tracing API. This API > was added as experimental in JDK7 [2][3] and haven't seen any > significant uptake since then. > > JSDT builds on top of the com.sun.tracing API and provides the bridge > to DTrace. It allows Java programs to fire custom DTrace probes. As an > implementation of com.sun.tracing API it needs to be removed before > proceeding with the removal of that API. > > This change is basically an anti-delta to [4] - minus the changes not > related to JSDT and/or DTrace. > > This change passes all the regression and JCK tests. > > Thanks, > > -JB- > > > [1] https://bugs.openjdk.java.net/browse/JDK-8062303 > [2] https://bugs.openjdk.java.net/browse/JDK-6537506 > [3] https://bugs.openjdk.java.net/browse/JDK-7037081 > [4] http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/rev/018d5b58dd4f From david.holmes at oracle.com Fri Jan 30 03:27:12 2015 From: david.holmes at oracle.com (David Holmes) Date: Fri, 30 Jan 2015 13:27:12 +1000 Subject: Request or comment: 8071690: Include local HotSpot headers before the system headers In-Reply-To: References: <54C863D7.7000808@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF726BF@DEWDFEMB12A.global.corp.sap> <54C8AC0E.2040500@oracle.com> Message-ID: <54CAFA10.9000206@oracle.com> On 28/01/2015 8:26 PM, Volker Simonis wrote: > On Wed, Jan 28, 2015 at 10:29 AM, David Holmes wrote: >> Hi Goetz, >> >> On 28/01/2015 6:26 PM, Lindenmaier, Goetz wrote: >>> >>> Hi David, >>> >>> but by enforcing that rule, the effect is limited to the header >>> that includes other system headers. You can fix the problem >>> by adding globalDefinitions.hpp in that one header foo.h, and whoever >>> includes foo.h get's it right. >> >> >> Maybe we need to back up a step - what is the rule about header files >> including other header files? >> > > I think the problem is that there is no general rule. There are just > hints like "every header file should be self-contained", i.e. it > should include all the other headers it depends on. Following this > rule should allow any arbitrary inclusion order. Right - which implies every header file should include the common headers like globalDefinitions.hpp, macros.hpp etc - or as someone suggested define one header to include all those and have each header include this new "master" file of commonly needed includes. That would also fix the conditional include problem that was recently fixed by moving all conditional includes to the end of the list. The suggested order of locals first then system simply makes no sense to me because local headers often need system headers and so should include them directly. Then the higher-level header file gives the illusion of including locals before system files, but in practice they are all mixed together anyway. > But in our case, we rely on a specific macro ("__STDC_FORMAT_MACROS") > being defined before we include a system header. So we have three > possibilities: > 1. define the macro at the top of every file before including anything. > 2. define the macro on the command line (actually a more elegant version of 1.) > 3. define the macro once in a local header and make sure the local > header is always included before any system header. > > Solution 1 and 2 should still allow an arbitrary inclusion order if > all headers are self-contained. > > The more I think about the problem the more I come to the conclusion > that for this specific issue solution 2 (defining the macro on the > command line) is the right solution. We already do this in other cases > in order to control how and what will be included from the system > headers (e.g. -D_GNU_SOURCE, -D_REENTRANT on Linux) and > -D__STDC_FORMAT_MACROS would perfectly fit into this list. It is > unclear to me why the latter was defined in a local header file. Probably simpler than fixing a bunch of makefiles to pass the -D flag. But this is a specific issue that needs fixing. I don't think the suggestion to include locals first necessarily helps it, or is generally applicable. > By the way, the example you mentioned, where we rely on a system > include coming first: > > #ifndef SOME_SYSTEM_THING > #define SOME_SYSTEM_THING xxx > #endif > > can be easily solved by having a local wrapper which only includes the > system header and afterwards performs the required configuration > steps. Other files of course will have to include the local wrapper > instead of the system header. Sounds awful :) BTW for those discussing precompiled headers also see: https://bugs.openjdk.java.net/browse/JDK-7008747 :) Cheers, David > Regards, > Volker > >> David >> >> >>> If fcntl.h had been added to foo.h in first place, all files that include >>> foo.h must include globalDefinitions.hpp before it ... not very easy to >>> catch. >>> >>> Best regards, >>> Goetz. >>> >>> >>> >>> >>> >>> -----Original Message----- >>> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf >>> Of David Holmes >>> Sent: Wednesday, January 28, 2015 5:22 AM >>> To: Volker Simonis; HotSpot Open Source Developers >>> Subject: Re: Request or comment: 8071690: Include local HotSpot headers >>> before the system headers >>> >>> Hi Volker, >>> >>> On 28/01/2015 3:29 AM, Volker Simonis wrote: >>>> >>>> Hi, >>>> >>>> I've just opened "8071690: Include local HotSpot headers before the >>>> system headers" but before I start working on it I'd like to hear what >>>> others think about the problem. If there's a consensus that it will be >>>> worth while doing this change I'll be happy to start working on it. >>> >>> >>> As I wrote in the bug report: >>> >>> I don't see how you can apply this as a general rule - or at least not >>> without some further rules. If local foo.h depends on a system header >>> then it will #include it, so a file that #includes foo.h can't control >>> the order of that system header include. >>> >>> We need to recognize (and detect!) where we have implicit ordering >>> constraints but I don't think a general rule actually helps with that. >>> And there may be cases where we rely on a system include coming first eg: >>> >>> #ifndef SOME_SYSTEM_THING >>> #define SOME_SYSTEM_THING xxx >>> #endif >>> >>> David >>> ----- >>> >>>> https://bugs.openjdk.java.net/browse/JDK-8071690 >>>> >>>> The following description is copied from the bug report for your >>>> convenience: >>>> >>>> There's no general rule in which order header files should be included >>>> but I think a good hint is to include local, project specific headers >>>> before system headers. This is mainly for two reasons: >>>> >>>> 1. It prevents that dependencies from local header files to system >>>> headers are hidden because a local header is included after a system >>>> header by chance. Instead, every local header should explicitly >>>> specify the system headers it depends on. >>>> >>>> 2. It enables the definition of local macros which control the >>>> behaviour of the system headers which are included later on. >>>> >>>> Point two may be considered bad style, but we actually use it for >>>> example in src/share/vm/utilities/globalDefinitions.hpp where we >>>> define "__STDC_FORMAT_MACROS" before we include "" in the >>>> compiler specific global definitions file. >>>> >>>> "__STDC_FORMAT_MACROS" controls the definition of the printf format >>>> macros in "" but this can only work if "" is >>>> really included AFTER the definition of "__STDC_FORMAT_MACROS". And >>>> this can only wok if every file includes the local HotSpot headers >>>> before any system headers, because otherwise "" may be >>>> indirectly included by a system header before we had a chance to >>>> define "__STDC_FORMAT_MACROS". >>>> >>>> This is exactly what happened after the integration of 8050807 which >>>> added the system include "" to vmError.cpp as follows: >>>> >>>> #include >>>> #include "precompiled.hpp" >>>> #include "code/codeCache.hpp" >>>> >>>> This change broke the build on AIX because "" indirectly >>>> included "" BEFORE we defined "__STDC_FORMAT_MACROS". >>>> >>>> To prevent such errors in the future I propose to change all HotSpot >>>> source files to always include the system headers AFTER the inclusion >>>> of the project specific HotSpot headers. >>>> >>>> I?ve attached a small Pythin script to this bug report which can be >>>> used as follows to detect the files which are currently violating this >>>> rule: >>>> >>>> find src/ \( -name "*.cpp" -o -name "*.hpp" \) -type f -exec python >>>> include.py {} \; >>>> >>>> src/os/solaris/dtrace/generateJvmOffsets.cpp: system header #include >>>> included before local header #include >>>> "code/codeBlob.hpp" >>>> src/os/windows/vm/decoder_windows.hpp: system header #include >>>> included before local header #include >>>> "utilities/decoder.hpp" >>>> src/os_cpu/bsd_zero/vm/os_bsd_zero.cpp: system header # include >>>> included before local header #include >>>> "assembler_zero.inline.hpp" >>>> src/share/vm/adlc/adlc.hpp: system header #include included >>>> before local header #include "string.h" >>>> src/share/vm/libadt/port.hpp: system header #include >>>> included before local header #include "port_tandem.hpp" >>>> src/share/vm/runtime/os.hpp: system header # include >>>> included before local header # include "jvm_solaris.h" >>>> src/share/vm/trace/traceDataTypes.hpp: system header #include >>>> included before local header #include >>>> "utilities/globalDefinitions.hpp" >>>> src/share/vm/utilities/dtrace.hpp: system header #include >>>> included before local header #include >>>> "dtracefiles/hotspot.h" >>>> src/share/vm/utilities/elfFile.cpp: system header #include >>>> included before local header #include "memory/allocation.inline.hpp" >>>> src/share/vm/utilities/elfFile.hpp: system header #include >>>> included before local header #include "globalDefinitions.hpp" >>>> src/share/vm/utilities/vmError.cpp: system header #include >>>> included before local header #include "precompiled.hpp" >>>> >>>> The script is written in Python intentionally such that it can be >>>> easily added as an automated hook to jcheck to prevent violations of >>>> this inclusion rule in the future. >>>> >>>> Of course we can also try to not rely on the inclusion order at all. >>>> IMHO it actually seems that this is the cleanest solution, but it may >>>> require moving some macro definitions from header files right into the >>>> command line (e.g. -D__STDC_FORMAT_MACROS for the example above). By >>>> the way, that's actually the way how I've fixed the above mentioned >>>> compile error on AIX without the need to touch any shared files. >>>> >>>> What do you think? >>>> >>>> Regards, >>>> Volker >>>> >> From dean.long at oracle.com Fri Jan 30 07:06:39 2015 From: dean.long at oracle.com (Dean Long) Date: Thu, 29 Jan 2015 23:06:39 -0800 Subject: RFR: AARCH64: 8064594: Top-level JDK changes In-Reply-To: <546348F8.9060900@redhat.com> References: <546348F8.9060900@redhat.com> Message-ID: <54CB2D7F.2000607@oracle.com> Sorry for the late question, but how is src/java.base/unix/native/libjli/aarch64/jvm.cfg different from src/java.base/unix/conf/aarch64/jvm.cfg? I can't find where the former is used. thanks, dl From aph at redhat.com Fri Jan 30 08:31:05 2015 From: aph at redhat.com (Andrew Haley) Date: Fri, 30 Jan 2015 08:31:05 +0000 Subject: RFR: AARCH64: 8064594: Top-level JDK changes In-Reply-To: <54CB2D7F.2000607@oracle.com> References: <546348F8.9060900@redhat.com> <54CB2D7F.2000607@oracle.com> Message-ID: <54CB4149.7080503@redhat.com> On 30/01/15 07:06, Dean Long wrote: > Sorry for the late question, but how is > src/java.base/unix/native/libjli/aarch64/jvm.cfg different from > src/java.base/unix/conf/aarch64/jvm.cfg? I can't find where the former > is used. I'm sorry, I have no idea! I can investigate next week when I get back from FOSDEM. Andrew. From adinn at redhat.com Fri Jan 30 08:48:01 2015 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 30 Jan 2015 08:48:01 +0000 Subject: [aarch64-port-dev ] RFR: AARCH64: 8064594: Top-level JDK changes In-Reply-To: <54CB4149.7080503@redhat.com> References: <546348F8.9060900@redhat.com> <54CB2D7F.2000607@oracle.com> <54CB4149.7080503@redhat.com> Message-ID: <54CB4541.7030407@redhat.com> On 30/01/15 08:31, Andrew Haley wrote: > On 30/01/15 07:06, Dean Long wrote: >> Sorry for the late question, but how is >> src/java.base/unix/native/libjli/aarch64/jvm.cfg different from >> src/java.base/unix/conf/aarch64/jvm.cfg? I can't find where the former >> is used. > > I'm sorry, I have no idea! I can investigate next week when I > get back from FOSDEM. It looks like that has been copied there by mistake. There is no such file in the aarch64-port jdk9 tree from which stage was derived nor is there one in the arch64-port jdk8 tree. regards, Andrew Dinn ----------- From goetz.lindenmaier at sap.com Fri Jan 30 09:35:33 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 30 Jan 2015 09:35:33 +0000 Subject: RFR(S): 8071822: [TEST_BUG] Adapt collectorPolicy internal tests to support 64K pages Message-ID: <4295855A5C1DE049A61835A1887419CC2CF89D44@DEWDFEMB12A.global.corp.sap> Hi, this fixes a final problem with 64-bit page size in the jtreg tests. It addresses the internal VM tests of the collector policies. Those tests set a row of fixed flag values, and one special value to test. Then they call the heap ergonomics and check the expected result of the special value. With 64K page size, the heap ergonomics adapt more values, breaking the tests. Wrt. the adapted flag values the tests are correct though. This change makes the expected values depend on the adapted flags, and the tests pass. They only depend on adapted flags that are not subject of the corresponding test. Please review this test. I please need a sponsor. http://cr.openjdk.java.net/~goetz/webrevs/8071822-jtregColPol/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8071822 Best regards, Goetz. From mikael.auno at oracle.com Fri Jan 30 09:44:27 2015 From: mikael.auno at oracle.com (Mikael Auno) Date: Fri, 30 Jan 2015 10:44:27 +0100 Subject: RFR(L): 8071908, 8071909: Port internal Diagnostic Command tests and test framework to jtreg (plus some testlibrary changes) Message-ID: <54CB527B.4060503@oracle.com> Hi, could I please some reviews for this test port? Issues: https://bugs.openjdk.java.net/browse/JDK-8071908 https://bugs.openjdk.java.net/browse/JDK-8071909 Webrev: http://cr.openjdk.java.net/~miauno/8071908_8071909/webrev.00/ Read on for the rationale on a few questions that might arise. * Why two issues? These changes are mainly a port of the Diagnostic Command (DCMD) tests and corresponding framework/utility classes from an internal (non-open) test framework to jtreg. The reason for the two issues is that the changes depend on a few modifications to testlibrary that are available in jdk/test but not yet in hotspot/test, as well as a small new addition to OutputAnalyzer, that are not specific to main subject (i.e. the DCMD tests). To keep the history as clean and coherent as possible, those changes will go in under JDK-8071909 while the new tests and DCMD-related additions to testlibrary go in under JDK-8071908. * Isn't there already utility classes for calling Diagnostic Commands? The main idea with the new utility classes in testlibrary is to provide a single interface to call Diagnostic Commands from tests, regardless of the transport used (e.g. jcmd or JMX). There are a few tests scattered around jdk/test and hotspot/test today that already utilize Diagnostic Commands in some way, but they either use different utility classes for this in different places or just do it directly in the test. Also, some of these utility classes or tests go through jcmd and some through JMX (most often without any real requirement for one transport over the other in the test). All this means that there are, today, numerous different implementations for calling Diagnostic Commands, and consequently a lot of code duplication. These utility classes are meant to replace all of these implementations, and with a single interface regardless of the transport at that. * You've missed this or that test using one of the existing utility classes! This is "by design". In order to keep the change at a more manageable size and to get the core of this change in sooner, we've chosen to do this transition in multiple steps. The first of those steps is what is in this review request; the core utility classes, the tests ported from the internal test framework and the adaption of the tests already in hotspot/test/serviceability/dcmd (since they happened to reside in the directory where we wanted to put the ported tests). When this is integrated and have gone a few rounds through nightly testing, the adaption of other tests in hotspot/test will follow, and after that jdk/test. Thanks, Mikael From jaroslav.bachorik at oracle.com Fri Jan 30 10:18:08 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Fri, 30 Jan 2015 11:18:08 +0100 Subject: RFR(L): 8071908, 8071909: Port internal Diagnostic Command tests and test framework to jtreg (plus some testlibrary changes) In-Reply-To: <54CB527B.4060503@oracle.com> References: <54CB527B.4060503@oracle.com> Message-ID: <54CB5A60.8020401@oracle.com> Hi Mikael, it's great to see this moving forward! Comments follow: * instead of throwing a RuntimeException from within the test classes you could use o.testng.Assert.fail(...) method * all the newly introduced tests are missing @summary * test/serviceability/dcmd/compiler/CodelistTest.java L43 - unused import L68 - an unused, commented out, line * test/testlibrary/com/oracle/java/testlibrary/OutputAnalyzer.java L436-438 - use Arrays.asList() * test/serviceability/dcmd/vm/UptimeTest.java L44 - spurious wakeups may cause the test fail intermittently; should make sure the wait was at least 'someUptime' seconds long -JB- On 30.1.2015 10:44, Mikael Auno wrote: > Hi, could I please some reviews for this test port? > > Issues: https://bugs.openjdk.java.net/browse/JDK-8071908 > https://bugs.openjdk.java.net/browse/JDK-8071909 > Webrev: http://cr.openjdk.java.net/~miauno/8071908_8071909/webrev.00/ > > Read on for the rationale on a few questions that might arise. > > * Why two issues? > > These changes are mainly a port of the Diagnostic Command (DCMD) tests > and corresponding framework/utility classes from an internal (non-open) > test framework to jtreg. The reason for the two issues is that the > changes depend on a few modifications to testlibrary that are available > in jdk/test but not yet in hotspot/test, as well as a small new addition > to OutputAnalyzer, that are not specific to main subject (i.e. the DCMD > tests). To keep the history as clean and coherent as possible, those > changes will go in under JDK-8071909 while the new tests and > DCMD-related additions to testlibrary go in under JDK-8071908. > > * Isn't there already utility classes for calling Diagnostic Commands? > > The main idea with the new utility classes in testlibrary is to provide > a single interface to call Diagnostic Commands from tests, regardless of > the transport used (e.g. jcmd or JMX). There are a few tests scattered > around jdk/test and hotspot/test today that already utilize Diagnostic > Commands in some way, but they either use different utility classes for > this in different places or just do it directly in the test. Also, some > of these utility classes or tests go through jcmd and some through JMX > (most often without any real requirement for one transport over the > other in the test). All this means that there are, today, numerous > different implementations for calling Diagnostic Commands, and > consequently a lot of code duplication. These utility classes are meant > to replace all of these implementations, and with a single interface > regardless of the transport at that. > > * You've missed this or that test using one of the existing utility classes! > > This is "by design". In order to keep the change at a more manageable > size and to get the core of this change in sooner, we've chosen to do > this transition in multiple steps. The first of those steps is what is > in this review request; the core utility classes, the tests ported from > the internal test framework and the adaption of the tests already in > hotspot/test/serviceability/dcmd (since they happened to reside in the > directory where we wanted to put the ported tests). When this is > integrated and have gone a few rounds through nightly testing, the > adaption of other tests in hotspot/test will follow, and after that > jdk/test. > > Thanks, > Mikael > From erik.gahlin at oracle.com Fri Jan 30 13:20:10 2015 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Fri, 30 Jan 2015 14:20:10 +0100 Subject: RFR(L): 8071908, 8071909: Port internal Diagnostic Command tests and test framework to jtreg (plus some testlibrary changes) In-Reply-To: <54CB527B.4060503@oracle.com> References: <54CB527B.4060503@oracle.com> Message-ID: <54CB850A.9050305@oracle.com> Looks good! Erik Mikael Auno skrev 2015-01-30 10:44: > Hi, could I please some reviews for this test port? > > Issues: https://bugs.openjdk.java.net/browse/JDK-8071908 > https://bugs.openjdk.java.net/browse/JDK-8071909 > Webrev: http://cr.openjdk.java.net/~miauno/8071908_8071909/webrev.00/ > > Read on for the rationale on a few questions that might arise. > > * Why two issues? > > These changes are mainly a port of the Diagnostic Command (DCMD) tests > and corresponding framework/utility classes from an internal (non-open) > test framework to jtreg. The reason for the two issues is that the > changes depend on a few modifications to testlibrary that are available > in jdk/test but not yet in hotspot/test, as well as a small new addition > to OutputAnalyzer, that are not specific to main subject (i.e. the DCMD > tests). To keep the history as clean and coherent as possible, those > changes will go in under JDK-8071909 while the new tests and > DCMD-related additions to testlibrary go in under JDK-8071908. > > * Isn't there already utility classes for calling Diagnostic Commands? > > The main idea with the new utility classes in testlibrary is to provide > a single interface to call Diagnostic Commands from tests, regardless of > the transport used (e.g. jcmd or JMX). There are a few tests scattered > around jdk/test and hotspot/test today that already utilize Diagnostic > Commands in some way, but they either use different utility classes for > this in different places or just do it directly in the test. Also, some > of these utility classes or tests go through jcmd and some through JMX > (most often without any real requirement for one transport over the > other in the test). All this means that there are, today, numerous > different implementations for calling Diagnostic Commands, and > consequently a lot of code duplication. These utility classes are meant > to replace all of these implementations, and with a single interface > regardless of the transport at that. > > * You've missed this or that test using one of the existing utility classes! > > This is "by design". In order to keep the change at a more manageable > size and to get the core of this change in sooner, we've chosen to do > this transition in multiple steps. The first of those steps is what is > in this review request; the core utility classes, the tests ported from > the internal test framework and the adaption of the tests already in > hotspot/test/serviceability/dcmd (since they happened to reside in the > directory where we wanted to put the ported tests). When this is > integrated and have gone a few rounds through nightly testing, the > adaption of other tests in hotspot/test will follow, and after that > jdk/test. > > Thanks, > Mikael From zoltan.majo at oracle.com Fri Jan 30 14:25:00 2015 From: zoltan.majo at oracle.com (=?UTF-8?B?Wm9sdMOhbiBNYWrDsw==?=) Date: Fri, 30 Jan 2015 15:25:00 +0100 Subject: [8u60] Request for approval: Backport of 8071818(S) Message-ID: <54CB943C.9000903@oracle.com> Hi, I would like to request the backport of the fix for JDK-8071818 to 8u60. Original (9) bug: https://bugs.openjdk.java.net/browse/JDK-8071818 Original webrev: http://javaweb.us.oracle.com/~zmajo/8071818/webrev.00/ The fix applies cleanly. Testing: manual testing of failing test case, JPRT tests on Solaris SPARC. Thank you! Best regards, Zoltan From zoltan.majo at oracle.com Fri Jan 30 14:35:38 2015 From: zoltan.majo at oracle.com (=?UTF-8?B?Wm9sdMOhbiBNYWrDsw==?=) Date: Fri, 30 Jan 2015 15:35:38 +0100 Subject: [8u60] Request for approval: Backport of 8071818(S) In-Reply-To: <54CB943C.9000903@oracle.com> References: <54CB943C.9000903@oracle.com> Message-ID: <54CB96BA.8050206@oracle.com> Hi, On 01/30/2015 03:25 PM, Zolt?n Maj? wrote: > Original webrev: http://javaweb.us.oracle.com/~zmajo/8071818/webrev.00/ The correct webrev URL is: http://cr.openjdk.java.net/~zmajo/8071818/webrev.00/ Thank you and best regards, Zoltan > > The fix applies cleanly. > > Testing: manual testing of failing test case, JPRT tests on Solaris > SPARC. > > Thank you! > > Best regards, > > > Zoltan > From sergey.sprogis at oracle.com Fri Jan 30 00:45:53 2015 From: sergey.sprogis at oracle.com (Sergey Sprogis) Date: Thu, 29 Jan 2015 16:45:53 -0800 Subject: jdk9_b48 Solaris 5.12 on X86, hotspot/src/share/vm/opto/phase.cpp, Error: dd_fd is not a member of DIR. Message-ID: <54CAD441.8030307@oracle.com> To test new Oracle Studio C++ compiler on Solaris S12 X86 system I downloaded jdk9_b48 and the proper hotspot sources. But my build fails as shown below: ===================== CC -DSOLARIS -DSPARC_WORKS -DAMD64 -DASSERT -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/closed/share/vm/prims -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/share/vm/prims -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/closed/share/vm -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/share/vm -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/share/vm/precompiled -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/closed/cpu/x86/vm -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/cpu/x86/vm -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/os_cpu/solaris_x86/vm -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/closed/os/solaris/vm -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/os/solaris/vm -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/closed/os/posix/vm -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/os/posix/vm -I../generated -DHOTSPOT_BUILD_USER="\"ssprogis\"" -DHOTSPOT_LIB_ARCH=\"amd64\" -DHOTSPOT_VM_DISTRO="\"Java HotSpot(TM)\"" -DTARGET_OS_FAMILY_solaris -DTARGET_ARCH_x86 -DTARGET_ARCH_MODEL_x86_64 -DTARGET_OS_ARCH_solaris_x86 -DTARGET_OS_ARCH_MODEL_solaris_x86_64 -DTARGET_COMPILER_sparcWorks -DCOMPILER2 -DCOMPILER1 -DDONT_USE_PRECOMPILED_HEADER -m64 -m64 /export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/os_cpu/solaris_x86/vm/solaris_x86_64.il -features=no%split_init -D_Crun_inline_placement -library=%none -KPIC -mt -xO4 -g0 -xs -g0 -xs -DVM_LITTLE_ENDIAN -xwe -features=no%except -DHAVE_DTRACE_H -DDTRACE_ENABLED -c -xMMD -xMF ../generated/dependencies/phase.o.d -o phase.o /export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/share/vm/opto/phase.cpp "/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/os/solaris/vm/perfMemory_solaris.cpp", line 337: Error: dd_fd is not a member of DIR. "/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/os/solaris/vm/perfMemory_solaris.cpp", line 369: Error: dd_fd is not a member of DIR. 2 Error(s) detected. ============================ Probably HS developers do not use Solaris S12 systems for their build yet, but anyway could anyone tell me whether there is any way to workaround that failure. Is that dd_fd object the same one which is defined under: /usr/include/dirent.h, or it's just a coincidence? May be all I need is just to insert # include into hotspot/src/share/vm/opto/phase.cpp? Thanks, Sergey From vladimir.kozlov at oracle.com Fri Jan 30 17:22:22 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 30 Jan 2015 09:22:22 -0800 Subject: [8u60] Request for approval: Backport of 8071818(S) In-Reply-To: <54CB96BA.8050206@oracle.com> References: <54CB943C.9000903@oracle.com> <54CB96BA.8050206@oracle.com> Message-ID: <54CBBDCE.2060707@oracle.com> Good. thanks, Vladimir On 1/30/15 6:35 AM, Zolt?n Maj? wrote: > Hi, > > > On 01/30/2015 03:25 PM, Zolt?n Maj? wrote: >> Original webrev: http://javaweb.us.oracle.com/~zmajo/8071818/webrev.00/ > > The correct webrev URL is: http://cr.openjdk.java.net/~zmajo/8071818/webrev.00/ > > Thank you and best regards, > > > Zoltan > >> >> The fix applies cleanly. >> >> Testing: manual testing of failing test case, JPRT tests on Solaris SPARC. >> >> Thank you! >> >> Best regards, >> >> >> Zoltan >> > From mikael.auno at oracle.com Fri Jan 30 18:01:50 2015 From: mikael.auno at oracle.com (Mikael Auno) Date: Fri, 30 Jan 2015 19:01:50 +0100 Subject: RFR(L): 8071908, 8071909: Port internal Diagnostic Command tests and test framework to jtreg (plus some testlibrary changes) In-Reply-To: <54CB527B.4060503@oracle.com> References: <54CB527B.4060503@oracle.com> Message-ID: <54CBC70E.40005@oracle.com> I found an issue on Windows. DCMD output has Unix newlines even on Windows (and actually mixed Unix and Windows newlines when running through jcmd), so I've updated OutputAnalyzer.asLines() to split on "(\\r\\n|\\n|\\r)" instead of line.separator. I've also added better debug output in CodeCacheTest.java. New webrev: http://cr.openjdk.java.net/~miauno/8071908_8071909/webrev.01/ Thanks, Mikael On 2015-01-30 10:44, Mikael Auno wrote: > Hi, could I please some reviews for this test port? > > Issues: https://bugs.openjdk.java.net/browse/JDK-8071908 > https://bugs.openjdk.java.net/browse/JDK-8071909 > Webrev: http://cr.openjdk.java.net/~miauno/8071908_8071909/webrev.00/ > > Read on for the rationale on a few questions that might arise. > > * Why two issues? > > These changes are mainly a port of the Diagnostic Command (DCMD) tests > and corresponding framework/utility classes from an internal (non-open) > test framework to jtreg. The reason for the two issues is that the > changes depend on a few modifications to testlibrary that are available > in jdk/test but not yet in hotspot/test, as well as a small new addition > to OutputAnalyzer, that are not specific to main subject (i.e. the DCMD > tests). To keep the history as clean and coherent as possible, those > changes will go in under JDK-8071909 while the new tests and > DCMD-related additions to testlibrary go in under JDK-8071908. > > * Isn't there already utility classes for calling Diagnostic Commands? > > The main idea with the new utility classes in testlibrary is to provide > a single interface to call Diagnostic Commands from tests, regardless of > the transport used (e.g. jcmd or JMX). There are a few tests scattered > around jdk/test and hotspot/test today that already utilize Diagnostic > Commands in some way, but they either use different utility classes for > this in different places or just do it directly in the test. Also, some > of these utility classes or tests go through jcmd and some through JMX > (most often without any real requirement for one transport over the > other in the test). All this means that there are, today, numerous > different implementations for calling Diagnostic Commands, and > consequently a lot of code duplication. These utility classes are meant > to replace all of these implementations, and with a single interface > regardless of the transport at that. > > * You've missed this or that test using one of the existing utility classes! > > This is "by design". In order to keep the change at a more manageable > size and to get the core of this change in sooner, we've chosen to do > this transition in multiple steps. The first of those steps is what is > in this review request; the core utility classes, the tests ported from > the internal test framework and the adaption of the tests already in > hotspot/test/serviceability/dcmd (since they happened to reside in the > directory where we wanted to put the ported tests). When this is > integrated and have gone a few rounds through nightly testing, the > adaption of other tests in hotspot/test will follow, and after that > jdk/test. > > Thanks, > Mikael > From zoltan.majo at oracle.com Fri Jan 30 18:41:22 2015 From: zoltan.majo at oracle.com (=?UTF-8?B?Wm9sdMOhbiBNYWrDsw==?=) Date: Fri, 30 Jan 2015 19:41:22 +0100 Subject: [8u60] Request for approval: Backport of 8071818(S) In-Reply-To: <54CBBDCE.2060707@oracle.com> References: <54CB943C.9000903@oracle.com> <54CB96BA.8050206@oracle.com> <54CBBDCE.2060707@oracle.com> Message-ID: <54CBD052.1060706@oracle.com> Thank you, Vladimir! Best regards, Zoltan On 01/30/2015 06:22 PM, Vladimir Kozlov wrote: > Good. > > thanks, > Vladimir > > On 1/30/15 6:35 AM, Zolt?n Maj? wrote: >> Hi, >> >> >> On 01/30/2015 03:25 PM, Zolt?n Maj? wrote: >>> Original webrev: http://javaweb.us.oracle.com/~zmajo/8071818/webrev.00/ >> >> The correct webrev URL is: >> http://cr.openjdk.java.net/~zmajo/8071818/webrev.00/ >> >> Thank you and best regards, >> >> >> Zoltan >> >>> >>> The fix applies cleanly. >>> >>> Testing: manual testing of failing test case, JPRT tests on Solaris >>> SPARC. >>> >>> Thank you! >>> >>> Best regards, >>> >>> >>> Zoltan >>> >> From mikael.auno at oracle.com Fri Jan 30 19:32:30 2015 From: mikael.auno at oracle.com (Mikael Auno) Date: Fri, 30 Jan 2015 20:32:30 +0100 Subject: RFR(L): 8071908, 8071909: Port internal Diagnostic Command tests and test framework to jtreg (plus some testlibrary changes) In-Reply-To: <54CB5A60.8020401@oracle.com> References: <54CB527B.4060503@oracle.com> <54CB5A60.8020401@oracle.com> Message-ID: <54CBDC4E.2010002@oracle.com> Jaroslav, First of all, thanks for the quick response and sorry for my slow one (your message didn't sort into the mail folder I expected, so didn't see it until now). Secondly, all good comments; all fixed. Updated webrev: http://cr.openjdk.java.net/~miauno/8071908_8071909/webrev.02/ Thanks, Mikael On 2015-01-30 11:18, Jaroslav Bachorik wrote: > Hi Mikael, > > it's great to see this moving forward! > > Comments follow: > > * instead of throwing a RuntimeException from within the test classes > you could use o.testng.Assert.fail(...) method > > * all the newly introduced tests are missing @summary > > * test/serviceability/dcmd/compiler/CodelistTest.java > L43 - unused import > L68 - an unused, commented out, line > > * test/testlibrary/com/oracle/java/testlibrary/OutputAnalyzer.java > L436-438 - use Arrays.asList() > > * test/serviceability/dcmd/vm/UptimeTest.java > L44 - spurious wakeups may cause the test fail intermittently; should > make sure the wait was at least 'someUptime' seconds long > > > -JB- > > On 30.1.2015 10:44, Mikael Auno wrote: >> Hi, could I please some reviews for this test port? >> >> Issues: https://bugs.openjdk.java.net/browse/JDK-8071908 >> https://bugs.openjdk.java.net/browse/JDK-8071909 >> Webrev: http://cr.openjdk.java.net/~miauno/8071908_8071909/webrev.00/ >> >> Read on for the rationale on a few questions that might arise. >> >> * Why two issues? >> >> These changes are mainly a port of the Diagnostic Command (DCMD) tests >> and corresponding framework/utility classes from an internal (non-open) >> test framework to jtreg. The reason for the two issues is that the >> changes depend on a few modifications to testlibrary that are available >> in jdk/test but not yet in hotspot/test, as well as a small new addition >> to OutputAnalyzer, that are not specific to main subject (i.e. the DCMD >> tests). To keep the history as clean and coherent as possible, those >> changes will go in under JDK-8071909 while the new tests and >> DCMD-related additions to testlibrary go in under JDK-8071908. >> >> * Isn't there already utility classes for calling Diagnostic Commands? >> >> The main idea with the new utility classes in testlibrary is to provide >> a single interface to call Diagnostic Commands from tests, regardless of >> the transport used (e.g. jcmd or JMX). There are a few tests scattered >> around jdk/test and hotspot/test today that already utilize Diagnostic >> Commands in some way, but they either use different utility classes for >> this in different places or just do it directly in the test. Also, some >> of these utility classes or tests go through jcmd and some through JMX >> (most often without any real requirement for one transport over the >> other in the test). All this means that there are, today, numerous >> different implementations for calling Diagnostic Commands, and >> consequently a lot of code duplication. These utility classes are meant >> to replace all of these implementations, and with a single interface >> regardless of the transport at that. >> >> * You've missed this or that test using one of the existing utility >> classes! >> >> This is "by design". In order to keep the change at a more manageable >> size and to get the core of this change in sooner, we've chosen to do >> this transition in multiple steps. The first of those steps is what is >> in this review request; the core utility classes, the tests ported from >> the internal test framework and the adaption of the tests already in >> hotspot/test/serviceability/dcmd (since they happened to reside in the >> directory where we wanted to put the ported tests). When this is >> integrated and have gone a few rounds through nightly testing, the >> adaption of other tests in hotspot/test will follow, and after that >> jdk/test. >> >> Thanks, >> Mikael >> > From jaroslav.bachorik at oracle.com Fri Jan 30 19:48:50 2015 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Fri, 30 Jan 2015 20:48:50 +0100 Subject: RFR(L): 8071908, 8071909: Port internal Diagnostic Command tests and test framework to jtreg (plus some testlibrary changes) In-Reply-To: <54CBDC4E.2010002@oracle.com> References: <54CB527B.4060503@oracle.com> <54CB5A60.8020401@oracle.com> <54CBDC4E.2010002@oracle.com> Message-ID: <54CBE022.20502@oracle.com> Nice! For me it's good to go. -JB- On 30.1.2015 20:32, Mikael Auno wrote: > Jaroslav, > > First of all, thanks for the quick response and sorry for my slow one > (your message didn't sort into the mail folder I expected, so didn't see > it until now). > > Secondly, all good comments; all fixed. > > Updated webrev: > http://cr.openjdk.java.net/~miauno/8071908_8071909/webrev.02/ > > Thanks, > Mikael > > On 2015-01-30 11:18, Jaroslav Bachorik wrote: >> Hi Mikael, >> >> it's great to see this moving forward! >> >> Comments follow: >> >> * instead of throwing a RuntimeException from within the test classes >> you could use o.testng.Assert.fail(...) method >> >> * all the newly introduced tests are missing @summary >> >> * test/serviceability/dcmd/compiler/CodelistTest.java >> L43 - unused import >> L68 - an unused, commented out, line >> >> * test/testlibrary/com/oracle/java/testlibrary/OutputAnalyzer.java >> L436-438 - use Arrays.asList() >> >> * test/serviceability/dcmd/vm/UptimeTest.java >> L44 - spurious wakeups may cause the test fail intermittently; should >> make sure the wait was at least 'someUptime' seconds long >> >> >> -JB- >> >> On 30.1.2015 10:44, Mikael Auno wrote: >>> Hi, could I please some reviews for this test port? >>> >>> Issues: https://bugs.openjdk.java.net/browse/JDK-8071908 >>> https://bugs.openjdk.java.net/browse/JDK-8071909 >>> Webrev: http://cr.openjdk.java.net/~miauno/8071908_8071909/webrev.00/ >>> >>> Read on for the rationale on a few questions that might arise. >>> >>> * Why two issues? >>> >>> These changes are mainly a port of the Diagnostic Command (DCMD) tests >>> and corresponding framework/utility classes from an internal (non-open) >>> test framework to jtreg. The reason for the two issues is that the >>> changes depend on a few modifications to testlibrary that are available >>> in jdk/test but not yet in hotspot/test, as well as a small new addition >>> to OutputAnalyzer, that are not specific to main subject (i.e. the DCMD >>> tests). To keep the history as clean and coherent as possible, those >>> changes will go in under JDK-8071909 while the new tests and >>> DCMD-related additions to testlibrary go in under JDK-8071908. >>> >>> * Isn't there already utility classes for calling Diagnostic Commands? >>> >>> The main idea with the new utility classes in testlibrary is to provide >>> a single interface to call Diagnostic Commands from tests, regardless of >>> the transport used (e.g. jcmd or JMX). There are a few tests scattered >>> around jdk/test and hotspot/test today that already utilize Diagnostic >>> Commands in some way, but they either use different utility classes for >>> this in different places or just do it directly in the test. Also, some >>> of these utility classes or tests go through jcmd and some through JMX >>> (most often without any real requirement for one transport over the >>> other in the test). All this means that there are, today, numerous >>> different implementations for calling Diagnostic Commands, and >>> consequently a lot of code duplication. These utility classes are meant >>> to replace all of these implementations, and with a single interface >>> regardless of the transport at that. >>> >>> * You've missed this or that test using one of the existing utility >>> classes! >>> >>> This is "by design". In order to keep the change at a more manageable >>> size and to get the core of this change in sooner, we've chosen to do >>> this transition in multiple steps. The first of those steps is what is >>> in this review request; the core utility classes, the tests ported from >>> the internal test framework and the adaption of the tests already in >>> hotspot/test/serviceability/dcmd (since they happened to reside in the >>> directory where we wanted to put the ported tests). When this is >>> integrated and have gone a few rounds through nightly testing, the >>> adaption of other tests in hotspot/test will follow, and after that >>> jdk/test. >>> >>> Thanks, >>> Mikael >>> >> > From daniel.daugherty at oracle.com Fri Jan 30 19:27:50 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 30 Jan 2015 12:27:50 -0700 Subject: jdk9_b48 Solaris 5.12 on X86, hotspot/src/share/vm/opto/phase.cpp, Error: dd_fd is not a member of DIR. In-Reply-To: <54CAD441.8030307@oracle.com> References: <54CAD441.8030307@oracle.com> Message-ID: <54CBDB36.4010400@oracle.com> Sergey, This issue is being tracked by this bug: JDK-8071501 perfMemory_solaris.cpp failing to compile with "Error: dd_fd is not a member of DIR." https://bugs.openjdk.java.net/browse/JDK-8071501 Dan On 1/29/15 5:45 PM, Sergey Sprogis wrote: > To test new Oracle Studio C++ compiler on Solaris S12 X86 system I > downloaded jdk9_b48 and the proper hotspot sources. > But my build fails as shown below: > > ===================== > CC -DSOLARIS -DSPARC_WORKS -DAMD64 -DASSERT > -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/closed/share/vm/prims > -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/share/vm/prims > -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/closed/share/vm > -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/share/vm > -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/share/vm/precompiled > -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/closed/cpu/x86/vm > -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/cpu/x86/vm > -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/os_cpu/solaris_x86/vm > -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/closed/os/solaris/vm > -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/os/solaris/vm > -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/closed/os/posix/vm > -I/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/os/posix/vm > -I../generated -DHOTSPOT_BUILD_USER="\"ssprogis\"" > -DHOTSPOT_LIB_ARCH=\"amd64\" -DHOTSPOT_VM_DISTRO="\"Java > HotSpot(TM)\"" -DTARGET_OS_FAMILY_solaris -DTARGET_ARCH_x86 > -DTARGET_ARCH_MODEL_x86_64 -DTARGET_OS_ARCH_solaris_x86 > -DTARGET_OS_ARCH_MODEL_solaris_x86_64 -DTARGET_COMPILER_sparcWorks > -DCOMPILER2 -DCOMPILER1 -DDONT_USE_PRECOMPILED_HEADER -m64 -m64 > /export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/os_cpu/solaris_x86/vm/solaris_x86_64.il > -features=no%split_init -D_Crun_inline_placement -library=%none -KPIC > -mt -xO4 -g0 -xs -g0 -xs -DVM_LITTLE_ENDIAN -xwe > -features=no%except -DHAVE_DTRACE_H -DDTRACE_ENABLED -c -xMMD -xMF > ../generated/dependencies/phase.o.d -o phase.o > /export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/share/vm/opto/phase.cpp > > "/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/os/solaris/vm/perfMemory_solaris.cpp", > line 337: Error: dd_fd is not a member of DIR. > "/export/home/jvm_testing_intel/apps/apps_col_run/hotspot_b48/hotspot/src/os/solaris/vm/perfMemory_solaris.cpp", > line 369: Error: dd_fd is not a member of DIR. > 2 Error(s) detected. > ============================ > > Probably HS developers do not use Solaris S12 systems for their build > yet, but anyway could > anyone tell me whether there is any way to workaround that failure. > Is that dd_fd object the same one which is defined under: > /usr/include/dirent.h, or it's just a coincidence? > May be all I need is just to insert > > # include > > into hotspot/src/share/vm/opto/phase.cpp? > > Thanks, > > Sergey From mikael.auno at oracle.com Fri Jan 30 20:19:23 2015 From: mikael.auno at oracle.com (Mikael Auno) Date: Fri, 30 Jan 2015 21:19:23 +0100 Subject: RFR(L): 8071908, 8071909: Port internal Diagnostic Command tests and test framework to jtreg (plus some testlibrary changes) In-Reply-To: <54CBE022.20502@oracle.com> References: <54CB527B.4060503@oracle.com> <54CB5A60.8020401@oracle.com> <54CBDC4E.2010002@oracle.com> <54CBE022.20502@oracle.com> Message-ID: <54CBE74B.7040104@oracle.com> Thanks Jaroslav! I'll leave this open for further comments if someone else has them until Monday afternoon (CET). If there are no more comments by then I'll get it pushed at that time. Mikael On 2015-01-30 20:48, Jaroslav Bachorik wrote: > Nice! For me it's good to go. > > -JB- > > On 30.1.2015 20:32, Mikael Auno wrote: >> Jaroslav, >> >> First of all, thanks for the quick response and sorry for my slow one >> (your message didn't sort into the mail folder I expected, so didn't see >> it until now). >> >> Secondly, all good comments; all fixed. >> >> Updated webrev: >> http://cr.openjdk.java.net/~miauno/8071908_8071909/webrev.02/ >> >> Thanks, >> Mikael >> >> On 2015-01-30 11:18, Jaroslav Bachorik wrote: >>> Hi Mikael, >>> >>> it's great to see this moving forward! >>> >>> Comments follow: >>> >>> * instead of throwing a RuntimeException from within the test classes >>> you could use o.testng.Assert.fail(...) method >>> >>> * all the newly introduced tests are missing @summary >>> >>> * test/serviceability/dcmd/compiler/CodelistTest.java >>> L43 - unused import >>> L68 - an unused, commented out, line >>> >>> * test/testlibrary/com/oracle/java/testlibrary/OutputAnalyzer.java >>> L436-438 - use Arrays.asList() >>> >>> * test/serviceability/dcmd/vm/UptimeTest.java >>> L44 - spurious wakeups may cause the test fail intermittently; should >>> make sure the wait was at least 'someUptime' seconds long >>> >>> >>> -JB- >>> >>> On 30.1.2015 10:44, Mikael Auno wrote: >>>> Hi, could I please some reviews for this test port? >>>> >>>> Issues: https://bugs.openjdk.java.net/browse/JDK-8071908 >>>> https://bugs.openjdk.java.net/browse/JDK-8071909 >>>> Webrev: http://cr.openjdk.java.net/~miauno/8071908_8071909/webrev.00/ >>>> >>>> Read on for the rationale on a few questions that might arise. >>>> >>>> * Why two issues? >>>> >>>> These changes are mainly a port of the Diagnostic Command (DCMD) tests >>>> and corresponding framework/utility classes from an internal (non-open) >>>> test framework to jtreg. The reason for the two issues is that the >>>> changes depend on a few modifications to testlibrary that are available >>>> in jdk/test but not yet in hotspot/test, as well as a small new >>>> addition >>>> to OutputAnalyzer, that are not specific to main subject (i.e. the DCMD >>>> tests). To keep the history as clean and coherent as possible, those >>>> changes will go in under JDK-8071909 while the new tests and >>>> DCMD-related additions to testlibrary go in under JDK-8071908. >>>> >>>> * Isn't there already utility classes for calling Diagnostic Commands? >>>> >>>> The main idea with the new utility classes in testlibrary is to provide >>>> a single interface to call Diagnostic Commands from tests, >>>> regardless of >>>> the transport used (e.g. jcmd or JMX). There are a few tests scattered >>>> around jdk/test and hotspot/test today that already utilize Diagnostic >>>> Commands in some way, but they either use different utility classes for >>>> this in different places or just do it directly in the test. Also, some >>>> of these utility classes or tests go through jcmd and some through JMX >>>> (most often without any real requirement for one transport over the >>>> other in the test). All this means that there are, today, numerous >>>> different implementations for calling Diagnostic Commands, and >>>> consequently a lot of code duplication. These utility classes are meant >>>> to replace all of these implementations, and with a single interface >>>> regardless of the transport at that. >>>> >>>> * You've missed this or that test using one of the existing utility >>>> classes! >>>> >>>> This is "by design". In order to keep the change at a more manageable >>>> size and to get the core of this change in sooner, we've chosen to do >>>> this transition in multiple steps. The first of those steps is what is >>>> in this review request; the core utility classes, the tests ported from >>>> the internal test framework and the adaption of the tests already in >>>> hotspot/test/serviceability/dcmd (since they happened to reside in the >>>> directory where we wanted to put the ported tests). When this is >>>> integrated and have gone a few rounds through nightly testing, the >>>> adaption of other tests in hotspot/test will follow, and after that >>>> jdk/test. >>>> >>>> Thanks, >>>> Mikael >>>> >>> >> > From kim.barrett at oracle.com Sat Jan 31 01:53:51 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 30 Jan 2015 20:53:51 -0500 Subject: Request or comment: 8071690: Include local HotSpot headers before the system headers In-Reply-To: <54C8BEE9.50607@oracle.com> References: <54C863D7.7000808@oracle.com> <4295855A5C1DE049A61835A1887419CC2CF726BF@DEWDFEMB12A.global.corp.sap> <54C8AC0E.2040500@oracle.com> <54C8BEE9.50607@oracle.com> Message-ID: On Jan 28, 2015, at 5:50 AM, David Holmes wrote: > > I thought the double underscore macros were only for compiler use - so what normally sets it and what does it get used for? __STDC_FORMAT_MACROS, __STDC_CONSTANT_MACROS, and __STDC_LIMIT_MACROS are "user-level" configuration macros for and that are suggested by the C99 standard. [Note that I say "suggested" because they are only ever mentioned in footnotes, which are non-normative.] They are supposed to control whether those headers define certain sets of macros when those headers are processed by a C++ implementation. [One might wonder what business a C standard has in saying anything at all about what C++ implementations should do.] So they are explicitly intended to be defined by application code. I've seen problems with dealing with these in other code bases. I've only seen two methods for dealing with them that seemed to work reliably: - Forbid the use of the affected macros in headers. In translation units that want to use the affected macros, define the associated enabling macro at the top, prior to any includes. - Globally define the desired enabling macros via the build system. From dean.long at oracle.com Sat Jan 31 21:56:09 2015 From: dean.long at oracle.com (Dean Long) Date: Sat, 31 Jan 2015 13:56:09 -0800 Subject: [aarch64-port-dev ] RFR: AARCH64: 8064594: Top-level JDK changes In-Reply-To: <54CB4541.7030407@redhat.com> References: <546348F8.9060900@redhat.com> <54CB2D7F.2000607@oracle.com> <54CB4149.7080503@redhat.com> <54CB4541.7030407@redhat.com> Message-ID: <54CD4F79.7060202@oracle.com> On 1/30/2015 12:48 AM, Andrew Dinn wrote: > > On 30/01/15 08:31, Andrew Haley wrote: >> On 30/01/15 07:06, Dean Long wrote: >>> Sorry for the late question, but how is >>> src/java.base/unix/native/libjli/aarch64/jvm.cfg different from >>> src/java.base/unix/conf/aarch64/jvm.cfg? I can't find where the former >>> is used. >> I'm sorry, I have no idea! I can investigate next week when I >> get back from FOSDEM. > It looks like that has been copied there by mistake. There is no such > file in the aarch64-port jdk9 tree from which stage was derived nor is > there one in the arch64-port jdk8 tree. > > regards, > > > Andrew Dinn > ----------- > I filed a bug: https://bugs.openjdk.java.net/browse/JDK-8072053 dl