From coleen.phillimore at oracle.com Sat Feb 1 00:08:12 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 31 Jan 2020 19:08:12 -0500 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> Message-ID: <3c04824c-309d-69b0-94d6-4b6ef5a810f7@oracle.com> On 1/31/20 11:03 AM, Frederic Parain wrote: > Hi David, > > I?ve fixed all the issues you mentioned below. > > According to the discussion about the VM flags, the following modifications > have also been made: > > globals.hpp: > UseNewFieldLayout is now a deprecated product flag > > fieldLayoutBuilder.cpp > lines 350-362: > UseEmptySlotsInSupers VM option now controls both new optimizations > lines 449-464 > Fix an issue with layout printing (inherited fields were printed > only for the direct super-class, now all inherited fields are printed) I see the bug.? There's an InstanceKlass::java_super() call that does this. 465 ik = ik->super() == NULL ? NULL : InstanceKlass::cast(ik->super()); > > New webrev: > http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.08/index.html Looks good! Coleen > > > CR and CSR have been updated accordingly (new VM flags explanations in the CSR). > > Thank you, > > Fred > > >> On Jan 29, 2020, at 02:06, David Holmes wrote: >> >> Hi Fred, >> >> I've looked at the v7 version. A few more stylistic comments on that first. Note, no need for an item by item response unless that makes it easier for you to track :) >> >> src/hotspot/share/classfile/classFileParser.cpp >> >> #include "classfile/defaultMethods.hpp" >> +#include "classfile/fieldLayoutBuilder.hpp" >> #include "classfile/dictionary.hpp" >> >> Include files are not in alphabetical order. >> >> + * This may well change: FixMe if doesn't, >> >> s/if/if it/ >> >> + //Make a temp copy, and iterate through and copy back into the orig >> >> Space after // >> >> s/orig/original/ >> >> + OopMapBlock* nonstatic_oop_map = _nonstatic_oop_maps; >> >> Extra space after * >> >> + assert(ik->nonstatic_oop_map_count() == _field_info->oop_map_blocks->_nonstatic_oop_map_count, >> + "sanity"); >> >> Second line needs to be indented further: >> >> assert(ik->nonstatic_oop_map_count() == _field_info->oop_map_blocks->_nonstatic_oop_map_count, >> "sanity"); >> >> --- >> >> src/hotspot/share/classfile/classFileParser.hpp >> >> +public: >> + OopMapBlock* _nonstatic_oop_maps; >> + unsigned int _nonstatic_oop_map_count; >> + unsigned int _max_nonstatic_oop_maps; >> + >> + public: >> >> Second public uneeded. First public may be indented wrong (I'm not sure what the rule is - single space indent?) >> >> class ClassFileParser { >> + friend class FieldLayoutBuilder; >> + friend class FieldLayout; >> >> class ClassAnnotationCollector; >> class FieldAllocationCount; >> class FieldAnnotationCollector; >> >> Indents are different. I think the class forward declarations should have extra space. >> >> --- >> >> src/hotspot/share/oops/instanceKlass.hpp >> >> + void increment_count(int diff) { _count += diff; } >> >> Extra spaces before { >> >> --- >> >> src/hotspot/share/runtime/globals.hpp >> >> + diagnostic(bool, UseNewFieldLayout, true, \ >> + "Use new algorithm to compute layouts") \ >> + \ >> + product(bool, UseEmptySlotsInSupers, true, >> >> Not sure I see why one flag is diagnostic and the other product. Do you expect people to need to disable using empty slots more so than needing to disable using the new field layout altogether? >> >> --- >> >> src/hotspot/share/classfile/fieldLayoutBuilder.cpp >> >> + assert(kind == EMPTY || kind == RESERVED || kind == PADDING || kind == INHERITED, >> + "Otherwise, should use the constructor with a field index argument"); >> >> Indentation of second line is wrong. >> >> + assert(kind == REGULAR || kind == FLATTENED || kind == INHERITED, >> + "Other kind do not have a field index"); >> >> Ditto. >> >> >> + if (list == NULL) return; >> + if (start == NULL) { >> + start = this->_start; >> + } >> >> Inconsistent style for single statement if-blocks. Same thing later in the file. >> >> + output->print_cr(" @%d \"%s\" %s %d/%d %s", >> + b->offset(), >> + fi->name(_cp)->as_C_string(), >> + fi->signature(_cp)->as_C_string(), >> + b->size(), >> + b->alignment(), >> + "REGULAR"); >> >> Incorrect identation of continuing line. Same for all the following print blocks. >> >> + } else if (_classname == vmSymbols::java_lang_Boolean() || >> + _classname == vmSymbols::java_lang_Character() || >> + _classname == vmSymbols::java_lang_Float() || >> + _classname == vmSymbols::java_lang_Double() || >> + _classname == vmSymbols::java_lang_Byte() || >> + _classname == vmSymbols::java_lang_Short() || >> + _classname == vmSymbols::java_lang_Integer() || >> + _classname == vmSymbols::java_lang_Long()) { >> >> Incorrect identation of continuing line. >> >> --- >> >> src/hotspot/share/classfile/fieldLayoutBuilder.hpp >> >> +// and the boxing classes). The rational for having multiple methods >> >> s/rational/rationale/ >> >> + FieldLayoutBuilder(const Symbol* classname, const InstanceKlass* super_klass, ConstantPool* constant_pool, >> + Array* fields, bool is_contended, FieldLayoutInfo* info); >> >> Indentation wrong for continuing line. >> >> + int get_alignment() { >> + assert(_alignment != -1, "Uninitialized"); >> + return _alignment; >> + } >> >> Indenting appears off by one. >> >> --- >> >> test/hotspot/jtreg/runtime/FieldLayout/FieldDensityTest.java >> >> + * @run main/othervm -XX:+UseCompressedOops -XX:+UseCompressedClassPointers FieldDensityTest >> + * @run main/othervm -XX:+UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest >> + * @run main/othervm -XX:-UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest >> >> The test won't run on 32-bit platforms as the compressed oops flags won't exist. >> >> --- >> >> Some follow up comments below ... >> >> With trimming ... >> >> On 25/01/2020 3:20 am, Frederic Parain wrote: >>>> On Jan 24, 2020, at 08:19, David Holmes wrote: >>>> >>>> 466 int super_flen = super->nof_nonstatic_fields(); >>>> >>>> Could be folded directly into the assert so we don't call in product. >>> Calling not_nonstatic_fields() has the side effect to compute non-static fields, >>> which is required to get a correct value when reading super->_nonstatic_fields, >>> so the call is needed even in product builds. >> Yuck! That's a horrible side-effect - but not your fault obviously. :) It would be better to have a nonstatic_fields() accessor that has the same lazy initialization side-effect. >> >>>> General style issue: when breaking a long line with a method call, the new line (containing arguments) should be indented to the opening ( of the method call e.g. >> ... >>>> etc. This applies across all files. >>> Fixes applied lines 4003, 4011, 4041, 4138, 4143. >> Fix was also needed in other files. Current issues highlighted above. >> >>>> src/hotspot/share/oops/instanceKlass.hpp >>>> >>>> You need to be careful with _extra_flags usage if there can be concurrently updated bits. At the moment it looks like redefinition is a mutable dynamic property, whilst "contended annotations" should be a static immutable property - is that right? >>> Correct, _has_contended_annotations is a static immutable property, while _is_being_redefined is a mutable one. >> Good to know. My concern is that if someone adds a new mutable flag bit the need for atomic updates may not be noticed. We got bitten by this in the past with a flag field and I think we eventually migrated all of the mutable bits out into their own field. (Coleen should recall that :) ). >> >>>> 61 FLATTENED, // flattened field >>>> >>>> Does this have any meaning before inline types come in? >>> Yes, I wanted to reserved the entry in the enum. >> Hmmm a tenuous "okay". Seems odd to require this now to support code that is still some way from joining mainline. >> >>>> In FieldLayoutBuilder::epilogue you have a number of calls to Thread::current() as well as an implicit call when you use ResourceMarks. You should capture the current thread once in a local and reuse it. >>> Fixed >> It seems that this fix is now not needed as there is only one use left of the new "thread" variable in the ResourceMark. So that can return to being: >> >> ResourceMark rm; >> >> Thanks, >> David >> ----- >> From david.holmes at oracle.com Mon Feb 3 02:37:27 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 3 Feb 2020 12:37:27 +1000 Subject: RFR: 8238284: [macos] Zero VM build fails due to an obvious typo In-Reply-To: <85817F78-9D26-4FA0-8ABD-58830A714B15@tencent.com> References: <85817F78-9D26-4FA0-8ABD-58830A714B15@tencent.com> Message-ID: Hi Jie, On 31/01/2020 11:55 pm, jiefu(??) wrote: > Hi David, > > Thanks for your review and nice suggestion. > > Updated: http://cr.openjdk.java.net/~jiefu/8238284/webrev.01/ > > Testing: > - zero build tests on both macos and linux/x64. That all seems fine to me. Thanks, David > Thanks a lot. > Best regards, > Jie > > > ?On 2020/1/31, 8:00 PM, "David Holmes" wrote: > > Hi Jie, > > I note however that the same problem regarding lack of const exists in > other zero implementations. Should they also be fixed? > > Fixed. Thanks. > > From jiefu at tencent.com Mon Feb 3 02:53:08 2020 From: jiefu at tencent.com (=?utf-8?B?amllZnUo5YKF5p2wKQ==?=) Date: Mon, 3 Feb 2020 02:53:08 +0000 Subject: RFR: 8238284: [macos] Zero VM build fails due to an obvious typo Message-ID: <6898E027-6EA3-4807-8B31-05894E114F0F@tencent.com> Thanks David. Is it OK to be pushed? Or shall we wait for one more review? Thanks a lot. Best regards, Jie ?On 2020/2/3, 10:38 AM, "David Holmes" wrote: Hi Jie, On 31/01/2020 11:55 pm, jiefu(??) wrote: > Hi David, > > Thanks for your review and nice suggestion. > > Updated: http://cr.openjdk.java.net/~jiefu/8238284/webrev.01/ > > Testing: > - zero build tests on both macos and linux/x64. That all seems fine to me. Thanks, David > Thanks a lot. > Best regards, > Jie > > > On 2020/1/31, 8:00 PM, "David Holmes" wrote: > > Hi Jie, > > I note however that the same problem regarding lack of const exists in > other zero implementations. Should they also be fixed? > > Fixed. Thanks. > > From david.holmes at oracle.com Mon Feb 3 03:26:12 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 3 Feb 2020 13:26:12 +1000 Subject: RFR: 8238284: [macos] Zero VM build fails due to an obvious typo In-Reply-To: <6898E027-6EA3-4807-8B31-05894E114F0F@tencent.com> References: <6898E027-6EA3-4807-8B31-05894E114F0F@tencent.com> Message-ID: <1dac70ee-eefd-1ea2-3763-09d2d641d942@oracle.com> Hi Jie, On 3/02/2020 12:53 pm, jiefu(??) wrote: > Thanks David. > > Is it OK to be pushed? > Or shall we wait for one more review? This seems pretty trivial to me so I think it is okay to push. Thanks, David > Thanks a lot. > Best regards, > Jie > > ?On 2020/2/3, 10:38 AM, "David Holmes" wrote: > > Hi Jie, > > On 31/01/2020 11:55 pm, jiefu(??) wrote: > > Hi David, > > > > Thanks for your review and nice suggestion. > > > > Updated: http://cr.openjdk.java.net/~jiefu/8238284/webrev.01/ > > > > Testing: > > - zero build tests on both macos and linux/x64. > > That all seems fine to me. > > Thanks, > David > > > Thanks a lot. > > Best regards, > > Jie > > > > > > On 2020/1/31, 8:00 PM, "David Holmes" wrote: > > > > Hi Jie, > > > > I note however that the same problem regarding lack of const exists in > > other zero implementations. Should they also be fixed? > > > > Fixed. Thanks. > > > > > > > From jiefu at tencent.com Mon Feb 3 05:31:12 2020 From: jiefu at tencent.com (=?utf-8?B?amllZnUo5YKF5p2wKQ==?=) Date: Mon, 3 Feb 2020 05:31:12 +0000 Subject: RFR: 8238284: [macos] Zero VM build fails due to an obvious typo Message-ID: <35217624-2346-42A3-B65A-BCF620F661B5@tencent.com> Pushed. Thanks David. ?On 2020/2/3, 11:29 AM, "David Holmes" wrote: Hi Jie, On 3/02/2020 12:53 pm, jiefu(??) wrote: > Thanks David. > > Is it OK to be pushed? > Or shall we wait for one more review? This seems pretty trivial to me so I think it is okay to push. Thanks, David > Thanks a lot. > Best regards, > Jie > > On 2020/2/3, 10:38 AM, "David Holmes" wrote: > > Hi Jie, > > On 31/01/2020 11:55 pm, jiefu(??) wrote: > > Hi David, > > > > Thanks for your review and nice suggestion. > > > > Updated: http://cr.openjdk.java.net/~jiefu/8238284/webrev.01/ > > > > Testing: > > - zero build tests on both macos and linux/x64. > > That all seems fine to me. > > Thanks, > David > > > Thanks a lot. > > Best regards, > > Jie > > > > > > On 2020/1/31, 8:00 PM, "David Holmes" wrote: > > > > Hi Jie, > > > > I note however that the same problem regarding lack of const exists in > > other zero implementations. Should they also be fixed? > > > > Fixed. Thanks. > > > > > > > From david.holmes at oracle.com Mon Feb 3 06:24:36 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 3 Feb 2020 16:24:36 +1000 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> Message-ID: Hi Fred, On 1/02/2020 2:03 am, Frederic Parain wrote: > Hi David, > > I?ve fixed all the issues you mentioned below. > > According to the discussion about the VM flags, the following modifications > have also been made: > > globals.hpp: > UseNewFieldLayout is now a deprecated product flag You need to add it to the deprecated flag table in arguments.cpp and to the test test/hotspot/jtreg/runtime/CommandLine/VMDeprecatedOptions.java > fieldLayoutBuilder.cpp > lines 350-362: > UseEmptySlotsInSupers VM option now controls both new optimizations > lines 449-464 > Fix an issue with layout printing (inherited fields were printed > only for the direct super-class, now all inherited fields are printed) > > New webrev: > http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.08/index.html Looks good. A couple of minor follow ups ... src/hotspot/share/classfile/classFileParser.cpp ! OopMapBlock* nonstatic_oop_map = _nonstatic_oop_maps; Still an extra space after * + FieldLayoutBuilder lb(class_name(), super_klass(), _cp, _fields, + _parsed_annotations->is_contended(), _field_info); Indentation of continuing line is wrong. For the test, you have now restricted it to only run on 64-bit even though it has code that appears to allow for 32-bit. As this isn't a 64-bit specific feature I think it should run on all platforms. So this: /* * @test * @bug 8237767 * @requires vm.bits == "64" * @summary Verify behaviour of field layout algorithm * @library /test/lib * @modules java.base/jdk.internal.misc * java.management * @run main/othervm -XX:+UseCompressedOops -XX:+UseCompressedClassPointers FieldDensityTest * @run main/othervm -XX:+UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest * @run main/othervm -XX:-UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest */ can become: /* * @test * @bug 8237767 * @summary Verify behaviour of field layout algorithm * @library /test/lib * @modules java.base/jdk.internal.misc * java.management * @run main/othervm FieldDensityTest */ /* * @test * @requires vm.bits == "64" * @run main/othervm -XX:+UseCompressedOops -XX:+UseCompressedClassPointers FieldDensityTest * @run main/othervm -XX:+UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest */ > > CR and CSR have been updated accordingly (new VM flags explanations in the CSR). There is one small glitch in the CSR. Code removal associated with a flag occurs when it is obsoleted. So under the proposed timeline of deprecation in 15 and obsoletion in 16, the old code will be removed in 16 not 17. Comment added to CSR to that effect. Thanks, David > Thank you, > > Fred > > >> On Jan 29, 2020, at 02:06, David Holmes wrote: >> >> Hi Fred, >> >> I've looked at the v7 version. A few more stylistic comments on that first. Note, no need for an item by item response unless that makes it easier for you to track :) >> >> src/hotspot/share/classfile/classFileParser.cpp >> >> #include "classfile/defaultMethods.hpp" >> +#include "classfile/fieldLayoutBuilder.hpp" >> #include "classfile/dictionary.hpp" >> >> Include files are not in alphabetical order. >> >> + * This may well change: FixMe if doesn't, >> >> s/if/if it/ >> >> + //Make a temp copy, and iterate through and copy back into the orig >> >> Space after // >> >> s/orig/original/ >> >> + OopMapBlock* nonstatic_oop_map = _nonstatic_oop_maps; >> >> Extra space after * >> >> + assert(ik->nonstatic_oop_map_count() == _field_info->oop_map_blocks->_nonstatic_oop_map_count, >> + "sanity"); >> >> Second line needs to be indented further: >> >> assert(ik->nonstatic_oop_map_count() == _field_info->oop_map_blocks->_nonstatic_oop_map_count, >> "sanity"); >> >> --- >> >> src/hotspot/share/classfile/classFileParser.hpp >> >> +public: >> + OopMapBlock* _nonstatic_oop_maps; >> + unsigned int _nonstatic_oop_map_count; >> + unsigned int _max_nonstatic_oop_maps; >> + >> + public: >> >> Second public uneeded. First public may be indented wrong (I'm not sure what the rule is - single space indent?) >> >> class ClassFileParser { >> + friend class FieldLayoutBuilder; >> + friend class FieldLayout; >> >> class ClassAnnotationCollector; >> class FieldAllocationCount; >> class FieldAnnotationCollector; >> >> Indents are different. I think the class forward declarations should have extra space. >> >> --- >> >> src/hotspot/share/oops/instanceKlass.hpp >> >> + void increment_count(int diff) { _count += diff; } >> >> Extra spaces before { >> >> --- >> >> src/hotspot/share/runtime/globals.hpp >> >> + diagnostic(bool, UseNewFieldLayout, true, \ >> + "Use new algorithm to compute layouts") \ >> + \ >> + product(bool, UseEmptySlotsInSupers, true, >> >> Not sure I see why one flag is diagnostic and the other product. Do you expect people to need to disable using empty slots more so than needing to disable using the new field layout altogether? >> >> --- >> >> src/hotspot/share/classfile/fieldLayoutBuilder.cpp >> >> + assert(kind == EMPTY || kind == RESERVED || kind == PADDING || kind == INHERITED, >> + "Otherwise, should use the constructor with a field index argument"); >> >> Indentation of second line is wrong. >> >> + assert(kind == REGULAR || kind == FLATTENED || kind == INHERITED, >> + "Other kind do not have a field index"); >> >> Ditto. >> >> >> + if (list == NULL) return; >> + if (start == NULL) { >> + start = this->_start; >> + } >> >> Inconsistent style for single statement if-blocks. Same thing later in the file. >> >> + output->print_cr(" @%d \"%s\" %s %d/%d %s", >> + b->offset(), >> + fi->name(_cp)->as_C_string(), >> + fi->signature(_cp)->as_C_string(), >> + b->size(), >> + b->alignment(), >> + "REGULAR"); >> >> Incorrect identation of continuing line. Same for all the following print blocks. >> >> + } else if (_classname == vmSymbols::java_lang_Boolean() || >> + _classname == vmSymbols::java_lang_Character() || >> + _classname == vmSymbols::java_lang_Float() || >> + _classname == vmSymbols::java_lang_Double() || >> + _classname == vmSymbols::java_lang_Byte() || >> + _classname == vmSymbols::java_lang_Short() || >> + _classname == vmSymbols::java_lang_Integer() || >> + _classname == vmSymbols::java_lang_Long()) { >> >> Incorrect identation of continuing line. >> >> --- >> >> src/hotspot/share/classfile/fieldLayoutBuilder.hpp >> >> +// and the boxing classes). The rational for having multiple methods >> >> s/rational/rationale/ >> >> + FieldLayoutBuilder(const Symbol* classname, const InstanceKlass* super_klass, ConstantPool* constant_pool, >> + Array* fields, bool is_contended, FieldLayoutInfo* info); >> >> Indentation wrong for continuing line. >> >> + int get_alignment() { >> + assert(_alignment != -1, "Uninitialized"); >> + return _alignment; >> + } >> >> Indenting appears off by one. >> >> --- >> >> test/hotspot/jtreg/runtime/FieldLayout/FieldDensityTest.java >> >> + * @run main/othervm -XX:+UseCompressedOops -XX:+UseCompressedClassPointers FieldDensityTest >> + * @run main/othervm -XX:+UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest >> + * @run main/othervm -XX:-UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest >> >> The test won't run on 32-bit platforms as the compressed oops flags won't exist. >> >> --- >> >> Some follow up comments below ... >> >> With trimming ... >> >> On 25/01/2020 3:20 am, Frederic Parain wrote: >>>> On Jan 24, 2020, at 08:19, David Holmes wrote: >>>> >>>> 466 int super_flen = super->nof_nonstatic_fields(); >>>> >>>> Could be folded directly into the assert so we don't call in product. >>> Calling not_nonstatic_fields() has the side effect to compute non-static fields, >>> which is required to get a correct value when reading super->_nonstatic_fields, >>> so the call is needed even in product builds. >> >> Yuck! That's a horrible side-effect - but not your fault obviously. :) It would be better to have a nonstatic_fields() accessor that has the same lazy initialization side-effect. >> >>>> General style issue: when breaking a long line with a method call, the new line (containing arguments) should be indented to the opening ( of the method call e.g. >> ... >>>> etc. This applies across all files. >>> Fixes applied lines 4003, 4011, 4041, 4138, 4143. >> >> Fix was also needed in other files. Current issues highlighted above. >> >>>> >>>> src/hotspot/share/oops/instanceKlass.hpp >>>> >>>> You need to be careful with _extra_flags usage if there can be concurrently updated bits. At the moment it looks like redefinition is a mutable dynamic property, whilst "contended annotations" should be a static immutable property - is that right? >>> Correct, _has_contended_annotations is a static immutable property, while _is_being_redefined is a mutable one. >> >> Good to know. My concern is that if someone adds a new mutable flag bit the need for atomic updates may not be noticed. We got bitten by this in the past with a flag field and I think we eventually migrated all of the mutable bits out into their own field. (Coleen should recall that :) ). >> >>>> 61 FLATTENED, // flattened field >>>> >>>> Does this have any meaning before inline types come in? >>> Yes, I wanted to reserved the entry in the enum. >> >> Hmmm a tenuous "okay". Seems odd to require this now to support code that is still some way from joining mainline. >> >>>> In FieldLayoutBuilder::epilogue you have a number of calls to Thread::current() as well as an implicit call when you use ResourceMarks. You should capture the current thread once in a local and reuse it. >>> Fixed >> >> It seems that this fix is now not needed as there is only one use left of the new "thread" variable in the ResourceMark. So that can return to being: >> >> ResourceMark rm; >> >> Thanks, >> David >> ----- >> > From magnus.ihse.bursie at oracle.com Mon Feb 3 08:02:34 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 3 Feb 2020 09:02:34 +0100 Subject: RFR: JDK-8238281 Raise minimum gcc version needed to 5.0 Message-ID: JEP 347 "Adopt C++14 Language Features in HotSpot" (JDK-8208089) will require that all compilers support the C++14 language extension. The first gcc version to fully support C++14 is 5.0. We need to raise the bar from 4.8 to 5.0. Since removing support for old compilers, especially gcc, can be tricky to handle for some on old and odd setups, I think it's best to do this as a separate step from the C++14 upgrade. On the way, this will allow us to remove some workarounds/close some bugs for really old gcc versions. For hotspot, most notably this means: ?* The workarounds for no ATTRIBUTE_PRINTF are not needed anymore. * The workaround for ATTRIBUTE_ALIGNED is no longer needed. * The workaround in offset_of is not needed anymore. Instead, the appropriate warning is disabled. (In fact, now only xlc need special treatment for offset_of; if that could be fixed now, it would mean that offset_of can be replaced by offsetof in hotspot.) * [Bug fix] The define CAN_USE_NAN_DEFINE (which is used only in share/runtime/sharedRuntimeTrans.cpp), is now always defined for gcc. This was previously done only for gcc >= 4.3 and < 5. I believe this was a bug, and the intention was to allow for old-style NaN behavior in gcc 4.2 and older. Bug: https://bugs.openjdk.java.net/browse/JDK-8238281 WebRev: http://cr.openjdk.java.net/~ihse/JDK-8238281-make-gcc-5-minimum/webrev.01 From thomas.schatzl at oracle.com Mon Feb 3 09:01:46 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 03 Feb 2020 10:01:46 +0100 Subject: RFR[XS]: 8238272: Eliminate cast_from_oop to narrowOop* In-Reply-To: <0F754E34-462D-4183-8CE0-F2933D4F310F@oracle.com> References: <267AF7FC-B6EB-4232-A51F-035A7217BCC6@oracle.com> <44ad6bd5-b66a-3306-0cb9-f518d1b8e2c4@redhat.com> <805C4111-CF21-4EAF-AC78-742045A3DD73@oracle.com> <0F754E34-462D-4183-8CE0-F2933D4F310F@oracle.com> Message-ID: <6afd673387d694f87ea65049a4362865333cba87.camel@oracle.com> Hi, On Fri, 2020-01-31 at 17:12 -0500, Kim Barrett wrote: > > On Jan 31, 2020, at 2:35 AM, Kim Barrett > > wrote: > > > > > On Jan 31, 2020, at 2:07 AM, Aleksey Shipilev > > > wrote: > > > > > > On 1/31/20 7:51 AM, Kim Barrett wrote: > > > > Webrev: > > > > https://cr.openjdk.java.net/~kbarrett/8238272/open.00/ > > > > > > Hm, I am puzzled a bit. > > > > > > [...] > > The fix is to use the same hand-inlined address calculation for > > start as for end, perhaps > > packaging that up in a new function. I?ll get back to this next > > week. Sorry for the noise. > > Here's a new webrev: > > https://cr.openjdk.java.net/~kbarrett/8238272/open.01/ > > I didn't bother with an incremental webrev, since the new version is > just > a one-line change replacing the previous one-line change. > > Testing: tier1 > looks good. Thomas From shade at redhat.com Mon Feb 3 12:07:12 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 3 Feb 2020 13:07:12 +0100 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> Message-ID: <1830530c-b086-534b-9bef-bb7e762fd6c6@redhat.com> Hi again, On 1/31/20 5:03 PM, Frederic Parain wrote: > New webrev: > http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.08/index.html I have been running new layouter (at webrev.08) with the collection of class files I have around, and I believe there is a instance size regression. Look here: https://cr.openjdk.java.net/~shade/8237767/regression-1.txt -- Thanks, -Aleksey From matthias.baesken at sap.com Mon Feb 3 12:35:19 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 3 Feb 2020 12:35:19 +0000 Subject: RFR: 8238161: use os::fopen in HS code where possible In-Reply-To: References: Message-ID: Hi David, from what I understand it should behave in the same way it does when a file descriptor to a "normal" file is opened . At least checking for the FD_CLOEXEC flag for such an fd works as expected /* check that O_CLOEXEC works */ int fd = open64("/proc/cpuinfo", O_CLOEXEC, 0444); int flags; flags = ::fcntl(fd, F_GETFD); if (flags != -1) { if ((flags & FD_CLOEXEC) != 0) { printf("setting O_CLOEXEC with open seems to work.\n"); } else { printf("PROBLEM setting O_CLOEXEC with open !\n"); } } ... outputs : "setting O_CLOEXEC with open seems to work." Best regards, Matthias > Hi Matthias, > > Does close-on-exec have any meaning when applied to /proc filesystem ?? > > Thanks, > David > > On 31/01/2020 1:47 am, Baesken, Matthias wrote: > > Please review this change which adjusts a number of calls (in HS code) > from using fopen to os::fopen . > > > > There exists for some time a function os::fopen in HS code . This function > sets "close-on-exec" flags (when possible) to the opened filedescriptor. > > It might have some benefits to use os::fopen more in HS code, currently > there are still quite a few calls directly to fopen . > > (however there are also already a number of calls to os::fopen e.g. some > calls in os_linux.cpp /os_solaris.cpp should already look up os::fopen , and > logFileOutput.cpp uses os::fopen directly ) > > > > > > see also the thread > > https://mail.openjdk.java.net/pipermail/hotspot-dev/2020- > January/040641.html > > on hotspot-dev about the topic. > > > > > > > > Bug/webrev : > > > > https://bugs.openjdk.java.net/browse/JDK-8238161 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8238161.0/ > > > > > > Thanks, Matthias > > From matthias.baesken at sap.com Mon Feb 3 13:08:06 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 3 Feb 2020 13:08:06 +0000 Subject: RFR: 8238161: use os::fopen in HS code where possible Message-ID: A little bit more info about testing - I put the change in our internal build/test queue , so it means it went through JTREG , JCK and some benchmarks and SAP application related tests . Best regards, Matthias Please review this change which adjusts a number of calls (in HS code) from using fopen to os::fopen . There exists for some time a function os::fopen in HS code . This function sets "close-on-exec" flags (when possible) to the opened filedescriptor. It might have some benefits to use os::fopen more in HS code, currently there are still quite a few calls directly to fopen . (however there are also already a number of calls to os::fopen e.g. some calls in os_linux.cpp /os_solaris.cpp should already look up os::fopen , and logFileOutput.cpp uses os::fopen directly ) see also the thread https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-January/040641.html on hotspot-dev about the topic. Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8238161 http://cr.openjdk.java.net/~mbaesken/webrevs/8238161.0/ Thanks, Matthias From david.holmes at oracle.com Mon Feb 3 13:25:31 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 3 Feb 2020 23:25:31 +1000 Subject: RFR: 8238161: use os::fopen in HS code where possible In-Reply-To: References: Message-ID: <76c85895-d80b-964e-8887-44e309dc3959@oracle.com> On 3/02/2020 10:35 pm, Baesken, Matthias wrote: > Hi David, from what I understand it should behave in the same way it does when a file descriptor to a "normal" file is opened . > At least checking for the FD_CLOEXEC flag for such an fd works as expected > > /* check that O_CLOEXEC works */ > int fd = open64("/proc/cpuinfo", O_CLOEXEC, 0444); > int flags; > > flags = ::fcntl(fd, F_GETFD); > if (flags != -1) { > if ((flags & FD_CLOEXEC) != 0) { > printf("setting O_CLOEXEC with open seems to work.\n"); > } else { > printf("PROBLEM setting O_CLOEXEC with open !\n"); > } > } > > > ... outputs : "setting O_CLOEXEC with open seems to work." I expect it to be applicable but does it actually do anything such that it is necessary to go and make these changes for /proc filesystem usages? This is probably harmless but it also seems somewhat unnecessary to do. David > > Best regards, Matthias > > > > >> Hi Matthias, >> >> Does close-on-exec have any meaning when applied to /proc filesystem ?? >> >> Thanks, >> David >> >> On 31/01/2020 1:47 am, Baesken, Matthias wrote: >>> Please review this change which adjusts a number of calls (in HS code) >> from using fopen to os::fopen . >>> >>> There exists for some time a function os::fopen in HS code . This function >> sets "close-on-exec" flags (when possible) to the opened filedescriptor. >>> It might have some benefits to use os::fopen more in HS code, currently >> there are still quite a few calls directly to fopen . >>> (however there are also already a number of calls to os::fopen e.g. some >> calls in os_linux.cpp /os_solaris.cpp should already look up os::fopen , and >> logFileOutput.cpp uses os::fopen directly ) >>> >>> >>> see also the thread >>> https://mail.openjdk.java.net/pipermail/hotspot-dev/2020- >> January/040641.html >>> on hotspot-dev about the topic. >>> >>> >>> >>> Bug/webrev : >>> >>> https://bugs.openjdk.java.net/browse/JDK-8238161 >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8238161.0/ >>> >>> >>> Thanks, Matthias >>> From matthias.baesken at sap.com Mon Feb 3 14:02:54 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 3 Feb 2020 14:02:54 +0000 Subject: perfMemory_bsd.cpp get_user_name_slow misses check ? Message-ID: Hello, I noticed that perfMemory_bsd.cpp function static char* get_user_name_slow(int vmid, TRAPS) misses the // Since we don't create the backing store files in directories // pointed to by symbolic links, we also don't follow them when // looking for the files. We check for a symbolic link after the // call to opendir in order to eliminate a small window where the // symlink can be exploited. // if (!is_directory_secure(usrdir_name)) { FREE_C_HEAP_ARRAY(char, usrdir_name); os::closedir(subdirp); continue; } Check that aix/linux/solaris have . Should this be added ? Best Regards, Matthias From frederic.parain at oracle.com Mon Feb 3 16:00:39 2020 From: frederic.parain at oracle.com (Frederic Parain) Date: Mon, 3 Feb 2020 11:00:39 -0500 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: <1830530c-b086-534b-9bef-bb7e762fd6c6@redhat.com> References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> <1830530c-b086-534b-9bef-bb7e762fd6c6@redhat.com> Message-ID: Hi Aleksey, Thank you for reporting this issue. There?s definitively a regression if the layout of this class. The cause of this regression is the way the new algorithm allocates reference fields: it looks for a space big enough to allocate them contiguously (so there?s a single oopmap for all oops declared in the current class, and less work for the GC to find these oops). An unfortunate side effect of this strategy is that some unused slots big enough to store one or several oops are not considered as valid candidates because they cannot contain all oops. In the case you spotted, the abstract class has a long field and eight reference fields. The long field is allocated first, because of its size, creating a gap between the end of the header (with a compressed class pointer) and this first field. Then, the new algorithm tries to allocate the eight reference fields contiguously and look for a space big enough, and ignore this gap because it can only contain one oop. The story repeats itself with the sub-class which has a double field and twelve reference fields. The double field doesn?t fit in the gap, and the code tries to allocate all reference fields contiguously, ignoring the possibility to store an oop in this gap. A possible fix for this issue would be to drop the contiguous allocation of oops. Regards, Fred > On Feb 3, 2020, at 07:07, Aleksey Shipilev wrote: > > Hi again, > > On 1/31/20 5:03 PM, Frederic Parain wrote: >> New webrev: >> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.08/index.html > > I have been running new layouter (at webrev.08) with the collection of class files I have around, > and I believe there is a instance size regression. Look here: > https://cr.openjdk.java.net/~shade/8237767/regression-1.txt > > -- > Thanks, > -Aleksey > From shade at redhat.com Mon Feb 3 16:33:26 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 3 Feb 2020 17:33:26 +0100 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> <1830530c-b086-534b-9bef-bb7e762fd6c6@redhat.com> Message-ID: On 2/3/20 5:00 PM, Frederic Parain wrote: > The cause of this regression is the way the new algorithm allocates > reference fields: it looks for a space big enough to allocate them > contiguously (so there?s a single oopmap for all oops declared in > the current class, and less work for the GC to find these oops). Oh! Seems to me that "less work for GC to find these oops" is more theoretical than a practical loss of object density due to this artificial inflation. Single oop map per class is nice too, unless it leads to this... As I look through the layouter going through my local Maven Central snapshot, this case does not look rare for us not to care. Looks like every class with a long and 3+ oop fields is regressing in instance size, which is in line with your explanation. I believe this needs to be resolved before releasing the new layouter in the wild. -- Thanks, -Aleksey From erik.joelsson at oracle.com Mon Feb 3 16:33:27 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 3 Feb 2020 08:33:27 -0800 Subject: RFR: JDK-8238281 Raise minimum gcc version needed to 5.0 In-Reply-To: References: Message-ID: Looks good to me. /Erik On 2020-02-03 00:02, Magnus Ihse Bursie wrote: > JEP 347 "Adopt C++14 Language Features in HotSpot" (JDK-8208089) will > require that all compilers support the C++14 language extension. The > first gcc version to fully support C++14 is 5.0. We need to raise the > bar from 4.8 to 5.0. > > Since removing support for old compilers, especially gcc, can be > tricky to handle for some on old and odd setups, I think it's best to > do this as a separate step from the C++14 upgrade. > > On the way, this will allow us to remove some workarounds/close some > bugs for really old gcc versions. > > For hotspot, most notably this means: > > ?* The workarounds for no ATTRIBUTE_PRINTF are not needed anymore. > > * The workaround for ATTRIBUTE_ALIGNED is no longer needed. > > * The workaround in offset_of is not needed anymore. Instead, the > appropriate warning is disabled. (In fact, now only xlc need special > treatment for offset_of; if that could be fixed now, it would mean > that offset_of can be replaced by offsetof in hotspot.) > > * [Bug fix] The define CAN_USE_NAN_DEFINE (which is used only in > share/runtime/sharedRuntimeTrans.cpp), is now always defined for gcc. > This was previously done only for gcc >= 4.3 and < 5. I believe this > was a bug, and the intention was to allow for old-style NaN behavior > in gcc 4.2 and older. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8238281 > WebRev: > http://cr.openjdk.java.net/~ihse/JDK-8238281-make-gcc-5-minimum/webrev.01 > From gerard.ziemski at oracle.com Mon Feb 3 18:01:11 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Mon, 3 Feb 2020 12:01:11 -0600 Subject: perfMemory_bsd.cpp get_user_name_slow misses check ? In-Reply-To: References: Message-ID: <13afb679-8868-2f58-8795-8999a790d86e@oracle.com> hi Matthias, Looking at "open_directory_secure()" API, which is called just before "is_directory_secure()", seems to suggest that it is the "is_directory_secure()" call that is not needed here. So perhaps linux/win/solaris have it wrong and bsd got it right? // Open the directory of the given path and validate it. // Return a DIR * of the open directory. // static DIR *open_directory_secure(const char* dirname) { ? // Open the directory using open() so that it can be verified ? // to be secure by calling is_dirfd_secure(), opendir() and then check ? // to see if they are the same file system object.? This method does not ? // introduce a window of opportunity for the directory to be attacked that ? // calling opendir() and is_directory_secure() does. cheers On 2/3/20 8:02 AM, Baesken, Matthias wrote: > Hello, I noticed that perfMemory_bsd.cpp function > > static char* get_user_name_slow(int vmid, TRAPS) > > misses the > > // Since we don't create the backing store files in directories > // pointed to by symbolic links, we also don't follow them when > // looking for the files. We check for a symbolic link after the > // call to opendir in order to eliminate a small window where the > // symlink can be exploited. > // > if (!is_directory_secure(usrdir_name)) { > FREE_C_HEAP_ARRAY(char, usrdir_name); > os::closedir(subdirp); > continue; > } > > > Check that aix/linux/solaris have . > Should this be added ? > > > Best Regards, Matthias From shade at redhat.com Mon Feb 3 18:15:26 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 3 Feb 2020 19:15:26 +0100 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> <1830530c-b086-534b-9bef-bb7e762fd6c6@redhat.com> Message-ID: On 2/3/20 5:33 PM, Aleksey Shipilev wrote: > As I look through the layouter going through my local Maven Central snapshot, this case does not > look rare for us not to care. Looks like every class with a long and 3+ oop fields is regressing in > instance size, which is in line with your explanation. I believe this needs to be resolved before > releasing the new layouter in the wild. To add some more data, out of ~15M classes, about 14K (~0.01%) are affected: https://cr.openjdk.java.net/~shade/8237767/classes-regression-1.txt Of course, this does not mean we would have 0.01% footprint regression, because we don't know how many instances of the affected class would be present. But browsing the list, it looks like plenty of data classes, iterators, etc present. For fun, here is some friendly fire: org.openjdk.tools.javac.code.Symbol -- it has long Symbol.flags_field and plenty of oop fields. -- Thanks, -Aleksey From david.holmes at oracle.com Tue Feb 4 02:20:49 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Feb 2020 12:20:49 +1000 Subject: RFR: JDK-8238281 Raise minimum gcc version needed to 5.0 In-Reply-To: References: Message-ID: <01260258-6ead-b422-63f5-e42614a909ff@oracle.com> Hi Magnus, This all seems fine to me. Thanks, David On 3/02/2020 6:02 pm, Magnus Ihse Bursie wrote: > JEP 347 "Adopt C++14 Language Features in HotSpot" (JDK-8208089) will > require that all compilers support the C++14 language extension. The > first gcc version to fully support C++14 is 5.0. We need to raise the > bar from 4.8 to 5.0. > > Since removing support for old compilers, especially gcc, can be tricky > to handle for some on old and odd setups, I think it's best to do this > as a separate step from the C++14 upgrade. > > On the way, this will allow us to remove some workarounds/close some > bugs for really old gcc versions. > > For hotspot, most notably this means: > > ?* The workarounds for no ATTRIBUTE_PRINTF are not needed anymore. > > * The workaround for ATTRIBUTE_ALIGNED is no longer needed. > > * The workaround in offset_of is not needed anymore. Instead, the > appropriate warning is disabled. (In fact, now only xlc need special > treatment for offset_of; if that could be fixed now, it would mean that > offset_of can be replaced by offsetof in hotspot.) > > * [Bug fix] The define CAN_USE_NAN_DEFINE (which is used only in > share/runtime/sharedRuntimeTrans.cpp), is now always defined for gcc. > This was previously done only for gcc >= 4.3 and < 5. I believe this was > a bug, and the intention was to allow for old-style NaN behavior in gcc > 4.2 and older. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8238281 > WebRev: > http://cr.openjdk.java.net/~ihse/JDK-8238281-make-gcc-5-minimum/webrev.01 > From magnus.ihse.bursie at oracle.com Tue Feb 4 06:09:23 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 4 Feb 2020 07:09:23 +0100 Subject: RFR: JDK-8238281 Raise minimum gcc version needed to 5.0 In-Reply-To: <01260258-6ead-b422-63f5-e42614a909ff@oracle.com> References: <01260258-6ead-b422-63f5-e42614a909ff@oracle.com> Message-ID: <1E107AB6-8406-4FE0-BE7A-D4E7890EDB31@oracle.com> > 4 feb. 2020 kl. 03:20 skrev David Holmes : > > Hi Magnus, > > This all seems fine to me. Thank you David. /Magnus > > Thanks, > David > >> On 3/02/2020 6:02 pm, Magnus Ihse Bursie wrote: >> JEP 347 "Adopt C++14 Language Features in HotSpot" (JDK-8208089) will require that all compilers support the C++14 language extension. The first gcc version to fully support C++14 is 5.0. We need to raise the bar from 4.8 to 5.0. >> Since removing support for old compilers, especially gcc, can be tricky to handle for some on old and odd setups, I think it's best to do this as a separate step from the C++14 upgrade. >> On the way, this will allow us to remove some workarounds/close some bugs for really old gcc versions. >> For hotspot, most notably this means: >> * The workarounds for no ATTRIBUTE_PRINTF are not needed anymore. >> * The workaround for ATTRIBUTE_ALIGNED is no longer needed. >> * The workaround in offset_of is not needed anymore. Instead, the appropriate warning is disabled. (In fact, now only xlc need special treatment for offset_of; if that could be fixed now, it would mean that offset_of can be replaced by offsetof in hotspot.) >> * [Bug fix] The define CAN_USE_NAN_DEFINE (which is used only in share/runtime/sharedRuntimeTrans.cpp), is now always defined for gcc. This was previously done only for gcc >= 4.3 and < 5. I believe this was a bug, and the intention was to allow for old-style NaN behavior in gcc 4.2 and older. >> Bug: https://bugs.openjdk.java.net/browse/JDK-8238281 >> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8238281-make-gcc-5-minimum/webrev.01 From matthias.baesken at sap.com Tue Feb 4 08:20:05 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 4 Feb 2020 08:20:05 +0000 Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images References: <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com> Message-ID: > > Hi Erik, maybe we can just rename the configure option to > > --enable-stripped-pdbs-for-bundle > > AND make the default = no/false . > Then without setting the configure flag, everything stays as it is for JDK > vendors/distributors who do not want the stripped pdbs in the bundle. > > Others who set the flag, have to "teach" the developers that the bundle > already contains stripped pdbs that need to be replaced by full/"private" > pdbs in case better symbols/stacks are wanted . > I think that?s a good compromise. > > > Now if MS had been kind enough to define a separate file type for the > > stripped pdbs, so that they could live alongside the full pdbs, we > > wouldn't have this issue. > > Unfortunately that seems not to work, I tried to use the stripped pdb-files > with another extension but no success ? . > > An alternative could be to create 2 bundles when "--enable-stripped-pdbs- > for-bundle" is set to yes , one with one without stripped pdbs . > Hello, any more comment on this ? What about the option to create 2 bundles , one with one without stripped pdbs (should be very compatible to what we have and "nondisruptive" ) ? Best regards, Matthias From matthias.baesken at sap.com Tue Feb 4 11:17:48 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 4 Feb 2020 11:17:48 +0000 Subject: RFR: 8238161: use os::fopen in HS code where possible In-Reply-To: <76c85895-d80b-964e-8887-44e309dc3959@oracle.com> References: <76c85895-d80b-964e-8887-44e309dc3959@oracle.com> Message-ID: > > On 3/02/2020 10:35 pm, Baesken, Matthias wrote: > > Hi David, from what I understand it should behave in the same way it does > when a file descriptor to a "normal" file is opened . > > At least checking for the FD_CLOEXEC flag for such an fd works as > expected > > > > /* check that O_CLOEXEC works */ > > int fd = open64("/proc/cpuinfo", O_CLOEXEC, 0444); > > int flags; > > > > flags = ::fcntl(fd, F_GETFD); > > if (flags != -1) { > > if ((flags & FD_CLOEXEC) != 0) { > > printf("setting O_CLOEXEC with open seems to work.\n"); > > } else { > > printf("PROBLEM setting O_CLOEXEC with open !\n"); > > } > > } > > > > > > ... outputs : "setting O_CLOEXEC with open seems to work." > > I expect it to be applicable but does it actually do anything such that > it is necessary to go and make these changes for /proc filesystem > usages? This is probably harmless but it also seems somewhat unnecessary > to do. > Hi David , probably for my small example (or generally for /proc/cpuinfo) it would not be necessary. Regarding other fds dealing with /proc like cgroupSubsystem_linux.cpp 66 cgroups = fopen("/proc/cgroups", "r"); 121 cgroup = fopen("/proc/self/cgroup", "r"); 170 mntinfo = fopen("/proc/self/mountinfo", "r"); 224 mntinfo = fopen("/proc/self/mountinfo", "r"); Maybe the cgroup/container folks could comment how important/unimportant it is to have the close-on-exec set on the fds in this case? Best regards, Matthias From matthias.baesken at sap.com Tue Feb 4 12:26:22 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 4 Feb 2020 12:26:22 +0000 Subject: perfMemory_bsd.cpp get_user_name_slow misses check ? In-Reply-To: <13afb679-8868-2f58-8795-8999a790d86e@oracle.com> References: <13afb679-8868-2f58-8795-8999a790d86e@oracle.com> Message-ID: Hi Gerard , thanks for the input ! So I think we keep the mac/bsd version "as it is" . Best regards, Matthias > > hi Matthias, > > Looking at "open_directory_secure()" API, which is called just before > "is_directory_secure()", seems to suggest that it is the > "is_directory_secure()" call that is not needed here. > > So perhaps linux/win/solaris have it wrong and bsd got it right? > > > // Open the directory of the given path and validate it. > // Return a DIR * of the open directory. > // > static DIR *open_directory_secure(const char* dirname) { > ? // Open the directory using open() so that it can be verified > ? // to be secure by calling is_dirfd_secure(), opendir() and then check > ? // to see if they are the same file system object.? This method does not > ? // introduce a window of opportunity for the directory to be attacked > that > ? // calling opendir() and is_directory_secure() does. > > > > cheers > From lutz.schmidt at sap.com Tue Feb 4 15:52:47 2020 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Tue, 4 Feb 2020 15:52:47 +0000 Subject: 8237830: support O_CLOEXEC in os::open on other OS than Linux Message-ID: Hi Matthias, your change looks good. Please remember that I'm not a Reviewer! I have one little detail you could improve, though: Could you please use '{' (curly braces) to enclose the if and else blocks in os_bsd.cpp (as you did in os_aix.cpp)? I don't need to see a new wevrev. Thanks, Lutz ?On 28.01.20, 12:31, "hotspot-dev on behalf of Baesken, Matthias" wrote: New webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.1/ (one 'c' got lost somehow in the older webrev) Best regards, Matthias > > Hi David , on AIX I see O_CLOEXEC since AIX 7.1. Seems it was not > supported in 6.1, at least I cannot find it there . > > Regarding macOS, this thread > > https://groups.google.com/forum/#!topic/golang-dev/7mmT7o9GYb4 > > claims it is there since 10.7 . > Maybe someone can provide more detailed information ? > > Best regards, Matthias > > > From matthias.baesken at sap.com Tue Feb 4 15:56:25 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 4 Feb 2020 15:56:25 +0000 Subject: 8237830: support O_CLOEXEC in os::open on other OS than Linux In-Reply-To: References: Message-ID: Thanks for the review ! May I get a second review please ? Best regards, Matthias > Hi Matthias, > > your change looks good. Please remember that I'm not a Reviewer! > > I have one little detail you could improve, though: > Could you please use '{' (curly braces) to enclose the if and else blocks in > os_bsd.cpp (as you did in os_aix.cpp)? I don't need to see a new wevrev. > > Thanks, > Lutz > > ?On 28.01.20, 12:31, "hotspot-dev on behalf of Baesken, Matthias" dev-bounces at openjdk.java.net on behalf of matthias.baesken at sap.com> > wrote: > > New webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.1/ > > (one 'c' got lost somehow in the older webrev) > > Best regards, Matthias > > > > > > Hi David , on AIX I see O_CLOEXEC since AIX 7.1. Seems it was not > > supported in 6.1, at least I cannot find it there . > > > > Regarding macOS, this thread > > > > https://groups.google.com/forum/#!topic/golang-dev/7mmT7o9GYb4 > > > > claims it is there since 10.7 . > > Maybe someone can provide more detailed information ? > > > > Best regards, Matthias > > > > > > > > From magnus.ihse.bursie at oracle.com Tue Feb 4 21:48:07 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 4 Feb 2020 22:48:07 +0100 Subject: RFR: JDK-8238281 Raise minimum gcc version needed to 5.0 In-Reply-To: References: Message-ID: Can I have one more hotspot reviewer, please? /Magnus On 2020-02-03 09:02, Magnus Ihse Bursie wrote: > JEP 347 "Adopt C++14 Language Features in HotSpot" (JDK-8208089) will > require that all compilers support the C++14 language extension. The > first gcc version to fully support C++14 is 5.0. We need to raise the > bar from 4.8 to 5.0. > > Since removing support for old compilers, especially gcc, can be > tricky to handle for some on old and odd setups, I think it's best to > do this as a separate step from the C++14 upgrade. > > On the way, this will allow us to remove some workarounds/close some > bugs for really old gcc versions. > > For hotspot, most notably this means: > > ?* The workarounds for no ATTRIBUTE_PRINTF are not needed anymore. > > * The workaround for ATTRIBUTE_ALIGNED is no longer needed. > > * The workaround in offset_of is not needed anymore. Instead, the > appropriate warning is disabled. (In fact, now only xlc need special > treatment for offset_of; if that could be fixed now, it would mean > that offset_of can be replaced by offsetof in hotspot.) > > * [Bug fix] The define CAN_USE_NAN_DEFINE (which is used only in > share/runtime/sharedRuntimeTrans.cpp), is now always defined for gcc. > This was previously done only for gcc >= 4.3 and < 5. I believe this > was a bug, and the intention was to allow for old-style NaN behavior > in gcc 4.2 and older. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8238281 > WebRev: > http://cr.openjdk.java.net/~ihse/JDK-8238281-make-gcc-5-minimum/webrev.01 > From jesper.wilhelmsson at oracle.com Wed Feb 5 00:48:19 2020 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Wed, 5 Feb 2020 01:48:19 +0100 Subject: RFR: JDK-8238281 Raise minimum gcc version needed to 5.0 In-Reply-To: References: Message-ID: <8DB1E98B-385B-4D6E-A8AB-A2B9F28ACCF1@oracle.com> Looks good! In a few comments you refer to the version with just the number 5. Would it be more correct to write 5.0? (doc/building.html and doc/building.md) /Jesper > On 4 Feb 2020, at 22:48, Magnus Ihse Bursie wrote: > > Can I have one more hotspot reviewer, please? > > /Magnus > > On 2020-02-03 09:02, Magnus Ihse Bursie wrote: >> JEP 347 "Adopt C++14 Language Features in HotSpot" (JDK-8208089) will require that all compilers support the C++14 language extension. The first gcc version to fully support C++14 is 5.0. We need to raise the bar from 4.8 to 5.0. >> >> Since removing support for old compilers, especially gcc, can be tricky to handle for some on old and odd setups, I think it's best to do this as a separate step from the C++14 upgrade. >> >> On the way, this will allow us to remove some workarounds/close some bugs for really old gcc versions. >> >> For hotspot, most notably this means: >> >> * The workarounds for no ATTRIBUTE_PRINTF are not needed anymore. >> >> * The workaround for ATTRIBUTE_ALIGNED is no longer needed. >> >> * The workaround in offset_of is not needed anymore. Instead, the appropriate warning is disabled. (In fact, now only xlc need special treatment for offset_of; if that could be fixed now, it would mean that offset_of can be replaced by offsetof in hotspot.) >> >> * [Bug fix] The define CAN_USE_NAN_DEFINE (which is used only in share/runtime/sharedRuntimeTrans.cpp), is now always defined for gcc. This was previously done only for gcc >= 4.3 and < 5. I believe this was a bug, and the intention was to allow for old-style NaN behavior in gcc 4.2 and older. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8238281 >> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8238281-make-gcc-5-minimum/webrev.01 >> > From matthias.baesken at sap.com Wed Feb 5 08:19:23 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 5 Feb 2020 08:19:23 +0000 Subject: RFR: JDK-8238281 Raise minimum gcc version needed to 5.0 Message-ID: Hi Magnus, looks good to me ! > Date: Tue, 4 Feb 2020 22:48:07 +0100 > From: Magnus Ihse Bursie > To: build-dev at openjdk.java.net, hotspot-dev > > Subject: Re: RFR: JDK-8238281 Raise minimum gcc version needed to 5.0 > Message-ID: > Content-Type: text/plain; charset=utf-8; format=flowed > > Can I have one more hotspot reviewer, please? > > /Magnus > > On 2020-02-03 09:02, Magnus Ihse Bursie wrote: > > JEP 347 "Adopt C++14 Language Features in HotSpot" (JDK-8208089) will > > require that all compilers support the C++14 language extension. The > > first gcc version to fully support C++14 is 5.0. We need to raise the > > bar from 4.8 to 5.0. > > > > Since removing support for old compilers, especially gcc, can be > > tricky to handle for some on old and odd setups, I think it's best to > > do this as a separate step from the C++14 upgrade. > > > > On the way, this will allow us to remove some workarounds/close some > > bugs for really old gcc versions. > > > > For hotspot, most notably this means: > > > > ?* The workarounds for no ATTRIBUTE_PRINTF are not needed anymore. > > > > * The workaround for ATTRIBUTE_ALIGNED is no longer needed. > > > > * The workaround in offset_of is not needed anymore. Instead, the > > appropriate warning is disabled. (In fact, now only xlc need special > > treatment for offset_of; if that could be fixed now, it would mean > > that offset_of can be replaced by offsetof in hotspot.) > > > > * [Bug fix] The define CAN_USE_NAN_DEFINE (which is used only in > > share/runtime/sharedRuntimeTrans.cpp), is now always defined for gcc. > > This was previously done only for gcc >= 4.3 and < 5. I believe this > > was a bug, and the intention was to allow for old-style NaN behavior > > in gcc 4.2 and older. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8238281 > > WebRev: > > http://cr.openjdk.java.net/~ihse/JDK-8238281-make-gcc-5- > minimum/webrev.01 > > From magnus.ihse.bursie at oracle.com Wed Feb 5 09:46:38 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 5 Feb 2020 10:46:38 +0100 Subject: RFR: JDK-8238281 Raise minimum gcc version needed to 5.0 In-Reply-To: <8DB1E98B-385B-4D6E-A8AB-A2B9F28ACCF1@oracle.com> References: <8DB1E98B-385B-4D6E-A8AB-A2B9F28ACCF1@oracle.com> Message-ID: <1c54e747-36a8-779b-be8c-11cc5b605c84@oracle.com> On 2020-02-05 01:48, Jesper Wilhelmsson wrote: > Looks good! Thanks. > In a few comments you refer to the version with just the number 5. Would it be more correct to write 5.0? > (doc/building.html and doc/building.md) I agree, it looks a bit odd. I changed it to 5.0 before pushing. /Magnus > > /Jesper > >> On 4 Feb 2020, at 22:48, Magnus Ihse Bursie wrote: >> >> Can I have one more hotspot reviewer, please? >> >> /Magnus >> >> On 2020-02-03 09:02, Magnus Ihse Bursie wrote: >>> JEP 347 "Adopt C++14 Language Features in HotSpot" (JDK-8208089) will require that all compilers support the C++14 language extension. The first gcc version to fully support C++14 is 5.0. We need to raise the bar from 4.8 to 5.0. >>> >>> Since removing support for old compilers, especially gcc, can be tricky to handle for some on old and odd setups, I think it's best to do this as a separate step from the C++14 upgrade. >>> >>> On the way, this will allow us to remove some workarounds/close some bugs for really old gcc versions. >>> >>> For hotspot, most notably this means: >>> >>> * The workarounds for no ATTRIBUTE_PRINTF are not needed anymore. >>> >>> * The workaround for ATTRIBUTE_ALIGNED is no longer needed. >>> >>> * The workaround in offset_of is not needed anymore. Instead, the appropriate warning is disabled. (In fact, now only xlc need special treatment for offset_of; if that could be fixed now, it would mean that offset_of can be replaced by offsetof in hotspot.) >>> >>> * [Bug fix] The define CAN_USE_NAN_DEFINE (which is used only in share/runtime/sharedRuntimeTrans.cpp), is now always defined for gcc. This was previously done only for gcc >= 4.3 and < 5. I believe this was a bug, and the intention was to allow for old-style NaN behavior in gcc 4.2 and older. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8238281 >>> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8238281-make-gcc-5-minimum/webrev.01 >>> From thomas.stuefe at gmail.com Wed Feb 5 16:15:29 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 5 Feb 2020 17:15:29 +0100 Subject: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux In-Reply-To: References: Message-ID: Hi Matthias: + // Validate that the use of the O_CLOEXEC flag on open above worked. + static sig_atomic_t O_CLOEXEC_is_known_to_work = 0;+ if (!O_CLOEXEC_is_known_to_work) { int flags = ::fcntl(fd, F_GETFD);- if (flags != -1)+ if (flags != -1) {+ if ((flags & FD_CLOEXEC) != 0) {+ O_CLOEXEC_is_known_to_work = 1;+ } else { ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); }-#endif+ }+ } This is either too complicated or not complicated enough :) if we know for a fact that open(..O_CLOEXEC) did work at least once, we never again check. if we know for a fact that open(..O_CLOEXEC) failed to work, we recheck again and again. Do we expect it to suddenly start working? If yes, could it also suddenly start failing? Should we then not check always? I propose the following change: 1) If open(..O_CLOEXEC) could sometimes fail and sometimes not, or we are just not sure, I would do the fcntl test every time. Or, even just always set the flag again with fcntl. So, just remove the O_CLOEXEC_is_known_to_work conditional variable. Which would reduce the code to a one liner: + ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); 2) If you are sure that it either always works or fails, you need a conditional with three states: unknown, works, does not work. And only check if unknown. I personally would prefer (1) because it is way simpler and fcntl is not such an expensive operation that we have to save time (also, how often do we open files anyway?). ---- Otherwise looks fine. This is a useful fix. Cheers, Thomas On Tue, Jan 28, 2020 at 12:32 PM Baesken, Matthias wrote: > New webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.1/ > > (one 'c' got lost somehow in the older webrev) > > Best regards, Matthias > > > > > > Hi David , on AIX I see O_CLOEXEC since AIX 7.1. Seems it was not > > supported in 6.1, at least I cannot find it there . > > > > Regarding macOS, this thread > > > > https://groups.google.com/forum/#!topic/golang-dev/7mmT7o9GYb4 > > > > claims it is there since 10.7 . > > Maybe someone can provide more detailed information ? > > > > Best regards, Matthias > > > > > > > > From matthias.baesken at sap.com Thu Feb 6 07:55:24 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 6 Feb 2020 07:55:24 +0000 Subject: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux In-Reply-To: References: Message-ID: Hi Thomas , the coding was borrowed from linux, see : https://hg.openjdk.java.net/jdk/jdk/file/7b57401deb0c/src/hotspot/os/linux/os_linux.cpp#l5616 I would be fine with always calling fcntl (your suggestion 1) , do you think it was done otherwise back then in os_linux because of performance issues ? Probably os::open in os_linux should be adjusted then to the new ?always call fcntl? approach too, right ? Best regards, Matthias Hi Matthias: + // Validate that the use of the O_CLOEXEC flag on open above worked. + static sig_atomic_t O_CLOEXEC_is_known_to_work = 0; + if (!O_CLOEXEC_is_known_to_work) { int flags = ::fcntl(fd, F_GETFD); - if (flags != -1) + if (flags != -1) { + if ((flags & FD_CLOEXEC) != 0) { + O_CLOEXEC_is_known_to_work = 1; + } else { ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); } -#endif + } + } This is either too complicated or not complicated enough :) if we know for a fact that open(..O_CLOEXEC) did work at least once, we never again check. if we know for a fact that open(..O_CLOEXEC) failed to work, we recheck again and again. Do we expect it to suddenly start working? If yes, could it also suddenly start failing? Should we then not check always? I propose the following change: 1) If open(..O_CLOEXEC) could sometimes fail and sometimes not, or we are just not sure, I would do the fcntl test every time. Or, even just always set the flag again with fcntl. So, just remove the O_CLOEXEC_is_known_to_work conditional variable. Which would reduce the code to a one liner: + ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); 2) If you are sure that it either always works or fails, you need a conditional with three states: unknown, works, does not work. And only check if unknown. I personally would prefer (1) because it is way simpler and fcntl is not such an expensive operation that we have to save time (also, how often do we open files anyway?). ---- Otherwise looks fine. This is a useful fix. Cheers, Thomas On Tue, Jan 28, 2020 at 12:32 PM Baesken, Matthias > wrote: New webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.1/ (one 'c' got lost somehow in the older webrev) Best regards, Matthias From huangjia at loongson.cn Thu Feb 6 09:20:54 2020 From: huangjia at loongson.cn (Jia Huang) Date: Thu, 6 Feb 2020 17:20:54 +0800 Subject: RFR:JDK-8238586: [TESTBUG] vmTestbase/jit/tiered/Test.java failed when TieredCompilation is disabled Message-ID: <2b2bd9d6-d332-d079-5b26-06def034be38@loongson.cn> Hi all, JBS: https://bugs.openjdk.java.net/browse/JDK-8238586 When TieredCompilation is disabled, the test will fail. Please review the following fix which make the test pass when disabled TieredCompilation. ------------------------------------------------------ diff -r e45d8ca836b5 -r 67bef3d6692f test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java --- a/test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java? Mon Feb 03 22:10:59 2020 +0800 +++ b/test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java? Thu Feb 06 15:25:19 2020 +0800 @@ -46,7 +46,7 @@ ?import jdk.test.lib.process.ProcessTools; ?public class Test { -??? private static String UNSUPPORTED_OPTION_MESSAGE = "-XX:TieredCompilation not supported in this VM"; +??? private static String UNSUPPORTED_OPTION_MESSAGE = "-XX:+TieredCompilation not supported in this VM"; ???? private static String REGEXP = "^[0-9.]+: \\[compile level=\\d"; ???? public static void main(String[] args) throws Exception { ???????? { @@ -56,7 +56,7 @@ ???????????????????? "-XX:+PrintTieredEvents", ???????????????????? "-version"); ???????????? var output = new OutputAnalyzer(pb.start()); -??????????? if (output.getStdout().contains(UNSUPPORTED_OPTION_MESSAGE)) { +??????????? if (output.getStderr().contains(UNSUPPORTED_OPTION_MESSAGE)) { ???????????????? throw new SkippedException(UNSUPPORTED_OPTION_MESSAGE); ???????????? } ???????????? output.shouldHaveExitValue(0) ------------------------------------------------------ For more info, please see the JBS. Thanks a lot Best regards, Jia. From thomas.stuefe at gmail.com Thu Feb 6 10:29:52 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 6 Feb 2020 11:29:52 +0100 Subject: Question about JDK-8043780 (was: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux) In-Reply-To: References: Message-ID: Hi Martin, do you remember https://bugs.openjdk.java.net/browse/JDK-8043780 ? Some questions came up about your patch: Your original coding does this check-O_CLOEXEC-after-open-and-set-if-missing-dance, but if it worked ar least once it will stop doing that. However, if it fails to work it will retest over and over again. Was this intentional? Do we expect it to start working at some point? If yes, could it also stop working at some point? I wonder whether we could shorten the coding to a simple, unconditional ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); to be done after open. If open(CLOEXEC) worked, this would be a unnecessary noop. If it did not, we set it now. We'd always pay for a fcntl call but I think it is cheap and open is not called that often anyway. -- But then, I wonder whether we can scratch that coding altogether. O_CLOEXEC has been in the kernel since 2.6.23. This is 13 years old. 2.6 line topped out at 2.6.39 in 2011, and I would expect anyone still running on 2.6 to have at least a recent kernel. What do you think? Thank you! Cheers, Thomas On Thu, Feb 6, 2020 at 8:55 AM Baesken, Matthias wrote: > Hi Thomas , the coding was borrowed from linux, see : > > > > > https://hg.openjdk.java.net/jdk/jdk/file/7b57401deb0c/src/hotspot/os/linux/os_linux.cpp#l5616 > > > > I would be fine with always calling fcntl (your suggestion 1) , do you > think it was done otherwise back then in os_linux because of > performance issues ? > > Probably os::open in os_linux should be adjusted then to the new > ?always call fcntl? approach too, right ? > > > > > > Best regards, Matthias > > > > > > > > Hi Matthias: > > > > + // Validate that the use of the O_CLOEXEC flag on open above worked. > > + static sig_atomic_t O_CLOEXEC_is_known_to_work = 0; > > + if (!O_CLOEXEC_is_known_to_work) { > > int flags = ::fcntl(fd, F_GETFD); > > - if (flags != -1) > > + if (flags != -1) { > > + if ((flags & FD_CLOEXEC) != 0) { > > + O_CLOEXEC_is_known_to_work = 1; > > + } else { > > ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); > > } > > -#endif > > + } > > + } > > > > This is either too complicated or not complicated enough :) > > > if we know for a fact that open(..O_CLOEXEC) did work at least once, we > never again check. > if we know for a fact that open(..O_CLOEXEC) failed to work, we recheck > again and again. Do we expect it to suddenly start working? If yes, could > it also suddenly start failing? Should we then not check always? > > I propose the following change: > > 1) If open(..O_CLOEXEC) could sometimes fail and sometimes not, or we are > just not sure, I would do the fcntl test every time. Or, even just always > set the flag again with fcntl. So, just remove the > O_CLOEXEC_is_known_to_work conditional variable. > > Which would reduce the code to a one liner: > > + ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); > > 2) If you are sure that it either always works or fails, you need a > conditional with three states: unknown, works, does not work. And only > check if unknown. > > I personally would prefer (1) because it is way simpler and fcntl is not > such an expensive operation that we have to save time (also, how often do > we open files anyway?). > > ---- > > > > Otherwise looks fine. This is a useful fix. > > > Cheers, Thomas > > > > > > > > > > On Tue, Jan 28, 2020 at 12:32 PM Baesken, Matthias < > matthias.baesken at sap.com> wrote: > > New webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.1/ > > (one 'c' got lost somehow in the older webrev) > > Best regards, Matthias > > From christoph.langer at sap.com Thu Feb 6 11:36:20 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 6 Feb 2020 11:36:20 +0000 Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images In-Reply-To: References: <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com> Message-ID: Hi, let me chime in to this discussion at this point. So, first of all, Matthias, thanks for your work so far. Erik, I fully understand your points and I agree that it's probably a good idea to refactor this whole process of creating these different types of bundles. Our idea is to provide functionality in the build system to add "public" or stripped debug files to the delivery image of the JDK. This would provide better information in case of crashes and would hence allow for better supportability. That's something we'd probably want to enable in SapMachine binary distributions. So, can we get to add a configure option like the one proposed by Matthias into the current code base? The option should be turned off by default. If it is switched on, the images/jdk folder in the build directory will have both, the *stripped.pdb files and the standard *.pdb files. However, having *stripped.pdb files around should not matter in terms of functionality (for developers and testing) as they'd not be used in the presence of the "real" pdb files anyway. The actual JDK bundle for delivery would then contain the *stripped.pdb files, renamed to *.pdb. And the debug symbols bundle would have the full *.pdb files only. Both could then be overlaid as needed. I think you raised two concerns. One is that you'd rather like to refactor bundling for several reasons. But I guess, should you eventually get to your refactoring, it shouldn't be a problem to take the functionality of this new option along. The other was regarding JMODs. I believe it's correct, that JMODs have never carried external debug information. For platforms other than MS Windows that's actually not a big problem because debug information can be internalized. And jlink has gotten several additions to set flags for stripping that data to the right level. I assume if jlinked images on Windows should ever be enabled to carry debug information, inclusion of external debug files would have to be added to JMODs and jlink. But that's definitely out of scope here. So, what do you think? What speaks against adding this option (that is off by default)? Thanks Christoph > -----Original Message----- > From: build-dev On Behalf Of Erik > Joelsson > Sent: Donnerstag, 23. Januar 2020 18:49 > To: Baesken, Matthias ; David Holmes > ; 'build-dev at openjdk.java.net' dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' dev at openjdk.java.net> > Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows for > jdk images > > > On 2020-01-23 00:03, Baesken, Matthias wrote: > > Hi Erik, yes true sorry for answering your comments a bit late . > > > >> If a user runs jlink and includes all the jmods we ship with the JDK, the > result > >> should be essentially equivalent to the original JDK image. The way the > >> stripped pdb files are included in the bundles sort of at the last > >> second of the build here breaks this property. > > I think we should address this in a separate bug/CR . > Maybe. I realize that my proposal below is quite a big task. But on the > other hand, I don't think breaking the relationship between the jmods > and the distribution bundles is on the table really. > > Looking for example into a Linux build, I see a lot of debuginfo files in the > jdk image (more or less for every shared lib) . > > But when looking into the jmods of that jdk image , no debuginfo files are > in there ( I checked the java.base jmod). > > So putting the files with debug information into the jmods seems to be > something that was not done so far cross platform (or is there some build > switch for it that I did not check?) . > > Maybe to keep the jmods as small as possible . > > No, we do not put the debuginfo files in the jmods nor the bundles > because those are not intended to be shipped to customers. We are > currently overlaying them into images/jdk in the build output to make > them available for local debugging. (This is convoluted and I would very > much like to get away from this practice at some point so that there is > a 1-1 mapping between images/jdk and bundles/jdk*-bin.tar.gz.) The > stripped pdb files you are proposing are on the contrary intended for > shipping to customers (as I understand your proposal) so comparing them > with the debuginfo files is not relevant. > > Now if MS had been kind enough to define a separate file type for the > stripped pdbs, so that they could live alongside the full pdbs, we > wouldn't have this issue. The heart of the problem is that only one set > of files (either stripped or full) can be present and usable in > images/jdk at a time. We have 2 main uses for images/jdk. > > 1. Developer running and debugging locally > > 2. Serve as the source for generating the distribution bundles > > We currently have one image serving both of these purposes, which is > already creating complicated and convoluted build steps. To properly > solve this I would argue for splitting these two apart into two > different images for the two different purposes. The build procedure > would then be, first build the images for distribution, only containing > what should go into each bundle. Then create the developer jdk image by > copying files from the distribution images. On Windows, the first image > would contain the stripped pdbs and when building the second, those > would get overwritten with the full pdbs. > > Now that I figured out a working model that would solve a bunch of other > problems as well, I would love to implement it, but I doubt I will have > time in the near future. > > /Erik > > > > >> To properly implement this, care will need to be taken to juggle the two > >> sets of pdb files around, making sure each build and test use case has > >> the correct one in place where and when it's needed. Quite possibly, we > >> cannot cover all use cases with one build configuration. Developers > >> needing the full debug symbols when debugging locally would likely need > >> to disable the stripped symbols so they get the full symbols everywhere. > >> Possibly this would need to be the default for debug builds and > >> configurable for release builds. > > From my limited experience , the developers do not work with the > bundles (that would contain now after my patch the stripped pds) but with > a "normal" jdk image that is created my make all. > > > > Best regards, Matthias > > > > > > > >> This still does not address anything in my objection. > >> > >> /Erik > >> > >> On 2020-01-22 07:46, Baesken, Matthias wrote: > >>> Hello, here is an updated version : > >>> > >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/ > >>> > >>> this one supports a configure switch "--enable-stripped-pdbs" to > enable > >> the feature . > >>> Best regards, Matthias > >>> > >>> > >>>> -----Original Message----- > >>>> From: Baesken, Matthias > >>>> Sent: Dienstag, 21. Januar 2020 11:03 > >>>> To: 'David Holmes' ; 'build- > >>>> dev at openjdk.java.net' ; 'hotspot- > >>>> dev at openjdk.java.net' > >>>> Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows > for > >>>> jdk images > >>>> > >>>> > >>>> Hi David , yes I think it makes sense to have a configure option for this . > >>>> Not everyone would like to have a larger JDK (even it is only a bit > larger). > >>>> From magnus.ihse.bursie at oracle.com Thu Feb 6 12:48:10 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 6 Feb 2020 13:48:10 +0100 Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images In-Reply-To: References: <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com> Message-ID: <2b92559a-6a1a-119a-1e62-eed6f9d3bfc8@oracle.com> On 2020-02-06 12:36, Langer, Christoph wrote: > Hi, > > let me chime in to this discussion at this point. So, first of all, Matthias, thanks for your work so far. > > Erik, I fully understand your points and I agree that it's probably a good idea to refactor this whole process of creating these different types of bundles. > > Our idea is to provide functionality in the build system to add "public" or stripped debug files to the delivery image of the JDK. This would provide better information in case of crashes and would hence allow for better supportability. That's something we'd probably want to enable in SapMachine binary distributions. > > So, can we get to add a configure option like the one proposed by Matthias into the current code base? > The option should be turned off by default. If it is switched on, the images/jdk folder in the build directory will have both, the *stripped.pdb files and the standard *.pdb files. However, having *stripped.pdb files around should not matter in terms of functionality (for developers and testing) as they'd not be used in the presence of the "real" pdb files anyway. The actual JDK bundle for delivery would then contain the *stripped.pdb files, renamed to *.pdb. And the debug symbols bundle would have the full *.pdb files only. Both could then be overlaid as needed. > > I think you raised two concerns. > One is that you'd rather like to refactor bundling for several reasons. But I guess, should you eventually get to your refactoring, it shouldn't be a problem to take the functionality of this new option along. > The other was regarding JMODs. I believe it's correct, that JMODs have never carried external debug information. For platforms other than MS Windows that's actually not a big problem because debug information can be internalized. And jlink has gotten several additions to set flags for stripping that data to the right level. I assume if jlinked images on Windows should ever be enabled to carry debug information, inclusion of external debug files would have to be added to JMODs and jlink. But that's definitely out of scope here. > > So, what do you think? What speaks against adding this option (that is off by default)? While Erik will need to comment on this himself, from my POV this looks okay. The conditions are: * This is controlled by a separate option (or perhaps even better as a new argument to --with-native-debug-symbols), so without this, nothing will change. * The code need to make sure to separate stripped.pdb and normal pdb files, when enabled. * And there must not be a heavy penalty in added code complexity. /Magnus > > Thanks > Christoph > >> -----Original Message----- >> From: build-dev On Behalf Of Erik >> Joelsson >> Sent: Donnerstag, 23. Januar 2020 18:49 >> To: Baesken, Matthias ; David Holmes >> ; 'build-dev at openjdk.java.net' > dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' > dev at openjdk.java.net> >> Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows for >> jdk images >> >> >> On 2020-01-23 00:03, Baesken, Matthias wrote: >>> Hi Erik, yes true sorry for answering your comments a bit late . >>> >>>> If a user runs jlink and includes all the jmods we ship with the JDK, the >> result >>>> should be essentially equivalent to the original JDK image. The way the >>>> stripped pdb files are included in the bundles sort of at the last >>>> second of the build here breaks this property. >>> I think we should address this in a separate bug/CR . >> Maybe. I realize that my proposal below is quite a big task. But on the >> other hand, I don't think breaking the relationship between the jmods >> and the distribution bundles is on the table really. >>> Looking for example into a Linux build, I see a lot of debuginfo files in the >> jdk image (more or less for every shared lib) . >>> But when looking into the jmods of that jdk image , no debuginfo files are >> in there ( I checked the java.base jmod). >>> So putting the files with debug information into the jmods seems to be >> something that was not done so far cross platform (or is there some build >> switch for it that I did not check?) . >>> Maybe to keep the jmods as small as possible . >> No, we do not put the debuginfo files in the jmods nor the bundles >> because those are not intended to be shipped to customers. We are >> currently overlaying them into images/jdk in the build output to make >> them available for local debugging. (This is convoluted and I would very >> much like to get away from this practice at some point so that there is >> a 1-1 mapping between images/jdk and bundles/jdk*-bin.tar.gz.) The >> stripped pdb files you are proposing are on the contrary intended for >> shipping to customers (as I understand your proposal) so comparing them >> with the debuginfo files is not relevant. >> >> Now if MS had been kind enough to define a separate file type for the >> stripped pdbs, so that they could live alongside the full pdbs, we >> wouldn't have this issue. The heart of the problem is that only one set >> of files (either stripped or full) can be present and usable in >> images/jdk at a time. We have 2 main uses for images/jdk. >> >> 1. Developer running and debugging locally >> >> 2. Serve as the source for generating the distribution bundles >> >> We currently have one image serving both of these purposes, which is >> already creating complicated and convoluted build steps. To properly >> solve this I would argue for splitting these two apart into two >> different images for the two different purposes. The build procedure >> would then be, first build the images for distribution, only containing >> what should go into each bundle. Then create the developer jdk image by >> copying files from the distribution images. On Windows, the first image >> would contain the stripped pdbs and when building the second, those >> would get overwritten with the full pdbs. >> >> Now that I figured out a working model that would solve a bunch of other >> problems as well, I would love to implement it, but I doubt I will have >> time in the near future. >> >> /Erik >> >>>> To properly implement this, care will need to be taken to juggle the two >>>> sets of pdb files around, making sure each build and test use case has >>>> the correct one in place where and when it's needed. Quite possibly, we >>>> cannot cover all use cases with one build configuration. Developers >>>> needing the full debug symbols when debugging locally would likely need >>>> to disable the stripped symbols so they get the full symbols everywhere. >>>> Possibly this would need to be the default for debug builds and >>>> configurable for release builds. >>> From my limited experience , the developers do not work with the >> bundles (that would contain now after my patch the stripped pds) but with >> a "normal" jdk image that is created my make all. >>> Best regards, Matthias >>> >>> >>> >>>> This still does not address anything in my objection. >>>> >>>> /Erik >>>> >>>> On 2020-01-22 07:46, Baesken, Matthias wrote: >>>>> Hello, here is an updated version : >>>>> >>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/ >>>>> >>>>> this one supports a configure switch "--enable-stripped-pdbs" to >> enable >>>> the feature . >>>>> Best regards, Matthias >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: Baesken, Matthias >>>>>> Sent: Dienstag, 21. Januar 2020 11:03 >>>>>> To: 'David Holmes' ; 'build- >>>>>> dev at openjdk.java.net' ; 'hotspot- >>>>>> dev at openjdk.java.net' >>>>>> Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows >> for >>>>>> jdk images >>>>>> >>>>>> >>>>>> Hi David , yes I think it makes sense to have a configure option for this . >>>>>> Not everyone would like to have a larger JDK (even it is only a bit >> larger). From frederic.parain at oracle.com Thu Feb 6 16:28:45 2020 From: frederic.parain at oracle.com (Frederic Parain) Date: Thu, 6 Feb 2020 11:28:45 -0500 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> Message-ID: <7E7AEBD0-09DD-4EA0-B2E0-2102EDABCAD9@oracle.com> David, The deprecated flag has been added to the table and to the test. Indentation/spaces have been fixed. CSR has been updated too. New webrev: http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html The webrev also contains some modifications to address issues found by Aleksey. See my response to his e-mail to have the incremental webrev. Thank you, Fred > On Feb 3, 2020, at 01:24, David Holmes wrote: > > Hi Fred, > > On 1/02/2020 2:03 am, Frederic Parain wrote: >> Hi David, >> I?ve fixed all the issues you mentioned below. >> According to the discussion about the VM flags, the following modifications >> have also been made: >> globals.hpp: >> UseNewFieldLayout is now a deprecated product flag > > You need to add it to the deprecated flag table in arguments.cpp and to the test > > test/hotspot/jtreg/runtime/CommandLine/VMDeprecatedOptions.java > >> fieldLayoutBuilder.cpp >> lines 350-362: >> UseEmptySlotsInSupers VM option now controls both new optimizations >> lines 449-464 >> Fix an issue with layout printing (inherited fields were printed >> only for the direct super-class, now all inherited fields are printed) >> New webrev: >> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.08/index.html > > Looks good. A couple of minor follow ups ... > > src/hotspot/share/classfile/classFileParser.cpp > > ! OopMapBlock* nonstatic_oop_map = _nonstatic_oop_maps; > > Still an extra space after * > > + FieldLayoutBuilder lb(class_name(), super_klass(), _cp, _fields, > + _parsed_annotations->is_contended(), _field_info); > > Indentation of continuing line is wrong. > > For the test, you have now restricted it to only run on 64-bit even though it has code that appears to allow for 32-bit. As this isn't a 64-bit specific feature I think it should run on all platforms. So this: > > /* > * @test > * @bug 8237767 > * @requires vm.bits == "64" > * @summary Verify behaviour of field layout algorithm > * @library /test/lib > * @modules java.base/jdk.internal.misc > * java.management > * @run main/othervm -XX:+UseCompressedOops -XX:+UseCompressedClassPointers FieldDensityTest > * @run main/othervm -XX:+UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest > * @run main/othervm -XX:-UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest > */ > > can become: > > /* > * @test > * @bug 8237767 > * @summary Verify behaviour of field layout algorithm > * @library /test/lib > * @modules java.base/jdk.internal.misc > * java.management > * @run main/othervm FieldDensityTest > */ > > /* > * @test > * @requires vm.bits == "64" > * @run main/othervm -XX:+UseCompressedOops -XX:+UseCompressedClassPointers FieldDensityTest > * @run main/othervm -XX:+UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest > */ > >> CR and CSR have been updated accordingly (new VM flags explanations in the CSR). > > There is one small glitch in the CSR. Code removal associated with a flag occurs when it is obsoleted. So under the proposed timeline of deprecation in 15 and obsoletion in 16, the old code will be removed in 16 not 17. Comment added to CSR to that effect. > > Thanks, > David > >> Thank you, >> Fred >>> On Jan 29, 2020, at 02:06, David Holmes wrote: >>> >>> Hi Fred, >>> >>> I've looked at the v7 version. A few more stylistic comments on that first. Note, no need for an item by item response unless that makes it easier for you to track :) >>> >>> src/hotspot/share/classfile/classFileParser.cpp >>> >>> #include "classfile/defaultMethods.hpp" >>> +#include "classfile/fieldLayoutBuilder.hpp" >>> #include "classfile/dictionary.hpp" >>> >>> Include files are not in alphabetical order. >>> >>> + * This may well change: FixMe if doesn't, >>> >>> s/if/if it/ >>> >>> + //Make a temp copy, and iterate through and copy back into the orig >>> >>> Space after // >>> >>> s/orig/original/ >>> >>> + OopMapBlock* nonstatic_oop_map = _nonstatic_oop_maps; >>> >>> Extra space after * >>> >>> + assert(ik->nonstatic_oop_map_count() == _field_info->oop_map_blocks->_nonstatic_oop_map_count, >>> + "sanity"); >>> >>> Second line needs to be indented further: >>> >>> assert(ik->nonstatic_oop_map_count() == _field_info->oop_map_blocks->_nonstatic_oop_map_count, >>> "sanity"); >>> >>> --- >>> >>> src/hotspot/share/classfile/classFileParser.hpp >>> >>> +public: >>> + OopMapBlock* _nonstatic_oop_maps; >>> + unsigned int _nonstatic_oop_map_count; >>> + unsigned int _max_nonstatic_oop_maps; >>> + >>> + public: >>> >>> Second public uneeded. First public may be indented wrong (I'm not sure what the rule is - single space indent?) >>> >>> class ClassFileParser { >>> + friend class FieldLayoutBuilder; >>> + friend class FieldLayout; >>> >>> class ClassAnnotationCollector; >>> class FieldAllocationCount; >>> class FieldAnnotationCollector; >>> >>> Indents are different. I think the class forward declarations should have extra space. >>> >>> --- >>> >>> src/hotspot/share/oops/instanceKlass.hpp >>> >>> + void increment_count(int diff) { _count += diff; } >>> >>> Extra spaces before { >>> >>> --- >>> >>> src/hotspot/share/runtime/globals.hpp >>> >>> + diagnostic(bool, UseNewFieldLayout, true, \ >>> + "Use new algorithm to compute layouts") \ >>> + \ >>> + product(bool, UseEmptySlotsInSupers, true, >>> >>> Not sure I see why one flag is diagnostic and the other product. Do you expect people to need to disable using empty slots more so than needing to disable using the new field layout altogether? >>> >>> --- >>> >>> src/hotspot/share/classfile/fieldLayoutBuilder.cpp >>> >>> + assert(kind == EMPTY || kind == RESERVED || kind == PADDING || kind == INHERITED, >>> + "Otherwise, should use the constructor with a field index argument"); >>> >>> Indentation of second line is wrong. >>> >>> + assert(kind == REGULAR || kind == FLATTENED || kind == INHERITED, >>> + "Other kind do not have a field index"); >>> >>> Ditto. >>> >>> >>> + if (list == NULL) return; >>> + if (start == NULL) { >>> + start = this->_start; >>> + } >>> >>> Inconsistent style for single statement if-blocks. Same thing later in the file. >>> >>> + output->print_cr(" @%d \"%s\" %s %d/%d %s", >>> + b->offset(), >>> + fi->name(_cp)->as_C_string(), >>> + fi->signature(_cp)->as_C_string(), >>> + b->size(), >>> + b->alignment(), >>> + "REGULAR"); >>> >>> Incorrect identation of continuing line. Same for all the following print blocks. >>> >>> + } else if (_classname == vmSymbols::java_lang_Boolean() || >>> + _classname == vmSymbols::java_lang_Character() || >>> + _classname == vmSymbols::java_lang_Float() || >>> + _classname == vmSymbols::java_lang_Double() || >>> + _classname == vmSymbols::java_lang_Byte() || >>> + _classname == vmSymbols::java_lang_Short() || >>> + _classname == vmSymbols::java_lang_Integer() || >>> + _classname == vmSymbols::java_lang_Long()) { >>> >>> Incorrect identation of continuing line. >>> >>> --- >>> >>> src/hotspot/share/classfile/fieldLayoutBuilder.hpp >>> >>> +// and the boxing classes). The rational for having multiple methods >>> >>> s/rational/rationale/ >>> >>> + FieldLayoutBuilder(const Symbol* classname, const InstanceKlass* super_klass, ConstantPool* constant_pool, >>> + Array* fields, bool is_contended, FieldLayoutInfo* info); >>> >>> Indentation wrong for continuing line. >>> >>> + int get_alignment() { >>> + assert(_alignment != -1, "Uninitialized"); >>> + return _alignment; >>> + } >>> >>> Indenting appears off by one. >>> >>> --- >>> >>> test/hotspot/jtreg/runtime/FieldLayout/FieldDensityTest.java >>> >>> + * @run main/othervm -XX:+UseCompressedOops -XX:+UseCompressedClassPointers FieldDensityTest >>> + * @run main/othervm -XX:+UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest >>> + * @run main/othervm -XX:-UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest >>> >>> The test won't run on 32-bit platforms as the compressed oops flags won't exist. >>> >>> --- >>> >>> Some follow up comments below ... >>> >>> With trimming ... >>> >>> On 25/01/2020 3:20 am, Frederic Parain wrote: >>>>> On Jan 24, 2020, at 08:19, David Holmes wrote: >>>>> >>>>> 466 int super_flen = super->nof_nonstatic_fields(); >>>>> >>>>> Could be folded directly into the assert so we don't call in product. >>>> Calling not_nonstatic_fields() has the side effect to compute non-static fields, >>>> which is required to get a correct value when reading super->_nonstatic_fields, >>>> so the call is needed even in product builds. >>> >>> Yuck! That's a horrible side-effect - but not your fault obviously. :) It would be better to have a nonstatic_fields() accessor that has the same lazy initialization side-effect. >>> >>>>> General style issue: when breaking a long line with a method call, the new line (containing arguments) should be indented to the opening ( of the method call e.g. >>> ... >>>>> etc. This applies across all files. >>>> Fixes applied lines 4003, 4011, 4041, 4138, 4143. >>> >>> Fix was also needed in other files. Current issues highlighted above. >>> >>>>> >>>>> src/hotspot/share/oops/instanceKlass.hpp >>>>> >>>>> You need to be careful with _extra_flags usage if there can be concurrently updated bits. At the moment it looks like redefinition is a mutable dynamic property, whilst "contended annotations" should be a static immutable property - is that right? >>>> Correct, _has_contended_annotations is a static immutable property, while _is_being_redefined is a mutable one. >>> >>> Good to know. My concern is that if someone adds a new mutable flag bit the need for atomic updates may not be noticed. We got bitten by this in the past with a flag field and I think we eventually migrated all of the mutable bits out into their own field. (Coleen should recall that :) ). >>> >>>>> 61 FLATTENED, // flattened field >>>>> >>>>> Does this have any meaning before inline types come in? >>>> Yes, I wanted to reserved the entry in the enum. >>> >>> Hmmm a tenuous "okay". Seems odd to require this now to support code that is still some way from joining mainline. >>> >>>>> In FieldLayoutBuilder::epilogue you have a number of calls to Thread::current() as well as an implicit call when you use ResourceMarks. You should capture the current thread once in a local and reuse it. >>>> Fixed >>> >>> It seems that this fix is now not needed as there is only one use left of the new "thread" variable in the ResourceMark. So that can return to being: >>> >>> ResourceMark rm; >>> >>> Thanks, >>> David >>> ----- >>> From frederic.parain at oracle.com Thu Feb 6 16:36:24 2020 From: frederic.parain at oracle.com (Frederic Parain) Date: Thu, 6 Feb 2020 11:36:24 -0500 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> <1830530c-b086-534b-9bef-bb7e762fd6c6@redhat.com> Message-ID: Aleksey, Thank you very much for your experiments and the help you gave me. Here?s a new version of the code addressing these regressions: http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html An incremental webrev is available here: http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c-diff/index.html Changes are mostly in src/hotspot/share/classfile/fieldLayoutBuilder.cpp: line 68-71: bug fix line 171-200: replace the first-fit search with a best-fit search line 394-395: bug fix line 613: replace bulk allocation of reference fields with individual allocations Remaining changes are cleanup from the change in line 613. Regards, Fred > On Feb 3, 2020, at 13:15, Aleksey Shipilev wrote: > > On 2/3/20 5:33 PM, Aleksey Shipilev wrote: >> As I look through the layouter going through my local Maven Central snapshot, this case does not >> look rare for us not to care. Looks like every class with a long and 3+ oop fields is regressing in >> instance size, which is in line with your explanation. I believe this needs to be resolved before >> releasing the new layouter in the wild. > > To add some more data, out of ~15M classes, about 14K (~0.01%) are affected: > https://cr.openjdk.java.net/~shade/8237767/classes-regression-1.txt > > Of course, this does not mean we would have 0.01% footprint regression, because we don't know how > many instances of the affected class would be present. But browsing the list, it looks like plenty > of data classes, iterators, etc present. > > For fun, here is some friendly fire: org.openjdk.tools.javac.code.Symbol -- it has long > Symbol.flags_field and plenty of oop fields. > > -- > Thanks, > -Aleksey > From erik.joelsson at oracle.com Thu Feb 6 16:39:51 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 6 Feb 2020 08:39:51 -0800 Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images In-Reply-To: <2b92559a-6a1a-119a-1e62-eed6f9d3bfc8@oracle.com> References: <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com> <2b92559a-6a1a-119a-1e62-eed6f9d3bfc8@oracle.com> Message-ID: <1989082f-cddf-9d4f-262b-5f9072f46cf7@oracle.com> On 2020-02-06 04:48, Magnus Ihse Bursie wrote: > On 2020-02-06 12:36, Langer, Christoph wrote: >> Hi, >> >> let me chime in to this discussion at this point. So, first of all, Matthias, thanks for your work so far. >> >> Erik, I fully understand your points and I agree that it's probably a good idea to refactor this whole process of creating these different types of bundles. >> >> Our idea is to provide functionality in the build system to add "public" or stripped debug files to the delivery image of the JDK. This would provide better information in case of crashes and would hence allow for better supportability. That's something we'd probably want to enable in SapMachine binary distributions. I very much support the idea of using these stripped pdb files. It has been a long standing issue in hotspot on Windows to not have access to stacktraces. >> So, can we get to add a configure option like the one proposed by Matthias into the current code base? >> The option should be turned off by default. If it is switched on, the images/jdk folder in the build directory will have both, the *stripped.pdb files and the standard *.pdb files. However, having *stripped.pdb files around should not matter in terms of functionality (for developers and testing) as they'd not be used in the presence of the "real" pdb files anyway. The actual JDK bundle for delivery would then contain the *stripped.pdb files, renamed to *.pdb. And the debug symbols bundle would have the full *.pdb files only. Both could then be overlaid as needed. >> >> I think you raised two concerns. >> One is that you'd rather like to refactor bundling for several reasons. But I guess, should you eventually get to your refactoring, it shouldn't be a problem to take the functionality of this new option along. >> The other was regarding JMODs. I believe it's correct, that JMODs have never carried external debug information. For platforms other than MS Windows that's actually not a big problem because debug information can be internalized. And jlink has gotten several additions to set flags for stripping that data to the right level. I assume if jlinked images on Windows should ever be enabled to carry debug information, inclusion of external debug files would have to be added to JMODs and jlink. But that's definitely out of scope here. The argument "jmods have never carried external debug information" just doesn't apply here. Neither has the distribution bundles, for the exact same reason. You really should not compare these new stripped pdb files to the existing debug symbol files, they are different files with different purposes. One is meant to be shipped to customers, the other is not. You say you want to ship these new stripped pdb files with your distribution so that customers have them present when they use your distribution. If you then omit these new files from the jmods, any customer created jlinked image will not have these new stripped pdb files, which IMO is a very weird and unexpected behavior from a customer point of view. Jlinking new images is an integral and promoted way of using a JDK, so any mismatch between the original JDK distribution and what you are able to jlink is a serious discrepancy. >> So, what do you think? What speaks against adding this option (that is off by default)? My main objective is that you introduce further discrepancies between the original distribution JDK image and what's possible to generate using jlink from that distribution JDK image. My second objective is that the already convoluted bundles creation logic becomes even more convoluted. I believe there is a better possible solution in the build but it will require a lot more work to figure out. All that said, if you still wish to continue, I will stop standing in the way. > While Erik will need to comment on this himself, from my POV this > looks okay. The conditions are: > > * This is controlled by a separate option (or perhaps even better as a > new argument to --with-native-debug-symbols), so without this, nothing > will change. > I think it needs to be a separate option as it's all orthogonal to the main debug symbols file creation. > * The code need to make sure to separate stripped.pdb and normal pdb > files, when enabled. > > * And there must not be a heavy penalty in added code complexity. > /Erik > /Magnus > >> Thanks >> Christoph >> >>> -----Original Message----- >>> From: build-dev On Behalf Of Erik >>> Joelsson >>> Sent: Donnerstag, 23. Januar 2020 18:49 >>> To: Baesken, Matthias; David Holmes >>> ; 'build-dev at openjdk.java.net' >> dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' >> dev at openjdk.java.net> >>> Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows for >>> jdk images >>> >>> >>> On 2020-01-23 00:03, Baesken, Matthias wrote: >>>> Hi Erik, yes true sorry for answering your comments a bit late . >>>> >>>>> If a user runs jlink and includes all the jmods we ship with the JDK, the >>> result >>>>> should be essentially equivalent to the original JDK image. The way the >>>>> stripped pdb files are included in the bundles sort of at the last >>>>> second of the build here breaks this property. >>>> I think we should address this in a separate bug/CR . >>> Maybe. I realize that my proposal below is quite a big task. But on the >>> other hand, I don't think breaking the relationship between the jmods >>> and the distribution bundles is on the table really. >>>> Looking for example into a Linux build, I see a lot of debuginfo files in the >>> jdk image (more or less for every shared lib) . >>>> But when looking into the jmods of that jdk image , no debuginfo files are >>> in there ( I checked the java.base jmod). >>>> So putting the files with debug information into the jmods seems to be >>> something that was not done so far cross platform (or is there some build >>> switch for it that I did not check?) . >>>> Maybe to keep the jmods as small as possible . >>> No, we do not put the debuginfo files in the jmods nor the bundles >>> because those are not intended to be shipped to customers. We are >>> currently overlaying them into images/jdk in the build output to make >>> them available for local debugging. (This is convoluted and I would very >>> much like to get away from this practice at some point so that there is >>> a 1-1 mapping between images/jdk and bundles/jdk*-bin.tar.gz.) The >>> stripped pdb files you are proposing are on the contrary intended for >>> shipping to customers (as I understand your proposal) so comparing them >>> with the debuginfo files is not relevant. >>> >>> Now if MS had been kind enough to define a separate file type for the >>> stripped pdbs, so that they could live alongside the full pdbs, we >>> wouldn't have this issue. The heart of the problem is that only one set >>> of files (either stripped or full) can be present and usable in >>> images/jdk at a time. We have 2 main uses for images/jdk. >>> >>> 1. Developer running and debugging locally >>> >>> 2. Serve as the source for generating the distribution bundles >>> >>> We currently have one image serving both of these purposes, which is >>> already creating complicated and convoluted build steps. To properly >>> solve this I would argue for splitting these two apart into two >>> different images for the two different purposes. The build procedure >>> would then be, first build the images for distribution, only containing >>> what should go into each bundle. Then create the developer jdk image by >>> copying files from the distribution images. On Windows, the first image >>> would contain the stripped pdbs and when building the second, those >>> would get overwritten with the full pdbs. >>> >>> Now that I figured out a working model that would solve a bunch of other >>> problems as well, I would love to implement it, but I doubt I will have >>> time in the near future. >>> >>> /Erik >>> >>>>> To properly implement this, care will need to be taken to juggle the two >>>>> sets of pdb files around, making sure each build and test use case has >>>>> the correct one in place where and when it's needed. Quite possibly, we >>>>> cannot cover all use cases with one build configuration. Developers >>>>> needing the full debug symbols when debugging locally would likely need >>>>> to disable the stripped symbols so they get the full symbols everywhere. >>>>> Possibly this would need to be the default for debug builds and >>>>> configurable for release builds. >>>> From my limited experience , the developers do not work with the >>> bundles (that would contain now after my patch the stripped pds) but with >>> a "normal" jdk image that is created my make all. >>>> Best regards, Matthias >>>> >>>> >>>> >>>>> This still does not address anything in my objection. >>>>> >>>>> /Erik >>>>> >>>>> On 2020-01-22 07:46, Baesken, Matthias wrote: >>>>>> Hello, here is an updated version : >>>>>> >>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/ >>>>>> >>>>>> this one supports a configure switch "--enable-stripped-pdbs" to >>> enable >>>>> the feature . >>>>>> Best regards, Matthias >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Baesken, Matthias >>>>>>> Sent: Dienstag, 21. Januar 2020 11:03 >>>>>>> To: 'David Holmes'; 'build- >>>>>>> dev at openjdk.java.net'; 'hotspot- >>>>>>> dev at openjdk.java.net' >>>>>>> Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows >>> for >>>>>>> jdk images >>>>>>> >>>>>>> >>>>>>> Hi David , yes I think it makes sense to have a configure option for this . >>>>>>> Not everyone would like to have a larger JDK (even it is only a bit >>> larger). > From shade at redhat.com Thu Feb 6 18:05:52 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 6 Feb 2020 19:05:52 +0100 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> <1830530c-b086-534b-9bef-bb7e762fd6c6@redhat.com> Message-ID: <9a20124c-10fa-448d-0135-e7aaf594caa7@redhat.com> Hi, I have to say that seeing the new field layouter matching all the cases with perfect layouter simulator I did back in 2013 is very impressive! Good job. On 2/6/20 5:36 PM, Frederic Parain wrote: > Here?s a new version of the code addressing these regressions: > http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html Only the minor ones. I don't need another webrev, you might consider fixing them before push. *) UseEmptySlotsInSupers should also be deprecated right away? === fieldLayoutBuilder.cpp: *) Generally, it feels some method bodies would enjoy some new lines, to split logical parts of the method. For example, fillHoles() seems to break into five logical parts: https://paste.centos.org/view/1b7805f6 *) "(t)he requirements"? 179 // was successful. If he requirements were the same but the search failed, a new search will *) "INHERITED bloc(k)s"? 302 // INHERITED blocs are marked as non-reference because oop_maps are handled by their holder class *) What is so suspicious about this test? 389 if (block->prev_block() != NULL) { // suspicious test *) "!last_search_success" here? 181 else if (b->size() == last_size && b->alignment() == last_alignment && last_search_success == false) { *) Still "first fit"? 239 // The allocation logic uses a first fit strategy: the set of fields is allocated *) Still true? oop fields can now fill the gaps, no? 598 // - then oop fields are allocated contiguously (to reduce the number of oopmaps 599 // and reduce the work of the GC). *) Let's write this: 192 if (candidate == NULL) candidate = cursor; 193 else if (cursor->size() < candidate->size()) candidate = cursor; like this: if (candidate == NULL) { candidate = cursor; } else if (cursor->size() < candidate->size()) { candidate = cursor; } *) I believe indenting switch cases is good style here: 559 switch(type) { ... 570 case T_OBJECT: 571 case T_ARRAY: 572 if (group != _static_fields) _nonstatic_oopmap_count++; 573 group->add_oop_field(fs); ... I.e.: switch(type) { case T_OBJECT: case T_ARRAY: ... default: ... } -- Thanks, -Aleksey From frederic.parain at oracle.com Thu Feb 6 18:54:04 2020 From: frederic.parain at oracle.com (Frederic Parain) Date: Thu, 6 Feb 2020 13:54:04 -0500 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: <9a20124c-10fa-448d-0135-e7aaf594caa7@redhat.com> References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> <1830530c-b086-534b-9bef-bb7e762fd6c6@redhat.com> <9a20124c-10fa-448d-0135-e7aaf594caa7@redhat.com> Message-ID: Aleksey, Thank you the review, I?ve fixed all issues (except deprecating the option, see below), and updated the webrev in place: http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html > On Feb 6, 2020, at 13:05, Aleksey Shipilev wrote: > > Hi, > > I have to say that seeing the new field layouter matching all the cases with perfect layouter > simulator I did back in 2013 is very impressive! Good job. Thank you. Your simulator helped finding some bugs and inefficiencies. > > On 2/6/20 5:36 PM, Frederic Parain wrote: >> Here?s a new version of the code addressing these regressions: >> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html > > Only the minor ones. I don't need another webrev, you might consider fixing them before push. > > *) UseEmptySlotsInSupers should also be deprecated right away? The plan was to keep this option on the long term, for use cases where keeping fields from a given class close to each other is more important than having a smaller instance size. > > === fieldLayoutBuilder.cpp: > > *) Generally, it feels some method bodies would enjoy some new lines, to split logical parts of the > method. For example, fillHoles() seems to break into five logical parts: > https://paste.centos.org/view/1b7805f6 I?e added more new lines across this file to make the code easier to read. > > *) "(t)he requirements"? > > 179 // was successful. If he requirements were the same but the search failed, a new search will Fixed > > *) "INHERITED bloc(k)s"? > > 302 // INHERITED blocs are marked as non-reference because oop_maps are handled by their holder class Fixed > > *) What is so suspicious about this test? > > 389 if (block->prev_block() != NULL) { // suspicious test After careful review, this case is not suspicious, it matches one case of insertion. Comment removed. > > *) "!last_search_success" here? > > 181 else if (b->size() == last_size && b->alignment() == last_alignment && > last_search_success == false) { Fixed. > > *) Still "first fit"? > > 239 // The allocation logic uses a first fit strategy: the set of fields is allocated Yes, FieldLayout::add_contiguously() still uses a first-fit strategy, but it is only to allocate static fields. However, I?ve update the comment before FieldLayout::add() to clarify the best-fit search. > > *) Still true? oop fields can now fill the gaps, no? > > 598 // - then oop fields are allocated contiguously (to reduce the number of oopmaps > 599 // and reduce the work of the GC). Updated. > > *) Let's write this: > > 192 if (candidate == NULL) candidate = cursor; > 193 else if (cursor->size() < candidate->size()) candidate = cursor; > > like this: > > if (candidate == NULL) { > candidate = cursor; > } else if (cursor->size() < candidate->size()) { > candidate = cursor; > } > Definitively easier to read. > *) I believe indenting switch cases is good style here: > > 559 switch(type) { > ... > 570 case T_OBJECT: > 571 case T_ARRAY: > 572 if (group != _static_fields) _nonstatic_oopmap_count++; > 573 group->add_oop_field(fs); > ... > > I.e.: > > switch(type) { > case T_OBJECT: > case T_ARRAY: > ... > default: > ... > } > I?ve fixed indentation of several switch statements in this file. > -- > Thanks, > -Aleksey > Thank you, Fred From kim.barrett at oracle.com Thu Feb 6 19:05:08 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 6 Feb 2020 14:05:08 -0500 Subject: RFR[XS]: 8238272: Eliminate cast_from_oop to narrowOop* In-Reply-To: <6afd673387d694f87ea65049a4362865333cba87.camel@oracle.com> References: <267AF7FC-B6EB-4232-A51F-035A7217BCC6@oracle.com> <44ad6bd5-b66a-3306-0cb9-f518d1b8e2c4@redhat.com> <805C4111-CF21-4EAF-AC78-742045A3DD73@oracle.com> <0F754E34-462D-4183-8CE0-F2933D4F310F@oracle.com> <6afd673387d694f87ea65049a4362865333cba87.camel@oracle.com> Message-ID: > On Feb 3, 2020, at 4:01 AM, Thomas Schatzl wrote: > > Hi, > > On Fri, 2020-01-31 at 17:12 -0500, Kim Barrett wrote: >> Here's a new webrev: >> >> https://cr.openjdk.java.net/~kbarrett/8238272/open.01/ >> >> I didn't bother with an incremental webrev, since the new version is >> just >> a one-line change replacing the previous one-line change. >> >> Testing: tier1 >> > > looks good. > > Thomas Thanks. From shade at redhat.com Thu Feb 6 19:06:53 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 6 Feb 2020 20:06:53 +0100 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> <1830530c-b086-534b-9bef-bb7e762fd6c6@redhat.com> <9a20124c-10fa-448d-0135-e7aaf594caa7@redhat.com> Message-ID: On 2/6/20 7:54 PM, Frederic Parain wrote: > http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html Still looks good to me. -- Thanks, -Aleksey From lois.foltan at oracle.com Thu Feb 6 20:29:42 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 6 Feb 2020 15:29:42 -0500 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> <1830530c-b086-534b-9bef-bb7e762fd6c6@redhat.com> <9a20124c-10fa-448d-0135-e7aaf594caa7@redhat.com> Message-ID: <3fd207ba-9593-2b10-30c9-475e27375973@oracle.com> On 2/6/2020 2:06 PM, Aleksey Shipilev wrote: > On 2/6/20 7:54 PM, Frederic Parain wrote: >> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html > Still looks good to me. > Hi Fred, I have reviewed as well.? Looks great and thank you for the preliminary code walk through we had about this change. Lois From david.holmes at oracle.com Thu Feb 6 22:45:40 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 7 Feb 2020 08:45:40 +1000 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: <7E7AEBD0-09DD-4EA0-B2E0-2102EDABCAD9@oracle.com> References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> <7E7AEBD0-09DD-4EA0-B2E0-2102EDABCAD9@oracle.com> Message-ID: <83dbad70-8398-2209-ba34-2bb417e3788b@oracle.com> Hi Fred, Nothing further from me. Thanks, David On 7/02/2020 2:28 am, Frederic Parain wrote: > David, > > The deprecated flag has been added to the table and to the test. > Indentation/spaces have been fixed. > CSR has been updated too. > > New webrev: > http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html > > The webrev also contains some modifications to address issues found > by Aleksey. See my response to his e-mail to have the incremental > webrev. > > Thank you, > > Fred > > >> On Feb 3, 2020, at 01:24, David Holmes wrote: >> >> Hi Fred, >> >> On 1/02/2020 2:03 am, Frederic Parain wrote: >>> Hi David, >>> I?ve fixed all the issues you mentioned below. >>> According to the discussion about the VM flags, the following modifications >>> have also been made: >>> globals.hpp: >>> UseNewFieldLayout is now a deprecated product flag >> >> You need to add it to the deprecated flag table in arguments.cpp and to the test >> >> test/hotspot/jtreg/runtime/CommandLine/VMDeprecatedOptions.java >> >>> fieldLayoutBuilder.cpp >>> lines 350-362: >>> UseEmptySlotsInSupers VM option now controls both new optimizations >>> lines 449-464 >>> Fix an issue with layout printing (inherited fields were printed >>> only for the direct super-class, now all inherited fields are printed) >>> New webrev: >>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.08/index.html >> >> Looks good. A couple of minor follow ups ... >> >> src/hotspot/share/classfile/classFileParser.cpp >> >> ! OopMapBlock* nonstatic_oop_map = _nonstatic_oop_maps; >> >> Still an extra space after * >> >> + FieldLayoutBuilder lb(class_name(), super_klass(), _cp, _fields, >> + _parsed_annotations->is_contended(), _field_info); >> >> Indentation of continuing line is wrong. >> >> For the test, you have now restricted it to only run on 64-bit even though it has code that appears to allow for 32-bit. As this isn't a 64-bit specific feature I think it should run on all platforms. So this: >> >> /* >> * @test >> * @bug 8237767 >> * @requires vm.bits == "64" >> * @summary Verify behaviour of field layout algorithm >> * @library /test/lib >> * @modules java.base/jdk.internal.misc >> * java.management >> * @run main/othervm -XX:+UseCompressedOops -XX:+UseCompressedClassPointers FieldDensityTest >> * @run main/othervm -XX:+UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest >> * @run main/othervm -XX:-UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest >> */ >> >> can become: >> >> /* >> * @test >> * @bug 8237767 >> * @summary Verify behaviour of field layout algorithm >> * @library /test/lib >> * @modules java.base/jdk.internal.misc >> * java.management >> * @run main/othervm FieldDensityTest >> */ >> >> /* >> * @test >> * @requires vm.bits == "64" >> * @run main/othervm -XX:+UseCompressedOops -XX:+UseCompressedClassPointers FieldDensityTest >> * @run main/othervm -XX:+UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest >> */ >> >>> CR and CSR have been updated accordingly (new VM flags explanations in the CSR). >> >> There is one small glitch in the CSR. Code removal associated with a flag occurs when it is obsoleted. So under the proposed timeline of deprecation in 15 and obsoletion in 16, the old code will be removed in 16 not 17. Comment added to CSR to that effect. >> >> Thanks, >> David >> >>> Thank you, >>> Fred >>>> On Jan 29, 2020, at 02:06, David Holmes wrote: >>>> >>>> Hi Fred, >>>> >>>> I've looked at the v7 version. A few more stylistic comments on that first. Note, no need for an item by item response unless that makes it easier for you to track :) >>>> >>>> src/hotspot/share/classfile/classFileParser.cpp >>>> >>>> #include "classfile/defaultMethods.hpp" >>>> +#include "classfile/fieldLayoutBuilder.hpp" >>>> #include "classfile/dictionary.hpp" >>>> >>>> Include files are not in alphabetical order. >>>> >>>> + * This may well change: FixMe if doesn't, >>>> >>>> s/if/if it/ >>>> >>>> + //Make a temp copy, and iterate through and copy back into the orig >>>> >>>> Space after // >>>> >>>> s/orig/original/ >>>> >>>> + OopMapBlock* nonstatic_oop_map = _nonstatic_oop_maps; >>>> >>>> Extra space after * >>>> >>>> + assert(ik->nonstatic_oop_map_count() == _field_info->oop_map_blocks->_nonstatic_oop_map_count, >>>> + "sanity"); >>>> >>>> Second line needs to be indented further: >>>> >>>> assert(ik->nonstatic_oop_map_count() == _field_info->oop_map_blocks->_nonstatic_oop_map_count, >>>> "sanity"); >>>> >>>> --- >>>> >>>> src/hotspot/share/classfile/classFileParser.hpp >>>> >>>> +public: >>>> + OopMapBlock* _nonstatic_oop_maps; >>>> + unsigned int _nonstatic_oop_map_count; >>>> + unsigned int _max_nonstatic_oop_maps; >>>> + >>>> + public: >>>> >>>> Second public uneeded. First public may be indented wrong (I'm not sure what the rule is - single space indent?) >>>> >>>> class ClassFileParser { >>>> + friend class FieldLayoutBuilder; >>>> + friend class FieldLayout; >>>> >>>> class ClassAnnotationCollector; >>>> class FieldAllocationCount; >>>> class FieldAnnotationCollector; >>>> >>>> Indents are different. I think the class forward declarations should have extra space. >>>> >>>> --- >>>> >>>> src/hotspot/share/oops/instanceKlass.hpp >>>> >>>> + void increment_count(int diff) { _count += diff; } >>>> >>>> Extra spaces before { >>>> >>>> --- >>>> >>>> src/hotspot/share/runtime/globals.hpp >>>> >>>> + diagnostic(bool, UseNewFieldLayout, true, \ >>>> + "Use new algorithm to compute layouts") \ >>>> + \ >>>> + product(bool, UseEmptySlotsInSupers, true, >>>> >>>> Not sure I see why one flag is diagnostic and the other product. Do you expect people to need to disable using empty slots more so than needing to disable using the new field layout altogether? >>>> >>>> --- >>>> >>>> src/hotspot/share/classfile/fieldLayoutBuilder.cpp >>>> >>>> + assert(kind == EMPTY || kind == RESERVED || kind == PADDING || kind == INHERITED, >>>> + "Otherwise, should use the constructor with a field index argument"); >>>> >>>> Indentation of second line is wrong. >>>> >>>> + assert(kind == REGULAR || kind == FLATTENED || kind == INHERITED, >>>> + "Other kind do not have a field index"); >>>> >>>> Ditto. >>>> >>>> >>>> + if (list == NULL) return; >>>> + if (start == NULL) { >>>> + start = this->_start; >>>> + } >>>> >>>> Inconsistent style for single statement if-blocks. Same thing later in the file. >>>> >>>> + output->print_cr(" @%d \"%s\" %s %d/%d %s", >>>> + b->offset(), >>>> + fi->name(_cp)->as_C_string(), >>>> + fi->signature(_cp)->as_C_string(), >>>> + b->size(), >>>> + b->alignment(), >>>> + "REGULAR"); >>>> >>>> Incorrect identation of continuing line. Same for all the following print blocks. >>>> >>>> + } else if (_classname == vmSymbols::java_lang_Boolean() || >>>> + _classname == vmSymbols::java_lang_Character() || >>>> + _classname == vmSymbols::java_lang_Float() || >>>> + _classname == vmSymbols::java_lang_Double() || >>>> + _classname == vmSymbols::java_lang_Byte() || >>>> + _classname == vmSymbols::java_lang_Short() || >>>> + _classname == vmSymbols::java_lang_Integer() || >>>> + _classname == vmSymbols::java_lang_Long()) { >>>> >>>> Incorrect identation of continuing line. >>>> >>>> --- >>>> >>>> src/hotspot/share/classfile/fieldLayoutBuilder.hpp >>>> >>>> +// and the boxing classes). The rational for having multiple methods >>>> >>>> s/rational/rationale/ >>>> >>>> + FieldLayoutBuilder(const Symbol* classname, const InstanceKlass* super_klass, ConstantPool* constant_pool, >>>> + Array* fields, bool is_contended, FieldLayoutInfo* info); >>>> >>>> Indentation wrong for continuing line. >>>> >>>> + int get_alignment() { >>>> + assert(_alignment != -1, "Uninitialized"); >>>> + return _alignment; >>>> + } >>>> >>>> Indenting appears off by one. >>>> >>>> --- >>>> >>>> test/hotspot/jtreg/runtime/FieldLayout/FieldDensityTest.java >>>> >>>> + * @run main/othervm -XX:+UseCompressedOops -XX:+UseCompressedClassPointers FieldDensityTest >>>> + * @run main/othervm -XX:+UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest >>>> + * @run main/othervm -XX:-UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest >>>> >>>> The test won't run on 32-bit platforms as the compressed oops flags won't exist. >>>> >>>> --- >>>> >>>> Some follow up comments below ... >>>> >>>> With trimming ... >>>> >>>> On 25/01/2020 3:20 am, Frederic Parain wrote: >>>>>> On Jan 24, 2020, at 08:19, David Holmes wrote: >>>>>> >>>>>> 466 int super_flen = super->nof_nonstatic_fields(); >>>>>> >>>>>> Could be folded directly into the assert so we don't call in product. >>>>> Calling not_nonstatic_fields() has the side effect to compute non-static fields, >>>>> which is required to get a correct value when reading super->_nonstatic_fields, >>>>> so the call is needed even in product builds. >>>> >>>> Yuck! That's a horrible side-effect - but not your fault obviously. :) It would be better to have a nonstatic_fields() accessor that has the same lazy initialization side-effect. >>>> >>>>>> General style issue: when breaking a long line with a method call, the new line (containing arguments) should be indented to the opening ( of the method call e.g. >>>> ... >>>>>> etc. This applies across all files. >>>>> Fixes applied lines 4003, 4011, 4041, 4138, 4143. >>>> >>>> Fix was also needed in other files. Current issues highlighted above. >>>> >>>>>> >>>>>> src/hotspot/share/oops/instanceKlass.hpp >>>>>> >>>>>> You need to be careful with _extra_flags usage if there can be concurrently updated bits. At the moment it looks like redefinition is a mutable dynamic property, whilst "contended annotations" should be a static immutable property - is that right? >>>>> Correct, _has_contended_annotations is a static immutable property, while _is_being_redefined is a mutable one. >>>> >>>> Good to know. My concern is that if someone adds a new mutable flag bit the need for atomic updates may not be noticed. We got bitten by this in the past with a flag field and I think we eventually migrated all of the mutable bits out into their own field. (Coleen should recall that :) ). >>>> >>>>>> 61 FLATTENED, // flattened field >>>>>> >>>>>> Does this have any meaning before inline types come in? >>>>> Yes, I wanted to reserved the entry in the enum. >>>> >>>> Hmmm a tenuous "okay". Seems odd to require this now to support code that is still some way from joining mainline. >>>> >>>>>> In FieldLayoutBuilder::epilogue you have a number of calls to Thread::current() as well as an implicit call when you use ResourceMarks. You should capture the current thread once in a local and reuse it. >>>>> Fixed >>>> >>>> It seems that this fix is now not needed as there is only one use left of the new "thread" variable in the ResourceMark. So that can return to being: >>>> >>>> ResourceMark rm; >>>> >>>> Thanks, >>>> David >>>> ----- >>>> > From coleen.phillimore at oracle.com Thu Feb 6 22:59:58 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 6 Feb 2020 17:59:58 -0500 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> <1830530c-b086-534b-9bef-bb7e762fd6c6@redhat.com> <9a20124c-10fa-448d-0135-e7aaf594caa7@redhat.com> Message-ID: Except for the same comment as Aleksey about this: http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c-diff/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.udiff.html + while (cursor != start) { + if (cursor->kind() == LayoutRawBlock::EMPTY && cursor->fit(b->size(), b->alignment())) { + if (candidate == NULL) candidate = cursor; + else if (cursor->size() < candidate->size()) candidate = cursor; + } + cursor = cursor->prev_block(); + } Shouldn't it be: + while (cursor != start) { + if (cursor->kind() == LayoutRawBlock::EMPTY && cursor->fit(b->size(), b->alignment())) { + if (candidate == NULL || cursor->size() < candidate->size()) { + candidate = cursor; + } + } + cursor = cursor->prev_block(); + } and should there be a 'break' once you've got your candidate? Thanks, Coleen On 2/6/20 1:54 PM, Frederic Parain wrote: > Aleksey, > > Thank you the review, I?ve fixed all issues (except deprecating the > option, see below), and updated the webrev in place: > > http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html > > >> On Feb 6, 2020, at 13:05, Aleksey Shipilev wrote: >> >> Hi, >> >> I have to say that seeing the new field layouter matching all the cases with perfect layouter >> simulator I did back in 2013 is very impressive! Good job. > Thank you. Your simulator helped finding some bugs and inefficiencies. > >> On 2/6/20 5:36 PM, Frederic Parain wrote: >>> Here?s a new version of the code addressing these regressions: >>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html >> Only the minor ones. I don't need another webrev, you might consider fixing them before push. >> >> *) UseEmptySlotsInSupers should also be deprecated right away? > The plan was to keep this option on the long term, for use cases > where keeping fields from a given class close to each other is more > important than having a smaller instance size. > >> === fieldLayoutBuilder.cpp: >> >> *) Generally, it feels some method bodies would enjoy some new lines, to split logical parts of the >> method. For example, fillHoles() seems to break into five logical parts: >> https://paste.centos.org/view/1b7805f6 > I?e added more new lines across this file to make the code easier to read. > >> *) "(t)he requirements"? >> >> 179 // was successful. If he requirements were the same but the search failed, a new search will > Fixed > >> *) "INHERITED bloc(k)s"? >> >> 302 // INHERITED blocs are marked as non-reference because oop_maps are handled by their holder class > Fixed > >> *) What is so suspicious about this test? >> >> 389 if (block->prev_block() != NULL) { // suspicious test > After careful review, this case is not suspicious, it matches > one case of insertion. Comment removed. > >> *) "!last_search_success" here? >> >> 181 else if (b->size() == last_size && b->alignment() == last_alignment && >> last_search_success == false) { > Fixed. > > >> *) Still "first fit"? >> >> 239 // The allocation logic uses a first fit strategy: the set of fields is allocated > Yes, FieldLayout::add_contiguously() still uses a first-fit strategy, but it is only > to allocate static fields. > > However, I?ve update the comment before FieldLayout::add() to clarify the best-fit search. > >> *) Still true? oop fields can now fill the gaps, no? >> >> 598 // - then oop fields are allocated contiguously (to reduce the number of oopmaps >> 599 // and reduce the work of the GC). > Updated. > >> *) Let's write this: >> >> 192 if (candidate == NULL) candidate = cursor; >> 193 else if (cursor->size() < candidate->size()) candidate = cursor; >> >> like this: >> >> if (candidate == NULL) { >> candidate = cursor; >> } else if (cursor->size() < candidate->size()) { >> candidate = cursor; >> } >> > Definitively easier to read. > >> *) I believe indenting switch cases is good style here: >> >> 559 switch(type) { >> ... >> 570 case T_OBJECT: >> 571 case T_ARRAY: >> 572 if (group != _static_fields) _nonstatic_oopmap_count++; >> 573 group->add_oop_field(fs); >> ... >> >> I.e.: >> >> switch(type) { >> case T_OBJECT: >> case T_ARRAY: >> ... >> default: >> ... >> } >> > I?ve fixed indentation of several switch statements in this file. > >> -- >> Thanks, >> -Aleksey >> > Thank you, > > Fred > From coleen.phillimore at oracle.com Thu Feb 6 23:01:15 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 6 Feb 2020 18:01:15 -0500 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> <1830530c-b086-534b-9bef-bb7e762fd6c6@redhat.com> <9a20124c-10fa-448d-0135-e7aaf594caa7@redhat.com> Message-ID: <53be54e5-889c-3d06-0e5a-3927e8ad645b@oracle.com> I meant to say that I have no further comments.? Looks good. Coleen On 2/6/20 5:59 PM, coleen.phillimore at oracle.com wrote: > > Except for the same comment as Aleksey about this: > > http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c-diff/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.udiff.html > > > + while (cursor != start) { > + if (cursor->kind() == LayoutRawBlock::EMPTY && > cursor->fit(b->size(), b->alignment())) { > + if (candidate == NULL) candidate = cursor; > + else if (cursor->size() < candidate->size()) candidate = cursor; > + } > + cursor = cursor->prev_block(); > + } > > Shouldn't it be: > > + while (cursor != start) { > + if (cursor->kind() == LayoutRawBlock::EMPTY && > cursor->fit(b->size(), b->alignment())) { > + if (candidate == NULL || cursor->size() < candidate->size()) { + > candidate = cursor; > +????????? } > + } > + cursor = cursor->prev_block(); > + } > > > and should there be a 'break' once you've got your candidate? > > Thanks, > Coleen > > On 2/6/20 1:54 PM, Frederic Parain wrote: >> Aleksey, >> >> Thank you the review, I?ve fixed all issues (except deprecating the >> option, see below), and updated the webrev in place: >> >> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html >> >> >>> On Feb 6, 2020, at 13:05, Aleksey Shipilev wrote: >>> >>> Hi, >>> >>> I have to say that seeing the new field layouter matching all the >>> cases with perfect layouter >>> simulator I did back in 2013 is very impressive! Good job. >> Thank you. Your simulator helped finding some bugs and inefficiencies. >> >>> On 2/6/20 5:36 PM, Frederic Parain wrote: >>>> Here?s a new version of the code addressing these regressions: >>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html >>> Only the minor ones. I don't need another webrev, you might consider >>> fixing them before push. >>> >>> *) UseEmptySlotsInSupers should also be deprecated right away? >> The plan was to keep this option on the long term, for use cases >> where keeping fields from a given class close to each other is more >> important than having a smaller instance size. >> >>> === fieldLayoutBuilder.cpp: >>> >>> *) Generally, it feels some method bodies would enjoy some new >>> lines, to split logical parts of the >>> method. For example, fillHoles() seems to break into five logical >>> parts: >>> https://paste.centos.org/view/1b7805f6 >> I?e added more new lines across this file to make the code easier to >> read. >> >>> *) "(t)he requirements"? >>> >>> 179? // was successful. If he requirements were the same but the >>> search failed, a new search will >> Fixed >> >>> *) "INHERITED bloc(k)s"? >>> >>> 302? // INHERITED blocs are marked as non-reference because oop_maps >>> are handled by their holder class >> Fixed >> >>> *) What is so suspicious about this test? >>> >>> 389?? if (block->prev_block() != NULL) {?????? // suspicious test >> After careful review, this case is not suspicious, it matches >> one case of insertion. Comment removed. >> >>> *) "!last_search_success" here? >>> >>> ? 181???? else? if (b->size() == last_size && b->alignment() == >>> last_alignment && >>> last_search_success == false) { >> Fixed. >> >> >>> *) Still "first fit"? >>> >>> 239 // The allocation logic uses a first fit strategy: the set of >>> fields is allocated >> Yes, FieldLayout::add_contiguously() still uses a first-fit strategy, >> but it is only >> to allocate static fields. >> >> However, I?ve update the comment before FieldLayout::add() to clarify >> the best-fit search. >> >>> *) Still true? oop fields can now fill the gaps, no? >>> >>> 598 //?? - then oop fields are allocated contiguously (to reduce the >>> number of oopmaps >>> 599 //???? and reduce the work of the GC). >> Updated. >> >>> *) Let's write this: >>> >>> 192?????????? if (candidate == NULL) candidate = cursor; >>> 193?????????? else if (cursor->size() < candidate->size()) candidate >>> = cursor; >>> >>> like this: >>> >>> ??? if (candidate == NULL) { >>> ?????? candidate = cursor; >>> ??? } else if (cursor->size() < candidate->size()) { >>> ?????? candidate = cursor; >>> ??? } >>> >> Definitively easier to read. >> >>> *) I believe indenting switch cases is good style here: >>> >>> 559???? switch(type) { >>> ... >>> 570???? case T_OBJECT: >>> 571???? case T_ARRAY: >>> 572?????? if (group != _static_fields) _nonstatic_oopmap_count++; >>> 573?????? group->add_oop_field(fs); >>> ... >>> >>> I.e.: >>> >>> ?? switch(type) { >>> ???? case T_OBJECT: >>> ???? case T_ARRAY: >>> ?????? ... >>> ???? default: >>> ?????? ... >>> ?? } >>> >> I?ve fixed indentation of several switch statements in this file. >> >>> -- >>> Thanks, >>> -Aleksey >>> >> Thank you, >> >> Fred >> > From igor.ignatyev at oracle.com Fri Feb 7 06:06:03 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 6 Feb 2020 22:06:03 -0800 Subject: RFR:JDK-8238586: [TESTBUG] vmTestbase/jit/tiered/Test.java failed when TieredCompilation is disabled In-Reply-To: <2b2bd9d6-d332-d079-5b26-06def034be38@loongson.cn> References: <2b2bd9d6-d332-d079-5b26-06def034be38@loongson.cn> Message-ID: <9C64491C-E122-4F61-A006-3B9DBB4F835C@oracle.com> Hi Jia, LGTM -- Igor > On Feb 6, 2020, at 1:20 AM, Jia Huang wrote: > > Hi all, > > JBS: https://bugs.openjdk.java.net/browse/JDK-8238586 > > When TieredCompilation is disabled, the test will fail. > Please review the following fix which make the test pass when disabled TieredCompilation. > ------------------------------------------------------ > diff -r e45d8ca836b5 -r 67bef3d6692f test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java > --- a/test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java Mon Feb 03 22:10:59 2020 +0800 > +++ b/test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java Thu Feb 06 15:25:19 2020 +0800 > @@ -46,7 +46,7 @@ > import jdk.test.lib.process.ProcessTools; > > public class Test { > - private static String UNSUPPORTED_OPTION_MESSAGE = "-XX:TieredCompilation not supported in this VM"; > + private static String UNSUPPORTED_OPTION_MESSAGE = "-XX:+TieredCompilation not supported in this VM"; > private static String REGEXP = "^[0-9.]+: \\[compile level=\\d"; > public static void main(String[] args) throws Exception { > { > @@ -56,7 +56,7 @@ > "-XX:+PrintTieredEvents", > "-version"); > var output = new OutputAnalyzer(pb.start()); > - if (output.getStdout().contains(UNSUPPORTED_OPTION_MESSAGE)) { > + if (output.getStderr().contains(UNSUPPORTED_OPTION_MESSAGE)) { > throw new SkippedException(UNSUPPORTED_OPTION_MESSAGE); > } > output.shouldHaveExitValue(0) > ------------------------------------------------------ > For more info, please see the JBS. > > Thanks a lot > > Best regards, > Jia. > From huangjia at loongson.cn Fri Feb 7 07:22:01 2020 From: huangjia at loongson.cn (Jia Huang) Date: Fri, 7 Feb 2020 15:22:01 +0800 Subject: RFR: JDK-8238586: [TESTBUG] vmTestbase/jit/tiered/Test.java failed when TieredCompilation is disabled In-Reply-To: <9C64491C-E122-4F61-A006-3B9DBB4F835C@oracle.com> References: <2b2bd9d6-d332-d079-5b26-06def034be38@loongson.cn> <9C64491C-E122-4F61-A006-3B9DBB4F835C@oracle.com> Message-ID: Thanks Igor. Is it a trivial patch and OK to be pushed? Or shall we wait for one more review? Thanks a lot. Best regards, Jia. ? 2020/2/7 14:06, Igor Ignatyev ??: > Hi Jia, > > LGTM > > -- Igor > >> On Feb 6, 2020, at 1:20 AM, Jia Huang wrote: >> >> Hi all, >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8238586 >> >> When TieredCompilation is disabled, the test will fail. >> Please review the following fix which make the test pass when disabled TieredCompilation. >> ------------------------------------------------------ >> diff -r e45d8ca836b5 -r 67bef3d6692f test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java >> --- a/test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java Mon Feb 03 22:10:59 2020 +0800 >> +++ b/test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java Thu Feb 06 15:25:19 2020 +0800 >> @@ -46,7 +46,7 @@ >> import jdk.test.lib.process.ProcessTools; >> >> public class Test { >> - private static String UNSUPPORTED_OPTION_MESSAGE = "-XX:TieredCompilation not supported in this VM"; >> + private static String UNSUPPORTED_OPTION_MESSAGE = "-XX:+TieredCompilation not supported in this VM"; >> private static String REGEXP = "^[0-9.]+: \\[compile level=\\d"; >> public static void main(String[] args) throws Exception { >> { >> @@ -56,7 +56,7 @@ >> "-XX:+PrintTieredEvents", >> "-version"); >> var output = new OutputAnalyzer(pb.start()); >> - if (output.getStdout().contains(UNSUPPORTED_OPTION_MESSAGE)) { >> + if (output.getStderr().contains(UNSUPPORTED_OPTION_MESSAGE)) { >> throw new SkippedException(UNSUPPORTED_OPTION_MESSAGE); >> } >> output.shouldHaveExitValue(0) >> ------------------------------------------------------ >> For more info, please see the JBS. >> >> Thanks a lot >> >> Best regards, >> Jia. >> From matthias.baesken at sap.com Fri Feb 7 08:50:03 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 7 Feb 2020 08:50:03 +0000 Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images In-Reply-To: <1989082f-cddf-9d4f-262b-5f9072f46cf7@oracle.com> References: <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com> <2b92559a-6a1a-119a-1e62-eed6f9d3bfc8@oracle.com> <1989082f-cddf-9d4f-262b-5f9072f46cf7@oracle.com> Message-ID: Hello, here is a slightly changed new webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.4/ (adjusted $(JRE_STRIPPED_PDB_FILES) in Bundles.gmk, that was in the wrong place ) . > I think it needs to be a separate option as it's all orthogonal to the > main debug symbols file creation. Yes it is a separate option I agree that?s better . One has to set --enable-stripped-pdbs=yes . Best regards , Matthias > > On 2020-02-06 04:48, Magnus Ihse Bursie wrote: > > On 2020-02-06 12:36, Langer, Christoph wrote: > >> Hi, > >> > >> let me chime in to this discussion at this point. So, first of all, Matthias, > thanks for your work so far. > >> > >> Erik, I fully understand your points and I agree that it's probably a good > idea to refactor this whole process of creating these different types of > bundles. > >> > >> Our idea is to provide functionality in the build system to add "public" or > stripped debug files to the delivery image of the JDK. This would provide > better information in case of crashes and would hence allow for better > supportability. That's something we'd probably want to enable in > SapMachine binary distributions. > I very much support the idea of using these stripped pdb files. It has > been a long standing issue in hotspot on Windows to not have access to > stacktraces. > >> So, can we get to add a configure option like the one proposed by > Matthias into the current code base? > >> The option should be turned off by default. If it is switched on, the > images/jdk folder in the build directory will have both, the *stripped.pdb > files and the standard *.pdb files. However, having *stripped.pdb files > around should not matter in terms of functionality (for developers and > testing) as they'd not be used in the presence of the "real" pdb files anyway. > The actual JDK bundle for delivery would then contain the *stripped.pdb > files, renamed to *.pdb. And the debug symbols bundle would have the full > *.pdb files only. Both could then be overlaid as needed. > >> > >> I think you raised two concerns. > >> One is that you'd rather like to refactor bundling for several reasons. But I > guess, should you eventually get to your refactoring, it shouldn't be a > problem to take the functionality of this new option along. > >> The other was regarding JMODs. I believe it's correct, that JMODs have > never carried external debug information. For platforms other than MS > Windows that's actually not a big problem because debug information can be > internalized. And jlink has gotten several additions to set flags for stripping > that data to the right level. I assume if jlinked images on Windows should > ever be enabled to carry debug information, inclusion of external debug files > would have to be added to JMODs and jlink. But that's definitely out of scope > here. > The argument "jmods have never carried external debug information" just > doesn't apply here. Neither has the distribution bundles, for the exact > same reason. You really should not compare these new stripped pdb files > to the existing debug symbol files, they are different files with > different purposes. One is meant to be shipped to customers, the other > is not. You say you want to ship these new stripped pdb files with your > distribution so that customers have them present when they use your > distribution. If you then omit these new files from the jmods, any > customer created jlinked image will not have these new stripped pdb > files, which IMO is a very weird and unexpected behavior from a customer > point of view. Jlinking new images is an integral and promoted way of > using a JDK, so any mismatch between the original JDK distribution and > what you are able to jlink is a serious discrepancy. > >> So, what do you think? What speaks against adding this option (that is off > by default)? > > My main objective is that you introduce further discrepancies between > the original distribution JDK image and what's possible to generate > using jlink from that distribution JDK image. My second objective is > that the already convoluted bundles creation logic becomes even more > convoluted. I believe there is a better possible solution in the build > but it will require a lot more work to figure out. > > All that said, if you still wish to continue, I will stop standing in > the way. > > > While Erik will need to comment on this himself, from my POV this > > looks okay. The conditions are: > > > > * This is controlled by a separate option (or perhaps even better as a > > new argument to --with-native-debug-symbols), so without this, nothing > > will change. > > > I think it needs to be a separate option as it's all orthogonal to the > main debug symbols file creation. > > * The code need to make sure to separate stripped.pdb and normal pdb > > files, when enabled. > > > > * And there must not be a heavy penalty in added code complexity. > > > /Erik > > /Magnus > > > >> Thanks > >> Christoph > >> > >>> -----Original Message----- > >>> From: build-dev On Behalf Of > Erik > >>> Joelsson > >>> Sent: Donnerstag, 23. Januar 2020 18:49 > >>> To: Baesken, Matthias; David Holmes > >>> ; 'build-dev at openjdk.java.net' >>> dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' >>> dev at openjdk.java.net> > >>> Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows > for > >>> jdk images > >>> > >>> > >>> On 2020-01-23 00:03, Baesken, Matthias wrote: > >>>> Hi Erik, yes true sorry for answering your comments a bit late . > >>>> > >>>>> If a user runs jlink and includes all the jmods we ship with the JDK, the > >>> result > >>>>> should be essentially equivalent to the original JDK image. The way > the > >>>>> stripped pdb files are included in the bundles sort of at the last > >>>>> second of the build here breaks this property. > >>>> I think we should address this in a separate bug/CR . > >>> Maybe. I realize that my proposal below is quite a big task. But on the > >>> other hand, I don't think breaking the relationship between the jmods > >>> and the distribution bundles is on the table really. > >>>> Looking for example into a Linux build, I see a lot of debuginfo files in > the > >>> jdk image (more or less for every shared lib) . > >>>> But when looking into the jmods of that jdk image , no debuginfo files > are > >>> in there ( I checked the java.base jmod). > >>>> So putting the files with debug information into the jmods seems to > be > >>> something that was not done so far cross platform (or is there some > build > >>> switch for it that I did not check?) . > >>>> Maybe to keep the jmods as small as possible . > >>> No, we do not put the debuginfo files in the jmods nor the bundles > >>> because those are not intended to be shipped to customers. We are > >>> currently overlaying them into images/jdk in the build output to make > >>> them available for local debugging. (This is convoluted and I would very > >>> much like to get away from this practice at some point so that there is > >>> a 1-1 mapping between images/jdk and bundles/jdk*-bin.tar.gz.) The > >>> stripped pdb files you are proposing are on the contrary intended for > >>> shipping to customers (as I understand your proposal) so comparing > them > >>> with the debuginfo files is not relevant. > >>> > >>> Now if MS had been kind enough to define a separate file type for the > >>> stripped pdbs, so that they could live alongside the full pdbs, we > >>> wouldn't have this issue. The heart of the problem is that only one set > >>> of files (either stripped or full) can be present and usable in > >>> images/jdk at a time. We have 2 main uses for images/jdk. > >>> > >>> 1. Developer running and debugging locally > >>> > >>> 2. Serve as the source for generating the distribution bundles > >>> > >>> We currently have one image serving both of these purposes, which is > >>> already creating complicated and convoluted build steps. To properly > >>> solve this I would argue for splitting these two apart into two > >>> different images for the two different purposes. The build procedure > >>> would then be, first build the images for distribution, only containing > >>> what should go into each bundle. Then create the developer jdk image > by > >>> copying files from the distribution images. On Windows, the first image > >>> would contain the stripped pdbs and when building the second, those > >>> would get overwritten with the full pdbs. > >>> > >>> Now that I figured out a working model that would solve a bunch of > other > >>> problems as well, I would love to implement it, but I doubt I will have > >>> time in the near future. > >>> > >>> /Erik > >>> > >>>>> To properly implement this, care will need to be taken to juggle the > two > >>>>> sets of pdb files around, making sure each build and test use case has > >>>>> the correct one in place where and when it's needed. Quite possibly, > we > >>>>> cannot cover all use cases with one build configuration. Developers > >>>>> needing the full debug symbols when debugging locally would likely > need > >>>>> to disable the stripped symbols so they get the full symbols > everywhere. > >>>>> Possibly this would need to be the default for debug builds and > >>>>> configurable for release builds. > >>>> From my limited experience , the developers do not work with the > >>> bundles (that would contain now after my patch the stripped pds) but > with > >>> a "normal" jdk image that is created my make all. > >>>> Best regards, Matthias > >>>> > >>>> > >>>> > >>>>> This still does not address anything in my objection. > >>>>> > >>>>> /Erik > >>>>> > >>>>> On 2020-01-22 07:46, Baesken, Matthias wrote: > >>>>>> Hello, here is an updated version : > >>>>>> > >>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/ > >>>>>> > >>>>>> this one supports a configure switch "--enable-stripped-pdbs" to > >>> enable > >>>>> the feature . > >>>>>> Best regards, Matthias > >>>>>> > >>>>>> > >>>>>>> -----Original Message----- > >>>>>>> From: Baesken, Matthias > >>>>>>> Sent: Dienstag, 21. Januar 2020 11:03 > >>>>>>> To: 'David Holmes'; 'build- > >>>>>>> dev at openjdk.java.net'; 'hotspot- > >>>>>>> dev at openjdk.java.net' > >>>>>>> Subject: RE: RFR: 8237192: Generate stripped/public pdbs on > Windows > >>> for > >>>>>>> jdk images > >>>>>>> > >>>>>>> > >>>>>>> Hi David , yes I think it makes sense to have a configure option for > this . > >>>>>>> Not everyone would like to have a larger JDK (even it is only a bit > >>> larger). > > From frederic.parain at oracle.com Fri Feb 7 12:59:59 2020 From: frederic.parain at oracle.com (Frederic Parain) Date: Fri, 7 Feb 2020 07:59:59 -0500 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> <1830530c-b086-534b-9bef-bb7e762fd6c6@redhat.com> <9a20124c-10fa-448d-0135-e7aaf594caa7@redhat.com> Message-ID: <41A9FAF2-33C9-4626-9364-2C51149945C3@oracle.com> Coleen, I?ve rewritten the code as you suggested. There?s no break because it is now a best-fit search, the loop doesn?t stop when a first candidate is found, but continues up to the start block to find the smallest candidate. Thank you, Fred > On Feb 6, 2020, at 17:59, coleen.phillimore at oracle.com wrote: > > > Except for the same comment as Aleksey about this: > > http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c-diff/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.udiff.html > > + while (cursor != start) { > + if (cursor->kind() == LayoutRawBlock::EMPTY && cursor->fit(b->size(), b->alignment())) { > + if (candidate == NULL) candidate = cursor; > + else if (cursor->size() < candidate->size()) candidate = cursor; > + } > + cursor = cursor->prev_block(); > + } > > Shouldn't it be: > > + while (cursor != start) { > + if (cursor->kind() == LayoutRawBlock::EMPTY && cursor->fit(b->size(), b->alignment())) { > + if (candidate == NULL || cursor->size() < candidate->size()) { + candidate = cursor; > + } > + } > + cursor = cursor->prev_block(); > + } > > > and should there be a 'break' once you've got your candidate? > > Thanks, > Coleen > > On 2/6/20 1:54 PM, Frederic Parain wrote: >> Aleksey, >> >> Thank you the review, I?ve fixed all issues (except deprecating the >> option, see below), and updated the webrev in place: >> >> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html >> >> >>> On Feb 6, 2020, at 13:05, Aleksey Shipilev wrote: >>> >>> Hi, >>> >>> I have to say that seeing the new field layouter matching all the cases with perfect layouter >>> simulator I did back in 2013 is very impressive! Good job. >> Thank you. Your simulator helped finding some bugs and inefficiencies. >> >>> On 2/6/20 5:36 PM, Frederic Parain wrote: >>>> Here?s a new version of the code addressing these regressions: >>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html >>> Only the minor ones. I don't need another webrev, you might consider fixing them before push. >>> >>> *) UseEmptySlotsInSupers should also be deprecated right away? >> The plan was to keep this option on the long term, for use cases >> where keeping fields from a given class close to each other is more >> important than having a smaller instance size. >> >>> === fieldLayoutBuilder.cpp: >>> >>> *) Generally, it feels some method bodies would enjoy some new lines, to split logical parts of the >>> method. For example, fillHoles() seems to break into five logical parts: >>> https://paste.centos.org/view/1b7805f6 >> I?e added more new lines across this file to make the code easier to read. >> >>> *) "(t)he requirements"? >>> >>> 179 // was successful. If he requirements were the same but the search failed, a new search will >> Fixed >> >>> *) "INHERITED bloc(k)s"? >>> >>> 302 // INHERITED blocs are marked as non-reference because oop_maps are handled by their holder class >> Fixed >> >>> *) What is so suspicious about this test? >>> >>> 389 if (block->prev_block() != NULL) { // suspicious test >> After careful review, this case is not suspicious, it matches >> one case of insertion. Comment removed. >> >>> *) "!last_search_success" here? >>> >>> 181 else if (b->size() == last_size && b->alignment() == last_alignment && >>> last_search_success == false) { >> Fixed. >> >> >>> *) Still "first fit"? >>> >>> 239 // The allocation logic uses a first fit strategy: the set of fields is allocated >> Yes, FieldLayout::add_contiguously() still uses a first-fit strategy, but it is only >> to allocate static fields. >> >> However, I?ve update the comment before FieldLayout::add() to clarify the best-fit search. >> >>> *) Still true? oop fields can now fill the gaps, no? >>> >>> 598 // - then oop fields are allocated contiguously (to reduce the number of oopmaps >>> 599 // and reduce the work of the GC). >> Updated. >> >>> *) Let's write this: >>> >>> 192 if (candidate == NULL) candidate = cursor; >>> 193 else if (cursor->size() < candidate->size()) candidate = cursor; >>> >>> like this: >>> >>> if (candidate == NULL) { >>> candidate = cursor; >>> } else if (cursor->size() < candidate->size()) { >>> candidate = cursor; >>> } >>> >> Definitively easier to read. >> >>> *) I believe indenting switch cases is good style here: >>> >>> 559 switch(type) { >>> ... >>> 570 case T_OBJECT: >>> 571 case T_ARRAY: >>> 572 if (group != _static_fields) _nonstatic_oopmap_count++; >>> 573 group->add_oop_field(fs); >>> ... >>> >>> I.e.: >>> >>> switch(type) { >>> case T_OBJECT: >>> case T_ARRAY: >>> ... >>> default: >>> ... >>> } >>> >> I?ve fixed indentation of several switch statements in this file. >> >>> -- >>> Thanks, >>> -Aleksey >>> >> Thank you, >> >> Fred >> > From magnus.ihse.bursie at oracle.com Fri Feb 7 13:09:28 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 7 Feb 2020 14:09:28 +0100 Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images In-Reply-To: References: <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com> <2b92559a-6a1a-119a-1e62-eed6f9d3bfc8@oracle.com> <1989082f-cddf-9d4f-262b-5f9072f46cf7@oracle.com> Message-ID: <9cd2165f-64f0-a096-9b99-aa8bc0798b34@oracle.com> On 2020-02-07 09:50, Baesken, Matthias wrote: > Hello, here is a slightly changed new webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.4/ In Bundles.gmk, this line: $(ECHO) found stripped pdb file $$$${f}, we rename it to: $$$${f%stripped.pdb}pdb; \ It looks almost like left-over debug output. If you want to keep it, please rephrase to something more terse, that fits better with the existing style of build messages. Also, it should probably be on the LOG=info level, so add a $(LOG_INFO). In NativeCompilation.gmk: Why not just a simple, ????????? ifeq ($(ENABLE_STRIPPED_PDBS), true) ??????????? $1_EXTRA_LDFLAGS += "-pdbstripped:$$($1_SYMBOLS_DIR)/$$($1_NOSUFFIX).stripped.pdb" ????????? endif ? I see no reason to duplicate code. In jdk-options.m4: I'm not 100% sure about the name of the new option. --enable-stripped-pdb does not fully convey the fact that we do not strip the *existing* pdb:s, but instead also add a new type. Maybe --enable-bundle-stripped-pdb? /Magnus > > (adjusted $(JRE_STRIPPED_PDB_FILES) in Bundles.gmk, that was in the wrong place ) . > >> I think it needs to be a separate option as it's all orthogonal to the >> main debug symbols file creation. > Yes it is a separate option I agree that?s better . One has to set --enable-stripped-pdbs=yes . > > Best regards , Matthias > > >> On 2020-02-06 04:48, Magnus Ihse Bursie wrote: >>> On 2020-02-06 12:36, Langer, Christoph wrote: >>>> Hi, >>>> >>>> let me chime in to this discussion at this point. So, first of all, Matthias, >> thanks for your work so far. >>>> Erik, I fully understand your points and I agree that it's probably a good >> idea to refactor this whole process of creating these different types of >> bundles. >>>> Our idea is to provide functionality in the build system to add "public" or >> stripped debug files to the delivery image of the JDK. This would provide >> better information in case of crashes and would hence allow for better >> supportability. That's something we'd probably want to enable in >> SapMachine binary distributions. >> I very much support the idea of using these stripped pdb files. It has >> been a long standing issue in hotspot on Windows to not have access to >> stacktraces. >>>> So, can we get to add a configure option like the one proposed by >> Matthias into the current code base? >>>> The option should be turned off by default. If it is switched on, the >> images/jdk folder in the build directory will have both, the *stripped.pdb >> files and the standard *.pdb files. However, having *stripped.pdb files >> around should not matter in terms of functionality (for developers and >> testing) as they'd not be used in the presence of the "real" pdb files anyway. >> The actual JDK bundle for delivery would then contain the *stripped.pdb >> files, renamed to *.pdb. And the debug symbols bundle would have the full >> *.pdb files only. Both could then be overlaid as needed. >>>> I think you raised two concerns. >>>> One is that you'd rather like to refactor bundling for several reasons. But I >> guess, should you eventually get to your refactoring, it shouldn't be a >> problem to take the functionality of this new option along. >>>> The other was regarding JMODs. I believe it's correct, that JMODs have >> never carried external debug information. For platforms other than MS >> Windows that's actually not a big problem because debug information can be >> internalized. And jlink has gotten several additions to set flags for stripping >> that data to the right level. I assume if jlinked images on Windows should >> ever be enabled to carry debug information, inclusion of external debug files >> would have to be added to JMODs and jlink. But that's definitely out of scope >> here. >> The argument "jmods have never carried external debug information" just >> doesn't apply here. Neither has the distribution bundles, for the exact >> same reason. You really should not compare these new stripped pdb files >> to the existing debug symbol files, they are different files with >> different purposes. One is meant to be shipped to customers, the other >> is not. You say you want to ship these new stripped pdb files with your >> distribution so that customers have them present when they use your >> distribution. If you then omit these new files from the jmods, any >> customer created jlinked image will not have these new stripped pdb >> files, which IMO is a very weird and unexpected behavior from a customer >> point of view. Jlinking new images is an integral and promoted way of >> using a JDK, so any mismatch between the original JDK distribution and >> what you are able to jlink is a serious discrepancy. >>>> So, what do you think? What speaks against adding this option (that is off >> by default)? >> >> My main objective is that you introduce further discrepancies between >> the original distribution JDK image and what's possible to generate >> using jlink from that distribution JDK image. My second objective is >> that the already convoluted bundles creation logic becomes even more >> convoluted. I believe there is a better possible solution in the build >> but it will require a lot more work to figure out. >> >> All that said, if you still wish to continue, I will stop standing in >> the way. >> >>> While Erik will need to comment on this himself, from my POV this >>> looks okay. The conditions are: >>> >>> * This is controlled by a separate option (or perhaps even better as a >>> new argument to --with-native-debug-symbols), so without this, nothing >>> will change. >>> >> I think it needs to be a separate option as it's all orthogonal to the >> main debug symbols file creation. >>> * The code need to make sure to separate stripped.pdb and normal pdb >>> files, when enabled. >>> >>> * And there must not be a heavy penalty in added code complexity. >>> >> /Erik >>> /Magnus >>> >>>> Thanks >>>> Christoph >>>> >>>>> -----Original Message----- >>>>> From: build-dev On Behalf Of >> Erik >>>>> Joelsson >>>>> Sent: Donnerstag, 23. Januar 2020 18:49 >>>>> To: Baesken, Matthias; David Holmes >>>>> ; 'build-dev at openjdk.java.net' >>>> dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' >>>> dev at openjdk.java.net> >>>>> Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows >> for >>>>> jdk images >>>>> >>>>> >>>>> On 2020-01-23 00:03, Baesken, Matthias wrote: >>>>>> Hi Erik, yes true sorry for answering your comments a bit late . >>>>>> >>>>>>> If a user runs jlink and includes all the jmods we ship with the JDK, the >>>>> result >>>>>>> should be essentially equivalent to the original JDK image. The way >> the >>>>>>> stripped pdb files are included in the bundles sort of at the last >>>>>>> second of the build here breaks this property. >>>>>> I think we should address this in a separate bug/CR . >>>>> Maybe. I realize that my proposal below is quite a big task. But on the >>>>> other hand, I don't think breaking the relationship between the jmods >>>>> and the distribution bundles is on the table really. >>>>>> Looking for example into a Linux build, I see a lot of debuginfo files in >> the >>>>> jdk image (more or less for every shared lib) . >>>>>> But when looking into the jmods of that jdk image , no debuginfo files >> are >>>>> in there ( I checked the java.base jmod). >>>>>> So putting the files with debug information into the jmods seems to >> be >>>>> something that was not done so far cross platform (or is there some >> build >>>>> switch for it that I did not check?) . >>>>>> Maybe to keep the jmods as small as possible . >>>>> No, we do not put the debuginfo files in the jmods nor the bundles >>>>> because those are not intended to be shipped to customers. We are >>>>> currently overlaying them into images/jdk in the build output to make >>>>> them available for local debugging. (This is convoluted and I would very >>>>> much like to get away from this practice at some point so that there is >>>>> a 1-1 mapping between images/jdk and bundles/jdk*-bin.tar.gz.) The >>>>> stripped pdb files you are proposing are on the contrary intended for >>>>> shipping to customers (as I understand your proposal) so comparing >> them >>>>> with the debuginfo files is not relevant. >>>>> >>>>> Now if MS had been kind enough to define a separate file type for the >>>>> stripped pdbs, so that they could live alongside the full pdbs, we >>>>> wouldn't have this issue. The heart of the problem is that only one set >>>>> of files (either stripped or full) can be present and usable in >>>>> images/jdk at a time. We have 2 main uses for images/jdk. >>>>> >>>>> 1. Developer running and debugging locally >>>>> >>>>> 2. Serve as the source for generating the distribution bundles >>>>> >>>>> We currently have one image serving both of these purposes, which is >>>>> already creating complicated and convoluted build steps. To properly >>>>> solve this I would argue for splitting these two apart into two >>>>> different images for the two different purposes. The build procedure >>>>> would then be, first build the images for distribution, only containing >>>>> what should go into each bundle. Then create the developer jdk image >> by >>>>> copying files from the distribution images. On Windows, the first image >>>>> would contain the stripped pdbs and when building the second, those >>>>> would get overwritten with the full pdbs. >>>>> >>>>> Now that I figured out a working model that would solve a bunch of >> other >>>>> problems as well, I would love to implement it, but I doubt I will have >>>>> time in the near future. >>>>> >>>>> /Erik >>>>> >>>>>>> To properly implement this, care will need to be taken to juggle the >> two >>>>>>> sets of pdb files around, making sure each build and test use case has >>>>>>> the correct one in place where and when it's needed. Quite possibly, >> we >>>>>>> cannot cover all use cases with one build configuration. Developers >>>>>>> needing the full debug symbols when debugging locally would likely >> need >>>>>>> to disable the stripped symbols so they get the full symbols >> everywhere. >>>>>>> Possibly this would need to be the default for debug builds and >>>>>>> configurable for release builds. >>>>>> From my limited experience , the developers do not work with the >>>>> bundles (that would contain now after my patch the stripped pds) but >> with >>>>> a "normal" jdk image that is created my make all. >>>>>> Best regards, Matthias >>>>>> >>>>>> >>>>>> >>>>>>> This still does not address anything in my objection. >>>>>>> >>>>>>> /Erik >>>>>>> >>>>>>> On 2020-01-22 07:46, Baesken, Matthias wrote: >>>>>>>> Hello, here is an updated version : >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/ >>>>>>>> >>>>>>>> this one supports a configure switch "--enable-stripped-pdbs" to >>>>> enable >>>>>>> the feature . >>>>>>>> Best regards, Matthias >>>>>>>> >>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Baesken, Matthias >>>>>>>>> Sent: Dienstag, 21. Januar 2020 11:03 >>>>>>>>> To: 'David Holmes'; 'build- >>>>>>>>> dev at openjdk.java.net'; 'hotspot- >>>>>>>>> dev at openjdk.java.net' >>>>>>>>> Subject: RE: RFR: 8237192: Generate stripped/public pdbs on >> Windows >>>>> for >>>>>>>>> jdk images >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi David , yes I think it makes sense to have a configure option for >> this . >>>>>>>>> Not everyone would like to have a larger JDK (even it is only a bit >>>>> larger). From coleen.phillimore at oracle.com Fri Feb 7 13:13:09 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 7 Feb 2020 08:13:09 -0500 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: <41A9FAF2-33C9-4626-9364-2C51149945C3@oracle.com> References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> <1830530c-b086-534b-9bef-bb7e762fd6c6@redhat.com> <9a20124c-10fa-448d-0135-e7aaf594caa7@redhat.com> <41A9FAF2-33C9-4626-9364-2C51149945C3@oracle.com> Message-ID: On 2/7/20 7:59 AM, Frederic Parain wrote: > Coleen, > > I?ve rewritten the code as you suggested. > > There?s no break because it is now a best-fit search, > the loop doesn?t stop when a first candidate is found, > but continues up to the start block to find the smallest > candidate. Ok!? Sounds good. Coleen > Thank you, > > Fred > > >> On Feb 6, 2020, at 17:59, coleen.phillimore at oracle.com wrote: >> >> >> Except for the same comment as Aleksey about this: >> >> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c-diff/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.udiff.html >> >> + while (cursor != start) { >> + if (cursor->kind() == LayoutRawBlock::EMPTY && cursor->fit(b->size(), b->alignment())) { >> + if (candidate == NULL) candidate = cursor; >> + else if (cursor->size() < candidate->size()) candidate = cursor; >> + } >> + cursor = cursor->prev_block(); >> + } >> >> Shouldn't it be: >> >> + while (cursor != start) { >> + if (cursor->kind() == LayoutRawBlock::EMPTY && cursor->fit(b->size(), b->alignment())) { >> + if (candidate == NULL || cursor->size() < candidate->size()) { + candidate = cursor; >> + } >> + } >> + cursor = cursor->prev_block(); >> + } >> >> >> and should there be a 'break' once you've got your candidate? >> >> Thanks, >> Coleen >> >> On 2/6/20 1:54 PM, Frederic Parain wrote: >>> Aleksey, >>> >>> Thank you the review, I?ve fixed all issues (except deprecating the >>> option, see below), and updated the webrev in place: >>> >>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html >>> >>> >>>> On Feb 6, 2020, at 13:05, Aleksey Shipilev wrote: >>>> >>>> Hi, >>>> >>>> I have to say that seeing the new field layouter matching all the cases with perfect layouter >>>> simulator I did back in 2013 is very impressive! Good job. >>> Thank you. Your simulator helped finding some bugs and inefficiencies. >>> >>>> On 2/6/20 5:36 PM, Frederic Parain wrote: >>>>> Here?s a new version of the code addressing these regressions: >>>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html >>>> Only the minor ones. I don't need another webrev, you might consider fixing them before push. >>>> >>>> *) UseEmptySlotsInSupers should also be deprecated right away? >>> The plan was to keep this option on the long term, for use cases >>> where keeping fields from a given class close to each other is more >>> important than having a smaller instance size. >>> >>>> === fieldLayoutBuilder.cpp: >>>> >>>> *) Generally, it feels some method bodies would enjoy some new lines, to split logical parts of the >>>> method. For example, fillHoles() seems to break into five logical parts: >>>> https://paste.centos.org/view/1b7805f6 >>> I?e added more new lines across this file to make the code easier to read. >>> >>>> *) "(t)he requirements"? >>>> >>>> 179 // was successful. If he requirements were the same but the search failed, a new search will >>> Fixed >>> >>>> *) "INHERITED bloc(k)s"? >>>> >>>> 302 // INHERITED blocs are marked as non-reference because oop_maps are handled by their holder class >>> Fixed >>> >>>> *) What is so suspicious about this test? >>>> >>>> 389 if (block->prev_block() != NULL) { // suspicious test >>> After careful review, this case is not suspicious, it matches >>> one case of insertion. Comment removed. >>> >>>> *) "!last_search_success" here? >>>> >>>> 181 else if (b->size() == last_size && b->alignment() == last_alignment && >>>> last_search_success == false) { >>> Fixed. >>> >>> >>>> *) Still "first fit"? >>>> >>>> 239 // The allocation logic uses a first fit strategy: the set of fields is allocated >>> Yes, FieldLayout::add_contiguously() still uses a first-fit strategy, but it is only >>> to allocate static fields. >>> >>> However, I?ve update the comment before FieldLayout::add() to clarify the best-fit search. >>> >>>> *) Still true? oop fields can now fill the gaps, no? >>>> >>>> 598 // - then oop fields are allocated contiguously (to reduce the number of oopmaps >>>> 599 // and reduce the work of the GC). >>> Updated. >>> >>>> *) Let's write this: >>>> >>>> 192 if (candidate == NULL) candidate = cursor; >>>> 193 else if (cursor->size() < candidate->size()) candidate = cursor; >>>> >>>> like this: >>>> >>>> if (candidate == NULL) { >>>> candidate = cursor; >>>> } else if (cursor->size() < candidate->size()) { >>>> candidate = cursor; >>>> } >>>> >>> Definitively easier to read. >>> >>>> *) I believe indenting switch cases is good style here: >>>> >>>> 559 switch(type) { >>>> ... >>>> 570 case T_OBJECT: >>>> 571 case T_ARRAY: >>>> 572 if (group != _static_fields) _nonstatic_oopmap_count++; >>>> 573 group->add_oop_field(fs); >>>> ... >>>> >>>> I.e.: >>>> >>>> switch(type) { >>>> case T_OBJECT: >>>> case T_ARRAY: >>>> ... >>>> default: >>>> ... >>>> } >>>> >>> I?ve fixed indentation of several switch statements in this file. >>> >>>> -- >>>> Thanks, >>>> -Aleksey >>>> >>> Thank you, >>> >>> Fred >>> From igor.ignatyev at oracle.com Fri Feb 7 15:47:35 2020 From: igor.ignatyev at oracle.com (Igor Ignatev) Date: Fri, 7 Feb 2020 07:47:35 -0800 Subject: RFR: JDK-8238586: [TESTBUG] vmTestbase/jit/tiered/Test.java failed when TieredCompilation is disabled In-Reply-To: References: Message-ID: I?d consider it as trivial. ? Igor > On Feb 6, 2020, at 11:22 PM, Jia Huang wrote: > > ?Thanks Igor. > > Is it a trivial patch and OK to be pushed? > Or shall we wait for one more review? > > Thanks a lot. > Best regards, > Jia. > >> ? 2020/2/7 14:06, Igor Ignatyev ??: >> Hi Jia, >> >> LGTM >> >> -- Igor >> >>>> On Feb 6, 2020, at 1:20 AM, Jia Huang wrote: >>> >>> Hi all, >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238586 >>> >>> When TieredCompilation is disabled, the test will fail. >>> Please review the following fix which make the test pass when disabled TieredCompilation. >>> ------------------------------------------------------ >>> diff -r e45d8ca836b5 -r 67bef3d6692f test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java >>> --- a/test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java Mon Feb 03 22:10:59 2020 +0800 >>> +++ b/test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java Thu Feb 06 15:25:19 2020 +0800 >>> @@ -46,7 +46,7 @@ >>> import jdk.test.lib.process.ProcessTools; >>> >>> public class Test { >>> - private static String UNSUPPORTED_OPTION_MESSAGE = "-XX:TieredCompilation not supported in this VM"; >>> + private static String UNSUPPORTED_OPTION_MESSAGE = "-XX:+TieredCompilation not supported in this VM"; >>> private static String REGEXP = "^[0-9.]+: \\[compile level=\\d"; >>> public static void main(String[] args) throws Exception { >>> { >>> @@ -56,7 +56,7 @@ >>> "-XX:+PrintTieredEvents", >>> "-version"); >>> var output = new OutputAnalyzer(pb.start()); >>> - if (output.getStdout().contains(UNSUPPORTED_OPTION_MESSAGE)) { >>> + if (output.getStderr().contains(UNSUPPORTED_OPTION_MESSAGE)) { >>> throw new SkippedException(UNSUPPORTED_OPTION_MESSAGE); >>> } >>> output.shouldHaveExitValue(0) >>> ------------------------------------------------------ >>> For more info, please see the JBS. >>> >>> Thanks a lot >>> >>> Best regards, >>> Jia. >>> > From zgu at redhat.com Fri Feb 7 15:53:56 2020 From: zgu at redhat.com (Zhengyu Gu) Date: Fri, 7 Feb 2020 10:53:56 -0500 Subject: [15] RFR 8238633: JVMTI heap walk should consult GC for marking oops Message-ID: Hi, I would like purpose this change that allows GC to provide ObjectMarker during JVMTI heap walk. Currently, JVMTI heap walk uses oop markword's 'marked' pattern to indicate 'visited' oop. Unfortunately, it conflicts with Shenandoah, who uses the pattern to indicate 'forwarding'. When JVMTI heap walk occurs in some of Shenandoah's concurrent heap (e.g. concurrent evacuation or concurrent reference updating phases), it can result corrupted heap, as it tries to resolve a real oop header as a forwarding pointer. This patch allows GC to provide ObjectMarker for JVMTI to track 'visited' oop, and uses current implementation as default, so that, it has no impact to GCs other than Shenandoah, who provides its own implementation. Bug: https://bugs.openjdk.java.net/browse/JDK-8238633 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.00/index.html Test: hotspot_gc vmTestbase_nsk_jdi vmTestbase_nsk_jvmti Thanks, -Zhengyu From vladimir.x.ivanov at oracle.com Fri Feb 7 16:52:36 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 7 Feb 2020 19:52:36 +0300 Subject: [15] RFR (S): 8238696: x86: Enumerate all detected CPU features in VM_Version feature string Message-ID: http://cr.openjdk.java.net/~vlivanov/8238696/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8238696 Abstract_VM_Version::features_string() doesn't contain all the CPU features detected by the JVM on x86 (for example, for AVX-512 only AVX-512F is included as "evex"). The fix adds missing features to the list [1]. Testing: manual (-Xlog:os+cpu=trace output on different hardware), hs-precheckin-comp, hs-tier1, hs-tier2 Best regards, Vladimir Ivanov [1] $ java -Xlog:os+cpu=trace -XX:UseAVX=3 -version ... [0.010s][info][os,cpu] UseSSE=4 UseAVX=3 UseAES=1 MaxVectorSize=64 ... [0.011s][info][os,cpu] CPU:total 8 (initial active 8) (4 cores per cpu, 2 threads per core) family 6 model 85 stepping 4, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, vzeroupper, avx, avx2, aes, clmul, erms, rtm, 3dnowpref, lzcnt, ht, tsc, bmi1, bmi2, adx, avx512f, avx512dq, avx512cd, avx512bw, avx512vl, fma, clflush, clflushopt, clwb ... [0.011s][info][os,cpu] flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat umip pku ospke md_clear ... From frederic.parain at oracle.com Fri Feb 7 18:52:09 2020 From: frederic.parain at oracle.com (Frederic Parain) Date: Fri, 7 Feb 2020 13:52:09 -0500 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> <1830530c-b086-534b-9bef-bb7e762fd6c6@redhat.com> <9a20124c-10fa-448d-0135-e7aaf594caa7@redhat.com> <41A9FAF2-33C9-4626-9364-2C51149945C3@oracle.com> Message-ID: Thank you Coleen. Fred > On Feb 7, 2020, at 08:13, coleen.phillimore at oracle.com wrote: > > > > On 2/7/20 7:59 AM, Frederic Parain wrote: >> Coleen, >> >> I?ve rewritten the code as you suggested. >> >> There?s no break because it is now a best-fit search, >> the loop doesn?t stop when a first candidate is found, >> but continues up to the start block to find the smallest >> candidate. > > Ok! Sounds good. > Coleen > >> Thank you, >> >> Fred >> >> >>> On Feb 6, 2020, at 17:59, coleen.phillimore at oracle.com wrote: >>> >>> >>> Except for the same comment as Aleksey about this: >>> >>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c-diff/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.udiff.html >>> >>> + while (cursor != start) { >>> + if (cursor->kind() == LayoutRawBlock::EMPTY && cursor->fit(b->size(), b->alignment())) { >>> + if (candidate == NULL) candidate = cursor; >>> + else if (cursor->size() < candidate->size()) candidate = cursor; >>> + } >>> + cursor = cursor->prev_block(); >>> + } >>> >>> Shouldn't it be: >>> >>> + while (cursor != start) { >>> + if (cursor->kind() == LayoutRawBlock::EMPTY && cursor->fit(b->size(), b->alignment())) { >>> + if (candidate == NULL || cursor->size() < candidate->size()) { + candidate = cursor; >>> + } >>> + } >>> + cursor = cursor->prev_block(); >>> + } >>> >>> >>> and should there be a 'break' once you've got your candidate? >>> >>> Thanks, >>> Coleen >>> >>> On 2/6/20 1:54 PM, Frederic Parain wrote: >>>> Aleksey, >>>> >>>> Thank you the review, I?ve fixed all issues (except deprecating the >>>> option, see below), and updated the webrev in place: >>>> >>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html >>>> >>>> >>>>> On Feb 6, 2020, at 13:05, Aleksey Shipilev wrote: >>>>> >>>>> Hi, >>>>> >>>>> I have to say that seeing the new field layouter matching all the cases with perfect layouter >>>>> simulator I did back in 2013 is very impressive! Good job. >>>> Thank you. Your simulator helped finding some bugs and inefficiencies. >>>> >>>>> On 2/6/20 5:36 PM, Frederic Parain wrote: >>>>>> Here?s a new version of the code addressing these regressions: >>>>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html >>>>> Only the minor ones. I don't need another webrev, you might consider fixing them before push. >>>>> >>>>> *) UseEmptySlotsInSupers should also be deprecated right away? >>>> The plan was to keep this option on the long term, for use cases >>>> where keeping fields from a given class close to each other is more >>>> important than having a smaller instance size. >>>> >>>>> === fieldLayoutBuilder.cpp: >>>>> >>>>> *) Generally, it feels some method bodies would enjoy some new lines, to split logical parts of the >>>>> method. For example, fillHoles() seems to break into five logical parts: >>>>> https://paste.centos.org/view/1b7805f6 >>>> I?e added more new lines across this file to make the code easier to read. >>>> >>>>> *) "(t)he requirements"? >>>>> >>>>> 179 // was successful. If he requirements were the same but the search failed, a new search will >>>> Fixed >>>> >>>>> *) "INHERITED bloc(k)s"? >>>>> >>>>> 302 // INHERITED blocs are marked as non-reference because oop_maps are handled by their holder class >>>> Fixed >>>> >>>>> *) What is so suspicious about this test? >>>>> >>>>> 389 if (block->prev_block() != NULL) { // suspicious test >>>> After careful review, this case is not suspicious, it matches >>>> one case of insertion. Comment removed. >>>> >>>>> *) "!last_search_success" here? >>>>> >>>>> 181 else if (b->size() == last_size && b->alignment() == last_alignment && >>>>> last_search_success == false) { >>>> Fixed. >>>> >>>> >>>>> *) Still "first fit"? >>>>> >>>>> 239 // The allocation logic uses a first fit strategy: the set of fields is allocated >>>> Yes, FieldLayout::add_contiguously() still uses a first-fit strategy, but it is only >>>> to allocate static fields. >>>> >>>> However, I?ve update the comment before FieldLayout::add() to clarify the best-fit search. >>>> >>>>> *) Still true? oop fields can now fill the gaps, no? >>>>> >>>>> 598 // - then oop fields are allocated contiguously (to reduce the number of oopmaps >>>>> 599 // and reduce the work of the GC). >>>> Updated. >>>> >>>>> *) Let's write this: >>>>> >>>>> 192 if (candidate == NULL) candidate = cursor; >>>>> 193 else if (cursor->size() < candidate->size()) candidate = cursor; >>>>> >>>>> like this: >>>>> >>>>> if (candidate == NULL) { >>>>> candidate = cursor; >>>>> } else if (cursor->size() < candidate->size()) { >>>>> candidate = cursor; >>>>> } >>>>> >>>> Definitively easier to read. >>>> >>>>> *) I believe indenting switch cases is good style here: >>>>> >>>>> 559 switch(type) { >>>>> ... >>>>> 570 case T_OBJECT: >>>>> 571 case T_ARRAY: >>>>> 572 if (group != _static_fields) _nonstatic_oopmap_count++; >>>>> 573 group->add_oop_field(fs); >>>>> ... >>>>> >>>>> I.e.: >>>>> >>>>> switch(type) { >>>>> case T_OBJECT: >>>>> case T_ARRAY: >>>>> ... >>>>> default: >>>>> ... >>>>> } >>>>> >>>> I?ve fixed indentation of several switch statements in this file. >>>> >>>>> -- >>>>> Thanks, >>>>> -Aleksey >>>>> >>>> Thank you, >>>> >>>> Fred >>>> > From frederic.parain at oracle.com Fri Feb 7 18:52:25 2020 From: frederic.parain at oracle.com (Frederic Parain) Date: Fri, 7 Feb 2020 13:52:25 -0500 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: <83dbad70-8398-2209-ba34-2bb417e3788b@oracle.com> References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> <7E7AEBD0-09DD-4EA0-B2E0-2102EDABCAD9@oracle.com> <83dbad70-8398-2209-ba34-2bb417e3788b@oracle.com> Message-ID: <8782D960-4024-4E55-BEED-71EB0CD34328@oracle.com> Thank you David, Fred > On Feb 6, 2020, at 17:45, David Holmes wrote: > > Hi Fred, > > Nothing further from me. > > Thanks, > David > > On 7/02/2020 2:28 am, Frederic Parain wrote: >> David, >> The deprecated flag has been added to the table and to the test. >> Indentation/spaces have been fixed. >> CSR has been updated too. >> New webrev: >> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html >> The webrev also contains some modifications to address issues found >> by Aleksey. See my response to his e-mail to have the incremental >> webrev. >> Thank you, >> Fred >>> On Feb 3, 2020, at 01:24, David Holmes wrote: >>> >>> Hi Fred, >>> >>> On 1/02/2020 2:03 am, Frederic Parain wrote: >>>> Hi David, >>>> I?ve fixed all the issues you mentioned below. >>>> According to the discussion about the VM flags, the following modifications >>>> have also been made: >>>> globals.hpp: >>>> UseNewFieldLayout is now a deprecated product flag >>> >>> You need to add it to the deprecated flag table in arguments.cpp and to the test >>> >>> test/hotspot/jtreg/runtime/CommandLine/VMDeprecatedOptions.java >>> >>>> fieldLayoutBuilder.cpp >>>> lines 350-362: >>>> UseEmptySlotsInSupers VM option now controls both new optimizations >>>> lines 449-464 >>>> Fix an issue with layout printing (inherited fields were printed >>>> only for the direct super-class, now all inherited fields are printed) >>>> New webrev: >>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.08/index.html >>> >>> Looks good. A couple of minor follow ups ... >>> >>> src/hotspot/share/classfile/classFileParser.cpp >>> >>> ! OopMapBlock* nonstatic_oop_map = _nonstatic_oop_maps; >>> >>> Still an extra space after * >>> >>> + FieldLayoutBuilder lb(class_name(), super_klass(), _cp, _fields, >>> + _parsed_annotations->is_contended(), _field_info); >>> >>> Indentation of continuing line is wrong. >>> >>> For the test, you have now restricted it to only run on 64-bit even though it has code that appears to allow for 32-bit. As this isn't a 64-bit specific feature I think it should run on all platforms. So this: >>> >>> /* >>> * @test >>> * @bug 8237767 >>> * @requires vm.bits == "64" >>> * @summary Verify behaviour of field layout algorithm >>> * @library /test/lib >>> * @modules java.base/jdk.internal.misc >>> * java.management >>> * @run main/othervm -XX:+UseCompressedOops -XX:+UseCompressedClassPointers FieldDensityTest >>> * @run main/othervm -XX:+UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest >>> * @run main/othervm -XX:-UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest >>> */ >>> >>> can become: >>> >>> /* >>> * @test >>> * @bug 8237767 >>> * @summary Verify behaviour of field layout algorithm >>> * @library /test/lib >>> * @modules java.base/jdk.internal.misc >>> * java.management >>> * @run main/othervm FieldDensityTest >>> */ >>> >>> /* >>> * @test >>> * @requires vm.bits == "64" >>> * @run main/othervm -XX:+UseCompressedOops -XX:+UseCompressedClassPointers FieldDensityTest >>> * @run main/othervm -XX:+UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest >>> */ >>> >>>> CR and CSR have been updated accordingly (new VM flags explanations in the CSR). >>> >>> There is one small glitch in the CSR. Code removal associated with a flag occurs when it is obsoleted. So under the proposed timeline of deprecation in 15 and obsoletion in 16, the old code will be removed in 16 not 17. Comment added to CSR to that effect. >>> >>> Thanks, >>> David >>> >>>> Thank you, >>>> Fred >>>>> On Jan 29, 2020, at 02:06, David Holmes wrote: >>>>> >>>>> Hi Fred, >>>>> >>>>> I've looked at the v7 version. A few more stylistic comments on that first. Note, no need for an item by item response unless that makes it easier for you to track :) >>>>> >>>>> src/hotspot/share/classfile/classFileParser.cpp >>>>> >>>>> #include "classfile/defaultMethods.hpp" >>>>> +#include "classfile/fieldLayoutBuilder.hpp" >>>>> #include "classfile/dictionary.hpp" >>>>> >>>>> Include files are not in alphabetical order. >>>>> >>>>> + * This may well change: FixMe if doesn't, >>>>> >>>>> s/if/if it/ >>>>> >>>>> + //Make a temp copy, and iterate through and copy back into the orig >>>>> >>>>> Space after // >>>>> >>>>> s/orig/original/ >>>>> >>>>> + OopMapBlock* nonstatic_oop_map = _nonstatic_oop_maps; >>>>> >>>>> Extra space after * >>>>> >>>>> + assert(ik->nonstatic_oop_map_count() == _field_info->oop_map_blocks->_nonstatic_oop_map_count, >>>>> + "sanity"); >>>>> >>>>> Second line needs to be indented further: >>>>> >>>>> assert(ik->nonstatic_oop_map_count() == _field_info->oop_map_blocks->_nonstatic_oop_map_count, >>>>> "sanity"); >>>>> >>>>> --- >>>>> >>>>> src/hotspot/share/classfile/classFileParser.hpp >>>>> >>>>> +public: >>>>> + OopMapBlock* _nonstatic_oop_maps; >>>>> + unsigned int _nonstatic_oop_map_count; >>>>> + unsigned int _max_nonstatic_oop_maps; >>>>> + >>>>> + public: >>>>> >>>>> Second public uneeded. First public may be indented wrong (I'm not sure what the rule is - single space indent?) >>>>> >>>>> class ClassFileParser { >>>>> + friend class FieldLayoutBuilder; >>>>> + friend class FieldLayout; >>>>> >>>>> class ClassAnnotationCollector; >>>>> class FieldAllocationCount; >>>>> class FieldAnnotationCollector; >>>>> >>>>> Indents are different. I think the class forward declarations should have extra space. >>>>> >>>>> --- >>>>> >>>>> src/hotspot/share/oops/instanceKlass.hpp >>>>> >>>>> + void increment_count(int diff) { _count += diff; } >>>>> >>>>> Extra spaces before { >>>>> >>>>> --- >>>>> >>>>> src/hotspot/share/runtime/globals.hpp >>>>> >>>>> + diagnostic(bool, UseNewFieldLayout, true, \ >>>>> + "Use new algorithm to compute layouts") \ >>>>> + \ >>>>> + product(bool, UseEmptySlotsInSupers, true, >>>>> >>>>> Not sure I see why one flag is diagnostic and the other product. Do you expect people to need to disable using empty slots more so than needing to disable using the new field layout altogether? >>>>> >>>>> --- >>>>> >>>>> src/hotspot/share/classfile/fieldLayoutBuilder.cpp >>>>> >>>>> + assert(kind == EMPTY || kind == RESERVED || kind == PADDING || kind == INHERITED, >>>>> + "Otherwise, should use the constructor with a field index argument"); >>>>> >>>>> Indentation of second line is wrong. >>>>> >>>>> + assert(kind == REGULAR || kind == FLATTENED || kind == INHERITED, >>>>> + "Other kind do not have a field index"); >>>>> >>>>> Ditto. >>>>> >>>>> >>>>> + if (list == NULL) return; >>>>> + if (start == NULL) { >>>>> + start = this->_start; >>>>> + } >>>>> >>>>> Inconsistent style for single statement if-blocks. Same thing later in the file. >>>>> >>>>> + output->print_cr(" @%d \"%s\" %s %d/%d %s", >>>>> + b->offset(), >>>>> + fi->name(_cp)->as_C_string(), >>>>> + fi->signature(_cp)->as_C_string(), >>>>> + b->size(), >>>>> + b->alignment(), >>>>> + "REGULAR"); >>>>> >>>>> Incorrect identation of continuing line. Same for all the following print blocks. >>>>> >>>>> + } else if (_classname == vmSymbols::java_lang_Boolean() || >>>>> + _classname == vmSymbols::java_lang_Character() || >>>>> + _classname == vmSymbols::java_lang_Float() || >>>>> + _classname == vmSymbols::java_lang_Double() || >>>>> + _classname == vmSymbols::java_lang_Byte() || >>>>> + _classname == vmSymbols::java_lang_Short() || >>>>> + _classname == vmSymbols::java_lang_Integer() || >>>>> + _classname == vmSymbols::java_lang_Long()) { >>>>> >>>>> Incorrect identation of continuing line. >>>>> >>>>> --- >>>>> >>>>> src/hotspot/share/classfile/fieldLayoutBuilder.hpp >>>>> >>>>> +// and the boxing classes). The rational for having multiple methods >>>>> >>>>> s/rational/rationale/ >>>>> >>>>> + FieldLayoutBuilder(const Symbol* classname, const InstanceKlass* super_klass, ConstantPool* constant_pool, >>>>> + Array* fields, bool is_contended, FieldLayoutInfo* info); >>>>> >>>>> Indentation wrong for continuing line. >>>>> >>>>> + int get_alignment() { >>>>> + assert(_alignment != -1, "Uninitialized"); >>>>> + return _alignment; >>>>> + } >>>>> >>>>> Indenting appears off by one. >>>>> >>>>> --- >>>>> >>>>> test/hotspot/jtreg/runtime/FieldLayout/FieldDensityTest.java >>>>> >>>>> + * @run main/othervm -XX:+UseCompressedOops -XX:+UseCompressedClassPointers FieldDensityTest >>>>> + * @run main/othervm -XX:+UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest >>>>> + * @run main/othervm -XX:-UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest >>>>> >>>>> The test won't run on 32-bit platforms as the compressed oops flags won't exist. >>>>> >>>>> --- >>>>> >>>>> Some follow up comments below ... >>>>> >>>>> With trimming ... >>>>> >>>>> On 25/01/2020 3:20 am, Frederic Parain wrote: >>>>>>> On Jan 24, 2020, at 08:19, David Holmes wrote: >>>>>>> >>>>>>> 466 int super_flen = super->nof_nonstatic_fields(); >>>>>>> >>>>>>> Could be folded directly into the assert so we don't call in product. >>>>>> Calling not_nonstatic_fields() has the side effect to compute non-static fields, >>>>>> which is required to get a correct value when reading super->_nonstatic_fields, >>>>>> so the call is needed even in product builds. >>>>> >>>>> Yuck! That's a horrible side-effect - but not your fault obviously. :) It would be better to have a nonstatic_fields() accessor that has the same lazy initialization side-effect. >>>>> >>>>>>> General style issue: when breaking a long line with a method call, the new line (containing arguments) should be indented to the opening ( of the method call e.g. >>>>> ... >>>>>>> etc. This applies across all files. >>>>>> Fixes applied lines 4003, 4011, 4041, 4138, 4143. >>>>> >>>>> Fix was also needed in other files. Current issues highlighted above. >>>>> >>>>>>> >>>>>>> src/hotspot/share/oops/instanceKlass.hpp >>>>>>> >>>>>>> You need to be careful with _extra_flags usage if there can be concurrently updated bits. At the moment it looks like redefinition is a mutable dynamic property, whilst "contended annotations" should be a static immutable property - is that right? >>>>>> Correct, _has_contended_annotations is a static immutable property, while _is_being_redefined is a mutable one. >>>>> >>>>> Good to know. My concern is that if someone adds a new mutable flag bit the need for atomic updates may not be noticed. We got bitten by this in the past with a flag field and I think we eventually migrated all of the mutable bits out into their own field. (Coleen should recall that :) ). >>>>> >>>>>>> 61 FLATTENED, // flattened field >>>>>>> >>>>>>> Does this have any meaning before inline types come in? >>>>>> Yes, I wanted to reserved the entry in the enum. >>>>> >>>>> Hmmm a tenuous "okay". Seems odd to require this now to support code that is still some way from joining mainline. >>>>> >>>>>>> In FieldLayoutBuilder::epilogue you have a number of calls to Thread::current() as well as an implicit call when you use ResourceMarks. You should capture the current thread once in a local and reuse it. >>>>>> Fixed >>>>> >>>>> It seems that this fix is now not needed as there is only one use left of the new "thread" variable in the ResourceMark. So that can return to being: >>>>> >>>>> ResourceMark rm; >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> From frederic.parain at oracle.com Fri Feb 7 18:52:40 2020 From: frederic.parain at oracle.com (Frederic Parain) Date: Fri, 7 Feb 2020 13:52:40 -0500 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: <3fd207ba-9593-2b10-30c9-475e27375973@oracle.com> References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> <1830530c-b086-534b-9bef-bb7e762fd6c6@redhat.com> <9a20124c-10fa-448d-0135-e7aaf594caa7@redhat.com> <3fd207ba-9593-2b10-30c9-475e27375973@oracle.com> Message-ID: <000E3D95-FD3F-4AF5-9B91-A8AC35F5DFDA@oracle.com> Thank you Lois, Fred > On Feb 6, 2020, at 15:29, Lois Foltan wrote: > > On 2/6/2020 2:06 PM, Aleksey Shipilev wrote: >> On 2/6/20 7:54 PM, Frederic Parain wrote: >>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html >> Still looks good to me. >> > Hi Fred, > > I have reviewed as well. Looks great and thank you for the preliminary code walk through we had about this change. > > Lois From frederic.parain at oracle.com Fri Feb 7 18:52:58 2020 From: frederic.parain at oracle.com (Frederic Parain) Date: Fri, 7 Feb 2020 13:52:58 -0500 Subject: RFR[L]: 8237767 Field layout computation overhaul In-Reply-To: References: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com> <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com> <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com> <1830530c-b086-534b-9bef-bb7e762fd6c6@redhat.com> <9a20124c-10fa-448d-0135-e7aaf594caa7@redhat.com> Message-ID: <7F6538D1-F06E-4C4E-A16A-5A679B51404A@oracle.com> Thank you Aleksey, Fred > On Feb 6, 2020, at 14:06, Aleksey Shipilev wrote: > > On 2/6/20 7:54 PM, Frederic Parain wrote: >> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.09c/index.html > > Still looks good to me. > > -- > Thanks, > -Aleksey > From erik.joelsson at oracle.com Fri Feb 7 21:49:39 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 7 Feb 2020 13:49:39 -0800 Subject: ARM build broken? In-Reply-To: <04FD9482-B39A-4E35-9DE6-489F3E02D2AE@mountainminds.com> References: <04FD9482-B39A-4E35-9DE6-489F3E02D2AE@mountainminds.com> Message-ID: Adding hotspot-dev since this is caused by a source change in hotspot and not really a build issue. We see it in our internal arm builds as well. /Erik On 2020-02-07 13:38, Marc Hoffmann wrote: > Dear JDK developers, > > it looks like building http://hg.openjdk.java.net/jdk/jdk/ in ARM is broken since yesterday. This commit seems to be the first one that broke the build: > changeset: 57948:4a4d185098e2 > tag: tip > user: lfoltan > date: Thu Feb 06 14:29:57 2020 +0000 > summary: 8230199: consolidate signature parsing code in HotSpot sources > this one now leads to another error (see below): > changeset: 57949:d3caf06ac9ae > tag: tip > user: lfoltan > date: Thu Feb 06 15:28:37 2020 +0000 > summary: 8238600: Remove runtime/fieldType.hpp and fieldType.cpp > > Is this a problem with the source tree or do we need to adjust the build setup on ARM? > > Background info: I?m a developer of the JaCoCo project and we do CI builds on all JDKs to test them agains the JaCoCo integration tests. > > Best regards, > -marc > > > > > > > ERROR: Build failed for target 'images' in configuration 'linux-arm-server-release' (exit code 2) > Stopping sjavac server > > === Output from failing command(s) repeated here === > * For target hotspot_variant-server_libjvm_objs_interpreterRT_arm.o: > /workspace/src/hotspot/cpu/arm/interpreterRT_arm.cpp:54:27: error: 'int_parm' is not a member of 'SignatureIterator' > (u1) SignatureIterator::int_parm, // bool > ^~~~~~~~ > /workspace/src/hotspot/cpu/arm/interpreterRT_arm.cpp:55:27: error: 'int_parm' is not a member of 'SignatureIterator' > (u1) SignatureIterator::int_parm, // byte > ^~~~~~~~ > /workspace/src/hotspot/cpu/arm/interpreterRT_arm.cpp:56:27: error: 'int_parm' is not a member of 'SignatureIterator' > (u1) SignatureIterator::int_parm, // char > ^~~~~~~~ > /workspace/src/hotspot/cpu/arm/interpreterRT_arm.cpp:57:27: error: 'int_parm' is not a member of 'SignatureIterator' > (u1) SignatureIterator::int_parm, // short > ^~~~~~~~ > /workspace/src/hotspot/cpu/arm/interpreterRT_arm.cpp:58:27: error: 'int_parm' is not a member of 'SignatureIterator' > (u1) SignatureIterator::int_parm, // int > ^~~~~~~~ > ... (rest of output omitted) > > From david.holmes at oracle.com Fri Feb 7 22:49:53 2020 From: david.holmes at oracle.com (David Holmes) Date: Sat, 8 Feb 2020 08:49:53 +1000 Subject: ARM build broken? In-Reply-To: References: <04FD9482-B39A-4E35-9DE6-489F3E02D2AE@mountainminds.com> Message-ID: <36b3f707-1468-a862-4987-0c328c05063c@oracle.com> Hi, It is a known breakage: https://bugs.openjdk.java.net/browse/JDK-8238643 David On 8/02/2020 7:49 am, Erik Joelsson wrote: > Adding hotspot-dev since this is caused by a source change in hotspot > and not really a build issue. We see it in our internal arm builds as well. > > /Erik > > On 2020-02-07 13:38, Marc Hoffmann wrote: >> Dear JDK developers, >> >> it looks like building http://hg.openjdk.java.net/jdk/jdk/ >> in ARM is broken since >> yesterday. This commit seems to be the first one that broke the build: >> changeset:?? 57948:4a4d185098e2 >> tag:???????? tip >> user:??????? lfoltan >> date:??????? Thu Feb 06 14:29:57 2020 +0000 >> summary:???? 8230199: consolidate signature parsing code in HotSpot >> sources >> this one now leads to another error (see below): >> changeset:?? 57949:d3caf06ac9ae >> tag:???????? tip >> user:??????? lfoltan >> date:??????? Thu Feb 06 15:28:37 2020 +0000 >> summary:???? 8238600: Remove runtime/fieldType.hpp and fieldType.cpp >> >> Is this a problem with the source tree or do we need to adjust the >> build setup on ARM? >> >> Background info: I?m a developer of the JaCoCo project and we do CI >> builds on all JDKs to test them agains the JaCoCo integration tests. >> >> Best regards, >> -marc >> >> >> >> >> >> >> ERROR: Build failed for target 'images' in configuration >> 'linux-arm-server-release' (exit code 2) >> Stopping sjavac server >> >> === Output from failing command(s) repeated here === >> * For target hotspot_variant-server_libjvm_objs_interpreterRT_arm.o: >> /workspace/src/hotspot/cpu/arm/interpreterRT_arm.cpp:54:27: error: >> 'int_parm' is not a member of 'SignatureIterator' >> ??? (u1) SignatureIterator::int_parm, // bool >> ??????????????????????????? ^~~~~~~~ >> /workspace/src/hotspot/cpu/arm/interpreterRT_arm.cpp:55:27: error: >> 'int_parm' is not a member of 'SignatureIterator' >> ??? (u1) SignatureIterator::int_parm, // byte >> ??????????????????????????? ^~~~~~~~ >> /workspace/src/hotspot/cpu/arm/interpreterRT_arm.cpp:56:27: error: >> 'int_parm' is not a member of 'SignatureIterator' >> ??? (u1) SignatureIterator::int_parm, // char >> ??????????????????????????? ^~~~~~~~ >> /workspace/src/hotspot/cpu/arm/interpreterRT_arm.cpp:57:27: error: >> 'int_parm' is not a member of 'SignatureIterator' >> ??? (u1) SignatureIterator::int_parm, // short >> ??????????????????????????? ^~~~~~~~ >> /workspace/src/hotspot/cpu/arm/interpreterRT_arm.cpp:58:27: error: >> 'int_parm' is not a member of 'SignatureIterator' >> ??? (u1) SignatureIterator::int_parm, // int >> ??????????????????????????? ^~~~~~~~ >> ??? ... (rest of output omitted) >> >> From huangjia at loongson.cn Sat Feb 8 05:25:26 2020 From: huangjia at loongson.cn (Jia Huang) Date: Sat, 8 Feb 2020 13:25:26 +0800 Subject: RFR: JDK-8238586: [TESTBUG] vmTestbase/jit/tiered/Test.java failed when TieredCompilation is disabled In-Reply-To: References: Message-ID: <6178b0ab-67af-f7c1-dcff-aeb325c80384@loongson.cn> Hi Igor, Thanks for your review. Could you please sponsor it? Thanks a lot. Best regards, Jia. ? 2020/2/7 23:47, Igor Ignatev ??: > I?d consider it as trivial. > > ? Igor > >> On Feb 6, 2020, at 11:22 PM, Jia Huang wrote: >> >> ?Thanks Igor. >> >> Is it a trivial patch and OK to be pushed? >> Or shall we wait for one more review? >> >> Thanks a lot. >> Best regards, >> Jia. >> >>> ? 2020/2/7 14:06, Igor Ignatyev ??: >>> Hi Jia, >>> >>> LGTM >>> >>> -- Igor >>> >>>>> On Feb 6, 2020, at 1:20 AM, Jia Huang wrote: >>>> Hi all, >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238586 >>>> >>>> When TieredCompilation is disabled, the test will fail. >>>> Please review the following fix which make the test pass when disabled TieredCompilation. >>>> ------------------------------------------------------ >>>> diff -r e45d8ca836b5 -r 67bef3d6692f test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java >>>> --- a/test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java Mon Feb 03 22:10:59 2020 +0800 >>>> +++ b/test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java Thu Feb 06 15:25:19 2020 +0800 >>>> @@ -46,7 +46,7 @@ >>>> import jdk.test.lib.process.ProcessTools; >>>> >>>> public class Test { >>>> - private static String UNSUPPORTED_OPTION_MESSAGE = "-XX:TieredCompilation not supported in this VM"; >>>> + private static String UNSUPPORTED_OPTION_MESSAGE = "-XX:+TieredCompilation not supported in this VM"; >>>> private static String REGEXP = "^[0-9.]+: \\[compile level=\\d"; >>>> public static void main(String[] args) throws Exception { >>>> { >>>> @@ -56,7 +56,7 @@ >>>> "-XX:+PrintTieredEvents", >>>> "-version"); >>>> var output = new OutputAnalyzer(pb.start()); >>>> - if (output.getStdout().contains(UNSUPPORTED_OPTION_MESSAGE)) { >>>> + if (output.getStderr().contains(UNSUPPORTED_OPTION_MESSAGE)) { >>>> throw new SkippedException(UNSUPPORTED_OPTION_MESSAGE); >>>> } >>>> output.shouldHaveExitValue(0) >>>> ------------------------------------------------------ >>>> For more info, please see the JBS. >>>> >>>> Thanks a lot >>>> >>>> Best regards, >>>> Jia. >>>> From igor.ignatyev at oracle.com Sat Feb 8 08:18:25 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Sat, 8 Feb 2020 00:18:25 -0800 Subject: RFR: JDK-8238586: [TESTBUG] vmTestbase/jit/tiered/Test.java failed when TieredCompilation is disabled In-Reply-To: <6178b0ab-67af-f7c1-dcff-aeb325c80384@loongson.cn> References: <6178b0ab-67af-f7c1-dcff-aeb325c80384@loongson.cn> Message-ID: pushed. -- Igor > On Feb 7, 2020, at 9:25 PM, Jia Huang wrote: > > Hi Igor, > > Thanks for your review. > > Could you please sponsor it? > > Thanks a lot. > Best regards, > Jia. > > ? 2020/2/7 23:47, Igor Ignatev ??: >> I?d consider it as trivial. >> >> ? Igor >> >>> On Feb 6, 2020, at 11:22 PM, Jia Huang wrote: >>> >>> ?Thanks Igor. >>> >>> Is it a trivial patch and OK to be pushed? >>> Or shall we wait for one more review? >>> >>> Thanks a lot. >>> Best regards, >>> Jia. >>> >>>> ? 2020/2/7 14:06, Igor Ignatyev ??: >>>> Hi Jia, >>>> >>>> LGTM >>>> >>>> -- Igor >>>> >>>>>> On Feb 6, 2020, at 1:20 AM, Jia Huang wrote: >>>>> Hi all, >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238586 >>>>> >>>>> When TieredCompilation is disabled, the test will fail. >>>>> Please review the following fix which make the test pass when disabled TieredCompilation. >>>>> ------------------------------------------------------ >>>>> diff -r e45d8ca836b5 -r 67bef3d6692f test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java >>>>> --- a/test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java Mon Feb 03 22:10:59 2020 +0800 >>>>> +++ b/test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java Thu Feb 06 15:25:19 2020 +0800 >>>>> @@ -46,7 +46,7 @@ >>>>> import jdk.test.lib.process.ProcessTools; >>>>> >>>>> public class Test { >>>>> - private static String UNSUPPORTED_OPTION_MESSAGE = "-XX:TieredCompilation not supported in this VM"; >>>>> + private static String UNSUPPORTED_OPTION_MESSAGE = "-XX:+TieredCompilation not supported in this VM"; >>>>> private static String REGEXP = "^[0-9.]+: \\[compile level=\\d"; >>>>> public static void main(String[] args) throws Exception { >>>>> { >>>>> @@ -56,7 +56,7 @@ >>>>> "-XX:+PrintTieredEvents", >>>>> "-version"); >>>>> var output = new OutputAnalyzer(pb.start()); >>>>> - if (output.getStdout().contains(UNSUPPORTED_OPTION_MESSAGE)) { >>>>> + if (output.getStderr().contains(UNSUPPORTED_OPTION_MESSAGE)) { >>>>> throw new SkippedException(UNSUPPORTED_OPTION_MESSAGE); >>>>> } >>>>> output.shouldHaveExitValue(0) >>>>> ------------------------------------------------------ >>>>> For more info, please see the JBS. >>>>> >>>>> Thanks a lot >>>>> >>>>> Best regards, >>>>> Jia. >>>>> > From huangjia at loongson.cn Sat Feb 8 08:20:57 2020 From: huangjia at loongson.cn (Jia Huang) Date: Sat, 8 Feb 2020 16:20:57 +0800 Subject: RFR: JDK-8238586: [TESTBUG] vmTestbase/jit/tiered/Test.java failed when TieredCompilation is disabled In-Reply-To: References: <6178b0ab-67af-f7c1-dcff-aeb325c80384@loongson.cn> Message-ID: <7bcc5466-b72f-4fd7-dd15-083b0cdc51b7@loongson.cn> Thanks Igor. ? 2020/2/8 16:18, Igor Ignatyev ??: > pushed. > > -- Igor > >> On Feb 7, 2020, at 9:25 PM, Jia Huang wrote: >> >> Hi Igor, >> >> Thanks for your review. >> >> Could you please sponsor it? >> >> Thanks a lot. >> Best regards, >> Jia. >> >> ? 2020/2/7 23:47, Igor Ignatev ??: >>> I?d consider it as trivial. >>> >>> ? Igor >>> >>>> On Feb 6, 2020, at 11:22 PM, Jia Huang wrote: >>>> >>>> ?Thanks Igor. >>>> >>>> Is it a trivial patch and OK to be pushed? >>>> Or shall we wait for one more review? >>>> >>>> Thanks a lot. >>>> Best regards, >>>> Jia. >>>> >>>>> ? 2020/2/7 14:06, Igor Ignatyev ??: >>>>> Hi Jia, >>>>> >>>>> LGTM >>>>> >>>>> -- Igor >>>>> >>>>>>> On Feb 6, 2020, at 1:20 AM, Jia Huang wrote: >>>>>> Hi all, >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238586 >>>>>> >>>>>> When TieredCompilation is disabled, the test will fail. >>>>>> Please review the following fix which make the test pass when disabled TieredCompilation. >>>>>> ------------------------------------------------------ >>>>>> diff -r e45d8ca836b5 -r 67bef3d6692f test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java >>>>>> --- a/test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java Mon Feb 03 22:10:59 2020 +0800 >>>>>> +++ b/test/hotspot/jtreg/vmTestbase/jit/tiered/Test.java Thu Feb 06 15:25:19 2020 +0800 >>>>>> @@ -46,7 +46,7 @@ >>>>>> import jdk.test.lib.process.ProcessTools; >>>>>> >>>>>> public class Test { >>>>>> - private static String UNSUPPORTED_OPTION_MESSAGE = "-XX:TieredCompilation not supported in this VM"; >>>>>> + private static String UNSUPPORTED_OPTION_MESSAGE = "-XX:+TieredCompilation not supported in this VM"; >>>>>> private static String REGEXP = "^[0-9.]+: \\[compile level=\\d"; >>>>>> public static void main(String[] args) throws Exception { >>>>>> { >>>>>> @@ -56,7 +56,7 @@ >>>>>> "-XX:+PrintTieredEvents", >>>>>> "-version"); >>>>>> var output = new OutputAnalyzer(pb.start()); >>>>>> - if (output.getStdout().contains(UNSUPPORTED_OPTION_MESSAGE)) { >>>>>> + if (output.getStderr().contains(UNSUPPORTED_OPTION_MESSAGE)) { >>>>>> throw new SkippedException(UNSUPPORTED_OPTION_MESSAGE); >>>>>> } >>>>>> output.shouldHaveExitValue(0) >>>>>> ------------------------------------------------------ >>>>>> For more info, please see the JBS. >>>>>> >>>>>> Thanks a lot >>>>>> >>>>>> Best regards, >>>>>> Jia. >>>>>> From kim.barrett at oracle.com Sun Feb 9 22:26:00 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 9 Feb 2020 17:26:00 -0500 Subject: RFR[XS]: 8238272: Eliminate cast_from_oop to narrowOop* In-Reply-To: <0F754E34-462D-4183-8CE0-F2933D4F310F@oracle.com> References: <267AF7FC-B6EB-4232-A51F-035A7217BCC6@oracle.com> <44ad6bd5-b66a-3306-0cb9-f518d1b8e2c4@redhat.com> <805C4111-CF21-4EAF-AC78-742045A3DD73@oracle.com> <0F754E34-462D-4183-8CE0-F2933D4F310F@oracle.com> Message-ID: > On Jan 31, 2020, at 5:12 PM, Kim Barrett wrote: > > [?] > Here's a new webrev: > > https://cr.openjdk.java.net/~kbarrett/8238272/open.01/ > > I didn't bother with an incremental webrev, since the new version is just > a one-line change replacing the previous one-line change. > > Testing: tier1 Ping? Looking for a second reviewer. From david.holmes at oracle.com Mon Feb 10 04:04:17 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 10 Feb 2020 14:04:17 +1000 Subject: [15] RFR (S): 8238696: x86: Enumerate all detected CPU features in VM_Version feature string In-Reply-To: References: Message-ID: <98506aca-4a47-a337-a707-19c2a424ca01@oracle.com> Hi Vladimir, On 8/02/2020 2:52 am, Vladimir Ivanov wrote: > http://cr.openjdk.java.net/~vlivanov/8238696/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8238696 > > Abstract_VM_Version::features_string() doesn't contain all the CPU > features detected by the JVM on x86 (for example, for AVX-512 only > AVX-512F is included as "evex"). > > The fix adds missing features to the list [1]. The function naming that you have changed for supports_avx512_vpclmulqdq was explicitly requested by Vladimir K.: http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-December/036326.html If anything I would suggest supports_vpopcntdq() should have the avx512 part added. The actual expansion of the feature string seems okay. It does make me wonder why these items have been left out previously. Thanks, David > Testing: manual (-Xlog:os+cpu=trace output on different hardware), > ???????? hs-precheckin-comp, hs-tier1, hs-tier2 > > Best regards, > Vladimir Ivanov > > [1] $ java -Xlog:os+cpu=trace -XX:UseAVX=3 -version > > ... > [0.010s][info][os,cpu] UseSSE=4? UseAVX=3? UseAES=1? MaxVectorSize=64 > ... > [0.011s][info][os,cpu] CPU:total 8 (initial active 8) (4 cores per cpu, > 2 threads per core) family 6 model 85 stepping 4, cmov, cx8, fxsr, mmx, > sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, vzeroupper, avx, avx2, > aes, clmul, erms, rtm, 3dnowpref, lzcnt, ht, tsc, bmi1, bmi2, adx, > avx512f, avx512dq, avx512cd, avx512bw, avx512vl, fma, clflush, > clflushopt, clwb > ... > [0.011s][info][os,cpu] flags??????? : fpu vme de pse tsc msr pae mce cx8 > apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht > syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl > xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pcid > sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c > rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single > pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase > tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq > rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt > xsavec xgetbv1 xsaves arat umip pku ospke md_clear > ... From david.holmes at oracle.com Mon Feb 10 07:15:35 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 10 Feb 2020 17:15:35 +1000 Subject: Testing assistance requested: ARM/PPC/S390 and a little Aarch64 Message-ID: <1ee1410a-ad23-a092-7b98-6094df294ee2@oracle.com> Hi, I was hoping some folk may be able to take this preliminary patch for a spin for me as I don't have access to a full range of hardware: http://cr.openjdk.java.net/~dholmes/8234372/webrev/ The patch relates to: https://bugs.openjdk.java.net/browse/JDK-8234372 'Investigate use of Thread::stack_base() and queries for "in stack"' The main functional changes here are in frame::safe_for_sender where we bring ARM/PPC/S390/Sparc into line with x86 and Aarch64 with regards to incorrect uses of >= or <= when comparing values with Thread::stack_base(). In addition this standardises the main sp safety check to reuse Thread::is_in_usable_stack(). And finally (with regards to the testing request) it fixes the validity check for "locals" wrt. Thread::stack_base(). Thanks, David From vladimir.x.ivanov at oracle.com Mon Feb 10 14:12:54 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 10 Feb 2020 17:12:54 +0300 Subject: [15] RFR (S): 8238696: x86: Enumerate all detected CPU features in VM_Version feature string In-Reply-To: <98506aca-4a47-a337-a707-19c2a424ca01@oracle.com> References: <98506aca-4a47-a337-a707-19c2a424ca01@oracle.com> Message-ID: <3b882679-b8e1-8e2c-fd8e-261d48224001@oracle.com> Thanks for taking a look, David. Updated webrev: http://cr.openjdk.java.net/~vlivanov/8238696/webrev.01/ >> http://cr.openjdk.java.net/~vlivanov/8238696/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8238696 >> >> Abstract_VM_Version::features_string() doesn't contain all the CPU >> features detected by the JVM on x86 (for example, for AVX-512 only >> AVX-512F is included as "evex"). >> >> The fix adds missing features to the list [1]. > > The function naming that you have changed for supports_avx512_vpclmulqdq > was explicitly requested by Vladimir K.: > > http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-December/036326.html > > > If anything I would suggest supports_vpopcntdq() should have the avx512 > part added. Makes sense. Also, decided to rename VNNI, VAES, and VBMI2. They are listed in the manual as part of AVX512. > The actual expansion of the feature string seems okay. It does make me > wonder why these items have been left out previously. I believe it was just overlooked. Added a comment near cpu feature constant declarations. Also, introduced a check to ensure the temporary on-stack buffer is large enough. Best regards, Vladimir Ivanov > > Thanks, > David > >> Testing: manual (-Xlog:os+cpu=trace output on different hardware), >> ????????? hs-precheckin-comp, hs-tier1, hs-tier2 >> >> Best regards, >> Vladimir Ivanov >> >> [1] $ java -Xlog:os+cpu=trace -XX:UseAVX=3 -version >> >> ... >> [0.010s][info][os,cpu] UseSSE=4? UseAVX=3? UseAES=1? MaxVectorSize=64 >> ... >> [0.011s][info][os,cpu] CPU:total 8 (initial active 8) (4 cores per >> cpu, 2 threads per core) family 6 model 85 stepping 4, cmov, cx8, >> fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, vzeroupper, >> avx, avx2, aes, clmul, erms, rtm, 3dnowpref, lzcnt, ht, tsc, bmi1, >> bmi2, adx, avx512f, avx512dq, avx512cd, avx512bw, avx512vl, fma, >> clflush, clflushopt, clwb >> ... >> [0.011s][info][os,cpu] flags??????? : fpu vme de pse tsc msr pae mce >> cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss >> ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good >> nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 >> pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave >> avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault >> invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority >> ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm >> mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw >> avx512vl xsaveopt xsavec xgetbv1 xsaves arat umip pku ospke md_clear >> ... From vladimir.x.ivanov at oracle.com Mon Feb 10 14:19:02 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 10 Feb 2020 17:19:02 +0300 Subject: [15] RFR (M): 8238681: Make -XX:UseSSE flag x86-specific Message-ID: http://cr.openjdk.java.net/~vlivanov/8238681/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8238681 Turn UseSSE into x86-specific flag since it doesn't have any meaning on non-x86 platforms. As a cleanup, I made all the code which assumes UseSSE<2 x86-32 specific. (x86-64 ABI requires SSE2.) Also, as a precaution, I made UseSSE obsolete on non-x86 platforms. Testing: hs-precheckin-comp, tier1-5, linux x86-32 fastdebug build. Thanks! Best regards, Vladimir Ivanov From vladimir.x.ivanov at oracle.com Mon Feb 10 14:47:11 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 10 Feb 2020 17:47:11 +0300 Subject: [15] RFR (T): 8238762: Optimized build is broken Message-ID: <18e8206e-45c3-6611-d371-c3e6ebe88223@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8238762 Trivial fix for a build problem with optimized build: diff --git a/src/hotspot/share/runtime/mutex.hpp b/src/hotspot/share/runtime/mutex.hpp --- a/src/hotspot/share/runtime/mutex.hpp +++ b/src/hotspot/share/runtime/mutex.hpp @@ -81,7 +81,7 @@ char _name[MUTEX_NAME_LEN]; // Name of mutex/monitor // Debugging fields for naming, deadlock detection, etc. (some only used in debug mode) -#ifdef ASSERT +#ifndef PRODUCT bool _allow_vm_block; int _rank; // rank (to avoid/detect potential deadlocks) Mutex* _next; // Used by a Thread to link up owned locks Best regards, Vladimir Ivanov From aph at redhat.com Mon Feb 10 16:17:44 2020 From: aph at redhat.com (Andrew Haley) Date: Mon, 10 Feb 2020 16:17:44 +0000 Subject: Testing assistance requested: ARM/PPC/S390 and a little Aarch64 In-Reply-To: <1ee1410a-ad23-a092-7b98-6094df294ee2@oracle.com> References: <1ee1410a-ad23-a092-7b98-6094df294ee2@oracle.com> Message-ID: On 2/10/20 7:15 AM, David Holmes wrote: > I was hoping some folk may be able to take this preliminary patch for a > spin for me as I don't have access to a full range of hardware: > > http://cr.openjdk.java.net/~dholmes/8234372/webrev/ > > The patch relates to: > > https://bugs.openjdk.java.net/browse/JDK-8234372 > > 'Investigate use of Thread::stack_base() and queries for "in stack"' > > The main functional changes here are in frame::safe_for_sender where we > bring ARM/PPC/S390/Sparc into line with x86 and Aarch64 with regards to > incorrect uses of >= or <= when comparing values with Thread::stack_base(). > > In addition this standardises the main sp safety check to reuse > Thread::is_in_usable_stack(). > > And finally (with regards to the testing request) it fixes the validity > check for "locals" wrt. Thread::stack_base(). Looks OK. Passes all smoke tests; is there anything else you'd like me to do? -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From coleen.phillimore at oracle.com Mon Feb 10 17:43:28 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 10 Feb 2020 12:43:28 -0500 Subject: [15] RFR (T): 8238762: Optimized build is broken In-Reply-To: <18e8206e-45c3-6611-d371-c3e6ebe88223@oracle.com> References: <18e8206e-45c3-6611-d371-c3e6ebe88223@oracle.com> Message-ID: <5d4189b9-5ae1-8f9c-35e0-390d2c75748b@oracle.com> Looks good! Coleen On 2/10/20 9:47 AM, Vladimir Ivanov wrote: > https://bugs.openjdk.java.net/browse/JDK-8238762 > > Trivial fix for a build problem with optimized build: > > diff --git a/src/hotspot/share/runtime/mutex.hpp > b/src/hotspot/share/runtime/mutex.hpp > --- a/src/hotspot/share/runtime/mutex.hpp > +++ b/src/hotspot/share/runtime/mutex.hpp > @@ -81,7 +81,7 @@ > ?? char _name[MUTEX_NAME_LEN];??????????? // Name of mutex/monitor > > ?? // Debugging fields for naming, deadlock detection, etc. (some only > used in debug mode) > -#ifdef ASSERT > +#ifndef PRODUCT > ?? bool??? _allow_vm_block; > ?? int???? _rank;???????????????? // rank (to avoid/detect potential > deadlocks) > ?? Mutex*? _next;???????????????? // Used by a Thread to link up owned > locks > > Best regards, > Vladimir Ivanov From david.holmes at oracle.com Mon Feb 10 22:22:53 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Feb 2020 08:22:53 +1000 Subject: Testing assistance requested: ARM/PPC/S390 and a little Aarch64 In-Reply-To: References: <1ee1410a-ad23-a092-7b98-6094df294ee2@oracle.com> Message-ID: <96365fc2-e955-0b74-c094-d80385a0c298@oracle.com> Hi Andrew, On 11/02/2020 2:17 am, Andrew Haley wrote: > On 2/10/20 7:15 AM, David Holmes wrote: >> I was hoping some folk may be able to take this preliminary patch for a >> spin for me as I don't have access to a full range of hardware: >> >> http://cr.openjdk.java.net/~dholmes/8234372/webrev/ >> >> The patch relates to: >> >> https://bugs.openjdk.java.net/browse/JDK-8234372 >> >> 'Investigate use of Thread::stack_base() and queries for "in stack"' >> >> The main functional changes here are in frame::safe_for_sender where we >> bring ARM/PPC/S390/Sparc into line with x86 and Aarch64 with regards to >> incorrect uses of >= or <= when comparing values with Thread::stack_base(). >> >> In addition this standardises the main sp safety check to reuse >> Thread::is_in_usable_stack(). >> >> And finally (with regards to the testing request) it fixes the validity >> check for "locals" wrt. Thread::stack_base(). > > Looks OK. Passes all smoke tests; is there anything else you'd like me to > do? Many thanks for that. If you could chime in on the RFR when it comes out that would be good. :) Thanks, David From david.holmes at oracle.com Tue Feb 11 04:34:19 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Feb 2020 14:34:19 +1000 Subject: [15] RFR (S): 8238696: x86: Enumerate all detected CPU features in VM_Version feature string In-Reply-To: <3b882679-b8e1-8e2c-fd8e-261d48224001@oracle.com> References: <98506aca-4a47-a337-a707-19c2a424ca01@oracle.com> <3b882679-b8e1-8e2c-fd8e-261d48224001@oracle.com> Message-ID: Hi Vladimir, This all seems reasonable to me. Thanks, David On 11/02/2020 12:12 am, Vladimir Ivanov wrote: > Thanks for taking a look, David. > > Updated webrev: > ? http://cr.openjdk.java.net/~vlivanov/8238696/webrev.01/ > >>> http://cr.openjdk.java.net/~vlivanov/8238696/webrev.00/ >>> https://bugs.openjdk.java.net/browse/JDK-8238696 >>> >>> Abstract_VM_Version::features_string() doesn't contain all the CPU >>> features detected by the JVM on x86 (for example, for AVX-512 only >>> AVX-512F is included as "evex"). >>> >>> The fix adds missing features to the list [1]. >> >> The function naming that you have changed for >> supports_avx512_vpclmulqdq was explicitly requested by Vladimir K.: >> >> http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-December/036326.html >> >> >> If anything I would suggest supports_vpopcntdq() should have the >> avx512 part added. > > Makes sense. Also, decided to rename VNNI, VAES, and VBMI2. They are > listed in the manual as part of AVX512. > >> The actual expansion of the feature string seems okay. It does make me >> wonder why these items have been left out previously. > > I believe it was just overlooked. Added a comment near cpu feature > constant declarations. > > Also, introduced a check to ensure the temporary on-stack buffer is > large enough. > > Best regards, > Vladimir Ivanov >> >> Thanks, >> David >> >>> Testing: manual (-Xlog:os+cpu=trace output on different hardware), >>> ????????? hs-precheckin-comp, hs-tier1, hs-tier2 >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> [1] $ java -Xlog:os+cpu=trace -XX:UseAVX=3 -version >>> >>> ... >>> [0.010s][info][os,cpu] UseSSE=4? UseAVX=3? UseAES=1? MaxVectorSize=64 >>> ... >>> [0.011s][info][os,cpu] CPU:total 8 (initial active 8) (4 cores per >>> cpu, 2 threads per core) family 6 model 85 stepping 4, cmov, cx8, >>> fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, >>> vzeroupper, avx, avx2, aes, clmul, erms, rtm, 3dnowpref, lzcnt, ht, >>> tsc, bmi1, bmi2, adx, avx512f, avx512dq, avx512cd, avx512bw, >>> avx512vl, fma, clflush, clflushopt, clwb >>> ... >>> [0.011s][info][os,cpu] flags??????? : fpu vme de pse tsc msr pae mce >>> cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss >>> ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good >>> nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 >>> pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave >>> avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault >>> invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority >>> ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm >>> mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd >>> avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat umip pku ospke >>> md_clear >>> ... From david.holmes at oracle.com Tue Feb 11 05:08:05 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Feb 2020 15:08:05 +1000 Subject: [15] RFR (M): 8238681: Make -XX:UseSSE flag x86-specific In-Reply-To: References: Message-ID: <8be7a9f3-cfd5-cdaf-5d54-b8e97da5e317@oracle.com> Hi Vladimir, On 11/02/2020 12:19 am, Vladimir Ivanov wrote: > http://cr.openjdk.java.net/~vlivanov/8238681/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8238681 > > Turn UseSSE into x86-specific flag since it doesn't have any meaning on > non-x86 platforms. That part seems okay. > As a cleanup, I made all the code which assumes UseSSE<2 x86-32 > specific. (x86-64 ABI requires SSE2.) This made things more obscure for me so I'll leave it to compiler folk to validate. > Also, as a precaution, I made UseSSE obsolete on non-x86 platforms. I took the liberty of filing a very short CSR request for this and added myself as Reviewer. Please change it to "Finalized" and await approval before pushing. Thanks, David > Testing: hs-precheckin-comp, tier1-5, linux x86-32 fastdebug build. > > Thanks! > > Best regards, > Vladimir Ivanov From david.holmes at oracle.com Tue Feb 11 05:40:29 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Feb 2020 15:40:29 +1000 Subject: RFR: 8234372: Investigate use of Thread::stack_base() and queries for "in stack" Message-ID: <67b3bb43-b53c-0151-33ca-2f58202c29af@oracle.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8234372 webrev: http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/ Following on from JDK-8215355 I checked all uses of Thread::stack_base() to watch for range tests that should be exclusive but are inclusive, and vice-versa. And in addition clarified and streamlined the various "in stack" checks that are made. Summary of changes: src/hotspot/cpu/aarch64/frame_aarch64.cpp src/hotspot/cpu/arm/frame_arm.cpp src/hotspot/cpu/ppc/frame_ppc.cpp src/hotspot/cpu/s390/frame_s390.cpp src/hotspot/cpu/sparc/frame_sparc.cpp src/hotspot/cpu/x86/frame_x86.cpp In terms of actual bugs the implementation of frame::safe_for_sender on all platforms except x86 and aarch64 was using the wrong range test in a number of cases, so these are all now correct and consistent. All platforms had an incorrect range check in relation to the "locals". All platforms now use is_in_usable_stack to check for a valid sp, rather than duplicating (sometimes incorrectly) that logic. -- src/hotspot/os/linux/os_linux.cpp src/hotspot/os/windows/os_windows.cpp Replaced explicit range check with is_in_usable_stack src/hotspot/os_cpu/linux_arm/os_linux_arm.cpp src/hotspot/os_cpu/linux_s390/thread_linux_s390.cpp Replaced explicit range check with on_local_stack. --- src/hotspot/share/runtime/thread.?pp Moved is_in_usable_stack from Thread to JavaThread (guard regions are only relevant for JavaThreads). Clarified functionality and use of the three "in stack" variants. Removed redundant check from is_in_stack: ! // Allow non Java threads to call this without stack_base ! if (_stack_base == NULL) return true; As this is executed by the current thread, and the very first thing a thread does is set its stack base and size, it is impossible to find a NULL stack_base (which is already asserted inside stack_base()). [I tested this extensively just as a sanity check: tiers 1-5 plus hotspot runtime/serviceability/gc.] Misc cleanup to use stack_end() rather than recalulate it. --- There are some further possible cleanups here but I didn't want to go too far with things that would obscure the functional changes too much. As mentioned in the bug report the three "in stack" functions would benefit from some minor renamings so that their relationship is clearer. But I can leave that to a follow on RFE. Further, it may be possible to replace a lot of the remaining uses of stack_base() with a more constrained "in stack" function that takes a limit. For example, rather than something like: if (thread->stack_base() > fp && fp >= sp) have: if (thread->is_in_stack_range(fp /* addr*/, sp /*limit*/)) which checks the given addr against stack_base and the limit, and checks the limit against stack_end(). The difficultly may lie in determining whether checking against the limit should be a > or >= test, as it will be dependent on the context. Again this seems like something for a second RFE. --- Testing: Thanks to Andrew Haley for taking the frame changes for a spin on ARM/PPC/Aarch64/S390(?). I also ran our tier 1 to 3 testing on x86 and sparc. Thanks, David From david.holmes at oracle.com Tue Feb 11 05:52:21 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Feb 2020 15:52:21 +1000 Subject: [15] RFR (M): 8238681: Make -XX:UseSSE flag x86-specific In-Reply-To: <8be7a9f3-cfd5-cdaf-5d54-b8e97da5e317@oracle.com> References: <8be7a9f3-cfd5-cdaf-5d54-b8e97da5e317@oracle.com> Message-ID: PS. In test: open/test > vi ./hotspot/jtreg/compiler/cpuflags/TestAESIntrinsicsOnSupportedConfig.java it has @requires vm.cpu.features ~= ".*aes.*" rather than a text for x86/x64. Is that a more specific runtime test for x86 capabilities? Thanks, David On 11/02/2020 3:08 pm, David Holmes wrote: > Hi Vladimir, > > On 11/02/2020 12:19 am, Vladimir Ivanov wrote: >> http://cr.openjdk.java.net/~vlivanov/8238681/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8238681 >> >> Turn UseSSE into x86-specific flag since it doesn't have any meaning >> on non-x86 platforms. > > That part seems okay. > >> As a cleanup, I made all the code which assumes UseSSE<2 x86-32 >> specific. (x86-64 ABI requires SSE2.) > > This made things more obscure for me so I'll leave it to compiler folk > to validate. > >> Also, as a precaution, I made UseSSE obsolete on non-x86 platforms. > > I took the liberty of filing a very short CSR request for this and added > myself as Reviewer. Please change it to "Finalized" and await approval > before pushing. > > Thanks, > David > >> Testing: hs-precheckin-comp, tier1-5, linux x86-32 fastdebug build. >> >> Thanks! >> >> Best regards, >> Vladimir Ivanov From vladimir.x.ivanov at oracle.com Tue Feb 11 08:17:16 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 11 Feb 2020 11:17:16 +0300 Subject: [15] RFR (T): 8238762: Optimized build is broken In-Reply-To: <5d4189b9-5ae1-8f9c-35e0-390d2c75748b@oracle.com> References: <18e8206e-45c3-6611-d371-c3e6ebe88223@oracle.com> <5d4189b9-5ae1-8f9c-35e0-390d2c75748b@oracle.com> Message-ID: Thanks, Coleen. Best regards, Vladimir Ivanov On 10.02.2020 20:43, coleen.phillimore at oracle.com wrote: > Looks good! > Coleen > > On 2/10/20 9:47 AM, Vladimir Ivanov wrote: >> https://bugs.openjdk.java.net/browse/JDK-8238762 >> >> Trivial fix for a build problem with optimized build: >> >> diff --git a/src/hotspot/share/runtime/mutex.hpp >> b/src/hotspot/share/runtime/mutex.hpp >> --- a/src/hotspot/share/runtime/mutex.hpp >> +++ b/src/hotspot/share/runtime/mutex.hpp >> @@ -81,7 +81,7 @@ >> ?? char _name[MUTEX_NAME_LEN];??????????? // Name of mutex/monitor >> >> ?? // Debugging fields for naming, deadlock detection, etc. (some only >> used in debug mode) >> -#ifdef ASSERT >> +#ifndef PRODUCT >> ?? bool??? _allow_vm_block; >> ?? int???? _rank;???????????????? // rank (to avoid/detect potential >> deadlocks) >> ?? Mutex*? _next;???????????????? // Used by a Thread to link up owned >> locks >> >> Best regards, >> Vladimir Ivanov > From boris.ulasevich at bell-sw.com Tue Feb 11 08:28:40 2020 From: boris.ulasevich at bell-sw.com (boris) Date: Tue, 11 Feb 2020 11:28:40 +0300 Subject: Testing assistance requested: ARM/PPC/S390 and a little Aarch64 In-Reply-To: <1ee1410a-ad23-a092-7b98-6094df294ee2@oracle.com> References: <1ee1410a-ad23-a092-7b98-6094df294ee2@oracle.com> Message-ID: <76983c9d-64e9-20d0-b605-ead4fe851779@bell-sw.com> Hi, Thanks for the update. ARM32 and PPC build/tests are Ok. regards, Boris On 10.02.2020 10:15, David Holmes wrote: > Hi, > > I was hoping some folk may be able to take this preliminary patch for > a spin for me as I don't have access to a full range of hardware: > > http://cr.openjdk.java.net/~dholmes/8234372/webrev/ > > The patch relates to: > > https://bugs.openjdk.java.net/browse/JDK-8234372 > > 'Investigate use of Thread::stack_base() and queries for "in stack"' > > The main functional changes here are in frame::safe_for_sender where > we bring ARM/PPC/S390/Sparc into line with x86 and Aarch64 with > regards to incorrect uses of >= or <= when comparing values with > Thread::stack_base(). > > In addition this standardises the main sp safety check to reuse > Thread::is_in_usable_stack(). > > And finally (with regards to the testing request) it fixes the > validity check for "locals" wrt. Thread::stack_base(). > > Thanks, > David From david.holmes at oracle.com Tue Feb 11 08:32:37 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Feb 2020 18:32:37 +1000 Subject: Testing assistance requested: ARM/PPC/S390 and a little Aarch64 In-Reply-To: <76983c9d-64e9-20d0-b605-ead4fe851779@bell-sw.com> References: <1ee1410a-ad23-a092-7b98-6094df294ee2@oracle.com> <76983c9d-64e9-20d0-b605-ead4fe851779@bell-sw.com> Message-ID: <4bcbb80c-e853-023e-f3fb-cd424b3be64d@oracle.com> Thanks Boris! David On 11/02/2020 6:28 pm, boris wrote: > Hi, > > Thanks for the update. > ARM32 and PPC build/tests are Ok. > > regards, > Boris > > On 10.02.2020 10:15, David Holmes wrote: >> Hi, >> >> I was hoping some folk may be able to take this preliminary patch for >> a spin for me as I don't have access to a full range of hardware: >> >> http://cr.openjdk.java.net/~dholmes/8234372/webrev/ >> >> The patch relates to: >> >> https://bugs.openjdk.java.net/browse/JDK-8234372 >> >> 'Investigate use of Thread::stack_base() and queries for "in stack"' >> >> The main functional changes here are in frame::safe_for_sender where >> we bring ARM/PPC/S390/Sparc into line with x86 and Aarch64 with >> regards to incorrect uses of >= or <= when comparing values with >> Thread::stack_base(). >> >> In addition this standardises the main sp safety check to reuse >> Thread::is_in_usable_stack(). >> >> And finally (with regards to the testing request) it fixes the >> validity check for "locals" wrt. Thread::stack_base(). >> >> Thanks, >> David From thomas.stuefe at gmail.com Tue Feb 11 08:35:26 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 11 Feb 2020 09:35:26 +0100 Subject: RFR: 8234372: Investigate use of Thread::stack_base() and queries for "in stack" In-Reply-To: <67b3bb43-b53c-0151-33ca-2f58202c29af@oracle.com> References: <67b3bb43-b53c-0151-33ca-2f58202c29af@oracle.com> Message-ID: Hi David, I did not find anything wrong in your patch. Nice cleanup, and great archaeological work :) Only small nits and some bike shedding: --- So, if I get this right: is_in_stack -> is in live stack (base ... sp] on_local_stack -> is in (base...start] includes guard pages is_in_usable_stack -> is in (base...start - guard] excludes guard pages The naming is confusing but I saw you recommended renaming the functions in the JBS comments, and I like all your suggestions better than what we have now. ----- http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/src/hotspot/cpu/aarch64/frame_aarch64.cpp.udiff.html - if (locals > thread->stack_base() || locals < (address) fp()) return false; + if (locals >= thread->stack_base() || locals < (address) fp()) return false; This would be easier to read as a negated positive (also applies to all other frame_xxx.cpp). Just an idea, maybe we could add a function Thread::is_in_stack_limited_by(ptr, arbitrary_end_ptr) which could compare that ptr is between (base .. arbitrary_end_ptr] and based on that we could implement the other three stack functions. For cases like this we could then write: if (!thread->is_in_stack_limited_by(locals, fp()) But I am unsure, maybe I overthink things. ------- http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/src/hotspot/os/linux/os_linux.cpp.udiff.html - if (addr < t->stack_base() && addr >= t->stack_reserved_zone_base()) { + if (t->is_in_usable_stack(addr)) { First confused me but then I read Fredericks comment in JBS so I think it is okay. But it would be nice to be able to remove this manually-expand-stack-coding altogether :) ------- Cheers, Thomas On Tue, Feb 11, 2020 at 6:40 AM David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8234372 > webrev: http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/ > > Following on from JDK-8215355 I checked all uses of Thread::stack_base() > to watch for range tests that should be exclusive but are inclusive, and > vice-versa. And in addition clarified and streamlined the various "in > stack" checks that are made. > > Summary of changes: > > src/hotspot/cpu/aarch64/frame_aarch64.cpp > src/hotspot/cpu/arm/frame_arm.cpp > src/hotspot/cpu/ppc/frame_ppc.cpp > src/hotspot/cpu/s390/frame_s390.cpp > src/hotspot/cpu/sparc/frame_sparc.cpp > src/hotspot/cpu/x86/frame_x86.cpp > > In terms of actual bugs the implementation of frame::safe_for_sender on > all platforms except x86 and aarch64 was using the wrong range test in a > number of cases, so these are all now correct and consistent. > > All platforms had an incorrect range check in relation to the "locals". > > All platforms now use is_in_usable_stack to check for a valid sp, rather > than duplicating (sometimes incorrectly) that logic. > > -- > > src/hotspot/os/linux/os_linux.cpp > src/hotspot/os/windows/os_windows.cpp > > Replaced explicit range check with is_in_usable_stack > > src/hotspot/os_cpu/linux_arm/os_linux_arm.cpp > src/hotspot/os_cpu/linux_s390/thread_linux_s390.cpp > > Replaced explicit range check with on_local_stack. > > --- > > src/hotspot/share/runtime/thread.?pp > > Moved is_in_usable_stack from Thread to JavaThread (guard regions are > only relevant for JavaThreads). > > Clarified functionality and use of the three "in stack" variants. > > Removed redundant check from is_in_stack: > > ! // Allow non Java threads to call this without stack_base > ! if (_stack_base == NULL) return true; > > As this is executed by the current thread, and the very first thing a > thread does is set its stack base and size, it is impossible to find a > NULL stack_base (which is already asserted inside stack_base()). [I > tested this extensively just as a sanity check: tiers 1-5 plus hotspot > runtime/serviceability/gc.] > > Misc cleanup to use stack_end() rather than recalulate it. > > --- > > There are some further possible cleanups here but I didn't want to go > too far with things that would obscure the functional changes too much. > As mentioned in the bug report the three "in stack" functions would > benefit from some minor renamings so that their relationship is clearer. > But I can leave that to a follow on RFE. Further, it may be possible to > replace a lot of the remaining uses of stack_base() with a more > constrained "in stack" function that takes a limit. For example, rather > than something like: > > if (thread->stack_base() > fp && fp >= sp) > > have: > > if (thread->is_in_stack_range(fp /* addr*/, sp /*limit*/)) > > which checks the given addr against stack_base and the limit, and checks > the limit against stack_end(). The difficultly may lie in determining > whether checking against the limit should be a > or >= test, as it will > be dependent on the context. Again this seems like something for a > second RFE. > > --- > > Testing: > > Thanks to Andrew Haley for taking the frame changes for a spin on > ARM/PPC/Aarch64/S390(?). > > I also ran our tier 1 to 3 testing on x86 and sparc. > > Thanks, > David > From matthias.baesken at sap.com Tue Feb 11 09:08:58 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 11 Feb 2020 09:08:58 +0000 Subject: Question about JDK-8043780 (was: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux) In-Reply-To: References: Message-ID: Ping ? any comments on this ? https://bugs.openjdk.java.net/browse/JDK-8043780 mentions : ?Backward compatibility needs to be provided via #ifdef and checking (once) whether O_CLOEXEC works.? So it was intentional that just ***once*** it is checked whether O_CLOEXEC works (in the linux version) . Best regards, Matthias From: Thomas St?fe Sent: Donnerstag, 6. Februar 2020 11:30 To: Martin Buchholz Cc: David Holmes ; hotspot-dev at openjdk.java.net; Baesken, Matthias Subject: Question about JDK-8043780 (was: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux) Hi Martin, do you remember https://bugs.openjdk.java.net/browse/JDK-8043780 ? Some questions came up about your patch: Your original coding does this check-O_CLOEXEC-after-open-and-set-if-missing-dance, but if it worked ar least once it will stop doing that. However, if it fails to work it will retest over and over again. Was this intentional? Do we expect it to start working at some point? If yes, could it also stop working at some point? I wonder whether we could shorten the coding to a simple, unconditional ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); to be done after open. If open(CLOEXEC) worked, this would be a unnecessary noop. If it did not, we set it now. We'd always pay for a fcntl call but I think it is cheap and open is not called that often anyway. -- But then, I wonder whether we can scratch that coding altogether. O_CLOEXEC has been in the kernel since 2.6.23. This is 13 years old. 2.6 line topped out at 2.6.39 in 2011, and I would expect anyone still running on 2.6 to have at least a recent kernel. What do you think? Thank you! Cheers, Thomas On Thu, Feb 6, 2020 at 8:55 AM Baesken, Matthias > wrote: Hi Thomas , the coding was borrowed from linux, see : https://hg.openjdk.java.net/jdk/jdk/file/7b57401deb0c/src/hotspot/os/linux/os_linux.cpp#l5616 I would be fine with always calling fcntl (your suggestion 1) , do you think it was done otherwise back then in os_linux because of performance issues ? Probably os::open in os_linux should be adjusted then to the new ?always call fcntl? approach too, right ? Best regards, Matthias Hi Matthias: + // Validate that the use of the O_CLOEXEC flag on open above worked. + static sig_atomic_t O_CLOEXEC_is_known_to_work = 0; + if (!O_CLOEXEC_is_known_to_work) { int flags = ::fcntl(fd, F_GETFD); - if (flags != -1) + if (flags != -1) { + if ((flags & FD_CLOEXEC) != 0) { + O_CLOEXEC_is_known_to_work = 1; + } else { ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); } -#endif + } + } This is either too complicated or not complicated enough :) if we know for a fact that open(..O_CLOEXEC) did work at least once, we never again check. if we know for a fact that open(..O_CLOEXEC) failed to work, we recheck again and again. Do we expect it to suddenly start working? If yes, could it also suddenly start failing? Should we then not check always? I propose the following change: 1) If open(..O_CLOEXEC) could sometimes fail and sometimes not, or we are just not sure, I would do the fcntl test every time. Or, even just always set the flag again with fcntl. So, just remove the O_CLOEXEC_is_known_to_work conditional variable. Which would reduce the code to a one liner: + ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); 2) If you are sure that it either always works or fails, you need a conditional with three states: unknown, works, does not work. And only check if unknown. I personally would prefer (1) because it is way simpler and fcntl is not such an expensive operation that we have to save time (also, how often do we open files anyway?). ---- Otherwise looks fine. This is a useful fix. Cheers, Thomas On Tue, Jan 28, 2020 at 12:32 PM Baesken, Matthias > wrote: New webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.1/ (one 'c' got lost somehow in the older webrev) Best regards, Matthias From vladimir.x.ivanov at oracle.com Tue Feb 11 11:27:18 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 11 Feb 2020 14:27:18 +0300 Subject: [15] RFR (M): 8238681: Make -XX:UseSSE flag x86-specific In-Reply-To: References: <8be7a9f3-cfd5-cdaf-5d54-b8e97da5e317@oracle.com> Message-ID: Hi David, Thanks for the feedback and for filing CSR request. > open/test > vi > ./hotspot/jtreg/compiler/cpuflags/TestAESIntrinsicsOnSupportedConfig.java > > it has > > ?@requires vm.cpu.features ~= ".*aes.*" > > rather than a text for x86/x64. Is that a more specific runtime test for > x86 capabilities? AES intrinsics are available on other platforms as well (e.g., AARCH64 and SPARC have them) and CPU features are named uniformly [1] [2]. So, I don't see an immediate problem to fix here. But while looking at the test, I noted that verification I added in vm_version_x86.cpp requires some additional handling in CPUInfo [3] to ignore trailing dot. So, here's updated webrev: http://cr.openjdk.java.net/~vlivanov/8238681/webrev.01/ Best regards, Vladimir Ivanov [1] http://hg.openjdk.java.net/jdk/jdk/file/8e6fa89397ca/src/hotspot/cpu/aarch64/vm_version_aarch64.cpp#l285 [2] http://hg.openjdk.java.net/jdk/jdk/file/8e6fa89397ca/src/hotspot/cpu/sparc/vm_version_sparc.cpp#l233 [3] http://hg.openjdk.java.net/jdk/jdk/file/8e6fa89397ca/test/lib/sun/hotspot/cpuinfo/CPUInfo.java > > Thanks, > David > > On 11/02/2020 3:08 pm, David Holmes wrote: >> Hi Vladimir, >> >> On 11/02/2020 12:19 am, Vladimir Ivanov wrote: >>> http://cr.openjdk.java.net/~vlivanov/8238681/webrev.00/ >>> https://bugs.openjdk.java.net/browse/JDK-8238681 >>> >>> Turn UseSSE into x86-specific flag since it doesn't have any meaning >>> on non-x86 platforms. >> >> That part seems okay. >> >>> As a cleanup, I made all the code which assumes UseSSE<2 x86-32 >>> specific. (x86-64 ABI requires SSE2.) >> >> This made things more obscure for me so I'll leave it to compiler folk >> to validate. >> >>> Also, as a precaution, I made UseSSE obsolete on non-x86 platforms. >> >> I took the liberty of filing a very short CSR request for this and >> added myself as Reviewer. Please change it to "Finalized" and await >> approval before pushing. >> >> Thanks, >> David >> >>> Testing: hs-precheckin-comp, tier1-5, linux x86-32 fastdebug build. >>> >>> Thanks! >>> >>> Best regards, >>> Vladimir Ivanov From vladimir.x.ivanov at oracle.com Tue Feb 11 11:45:48 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 11 Feb 2020 14:45:48 +0300 Subject: [15] RFR (M): 8238681: Make -XX:UseSSE flag x86-specific In-Reply-To: References: <8be7a9f3-cfd5-cdaf-5d54-b8e97da5e317@oracle.com> Message-ID: >> open/test > vi >> ./hotspot/jtreg/compiler/cpuflags/TestAESIntrinsicsOnSupportedConfig.java >> >> it has >> >> ??@requires vm.cpu.features ~= ".*aes.*" >> >> rather than a text for x86/x64. Is that a more specific runtime test >> for x86 capabilities? > > AES intrinsics are available on other platforms as well (e.g., AARCH64 > and SPARC have them) and CPU features are named uniformly [1] [2]. So, I > don't see an immediate problem to fix here. > > But while looking at the test, I noted that verification I added in > vm_version_x86.cpp requires some additional handling in CPUInfo [3] to > ignore trailing dot. So, here's updated webrev: > > ? http://cr.openjdk.java.net/~vlivanov/8238681/webrev.01/ Oops, it should be 8238696 instead ("x86: Enumerate all detected CPU features in VM_Version feature string"): http://cr.openjdk.java.net/~vlivanov/8238696/webrev.02/ Sorry for the confusion. (Also, cross-posted updated webrev in the review thread for 8238696.) Best regards, Vladimir Ivanov > [1] > http://hg.openjdk.java.net/jdk/jdk/file/8e6fa89397ca/src/hotspot/cpu/aarch64/vm_version_aarch64.cpp#l285 > > > [2] > http://hg.openjdk.java.net/jdk/jdk/file/8e6fa89397ca/src/hotspot/cpu/sparc/vm_version_sparc.cpp#l233 > > > [3] > http://hg.openjdk.java.net/jdk/jdk/file/8e6fa89397ca/test/lib/sun/hotspot/cpuinfo/CPUInfo.java > > >> >> Thanks, >> David >> >> On 11/02/2020 3:08 pm, David Holmes wrote: >>> Hi Vladimir, >>> >>> On 11/02/2020 12:19 am, Vladimir Ivanov wrote: >>>> http://cr.openjdk.java.net/~vlivanov/8238681/webrev.00/ >>>> https://bugs.openjdk.java.net/browse/JDK-8238681 >>>> >>>> Turn UseSSE into x86-specific flag since it doesn't have any meaning >>>> on non-x86 platforms. >>> >>> That part seems okay. >>> >>>> As a cleanup, I made all the code which assumes UseSSE<2 x86-32 >>>> specific. (x86-64 ABI requires SSE2.) >>> >>> This made things more obscure for me so I'll leave it to compiler >>> folk to validate. >>> >>>> Also, as a precaution, I made UseSSE obsolete on non-x86 platforms. >>> >>> I took the liberty of filing a very short CSR request for this and >>> added myself as Reviewer. Please change it to "Finalized" and await >>> approval before pushing. >>> >>> Thanks, >>> David >>> >>>> Testing: hs-precheckin-comp, tier1-5, linux x86-32 fastdebug build. >>>> >>>> Thanks! >>>> >>>> Best regards, >>>> Vladimir Ivanov From vladimir.x.ivanov at oracle.com Tue Feb 11 11:47:07 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 11 Feb 2020 14:47:07 +0300 Subject: [15] RFR (S): 8238696: x86: Enumerate all detected CPU features in VM_Version feature string In-Reply-To: <3b882679-b8e1-8e2c-fd8e-261d48224001@oracle.com> References: <98506aca-4a47-a337-a707-19c2a424ca01@oracle.com> <3b882679-b8e1-8e2c-fd8e-261d48224001@oracle.com> Message-ID: <47bf9452-fb75-0293-0a54-3df9e3a3a716@oracle.com> Updated webrev: http://cr.openjdk.java.net/~vlivanov/8238696/webrev.02/ CPU feature string verification in vm_version_x86.cpp requires some additional handling in CPUInfo [3] to ignore trailing dot. Best regards, Vladimir Ivanov On 10.02.2020 17:12, Vladimir Ivanov wrote: > Thanks for taking a look, David. > > Updated webrev: > ? http://cr.openjdk.java.net/~vlivanov/8238696/webrev.01/ > >>> http://cr.openjdk.java.net/~vlivanov/8238696/webrev.00/ >>> https://bugs.openjdk.java.net/browse/JDK-8238696 >>> >>> Abstract_VM_Version::features_string() doesn't contain all the CPU >>> features detected by the JVM on x86 (for example, for AVX-512 only >>> AVX-512F is included as "evex"). >>> >>> The fix adds missing features to the list [1]. >> >> The function naming that you have changed for >> supports_avx512_vpclmulqdq was explicitly requested by Vladimir K.: >> >> http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-December/036326.html >> >> >> If anything I would suggest supports_vpopcntdq() should have the >> avx512 part added. > > Makes sense. Also, decided to rename VNNI, VAES, and VBMI2. They are > listed in the manual as part of AVX512. > >> The actual expansion of the feature string seems okay. It does make me >> wonder why these items have been left out previously. > > I believe it was just overlooked. Added a comment near cpu feature > constant declarations. > > Also, introduced a check to ensure the temporary on-stack buffer is > large enough. > > Best regards, > Vladimir Ivanov >> >> Thanks, >> David >> >>> Testing: manual (-Xlog:os+cpu=trace output on different hardware), >>> ????????? hs-precheckin-comp, hs-tier1, hs-tier2 >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> [1] $ java -Xlog:os+cpu=trace -XX:UseAVX=3 -version >>> >>> ... >>> [0.010s][info][os,cpu] UseSSE=4? UseAVX=3? UseAES=1? MaxVectorSize=64 >>> ... >>> [0.011s][info][os,cpu] CPU:total 8 (initial active 8) (4 cores per >>> cpu, 2 threads per core) family 6 model 85 stepping 4, cmov, cx8, >>> fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, >>> vzeroupper, avx, avx2, aes, clmul, erms, rtm, 3dnowpref, lzcnt, ht, >>> tsc, bmi1, bmi2, adx, avx512f, avx512dq, avx512cd, avx512bw, >>> avx512vl, fma, clflush, clflushopt, clwb >>> ... >>> [0.011s][info][os,cpu] flags??????? : fpu vme de pse tsc msr pae mce >>> cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss >>> ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good >>> nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 >>> pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave >>> avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault >>> invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority >>> ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm >>> mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd >>> avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat umip pku ospke >>> md_clear >>> ... From david.holmes at oracle.com Tue Feb 11 12:27:40 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Feb 2020 22:27:40 +1000 Subject: [15] RFR (M): 8238681: Make -XX:UseSSE flag x86-specific In-Reply-To: References: <8be7a9f3-cfd5-cdaf-5d54-b8e97da5e317@oracle.com> Message-ID: <6cf1765d-ca4e-656f-2da9-54a4f71d4cd4@oracle.com> On 11/02/2020 9:27 pm, Vladimir Ivanov wrote: > Hi David, > > Thanks for the feedback and for filing CSR request. > >> open/test > vi >> ./hotspot/jtreg/compiler/cpuflags/TestAESIntrinsicsOnSupportedConfig.java >> >> it has >> >> ??@requires vm.cpu.features ~= ".*aes.*" >> >> rather than a text for x86/x64. Is that a more specific runtime test >> for x86 capabilities? > > AES intrinsics are available on other platforms as well (e.g., AARCH64 > and SPARC have them) and CPU features are named uniformly [1] [2]. So, I > don't see an immediate problem to fix here. But the test uses the UseSSE flag so that would mean it is limited to execution on x86 only - even if other platforms have AES. ?? David > But while looking at the test, I noted that verification I added in > vm_version_x86.cpp requires some additional handling in CPUInfo [3] to > ignore trailing dot. So, here's updated webrev: > > ? http://cr.openjdk.java.net/~vlivanov/8238681/webrev.01/ > > Best regards, > Vladimir Ivanov > > [1] > http://hg.openjdk.java.net/jdk/jdk/file/8e6fa89397ca/src/hotspot/cpu/aarch64/vm_version_aarch64.cpp#l285 > > > [2] > http://hg.openjdk.java.net/jdk/jdk/file/8e6fa89397ca/src/hotspot/cpu/sparc/vm_version_sparc.cpp#l233 > > > [3] > http://hg.openjdk.java.net/jdk/jdk/file/8e6fa89397ca/test/lib/sun/hotspot/cpuinfo/CPUInfo.java > > >> >> Thanks, >> David >> >> On 11/02/2020 3:08 pm, David Holmes wrote: >>> Hi Vladimir, >>> >>> On 11/02/2020 12:19 am, Vladimir Ivanov wrote: >>>> http://cr.openjdk.java.net/~vlivanov/8238681/webrev.00/ >>>> https://bugs.openjdk.java.net/browse/JDK-8238681 >>>> >>>> Turn UseSSE into x86-specific flag since it doesn't have any meaning >>>> on non-x86 platforms. >>> >>> That part seems okay. >>> >>>> As a cleanup, I made all the code which assumes UseSSE<2 x86-32 >>>> specific. (x86-64 ABI requires SSE2.) >>> >>> This made things more obscure for me so I'll leave it to compiler >>> folk to validate. >>> >>>> Also, as a precaution, I made UseSSE obsolete on non-x86 platforms. >>> >>> I took the liberty of filing a very short CSR request for this and >>> added myself as Reviewer. Please change it to "Finalized" and await >>> approval before pushing. >>> >>> Thanks, >>> David >>> >>>> Testing: hs-precheckin-comp, tier1-5, linux x86-32 fastdebug build. >>>> >>>> Thanks! >>>> >>>> Best regards, >>>> Vladimir Ivanov From david.holmes at oracle.com Tue Feb 11 12:32:50 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Feb 2020 22:32:50 +1000 Subject: RFR: 8234372: Investigate use of Thread::stack_base() and queries for "in stack" In-Reply-To: References: <67b3bb43-b53c-0151-33ca-2f58202c29af@oracle.com> Message-ID: Hi Thomas, On 11/02/2020 6:35 pm, Thomas St?fe wrote: > Hi David, > > I did not find anything wrong in your patch. Nice cleanup, and great > archaeological work :) Thanks for taking a look, and doing the additional testing. > Only small nits and some bike shedding: > > --- > > So, if I get this right: > > is_in_stack -> is in live stack (base ... sp] > on_local_stack ?-> is in (base...start] includes guard pages > is_in_usable_stack -> is in (base...start - guard] excludes guard pages > > The naming is confusing but I saw you recommended renaming the functions > in the JBS comments, and I like all your suggestions better than what we > have now. Right - fix this in the next round. > ----- > > http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/src/hotspot/cpu/aarch64/frame_aarch64.cpp.udiff.html > > - ?if (locals > thread->stack_base() || locals < (address) fp()) return > false; > + ?if (locals >= thread->stack_base() || locals < (address) fp()) return > false; > > This would be easier to read as a negated positive (also applies to all > other frame_xxx.cpp). > > Just an idea, maybe we could add a function > Thread::is_in_stack_limited_by(ptr, arbitrary_end_ptr) which could > compare that ptr is between (base .. arbitrary_end_ptr] and based on > that we could implement the other three stack functions. > > For cases like this we could then write: > if (!thread->is_in_stack_limited_by(locals, fp()) > > But I am unsure, maybe I overthink things. I made the same kind of suggestion :) Again something for round 2. > ------- > > http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/src/hotspot/os/linux/os_linux.cpp.udiff.html > > - ?if (addr < ?t->stack_base() && addr >= t->stack_reserved_zone_base()) { > + ?if (t->is_in_usable_stack(addr)) { > > First confused me but then I read Fredericks comment in JBS so I think > it is okay. Yes not immediately obvious that the two expressions are the same but when you expand things out, they are. > But it would be nice to be able to remove this > manually-expand-stack-coding altogether :) :) Thanks, David ----- > ------- > > Cheers, Thomas > > On Tue, Feb 11, 2020 at 6:40 AM David Holmes > wrote: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8234372 > webrev: http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/ > > Following on from JDK-8215355 I checked all uses of > Thread::stack_base() > to watch for range tests that should be exclusive but are inclusive, > and > vice-versa. And in addition clarified and streamlined the various "in > stack" checks that are made. > > Summary of changes: > > src/hotspot/cpu/aarch64/frame_aarch64.cpp > src/hotspot/cpu/arm/frame_arm.cpp > src/hotspot/cpu/ppc/frame_ppc.cpp > src/hotspot/cpu/s390/frame_s390.cpp > src/hotspot/cpu/sparc/frame_sparc.cpp > src/hotspot/cpu/x86/frame_x86.cpp > > In terms of actual bugs the implementation of frame::safe_for_sender on > all platforms except x86 and aarch64 was using the wrong range test > in a > number of cases, so these are all now correct and consistent. > > All platforms had an incorrect range check in relation to the "locals". > > All platforms now use is_in_usable_stack to check for a valid sp, > rather > than duplicating (sometimes incorrectly) that logic. > > -- > > src/hotspot/os/linux/os_linux.cpp > src/hotspot/os/windows/os_windows.cpp > > Replaced explicit range check with is_in_usable_stack > > src/hotspot/os_cpu/linux_arm/os_linux_arm.cpp > src/hotspot/os_cpu/linux_s390/thread_linux_s390.cpp > > Replaced explicit range check with on_local_stack. > > --- > > src/hotspot/share/runtime/thread.?pp > > Moved is_in_usable_stack from Thread to JavaThread (guard regions are > only relevant for JavaThreads). > > Clarified functionality and use of the three "in stack" variants. > > Removed redundant check from is_in_stack: > > !? ?// Allow non Java threads to call this without stack_base > !? ?if (_stack_base == NULL) return true; > > As this is executed by the current thread, and the very first thing a > thread does is set its stack base and size, it is impossible to find a > NULL stack_base (which is already asserted inside stack_base()). [I > tested this extensively just as a sanity check: tiers 1-5 plus hotspot > runtime/serviceability/gc.] > > Misc cleanup to use stack_end() rather than recalulate it. > > --- > > There are some further possible cleanups here but I didn't want to go > too far with things that would obscure the functional changes too much. > As mentioned in the bug report the three "in stack" functions would > benefit from some minor renamings so that their relationship is > clearer. > But I can leave that to a follow on RFE. Further, it may be possible to > replace a lot of the remaining uses of stack_base() with a more > constrained "in stack" function that takes a limit. For example, rather > than something like: > > if (thread->stack_base() > fp && fp >= sp) > > have: > > if (thread->is_in_stack_range(fp /* addr*/, sp /*limit*/)) > > which checks the given addr against stack_base and the limit, and > checks > the limit against stack_end(). The difficultly may lie in determining > whether checking against the limit should be a > or >= test, as it will > be dependent on the context. Again this seems like something for a > second RFE. > > --- > > Testing: > > Thanks to Andrew Haley for taking the frame changes for a spin on > ARM/PPC/Aarch64/S390(?). > > I also ran our tier 1 to 3 testing on x86 and sparc. > > Thanks, > David > From vladimir.x.ivanov at oracle.com Tue Feb 11 12:36:13 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 11 Feb 2020 15:36:13 +0300 Subject: [15] RFR (M): 8238681: Make -XX:UseSSE flag x86-specific In-Reply-To: <6cf1765d-ca4e-656f-2da9-54a4f71d4cd4@oracle.com> References: <8be7a9f3-cfd5-cdaf-5d54-b8e97da5e317@oracle.com> <6cf1765d-ca4e-656f-2da9-54a4f71d4cd4@oracle.com> Message-ID: <7462f132-2219-3b87-5d8a-edd5b69bc0f0@oracle.com> >>> open/test > vi >>> ./hotspot/jtreg/compiler/cpuflags/TestAESIntrinsicsOnSupportedConfig.java >>> >>> >>> it has >>> >>> ??@requires vm.cpu.features ~= ".*aes.*" >>> >>> rather than a text for x86/x64. Is that a more specific runtime test >>> for x86 capabilities? >> >> AES intrinsics are available on other platforms as well (e.g., AARCH64 >> and SPARC have them) and CPU features are named uniformly [1] [2]. So, >> I don't see an immediate problem to fix here. > > But the test uses the UseSSE flag so that would mean it is limited to > execution on x86 only - even if other platforms have AES. ?? The test has dynamic checks in platform-specific test cases: 107 private void testUseAESUseSSE2() throws Throwable { 108 if (Platform.isX86() || Platform.isX64()) { 165 private void testUseAESUseVIS2() throws Throwable { 166 if (Platform.isSparc()) { Best regards, Vladimir Ivanov [1] http://hg.openjdk.java.net/jdk/jdk/file/tip/test/hotspot/jtreg/compiler/cpuflags/TestAESIntrinsicsOnSupportedConfig.java >> But while looking at the test, I noted that verification I added in >> vm_version_x86.cpp requires some additional handling in CPUInfo [3] to >> ignore trailing dot. So, here's updated webrev: >> >> ?? http://cr.openjdk.java.net/~vlivanov/8238681/webrev.01/ >> >> Best regards, >> Vladimir Ivanov >> >> [1] >> http://hg.openjdk.java.net/jdk/jdk/file/8e6fa89397ca/src/hotspot/cpu/aarch64/vm_version_aarch64.cpp#l285 >> >> >> [2] >> http://hg.openjdk.java.net/jdk/jdk/file/8e6fa89397ca/src/hotspot/cpu/sparc/vm_version_sparc.cpp#l233 >> >> >> [3] >> http://hg.openjdk.java.net/jdk/jdk/file/8e6fa89397ca/test/lib/sun/hotspot/cpuinfo/CPUInfo.java >> >> >>> >>> Thanks, >>> David >>> >>> On 11/02/2020 3:08 pm, David Holmes wrote: >>>> Hi Vladimir, >>>> >>>> On 11/02/2020 12:19 am, Vladimir Ivanov wrote: >>>>> http://cr.openjdk.java.net/~vlivanov/8238681/webrev.00/ >>>>> https://bugs.openjdk.java.net/browse/JDK-8238681 >>>>> >>>>> Turn UseSSE into x86-specific flag since it doesn't have any >>>>> meaning on non-x86 platforms. >>>> >>>> That part seems okay. >>>> >>>>> As a cleanup, I made all the code which assumes UseSSE<2 x86-32 >>>>> specific. (x86-64 ABI requires SSE2.) >>>> >>>> This made things more obscure for me so I'll leave it to compiler >>>> folk to validate. >>>> >>>>> Also, as a precaution, I made UseSSE obsolete on non-x86 platforms. >>>> >>>> I took the liberty of filing a very short CSR request for this and >>>> added myself as Reviewer. Please change it to "Finalized" and await >>>> approval before pushing. >>>> >>>> Thanks, >>>> David >>>> >>>>> Testing: hs-precheckin-comp, tier1-5, linux x86-32 fastdebug build. >>>>> >>>>> Thanks! >>>>> >>>>> Best regards, >>>>> Vladimir Ivanov From david.holmes at oracle.com Tue Feb 11 12:38:00 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Feb 2020 22:38:00 +1000 Subject: [15] RFR (S): 8238696: x86: Enumerate all detected CPU features in VM_Version feature string In-Reply-To: <47bf9452-fb75-0293-0a54-3df9e3a3a716@oracle.com> References: <98506aca-4a47-a337-a707-19c2a424ca01@oracle.com> <3b882679-b8e1-8e2c-fd8e-261d48224001@oracle.com> <47bf9452-fb75-0293-0a54-3df9e3a3a716@oracle.com> Message-ID: On 11/02/2020 9:47 pm, Vladimir Ivanov wrote: > Updated webrev: > ? http://cr.openjdk.java.net/~vlivanov/8238696/webrev.02/ > > CPU feature string verification in vm_version_x86.cpp requires some > additional handling in CPUInfo [3] to ignore trailing dot. Now I'm wondering about using the dot as a terminator. Doesn't the return value from jio_snprintf tell you whether you exceeded the buffer? David ----- > Best regards, > Vladimir Ivanov > > On 10.02.2020 17:12, Vladimir Ivanov wrote: >> Thanks for taking a look, David. >> >> Updated webrev: >> ?? http://cr.openjdk.java.net/~vlivanov/8238696/webrev.01/ >> >>>> http://cr.openjdk.java.net/~vlivanov/8238696/webrev.00/ >>>> https://bugs.openjdk.java.net/browse/JDK-8238696 >>>> >>>> Abstract_VM_Version::features_string() doesn't contain all the CPU >>>> features detected by the JVM on x86 (for example, for AVX-512 only >>>> AVX-512F is included as "evex"). >>>> >>>> The fix adds missing features to the list [1]. >>> >>> The function naming that you have changed for >>> supports_avx512_vpclmulqdq was explicitly requested by Vladimir K.: >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-December/036326.html >>> >>> >>> If anything I would suggest supports_vpopcntdq() should have the >>> avx512 part added. >> >> Makes sense. Also, decided to rename VNNI, VAES, and VBMI2. They are >> listed in the manual as part of AVX512. >> >>> The actual expansion of the feature string seems okay. It does make >>> me wonder why these items have been left out previously. >> >> I believe it was just overlooked. Added a comment near cpu feature >> constant declarations. >> >> Also, introduced a check to ensure the temporary on-stack buffer is >> large enough. >> >> Best regards, >> Vladimir Ivanov >>> >>> Thanks, >>> David >>> >>>> Testing: manual (-Xlog:os+cpu=trace output on different hardware), >>>> ????????? hs-precheckin-comp, hs-tier1, hs-tier2 >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>> [1] $ java -Xlog:os+cpu=trace -XX:UseAVX=3 -version >>>> >>>> ... >>>> [0.010s][info][os,cpu] UseSSE=4? UseAVX=3? UseAES=1? MaxVectorSize=64 >>>> ... >>>> [0.011s][info][os,cpu] CPU:total 8 (initial active 8) (4 cores per >>>> cpu, 2 threads per core) family 6 model 85 stepping 4, cmov, cx8, >>>> fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, >>>> vzeroupper, avx, avx2, aes, clmul, erms, rtm, 3dnowpref, lzcnt, ht, >>>> tsc, bmi1, bmi2, adx, avx512f, avx512dq, avx512cd, avx512bw, >>>> avx512vl, fma, clflush, clflushopt, clwb >>>> ... >>>> [0.011s][info][os,cpu] flags??????? : fpu vme de pse tsc msr pae mce >>>> cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 >>>> ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon >>>> rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 >>>> fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer >>>> aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch >>>> cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi >>>> flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 >>>> erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt >>>> clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat >>>> umip pku ospke md_clear >>>> ... From david.holmes at oracle.com Tue Feb 11 12:39:22 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Feb 2020 22:39:22 +1000 Subject: [15] RFR (M): 8238681: Make -XX:UseSSE flag x86-specific In-Reply-To: <7462f132-2219-3b87-5d8a-edd5b69bc0f0@oracle.com> References: <8be7a9f3-cfd5-cdaf-5d54-b8e97da5e317@oracle.com> <6cf1765d-ca4e-656f-2da9-54a4f71d4cd4@oracle.com> <7462f132-2219-3b87-5d8a-edd5b69bc0f0@oracle.com> Message-ID: On 11/02/2020 10:36 pm, Vladimir Ivanov wrote: > >>>> open/test > vi >>>> ./hotspot/jtreg/compiler/cpuflags/TestAESIntrinsicsOnSupportedConfig.java >>>> >>>> >>>> it has >>>> >>>> ??@requires vm.cpu.features ~= ".*aes.*" >>>> >>>> rather than a text for x86/x64. Is that a more specific runtime test >>>> for x86 capabilities? >>> >>> AES intrinsics are available on other platforms as well (e.g., >>> AARCH64 and SPARC have them) and CPU features are named uniformly [1] >>> [2]. So, I don't see an immediate problem to fix here. >> >> But the test uses the UseSSE flag so that would mean it is limited to >> execution on x86 only - even if other platforms have AES. ?? > > The test has dynamic checks in platform-specific test cases: > > ?? 107???? private void testUseAESUseSSE2() throws Throwable { > ?? 108???????? if (Platform.isX86() || Platform.isX64()) { > > > ?? 165???? private void testUseAESUseVIS2() throws Throwable { > ?? 166???????? if (Platform.isSparc()) { Ah! Thanks :) David > Best regards, > Vladimir Ivanov > > [1] > http://hg.openjdk.java.net/jdk/jdk/file/tip/test/hotspot/jtreg/compiler/cpuflags/TestAESIntrinsicsOnSupportedConfig.java > > >>> But while looking at the test, I noted that verification I added in >>> vm_version_x86.cpp requires some additional handling in CPUInfo [3] >>> to ignore trailing dot. So, here's updated webrev: >>> >>> ?? http://cr.openjdk.java.net/~vlivanov/8238681/webrev.01/ >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> [1] >>> http://hg.openjdk.java.net/jdk/jdk/file/8e6fa89397ca/src/hotspot/cpu/aarch64/vm_version_aarch64.cpp#l285 >>> >>> >>> [2] >>> http://hg.openjdk.java.net/jdk/jdk/file/8e6fa89397ca/src/hotspot/cpu/sparc/vm_version_sparc.cpp#l233 >>> >>> >>> [3] >>> http://hg.openjdk.java.net/jdk/jdk/file/8e6fa89397ca/test/lib/sun/hotspot/cpuinfo/CPUInfo.java >>> >>> >>>> >>>> Thanks, >>>> David >>>> >>>> On 11/02/2020 3:08 pm, David Holmes wrote: >>>>> Hi Vladimir, >>>>> >>>>> On 11/02/2020 12:19 am, Vladimir Ivanov wrote: >>>>>> http://cr.openjdk.java.net/~vlivanov/8238681/webrev.00/ >>>>>> https://bugs.openjdk.java.net/browse/JDK-8238681 >>>>>> >>>>>> Turn UseSSE into x86-specific flag since it doesn't have any >>>>>> meaning on non-x86 platforms. >>>>> >>>>> That part seems okay. >>>>> >>>>>> As a cleanup, I made all the code which assumes UseSSE<2 x86-32 >>>>>> specific. (x86-64 ABI requires SSE2.) >>>>> >>>>> This made things more obscure for me so I'll leave it to compiler >>>>> folk to validate. >>>>> >>>>>> Also, as a precaution, I made UseSSE obsolete on non-x86 platforms. >>>>> >>>>> I took the liberty of filing a very short CSR request for this and >>>>> added myself as Reviewer. Please change it to "Finalized" and await >>>>> approval before pushing. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> Testing: hs-precheckin-comp, tier1-5, linux x86-32 fastdebug build. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> Best regards, >>>>>> Vladimir Ivanov From richard.reingruber at sap.com Tue Feb 11 12:52:07 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Tue, 11 Feb 2020 12:52:07 +0000 Subject: RFR(S) 8238782: Cleanup Deoptimization::deoptimize(): remove unused RegisterMap argument and don't update RegisterMap in callers if UseBiasedLocking is enabled Message-ID: Hi, I would like to do a small clean-up removing the unused parameter "map" from Deoptimization::deoptimize(JavaThread* thread, frame fr, RegisterMap *map, DeoptReason reason). Additionally the callers are changed to use a non-updating RegisterMap to iterate the stack frames. The updating RegisterMap was needed only for revoking biases in Deoptimization::deoptimize(), and since JDK-8226705 this isn't done in deoptimize() anymore[1][2][3]. So please review Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238782/webrev.0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8238782 Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. Thanks, Richard. See also: [1] JDK-8226705 removes call to revoke_biases_of_monitors() from Deoptimization::deoptimize() http://hg.openjdk.java.net/jdk/jdk/rev/408c445d04e8#l21.320 [2] JDK-8226705 adds call to revoke_from_deopt_handler() to Deoptimization::fetch_unroll_info_helper() [3] http://hg.openjdk.java.net/jdk/jdk/rev/408c445d04e8#l21.199 [3] JDK-8226705: Deoptimization::revoke_from_deopt_handler() revokes biases just before replacing a compiled frame with equivalent interpreter frames. http://hg.openjdk.java.net/jdk/jdk/rev/408c445d04e8#l21.290 From matthias.baesken at sap.com Tue Feb 11 13:06:22 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 11 Feb 2020 13:06:22 +0000 Subject: Question about JDK-8043780 (was: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux) References: Message-ID: >I wonder whether we could shorten the coding to a simple, unconditional > >::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); Looks like we would need to call first fcntl with F_GETFD AND then F_SETFD (or where would flags come from?) : int flags = ::fcntl(fd, F_GETFD); ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); That?s 2 calls that are not really needed . Best regards, Matthias From: Baesken, Matthias Sent: Dienstag, 11. Februar 2020 10:09 To: 'Thomas St?fe' ; Martin Buchholz Cc: David Holmes ; hotspot-dev at openjdk.java.net Subject: RE: Question about JDK-8043780 (was: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux) Ping ? any comments on this ? https://bugs.openjdk.java.net/browse/JDK-8043780 mentions : ?Backward compatibility needs to be provided via #ifdef and checking (once) whether O_CLOEXEC works.? So it was intentional that just ***once*** it is checked whether O_CLOEXEC works (in the linux version) . Best regards, Matthias From: Thomas St?fe > Sent: Donnerstag, 6. Februar 2020 11:30 To: Martin Buchholz > Cc: David Holmes >; hotspot-dev at openjdk.java.net; Baesken, Matthias > Subject: Question about JDK-8043780 (was: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux) Hi Martin, do you remember https://bugs.openjdk.java.net/browse/JDK-8043780 ? Some questions came up about your patch: Your original coding does this check-O_CLOEXEC-after-open-and-set-if-missing-dance, but if it worked ar least once it will stop doing that. However, if it fails to work it will retest over and over again. Was this intentional? Do we expect it to start working at some point? If yes, could it also stop working at some point? I wonder whether we could shorten the coding to a simple, unconditional ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); to be done after open. If open(CLOEXEC) worked, this would be a unnecessary noop. If it did not, we set it now. We'd always pay for a fcntl call but I think it is cheap and open is not called that often anyway. -- But then, I wonder whether we can scratch that coding altogether. O_CLOEXEC has been in the kernel since 2.6.23. This is 13 years old. 2.6 line topped out at 2.6.39 in 2011, and I would expect anyone still running on 2.6 to have at least a recent kernel. What do you think? Thank you! Cheers, Thomas On Thu, Feb 6, 2020 at 8:55 AM Baesken, Matthias > wrote: Hi Thomas , the coding was borrowed from linux, see : https://hg.openjdk.java.net/jdk/jdk/file/7b57401deb0c/src/hotspot/os/linux/os_linux.cpp#l5616 I would be fine with always calling fcntl (your suggestion 1) , do you think it was done otherwise back then in os_linux because of performance issues ? Probably os::open in os_linux should be adjusted then to the new ?always call fcntl? approach too, right ? Best regards, Matthias Hi Matthias: + // Validate that the use of the O_CLOEXEC flag on open above worked. + static sig_atomic_t O_CLOEXEC_is_known_to_work = 0; + if (!O_CLOEXEC_is_known_to_work) { int flags = ::fcntl(fd, F_GETFD); - if (flags != -1) + if (flags != -1) { + if ((flags & FD_CLOEXEC) != 0) { + O_CLOEXEC_is_known_to_work = 1; + } else { ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); } -#endif + } + } This is either too complicated or not complicated enough :) if we know for a fact that open(..O_CLOEXEC) did work at least once, we never again check. if we know for a fact that open(..O_CLOEXEC) failed to work, we recheck again and again. Do we expect it to suddenly start working? If yes, could it also suddenly start failing? Should we then not check always? I propose the following change: 1) If open(..O_CLOEXEC) could sometimes fail and sometimes not, or we are just not sure, I would do the fcntl test every time. Or, even just always set the flag again with fcntl. So, just remove the O_CLOEXEC_is_known_to_work conditional variable. Which would reduce the code to a one liner: + ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); 2) If you are sure that it either always works or fails, you need a conditional with three states: unknown, works, does not work. And only check if unknown. I personally would prefer (1) because it is way simpler and fcntl is not such an expensive operation that we have to save time (also, how often do we open files anyway?). ---- Otherwise looks fine. This is a useful fix. Cheers, Thomas On Tue, Jan 28, 2020 at 12:32 PM Baesken, Matthias > wrote: New webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.1/ (one 'c' got lost somehow in the older webrev) Best regards, Matthias From vladimir.x.ivanov at oracle.com Tue Feb 11 13:58:07 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 11 Feb 2020 16:58:07 +0300 Subject: [15] RFR (S): 8238696: x86: Enumerate all detected CPU features in VM_Version feature string In-Reply-To: References: <98506aca-4a47-a337-a707-19c2a424ca01@oracle.com> <3b882679-b8e1-8e2c-fd8e-261d48224001@oracle.com> <47bf9452-fb75-0293-0a54-3df9e3a3a716@oracle.com> Message-ID: <2267f17c-2f81-eec3-d31f-4e031e34b615@oracle.com> > Now I'm wondering about using the dot as a terminator. Doesn't the > return value from jio_snprintf tell you whether you exceeded the buffer? Yes, very good point. How about the following? http://cr.openjdk.java.net/~vlivanov/8238696/webrev.03 Considering some tests have hard-coded feature names, I decided to enhance the new test and check that all reported features are known (at least, on x86). Best regards, Vladimir Ivanov >> On 10.02.2020 17:12, Vladimir Ivanov wrote: >>> Thanks for taking a look, David. >>> >>> Updated webrev: >>> ?? http://cr.openjdk.java.net/~vlivanov/8238696/webrev.01/ >>> >>>>> http://cr.openjdk.java.net/~vlivanov/8238696/webrev.00/ >>>>> https://bugs.openjdk.java.net/browse/JDK-8238696 >>>>> >>>>> Abstract_VM_Version::features_string() doesn't contain all the CPU >>>>> features detected by the JVM on x86 (for example, for AVX-512 only >>>>> AVX-512F is included as "evex"). >>>>> >>>>> The fix adds missing features to the list [1]. >>>> >>>> The function naming that you have changed for >>>> supports_avx512_vpclmulqdq was explicitly requested by Vladimir K.: >>>> >>>> http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-December/036326.html >>>> >>>> >>>> If anything I would suggest supports_vpopcntdq() should have the >>>> avx512 part added. >>> >>> Makes sense. Also, decided to rename VNNI, VAES, and VBMI2. They are >>> listed in the manual as part of AVX512. >>> >>>> The actual expansion of the feature string seems okay. It does make >>>> me wonder why these items have been left out previously. >>> >>> I believe it was just overlooked. Added a comment near cpu feature >>> constant declarations. >>> >>> Also, introduced a check to ensure the temporary on-stack buffer is >>> large enough. >>> >>> Best regards, >>> Vladimir Ivanov >>>> >>>> Thanks, >>>> David >>>> >>>>> Testing: manual (-Xlog:os+cpu=trace output on different hardware), >>>>> ????????? hs-precheckin-comp, hs-tier1, hs-tier2 >>>>> >>>>> Best regards, >>>>> Vladimir Ivanov >>>>> >>>>> [1] $ java -Xlog:os+cpu=trace -XX:UseAVX=3 -version >>>>> >>>>> ... >>>>> [0.010s][info][os,cpu] UseSSE=4? UseAVX=3? UseAES=1? MaxVectorSize=64 >>>>> ... >>>>> [0.011s][info][os,cpu] CPU:total 8 (initial active 8) (4 cores per >>>>> cpu, 2 threads per core) family 6 model 85 stepping 4, cmov, cx8, >>>>> fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, >>>>> vzeroupper, avx, avx2, aes, clmul, erms, rtm, 3dnowpref, lzcnt, ht, >>>>> tsc, bmi1, bmi2, adx, avx512f, avx512dq, avx512cd, avx512bw, >>>>> avx512vl, fma, clflush, clflushopt, clwb >>>>> ... >>>>> [0.011s][info][os,cpu] flags??????? : fpu vme de pse tsc msr pae >>>>> mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse >>>>> sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon >>>>> rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx >>>>> ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt >>>>> tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm >>>>> 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp >>>>> tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle >>>>> avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx >>>>> smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec >>>>> xgetbv1 xsaves arat umip pku ospke md_clear >>>>> ... From matthias.baesken at sap.com Tue Feb 11 14:39:52 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 11 Feb 2020 14:39:52 +0000 Subject: Question about JDK-8043780 (was: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux) References: Message-ID: Hello, new webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.3/ * Only touching os_aix.cpp ( I leave os_bsd.cpp as it is , might make the discussion more simple ) * Added handling of failing O_CLOEXEC ( O_CLOEXEC_is_known_to_work == -1 ? case ) Best regards, Matthias From: Baesken, Matthias Sent: Dienstag, 11. Februar 2020 14:06 To: 'Thomas St?fe' ; 'Martin Buchholz' Cc: 'David Holmes' ; 'hotspot-dev at openjdk.java.net' Subject: RE: Question about JDK-8043780 (was: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux) >I wonder whether we could shorten the coding to a simple, unconditional > >::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); Looks like we would need to call first fcntl with F_GETFD AND then F_SETFD (or where would flags come from?) : int flags = ::fcntl(fd, F_GETFD); ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); That?s 2 calls that are not really needed . Best regards, Matthias From: Baesken, Matthias Sent: Dienstag, 11. Februar 2020 10:09 To: 'Thomas St?fe' >; Martin Buchholz > Cc: David Holmes >; hotspot-dev at openjdk.java.net Subject: RE: Question about JDK-8043780 (was: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux) Ping ? any comments on this ? https://bugs.openjdk.java.net/browse/JDK-8043780 mentions : ?Backward compatibility needs to be provided via #ifdef and checking (once) whether O_CLOEXEC works.? So it was intentional that just ***once*** it is checked whether O_CLOEXEC works (in the linux version) . Best regards, Matthias From: Thomas St?fe > Sent: Donnerstag, 6. Februar 2020 11:30 To: Martin Buchholz > Cc: David Holmes >; hotspot-dev at openjdk.java.net; Baesken, Matthias > Subject: Question about JDK-8043780 (was: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux) Hi Martin, do you remember https://bugs.openjdk.java.net/browse/JDK-8043780 ? Some questions came up about your patch: Your original coding does this check-O_CLOEXEC-after-open-and-set-if-missing-dance, but if it worked ar least once it will stop doing that. However, if it fails to work it will retest over and over again. Was this intentional? Do we expect it to start working at some point? If yes, could it also stop working at some point? I wonder whether we could shorten the coding to a simple, unconditional ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); to be done after open. If open(CLOEXEC) worked, this would be a unnecessary noop. If it did not, we set it now. We'd always pay for a fcntl call but I think it is cheap and open is not called that often anyway. -- But then, I wonder whether we can scratch that coding altogether. O_CLOEXEC has been in the kernel since 2.6.23. This is 13 years old. 2.6 line topped out at 2.6.39 in 2011, and I would expect anyone still running on 2.6 to have at least a recent kernel. What do you think? Thank you! Cheers, Thomas On Thu, Feb 6, 2020 at 8:55 AM Baesken, Matthias > wrote: Hi Thomas , the coding was borrowed from linux, see : https://hg.openjdk.java.net/jdk/jdk/file/7b57401deb0c/src/hotspot/os/linux/os_linux.cpp#l5616 I would be fine with always calling fcntl (your suggestion 1) , do you think it was done otherwise back then in os_linux because of performance issues ? Probably os::open in os_linux should be adjusted then to the new ?always call fcntl? approach too, right ? Best regards, Matthias Hi Matthias: + // Validate that the use of the O_CLOEXEC flag on open above worked. + static sig_atomic_t O_CLOEXEC_is_known_to_work = 0; + if (!O_CLOEXEC_is_known_to_work) { int flags = ::fcntl(fd, F_GETFD); - if (flags != -1) + if (flags != -1) { + if ((flags & FD_CLOEXEC) != 0) { + O_CLOEXEC_is_known_to_work = 1; + } else { ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); } -#endif + } + } This is either too complicated or not complicated enough :) if we know for a fact that open(..O_CLOEXEC) did work at least once, we never again check. if we know for a fact that open(..O_CLOEXEC) failed to work, we recheck again and again. Do we expect it to suddenly start working? If yes, could it also suddenly start failing? Should we then not check always? I propose the following change: 1) If open(..O_CLOEXEC) could sometimes fail and sometimes not, or we are just not sure, I would do the fcntl test every time. Or, even just always set the flag again with fcntl. So, just remove the O_CLOEXEC_is_known_to_work conditional variable. Which would reduce the code to a one liner: + ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); 2) If you are sure that it either always works or fails, you need a conditional with three states: unknown, works, does not work. And only check if unknown. I personally would prefer (1) because it is way simpler and fcntl is not such an expensive operation that we have to save time (also, how often do we open files anyway?). ---- Otherwise looks fine. This is a useful fix. Cheers, Thomas On Tue, Jan 28, 2020 at 12:32 PM Baesken, Matthias > wrote: New webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.1/ (one 'c' got lost somehow in the older webrev) Best regards, Matthias From daniel.daugherty at oracle.com Tue Feb 11 17:12:20 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 11 Feb 2020 12:12:20 -0500 Subject: RFR: 8234372: Investigate use of Thread::stack_base() and queries for "in stack" In-Reply-To: <67b3bb43-b53c-0151-33ca-2f58202c29af@oracle.com> References: <67b3bb43-b53c-0151-33ca-2f58202c29af@oracle.com> Message-ID: <5b4d1d1f-1a67-8967-1452-9e371ae17666@oracle.com> Hi David, Very nice cleanup of some crufty/old code. On 2/11/20 12:40 AM, David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8234372 > webrev: http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/ src/hotspot/share/runtime/thread.hpp ??? L691: ? // Warning: the method can only be used on the running thread. ??????? Not fond of this existing comment wording. Perhaps: ???????? ?? // Warning: Can only be called by the calling thread on itself. src/hotspot/share/runtime/thread.cpp ??? L1822: // Check for adr in the usable portion of our stack. ??????? Not fond of "our stack" here. To me it implies the caller's ??????? stack.? Perhaps: ?????????? // Check for adr in the usable portion of the JavaThread's stack. ??????? This is the comment from the thread.hpp: ??????? L1733: ? // Check if address is in the usable part of the stack (excludes protected ??????? L1734: ? // guard pages). Can be applied to any thread and is an approximation for ??????? L1735: ? // using is_in_stack when the query has to happen from another thread. ??????? It is much more clear, but also very long. src/hotspot/os/linux/os_linux.cpp ??? Note: Old code is this: ??? L722:?? if (addr stack_base() && addr >= t->stack_reserved_zone_base()) { ??? so it is different than the Win* version. The new call is better. src/hotspot/os/windows/os_windows.cpp ??? Note: Old code is this: ??? L2545: ????????? if (addr > thread->stack_reserved_zone_base() && addr < thread->stack_base()) { ??? so it is different than the Linux version. The new call is better. src/hotspot/cpu/sparc/frame_sparc.cpp ??? No comments. src/hotspot/cpu/x86/frame_x86.cpp ??? No comments. src/hotspot/cpu/aarch64/frame_aarch64.cpp ??? No comments. src/hotspot/cpu/arm/frame_arm.cpp ??? L144: ????? bool saved_fp_safe = ((address)saved_fp < thread->stack_base()) && (saved_fp > sender_sp); ??? L174: ????? bool saved_fp_safe = ((address)saved_fp < thread->stack_base()) && (saved_fp >= sender_sp); ??????? L144 uses '>' and L174 uses '>=' with the same operands. Why? src/hotspot/os_cpu/linux_arm/os_linux_arm.cpp ??? No comments. src/hotspot/cpu/ppc/frame_ppc.cpp ??? L72: ? bool fp_safe = (fp < thread->stack_base()) &&? (fp > sp); ??????? existing nit: extra space before second (...) expression. src/hotspot/cpu/s390/frame_s390.cpp ??? L76: ? bool fp_safe = (fp < thread->stack_base()) &&? (fp > sp); ??????? existing nit: extra space before second (...) expression. src/hotspot/os_cpu/linux_s390/thread_linux_s390.cpp ??? No comments. Other than one possible typo in src/hotspot/cpu/arm/frame_arm.cpp everything else is a nit. I don't need to see another webrev. Thumbs up! As you like to say: The proof will be in the testing. And I'll change that last word to "testing over time" since some of these code paths might be rarely used... Dan > > Following on from JDK-8215355 I checked all uses of > Thread::stack_base() to watch for range tests that should be exclusive > but are inclusive, and vice-versa. And in addition clarified and > streamlined the various "in stack" checks that are made. > > Summary of changes: > > src/hotspot/cpu/aarch64/frame_aarch64.cpp > src/hotspot/cpu/arm/frame_arm.cpp > src/hotspot/cpu/ppc/frame_ppc.cpp > src/hotspot/cpu/s390/frame_s390.cpp > src/hotspot/cpu/sparc/frame_sparc.cpp > src/hotspot/cpu/x86/frame_x86.cpp > > In terms of actual bugs the implementation of frame::safe_for_sender > on all platforms except x86 and aarch64 was using the wrong range test > in a number of cases, so these are all now correct and consistent. > > All platforms had an incorrect range check in relation to the "locals". > > All platforms now use is_in_usable_stack to check for a valid sp, > rather than duplicating (sometimes incorrectly) that logic. > > -- > > src/hotspot/os/linux/os_linux.cpp > src/hotspot/os/windows/os_windows.cpp > > Replaced explicit range check with is_in_usable_stack > > src/hotspot/os_cpu/linux_arm/os_linux_arm.cpp > src/hotspot/os_cpu/linux_s390/thread_linux_s390.cpp > > Replaced explicit range check with on_local_stack. > > --- > > src/hotspot/share/runtime/thread.?pp > > Moved is_in_usable_stack from Thread to JavaThread (guard regions are > only relevant for JavaThreads). > > Clarified functionality and use of the three "in stack" variants. > > Removed redundant check from is_in_stack: > > !?? // Allow non Java threads to call this without stack_base > !?? if (_stack_base == NULL) return true; > > As this is executed by the current thread, and the very first thing a > thread does is set its stack base and size, it is impossible to find a > NULL stack_base (which is already asserted inside stack_base()). [I > tested this extensively just as a sanity check: tiers 1-5 plus hotspot > runtime/serviceability/gc.] > > Misc cleanup to use stack_end() rather than recalulate it. > > --- > > There are some further possible cleanups here but I didn't want to go > too far with things that would obscure the functional changes too > much. As mentioned in the bug report the three "in stack" functions > would benefit from some minor renamings so that their relationship is > clearer. But I can leave that to a follow on RFE. Further, it may be > possible to replace a lot of the remaining uses of stack_base() with a > more constrained "in stack" function that takes a limit. For example, > rather than something like: > > if (thread->stack_base() > fp && fp >= sp) > > have: > > if (thread->is_in_stack_range(fp /* addr*/, sp /*limit*/)) > > which checks the given addr against stack_base and the limit, and > checks the limit against stack_end(). The difficultly may lie in > determining whether checking against the limit should be a > or >= > test, as it will be dependent on the context. Again this seems like > something for a second RFE. > > --- > > Testing: > > Thanks to Andrew Haley for taking the frame changes for a spin on > ARM/PPC/Aarch64/S390(?). > > I also ran our tier 1 to 3 testing on x86 and sparc. > > Thanks, > David From david.holmes at oracle.com Wed Feb 12 01:21:53 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Feb 2020 11:21:53 +1000 Subject: RFR: 8234372: Investigate use of Thread::stack_base() and queries for "in stack" In-Reply-To: <5b4d1d1f-1a67-8967-1452-9e371ae17666@oracle.com> References: <67b3bb43-b53c-0151-33ca-2f58202c29af@oracle.com> <5b4d1d1f-1a67-8967-1452-9e371ae17666@oracle.com> Message-ID: <5eeabab3-c232-a715-d953-c6921018f208@oracle.com> Hi Dan, On 12/02/2020 3:12 am, Daniel D. Daugherty wrote: > Hi David, > > Very nice cleanup of some crufty/old code. Thanks for the review! Commenbts inlined ... > > On 2/11/20 12:40 AM, David Holmes wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8234372 >> webrev: http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/ > > src/hotspot/share/runtime/thread.hpp > ??? L691: ? // Warning: the method can only be used on the running thread. > ??????? Not fond of this existing comment wording. Perhaps: > > ???????? ?? // Warning: Can only be called by the calling thread on > itself. > > src/hotspot/share/runtime/thread.cpp > ??? L1822: // Check for adr in the usable portion of our stack. > ??????? Not fond of "our stack" here. To me it implies the caller's > ??????? stack.? Perhaps: > > ?????????? // Check for adr in the usable portion of the JavaThread's > stack. > > ??????? This is the comment from the thread.hpp: > > ??????? L1733: ? // Check if address is in the usable part of the stack > (excludes protected > ??????? L1734: ? // guard pages). Can be applied to any thread and is > an approximation for > ??????? L1735: ? // using is_in_stack when the query has to happen from > another thread. > > ??????? It is much more clear, but also very long. I'll tweak the comments to further improve clarity. > > src/hotspot/os/linux/os_linux.cpp > ??? Note: Old code is this: > > ??? L722:?? if (addr stack_base() && addr >= > t->stack_reserved_zone_base()) { > > ??? so it is different than the Win* version. The new call is better. > > src/hotspot/os/windows/os_windows.cpp > ??? Note: Old code is this: > > ??? L2545: ????????? if (addr > thread->stack_reserved_zone_base() && > addr < thread->stack_base()) { > > ??? so it is different than the Linux version. The new call is better. > > > src/hotspot/cpu/sparc/frame_sparc.cpp > ??? No comments. I missed a change here: 254 bool sender_fp_safe = (sender_fp <= thread->stack_base()) && 255 (sender_fp > _FP); Should be < not <=. Now fixed. > src/hotspot/cpu/x86/frame_x86.cpp > ??? No comments. > > > src/hotspot/cpu/aarch64/frame_aarch64.cpp > ??? No comments. > > src/hotspot/cpu/arm/frame_arm.cpp > ??? L144: ????? bool saved_fp_safe = ((address)saved_fp < > thread->stack_base()) && (saved_fp > sender_sp); > ??? L174: ????? bool saved_fp_safe = ((address)saved_fp < > thread->stack_base()) && (saved_fp >= sender_sp); > ??????? L144 uses '>' and L174 uses '>=' with the same operands. Why? Well spotted. I can't answer why. Looking at the equivalent code in frame_aarch64.cpp both statements use >, so I'm going to assume that the >= at L174 is a bug. The x86 seems to confirm that too, so I'm changing L174. > src/hotspot/os_cpu/linux_arm/os_linux_arm.cpp > ??? No comments. > > > src/hotspot/cpu/ppc/frame_ppc.cpp > ??? L72: ? bool fp_safe = (fp < thread->stack_base()) &&? (fp > sp); > ??????? existing nit: extra space before second (...) expression. > > > src/hotspot/cpu/s390/frame_s390.cpp > ??? L76: ? bool fp_safe = (fp < thread->stack_base()) &&? (fp > sp); > ??????? existing nit: extra space before second (...) expression. Fixed. > src/hotspot/os_cpu/linux_s390/thread_linux_s390.cpp > ??? No comments. > > > Other than one possible typo in src/hotspot/cpu/arm/frame_arm.cpp > everything else is a nit. I don't need to see another webrev. I have produced an updated version for other folk to test the ARM change: http://cr.openjdk.java.net/~dholmes/8234372/webrev.v3/ Thanks, David ----- > Thumbs up! > > As you like to say: The proof will be in the testing. > And I'll change that last word to "testing over time" since some > of these code paths might be rarely used... > > Dan > > > >> >> Following on from JDK-8215355 I checked all uses of >> Thread::stack_base() to watch for range tests that should be exclusive >> but are inclusive, and vice-versa. And in addition clarified and >> streamlined the various "in stack" checks that are made. >> >> Summary of changes: >> >> src/hotspot/cpu/aarch64/frame_aarch64.cpp >> src/hotspot/cpu/arm/frame_arm.cpp >> src/hotspot/cpu/ppc/frame_ppc.cpp >> src/hotspot/cpu/s390/frame_s390.cpp >> src/hotspot/cpu/sparc/frame_sparc.cpp >> src/hotspot/cpu/x86/frame_x86.cpp >> >> In terms of actual bugs the implementation of frame::safe_for_sender >> on all platforms except x86 and aarch64 was using the wrong range test >> in a number of cases, so these are all now correct and consistent. >> >> All platforms had an incorrect range check in relation to the "locals". >> >> All platforms now use is_in_usable_stack to check for a valid sp, >> rather than duplicating (sometimes incorrectly) that logic. >> >> -- >> >> src/hotspot/os/linux/os_linux.cpp >> src/hotspot/os/windows/os_windows.cpp >> >> Replaced explicit range check with is_in_usable_stack >> >> src/hotspot/os_cpu/linux_arm/os_linux_arm.cpp >> src/hotspot/os_cpu/linux_s390/thread_linux_s390.cpp >> >> Replaced explicit range check with on_local_stack. >> >> --- >> >> src/hotspot/share/runtime/thread.?pp >> >> Moved is_in_usable_stack from Thread to JavaThread (guard regions are >> only relevant for JavaThreads). >> >> Clarified functionality and use of the three "in stack" variants. >> >> Removed redundant check from is_in_stack: >> >> !?? // Allow non Java threads to call this without stack_base >> !?? if (_stack_base == NULL) return true; >> >> As this is executed by the current thread, and the very first thing a >> thread does is set its stack base and size, it is impossible to find a >> NULL stack_base (which is already asserted inside stack_base()). [I >> tested this extensively just as a sanity check: tiers 1-5 plus hotspot >> runtime/serviceability/gc.] >> >> Misc cleanup to use stack_end() rather than recalulate it. >> >> --- >> >> There are some further possible cleanups here but I didn't want to go >> too far with things that would obscure the functional changes too >> much. As mentioned in the bug report the three "in stack" functions >> would benefit from some minor renamings so that their relationship is >> clearer. But I can leave that to a follow on RFE. Further, it may be >> possible to replace a lot of the remaining uses of stack_base() with a >> more constrained "in stack" function that takes a limit. For example, >> rather than something like: >> >> if (thread->stack_base() > fp && fp >= sp) >> >> have: >> >> if (thread->is_in_stack_range(fp /* addr*/, sp /*limit*/)) >> >> which checks the given addr against stack_base and the limit, and >> checks the limit against stack_end(). The difficultly may lie in >> determining whether checking against the limit should be a > or >= >> test, as it will be dependent on the context. Again this seems like >> something for a second RFE. >> >> --- >> >> Testing: >> >> Thanks to Andrew Haley for taking the frame changes for a spin on >> ARM/PPC/Aarch64/S390(?). >> >> I also ran our tier 1 to 3 testing on x86 and sparc. >> >> Thanks, >> David > From david.holmes at oracle.com Wed Feb 12 01:49:53 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Feb 2020 11:49:53 +1000 Subject: Question about JDK-8043780 In-Reply-To: References: Message-ID: <9428db9f-efb9-c4d3-cadc-13f829442654@oracle.com> Hi Matthias, On 12/02/2020 12:39 am, Baesken, Matthias wrote: > Hello,? new webrev : Can you do this on the actual review thread please. It is very hard to keep track of this one. Thanks, David > http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.3/ > > * Only touching? os_aix.cpp? ( I leave os_bsd.cpp as it is , might > make the discussion?? more ?simple ) > * Added handling of? failing O_CLOEXEC?? ( O_CLOEXEC_is_known_to_work > == -1? ? case ) > > Best regards, Matthias > > *From:*Baesken, Matthias > *Sent:* Dienstag, 11. Februar 2020 14:06 > *To:* 'Thomas St?fe' ; 'Martin Buchholz' > > *Cc:* 'David Holmes' ; > 'hotspot-dev at openjdk.java.net' > *Subject:* RE: Question about JDK-8043780 (was: RFR: 8237830: support > O_CLOEXEC in os::open on other OS than Linux) > >>I wonder whether we could shorten the coding to a simple, unconditional > >> > >>::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); > > Looks like we would need to call first ?fcntl with ?F_GETFD? AND then > F_SETFD?? (or where? would flags come from?) : > > ? ??int flags = ::fcntl(fd, F_GETFD); > > ? ??::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); > > That?s 2? calls? that are not really needed . > > Best regards, Matthias > > *From:*Baesken, Matthias > *Sent:* Dienstag, 11. Februar 2020 10:09 > *To:* 'Thomas St?fe' >; Martin Buchholz > > *Cc:* David Holmes >; hotspot-dev at openjdk.java.net > > *Subject:* RE: Question about JDK-8043780 (was: RFR: 8237830: support > O_CLOEXEC in os::open on other OS than Linux) > > Ping ? any comments on this ? > > https://bugs.openjdk.java.net/browse/JDK-8043780 > > mentions :? ?Backward compatibility needs to be provided via #ifdef and > checking (once) whether O_CLOEXEC works.? > > So it was intentional that just ***once*** it is checked whether > O_CLOEXEC?? works (in the linux version) . > > Best regards, Matthias > > *From:*Thomas St?fe > > *Sent:* Donnerstag, 6. Februar 2020 11:30 > *To:* Martin Buchholz > > *Cc:* David Holmes >; hotspot-dev at openjdk.java.net > ; Baesken, Matthias > > > *Subject:* Question about JDK-8043780 (was: RFR: 8237830: support > O_CLOEXEC in os::open on other OS than Linux) > > Hi Martin, > > do you remember https://bugs.openjdk.java.net/browse/JDK-8043780?? > > Some questions came up about your patch: > > Your original coding does this > check-O_CLOEXEC-after-open-and-set-if-missing-dance, but if it worked ar > least once it will stop doing that. However, if it fails to work it will > retest over and over again. Was this intentional? Do we expect it to > start working at some point? If yes, could it also stop working at some > point? > > I wonder whether we could shorten the coding to a simple, unconditional > > ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); > > to be done after open. > > If open(CLOEXEC) worked, this would be a unnecessary noop. If it did > not, we set it now. We'd always pay for a fcntl call but I think it is > cheap and open is not called that often anyway. > > -- > > But then, I wonder whether we can scratch that coding altogether. > O_CLOEXEC has been in the kernel since 2.6.23. This is 13 years old. 2.6 > line topped out at?2.6.39 in 2011, and I would expect anyone still > running on 2.6 to have at least a recent kernel. > > What do you think? > > Thank you! > > Cheers, Thomas > > On Thu, Feb 6, 2020 at 8:55 AM Baesken, Matthias > > wrote: > > Hi Thomas , the coding was borrowed from linux, see : > > https://hg.openjdk.java.net/jdk/jdk/file/7b57401deb0c/src/hotspot/os/linux/os_linux.cpp#l5616 > > I would be fine with ?always calling fcntl (your suggestion 1) ,? do > you think? it was done otherwise ?back then?? in os_linux because of > performance issues ? > > Probably os::open in os_linux? should be adjusted then? to the? new > ?always call fcntl? approach too, right ? > > Best regards, Matthias > > Hi Matthias: > > + // Validate that the use of the O_CLOEXEC flag on open above worked. > > +? static sig_atomic_t O_CLOEXEC_is_known_to_work = 0; > > +? if (!O_CLOEXEC_is_known_to_work) { > > ???? int flags = ::fcntl(fd, F_GETFD); > > -??? if (flags != -1) > > +??? if (flags != -1) { > > +????? if ((flags & FD_CLOEXEC) != 0) { > > +??????? O_CLOEXEC_is_known_to_work = 1; > > +????? } else { > > ?????? ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); > > ?? } > > -#endif > > +??? } > > +? } > > This is either too complicated or not complicated enough :) > > > if we know for a fact that open(..O_CLOEXEC) did work at least once, > we never again check. > if we know for a fact that open(..O_CLOEXEC) failed to work, we > recheck again and again. Do we expect it to suddenly start working? > If yes, could it also suddenly start failing? Should we then not > check always? > > I propose the following change: > > 1) If open(..O_CLOEXEC) could sometimes fail and sometimes not, or > we are just not sure, I would do the fcntl test every time. Or, even > just always set the flag again with fcntl. So, just remove the > O_CLOEXEC_is_known_to_work conditional variable. > > Which would reduce the code to a one liner: > > + ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); > > 2) If you are sure that it either always works or fails, you need a > conditional with three states: unknown, works, does not work. And > only check if unknown. > > I personally would prefer (1) because it is way simpler and fcntl is > not such an expensive operation that we have to save time (also, how > often do we open files anyway?). > > ---- > > Otherwise looks fine. This is a useful fix. > > > Cheers, Thomas > > > > On Tue, Jan 28, 2020 at 12:32 PM Baesken, Matthias > > wrote: > > New webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.1/ > > (one 'c' got lost somehow in the older webrev) > > Best regards, Matthias > From david.holmes at oracle.com Wed Feb 12 02:01:52 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Feb 2020 12:01:52 +1000 Subject: [15] RFR (S): 8238696: x86: Enumerate all detected CPU features in VM_Version feature string In-Reply-To: <2267f17c-2f81-eec3-d31f-4e031e34b615@oracle.com> References: <98506aca-4a47-a337-a707-19c2a424ca01@oracle.com> <3b882679-b8e1-8e2c-fd8e-261d48224001@oracle.com> <47bf9452-fb75-0293-0a54-3df9e3a3a716@oracle.com> <2267f17c-2f81-eec3-d31f-4e031e34b615@oracle.com> Message-ID: On 11/02/2020 11:58 pm, Vladimir Ivanov wrote: > >> Now I'm wondering about using the dot as a terminator. Doesn't the >> return value from jio_snprintf tell you whether you exceeded the buffer? > > Yes, very good point. How about the following? > > ? http://cr.openjdk.java.net/~vlivanov/8238696/webrev.03 Functional part seems fine. > Considering some tests have hard-coded feature names, I decided to > enhance the new test and check that all reported features are known (at > least, on x86). Not sure about the test - I don't know enough about which features are "always" presents on x86/x64. I'd suggest using @requires x86 rather than the internal Platform checks. Thanks, David > Best regards, > Vladimir Ivanov > >>> On 10.02.2020 17:12, Vladimir Ivanov wrote: >>>> Thanks for taking a look, David. >>>> >>>> Updated webrev: >>>> ?? http://cr.openjdk.java.net/~vlivanov/8238696/webrev.01/ >>>> >>>>>> http://cr.openjdk.java.net/~vlivanov/8238696/webrev.00/ >>>>>> https://bugs.openjdk.java.net/browse/JDK-8238696 >>>>>> >>>>>> Abstract_VM_Version::features_string() doesn't contain all the CPU >>>>>> features detected by the JVM on x86 (for example, for AVX-512 only >>>>>> AVX-512F is included as "evex"). >>>>>> >>>>>> The fix adds missing features to the list [1]. >>>>> >>>>> The function naming that you have changed for >>>>> supports_avx512_vpclmulqdq was explicitly requested by Vladimir K.: >>>>> >>>>> http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-December/036326.html >>>>> >>>>> >>>>> If anything I would suggest supports_vpopcntdq() should have the >>>>> avx512 part added. >>>> >>>> Makes sense. Also, decided to rename VNNI, VAES, and VBMI2. They are >>>> listed in the manual as part of AVX512. >>>> >>>>> The actual expansion of the feature string seems okay. It does make >>>>> me wonder why these items have been left out previously. >>>> >>>> I believe it was just overlooked. Added a comment near cpu feature >>>> constant declarations. >>>> >>>> Also, introduced a check to ensure the temporary on-stack buffer is >>>> large enough. >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> Testing: manual (-Xlog:os+cpu=trace output on different hardware), >>>>>> ????????? hs-precheckin-comp, hs-tier1, hs-tier2 >>>>>> >>>>>> Best regards, >>>>>> Vladimir Ivanov >>>>>> >>>>>> [1] $ java -Xlog:os+cpu=trace -XX:UseAVX=3 -version >>>>>> >>>>>> ... >>>>>> [0.010s][info][os,cpu] UseSSE=4? UseAVX=3? UseAES=1? MaxVectorSize=64 >>>>>> ... >>>>>> [0.011s][info][os,cpu] CPU:total 8 (initial active 8) (4 cores per >>>>>> cpu, 2 threads per core) family 6 model 85 stepping 4, cmov, cx8, >>>>>> fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, >>>>>> vzeroupper, avx, avx2, aes, clmul, erms, rtm, 3dnowpref, lzcnt, >>>>>> ht, tsc, bmi1, bmi2, adx, avx512f, avx512dq, avx512cd, avx512bw, >>>>>> avx512vl, fma, clflush, clflushopt, clwb >>>>>> ... >>>>>> [0.011s][info][os,cpu] flags??????? : fpu vme de pse tsc msr pae >>>>>> mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse >>>>>> sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon >>>>>> rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx >>>>>> ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt >>>>>> tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm >>>>>> abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb >>>>>> stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust >>>>>> bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq >>>>>> rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl >>>>>> xsaveopt xsavec xgetbv1 xsaves arat umip pku ospke md_clear >>>>>> ... From david.holmes at oracle.com Wed Feb 12 03:39:57 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Feb 2020 13:39:57 +1000 Subject: RFR(S) 8238782: Cleanup Deoptimization::deoptimize(): remove unused RegisterMap argument and don't update RegisterMap in callers if UseBiasedLocking is enabled In-Reply-To: References: Message-ID: <0094e0ef-3108-b3d4-7c09-2787afddb790@oracle.com> Hi Richard, This cleanup looks good to me! Thanks, David On 11/02/2020 10:52 pm, Reingruber, Richard wrote: > Hi, > > I would like to do a small clean-up removing the unused parameter "map" from > Deoptimization::deoptimize(JavaThread* thread, frame fr, RegisterMap *map, DeoptReason reason). > > Additionally the callers are changed to use a non-updating RegisterMap to iterate the stack > frames. The updating RegisterMap was needed only for revoking biases in > Deoptimization::deoptimize(), and since JDK-8226705 this isn't done in deoptimize() anymore[1][2][3]. > > So please review > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238782/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8238782 > > Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. > > Thanks, Richard. > > See also: > > [1] JDK-8226705 removes call to revoke_biases_of_monitors() from Deoptimization::deoptimize() > http://hg.openjdk.java.net/jdk/jdk/rev/408c445d04e8#l21.320 > > [2] JDK-8226705 adds call to revoke_from_deopt_handler() to Deoptimization::fetch_unroll_info_helper() [3] > http://hg.openjdk.java.net/jdk/jdk/rev/408c445d04e8#l21.199 > > [3] JDK-8226705: Deoptimization::revoke_from_deopt_handler() revokes biases just before replacing a > compiled frame with equivalent interpreter frames. > http://hg.openjdk.java.net/jdk/jdk/rev/408c445d04e8#l21.290 > From vladimir.kozlov at oracle.com Wed Feb 12 03:59:07 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 11 Feb 2020 19:59:07 -0800 Subject: RFR(S) 8238782: Cleanup Deoptimization::deoptimize(): remove unused RegisterMap argument and don't update RegisterMap in callers if UseBiasedLocking is enabled In-Reply-To: References: Message-ID: <0e4d224c-55a2-6ed6-b463-45ff7f5d1413@oracle.com> It seems reg_map variable is not used anymore in whitebox.cpp code you modified. Otherwise changes look good. Thanks, Vladimir On 2/11/20 4:52 AM, Reingruber, Richard wrote: > Hi, > > I would like to do a small clean-up removing the unused parameter "map" from > Deoptimization::deoptimize(JavaThread* thread, frame fr, RegisterMap *map, DeoptReason reason). > > Additionally the callers are changed to use a non-updating RegisterMap to iterate the stack > frames. The updating RegisterMap was needed only for revoking biases in > Deoptimization::deoptimize(), and since JDK-8226705 this isn't done in deoptimize() anymore[1][2][3]. > > So please review > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238782/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8238782 > > Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. > > Thanks, Richard. > > See also: > > [1] JDK-8226705 removes call to revoke_biases_of_monitors() from Deoptimization::deoptimize() > http://hg.openjdk.java.net/jdk/jdk/rev/408c445d04e8#l21.320 > > [2] JDK-8226705 adds call to revoke_from_deopt_handler() to Deoptimization::fetch_unroll_info_helper() [3] > http://hg.openjdk.java.net/jdk/jdk/rev/408c445d04e8#l21.199 > > [3] JDK-8226705: Deoptimization::revoke_from_deopt_handler() revokes biases just before replacing a > compiled frame with equivalent interpreter frames. > http://hg.openjdk.java.net/jdk/jdk/rev/408c445d04e8#l21.290 > From thomas.stuefe at gmail.com Wed Feb 12 07:23:04 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 12 Feb 2020 08:23:04 +0100 Subject: RFR: 8234372: Investigate use of Thread::stack_base() and queries for "in stack" In-Reply-To: References: <67b3bb43-b53c-0151-33ca-2f58202c29af@oracle.com> Message-ID: Tests ran through, no visible regressions so far. Cheers, Thomas On Tue, Feb 11, 2020 at 9:35 AM Thomas St?fe wrote: > Hi David, > > I did not find anything wrong in your patch. Nice cleanup, and great > archaeological work :) > > Only small nits and some bike shedding: > > --- > > So, if I get this right: > > is_in_stack -> is in live stack (base ... sp] > on_local_stack -> is in (base...start] includes guard pages > is_in_usable_stack -> is in (base...start - guard] excludes guard pages > > The naming is confusing but I saw you recommended renaming the functions > in the JBS comments, and I like all your suggestions better than what we > have now. > > ----- > > > http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/src/hotspot/cpu/aarch64/frame_aarch64.cpp.udiff.html > > - if (locals > thread->stack_base() || locals < (address) fp()) return > false; > + if (locals >= thread->stack_base() || locals < (address) fp()) return > false; > > This would be easier to read as a negated positive (also applies to all > other frame_xxx.cpp). > > Just an idea, maybe we could add a function > Thread::is_in_stack_limited_by(ptr, arbitrary_end_ptr) which could compare > that ptr is between (base .. arbitrary_end_ptr] and based on that we could > implement the other three stack functions. > > For cases like this we could then write: > if (!thread->is_in_stack_limited_by(locals, fp()) > > But I am unsure, maybe I overthink things. > > ------- > > > http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/src/hotspot/os/linux/os_linux.cpp.udiff.html > > - if (addr < t->stack_base() && addr >= t->stack_reserved_zone_base()) { > + if (t->is_in_usable_stack(addr)) { > > First confused me but then I read Fredericks comment in JBS so I think it > is okay. > > But it would be nice to be able to remove this > manually-expand-stack-coding altogether :) > > ------- > > Cheers, Thomas > > On Tue, Feb 11, 2020 at 6:40 AM David Holmes > wrote: > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8234372 >> webrev: http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/ >> >> Following on from JDK-8215355 I checked all uses of Thread::stack_base() >> to watch for range tests that should be exclusive but are inclusive, and >> vice-versa. And in addition clarified and streamlined the various "in >> stack" checks that are made. >> >> Summary of changes: >> >> src/hotspot/cpu/aarch64/frame_aarch64.cpp >> src/hotspot/cpu/arm/frame_arm.cpp >> src/hotspot/cpu/ppc/frame_ppc.cpp >> src/hotspot/cpu/s390/frame_s390.cpp >> src/hotspot/cpu/sparc/frame_sparc.cpp >> src/hotspot/cpu/x86/frame_x86.cpp >> >> In terms of actual bugs the implementation of frame::safe_for_sender on >> all platforms except x86 and aarch64 was using the wrong range test in a >> number of cases, so these are all now correct and consistent. >> >> All platforms had an incorrect range check in relation to the "locals". >> >> All platforms now use is_in_usable_stack to check for a valid sp, rather >> than duplicating (sometimes incorrectly) that logic. >> >> -- >> >> src/hotspot/os/linux/os_linux.cpp >> src/hotspot/os/windows/os_windows.cpp >> >> Replaced explicit range check with is_in_usable_stack >> >> src/hotspot/os_cpu/linux_arm/os_linux_arm.cpp >> src/hotspot/os_cpu/linux_s390/thread_linux_s390.cpp >> >> Replaced explicit range check with on_local_stack. >> >> --- >> >> src/hotspot/share/runtime/thread.?pp >> >> Moved is_in_usable_stack from Thread to JavaThread (guard regions are >> only relevant for JavaThreads). >> >> Clarified functionality and use of the three "in stack" variants. >> >> Removed redundant check from is_in_stack: >> >> ! // Allow non Java threads to call this without stack_base >> ! if (_stack_base == NULL) return true; >> >> As this is executed by the current thread, and the very first thing a >> thread does is set its stack base and size, it is impossible to find a >> NULL stack_base (which is already asserted inside stack_base()). [I >> tested this extensively just as a sanity check: tiers 1-5 plus hotspot >> runtime/serviceability/gc.] >> >> Misc cleanup to use stack_end() rather than recalulate it. >> >> --- >> >> There are some further possible cleanups here but I didn't want to go >> too far with things that would obscure the functional changes too much. >> As mentioned in the bug report the three "in stack" functions would >> benefit from some minor renamings so that their relationship is clearer. >> But I can leave that to a follow on RFE. Further, it may be possible to >> replace a lot of the remaining uses of stack_base() with a more >> constrained "in stack" function that takes a limit. For example, rather >> than something like: >> >> if (thread->stack_base() > fp && fp >= sp) >> >> have: >> >> if (thread->is_in_stack_range(fp /* addr*/, sp /*limit*/)) >> >> which checks the given addr against stack_base and the limit, and checks >> the limit against stack_end(). The difficultly may lie in determining >> whether checking against the limit should be a > or >= test, as it will >> be dependent on the context. Again this seems like something for a >> second RFE. >> >> --- >> >> Testing: >> >> Thanks to Andrew Haley for taking the frame changes for a spin on >> ARM/PPC/Aarch64/S390(?). >> >> I also ran our tier 1 to 3 testing on x86 and sparc. >> >> Thanks, >> David >> > From stefan.karlsson at oracle.com Wed Feb 12 08:24:43 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 12 Feb 2020 09:24:43 +0100 Subject: [15] RFR 8238633: JVMTI heap walk should consult GC for marking oops In-Reply-To: References: Message-ID: <8ea20a15-bdb9-27b0-c306-154f539a3674@oracle.com> Hi Zhengyu, On 2020-02-07 16:53, Zhengyu Gu wrote: > Hi, > > I would like purpose this change that allows GC to provide ObjectMarker > during JVMTI heap walk. > > Currently, JVMTI heap walk uses oop markword's 'marked' pattern to > indicate 'visited' oop. > > Unfortunately, it conflicts with Shenandoah, who uses the pattern to > indicate 'forwarding'. When JVMTI heap walk occurs in some of > Shenandoah's concurrent heap (e.g. concurrent evacuation or concurrent > reference updating phases), it can result corrupted heap, as it tries to > resolve a real oop header as a forwarding pointer. > > This patch allows GC to provide ObjectMarker for JVMTI to track > 'visited' oop, and uses current implementation as default, so that, it > has no impact to GCs other than Shenandoah, who provides its own > implementation. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8238633 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.00/index.html > Would you mind if I asked you to move the object marker code to its own objectMarker.hpp/cpp files? --- Another suggestion is to move the "marking" out of the visit() function, and renamed ObjectMarker::visted() to ObjectMarker::marked(). - if (!ObjectMarker::visited(o)) { - if (!visit(o)) { + if (!marker.object_marker()->visited(o)) { + if (!visit(o, marker.object_marker())) { Would become: + if (!marker.object_marker()->mark(o)) { + if (!visit(o)) { This assert would be unnecessary: -bool VM_HeapWalkOperation::visit(oop o) { +bool VM_HeapWalkOperation::visit(oop o, ObjectMarker* object_marker) { // mark object as visited - assert(!ObjectMarker::visited(o), "can't visit same object more than once"); - ObjectMarker::mark(o); + assert(!object_marker->visited(o), "can't visit same object more than once"); + object_marker->mark(o); The name and comment would match: -// return true if object is marked -inline bool ObjectMarker::visited(oop o) { - return o->mark().is_marked(); -} --- Previously, the calls to 'mark' and 'visited' were inlineable, but now every GC has to take a virtual call when marking the objects. My guess is that this code is slow anyway, and that it doesn't matter too much, but did you measure the effect of that change with, for example, G1? Thanks, StefanK > Test: > ? hotspot_gc > ? vmTestbase_nsk_jdi > ? vmTestbase_nsk_jvmti > > Thanks, > > -Zhengyu > > From matthias.baesken at sap.com Wed Feb 12 08:31:38 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 12 Feb 2020 08:31:38 +0000 Subject: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux References: Message-ID: New webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.3/ * this time , only touching os_aix.cpp ( I leave os_bsd.cpp as it is , might make the discussion more simple ) * Added handling of failing O_CLOEXEC ( O_CLOEXEC_is_known_to_work == -1 ? case ) Best regards, Matthias From: Baesken, Matthias Sent: Donnerstag, 6. Februar 2020 08:55 To: 'Thomas St?fe' Cc: David Holmes ; hotspot-dev at openjdk.java.net Subject: RE: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux Hi Thomas , the coding was borrowed from linux, see : https://hg.openjdk.java.net/jdk/jdk/file/7b57401deb0c/src/hotspot/os/linux/os_linux.cpp#l5616 I would be fine with always calling fcntl (your suggestion 1) , do you think it was done otherwise back then in os_linux because of performance issues ? Probably os::open in os_linux should be adjusted then to the new ?always call fcntl? approach too, right ? Best regards, Matthias Hi Matthias: + // Validate that the use of the O_CLOEXEC flag on open above worked. + static sig_atomic_t O_CLOEXEC_is_known_to_work = 0; + if (!O_CLOEXEC_is_known_to_work) { int flags = ::fcntl(fd, F_GETFD); - if (flags != -1) + if (flags != -1) { + if ((flags & FD_CLOEXEC) != 0) { + O_CLOEXEC_is_known_to_work = 1; + } else { ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); } -#endif + } + } This is either too complicated or not complicated enough :) if we know for a fact that open(..O_CLOEXEC) did work at least once, we never again check. if we know for a fact that open(..O_CLOEXEC) failed to work, we recheck again and again. Do we expect it to suddenly start working? If yes, could it also suddenly start failing? Should we then not check always? I propose the following change: 1) If open(..O_CLOEXEC) could sometimes fail and sometimes not, or we are just not sure, I would do the fcntl test every time. Or, even just always set the flag again with fcntl. So, just remove the O_CLOEXEC_is_known_to_work conditional variable. Which would reduce the code to a one liner: + ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); 2) If you are sure that it either always works or fails, you need a conditional with three states: unknown, works, does not work. And only check if unknown. I personally would prefer (1) because it is way simpler and fcntl is not such an expensive operation that we have to save time (also, how often do we open files anyway?). ---- Otherwise looks fine. This is a useful fix. Cheers, Thomas On Tue, Jan 28, 2020 at 12:32 PM Baesken, Matthias > wrote: New webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.1/ (one 'c' got lost somehow in the older webrev) Best regards, Matthias From richard.reingruber at sap.com Wed Feb 12 08:44:29 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Wed, 12 Feb 2020 08:44:29 +0000 Subject: RFR(S) 8238782: Cleanup Deoptimization::deoptimize(): remove unused RegisterMap argument and don't update RegisterMap in callers if UseBiasedLocking is enabled In-Reply-To: <0094e0ef-3108-b3d4-7c09-2787afddb790@oracle.com> References: <0094e0ef-3108-b3d4-7c09-2787afddb790@oracle.com> Message-ID: Hi David, thanks for reviewing! Richard. -----Original Message----- From: David Holmes Sent: Mittwoch, 12. Februar 2020 04:40 To: Reingruber, Richard ; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(S) 8238782: Cleanup Deoptimization::deoptimize(): remove unused RegisterMap argument and don't update RegisterMap in callers if UseBiasedLocking is enabled Hi Richard, This cleanup looks good to me! Thanks, David On 11/02/2020 10:52 pm, Reingruber, Richard wrote: > Hi, > > I would like to do a small clean-up removing the unused parameter "map" from > Deoptimization::deoptimize(JavaThread* thread, frame fr, RegisterMap *map, DeoptReason reason). > > Additionally the callers are changed to use a non-updating RegisterMap to iterate the stack > frames. The updating RegisterMap was needed only for revoking biases in > Deoptimization::deoptimize(), and since JDK-8226705 this isn't done in deoptimize() anymore[1][2][3]. > > So please review > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238782/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8238782 > > Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. > > Thanks, Richard. > > See also: > > [1] JDK-8226705 removes call to revoke_biases_of_monitors() from Deoptimization::deoptimize() > http://hg.openjdk.java.net/jdk/jdk/rev/408c445d04e8#l21.320 > > [2] JDK-8226705 adds call to revoke_from_deopt_handler() to Deoptimization::fetch_unroll_info_helper() [3] > http://hg.openjdk.java.net/jdk/jdk/rev/408c445d04e8#l21.199 > > [3] JDK-8226705: Deoptimization::revoke_from_deopt_handler() revokes biases just before replacing a > compiled frame with equivalent interpreter frames. > http://hg.openjdk.java.net/jdk/jdk/rev/408c445d04e8#l21.290 > From richard.reingruber at sap.com Wed Feb 12 08:50:03 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Wed, 12 Feb 2020 08:50:03 +0000 Subject: RFR(S) 8238782: Cleanup Deoptimization::deoptimize(): remove unused RegisterMap argument and don't update RegisterMap in callers if UseBiasedLocking is enabled In-Reply-To: <0e4d224c-55a2-6ed6-b463-45ff7f5d1413@oracle.com> References: <0e4d224c-55a2-6ed6-b463-45ff7f5d1413@oracle.com> Message-ID: Hi Vladimir, > It seems reg_map variable is not used anymore in whitebox.cpp code you modified. Oh, certainly. Good you noticed it. I removed the line. > Otherwise changes look good. Thanks for the review, Richard. -----Original Message----- From: Vladimir Kozlov Sent: Mittwoch, 12. Februar 2020 04:59 To: Reingruber, Richard ; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(S) 8238782: Cleanup Deoptimization::deoptimize(): remove unused RegisterMap argument and don't update RegisterMap in callers if UseBiasedLocking is enabled It seems reg_map variable is not used anymore in whitebox.cpp code you modified. Otherwise changes look good. Thanks, Vladimir On 2/11/20 4:52 AM, Reingruber, Richard wrote: > Hi, > > I would like to do a small clean-up removing the unused parameter "map" from > Deoptimization::deoptimize(JavaThread* thread, frame fr, RegisterMap *map, DeoptReason reason). > > Additionally the callers are changed to use a non-updating RegisterMap to iterate the stack > frames. The updating RegisterMap was needed only for revoking biases in > Deoptimization::deoptimize(), and since JDK-8226705 this isn't done in deoptimize() anymore[1][2][3]. > > So please review > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238782/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8238782 > > Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. > > Thanks, Richard. > > See also: > > [1] JDK-8226705 removes call to revoke_biases_of_monitors() from Deoptimization::deoptimize() > http://hg.openjdk.java.net/jdk/jdk/rev/408c445d04e8#l21.320 > > [2] JDK-8226705 adds call to revoke_from_deopt_handler() to Deoptimization::fetch_unroll_info_helper() [3] > http://hg.openjdk.java.net/jdk/jdk/rev/408c445d04e8#l21.199 > > [3] JDK-8226705: Deoptimization::revoke_from_deopt_handler() revokes biases just before replacing a > compiled frame with equivalent interpreter frames. > http://hg.openjdk.java.net/jdk/jdk/rev/408c445d04e8#l21.290 > From richard.reingruber at sap.com Wed Feb 12 10:23:27 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Wed, 12 Feb 2020 10:23:27 +0000 Subject: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant Message-ID: // Repost including hotspot runtime and gc lists. // Dean Long suggested to do so, because the enhancement replaces a vm operation // with a handshake. // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html Hi, could I please get reviews for this small enhancement in hotspot's jvmti implementation: Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8238585 The change avoids making all compiled methods on stack not_entrant when switching a java thread to interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack. Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations. Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. Thanks, Richard. See also my question if anyone knows a reason for making the compiled methods not_entrant: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html From archana.nogriya at uk.ibm.com Wed Feb 12 12:04:19 2020 From: archana.nogriya at uk.ibm.com (Archana Nogriya) Date: Wed, 12 Feb 2020 12:04:19 +0000 Subject: Copyrights issue GPLV2 with no classpath exception in OpenJDK14 : Implement JFR Event Streaming Message-ID: Hi, We have noticed that, src/jdk.jfr/share/classes/jdk/jfr/internal/consumer/ConstantLookup.java src/jdk.jfr/share/classes/jdk/jfr/internal/consumer/FileAccess.java, files has been added couple of month back, and in new copyright added with GPLV2 but there is no mention of Classpath exception. Since this code is part of release and distributed please can I request you to add Classpath exception in the copyrights. JEP: https://bugs.openjdk.java.net/browse/JDK-8226511 Changeset : https://hg.openjdk.java.net/jdk/jdk/rev/c16ac7a2eba4 Many Thanks & Regards Archana Nogriya Java Runtimes Development and Project Manager IBM Hursley IBM United Kingdom Ltd internet email: archana.nogriya at uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From david.holmes at oracle.com Wed Feb 12 13:15:19 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Feb 2020 23:15:19 +1000 Subject: RFR: 8234372: Investigate use of Thread::stack_base() and queries for "in stack" In-Reply-To: References: <67b3bb43-b53c-0151-33ca-2f58202c29af@oracle.com> Message-ID: On 12/02/2020 5:23 pm, Thomas St?fe wrote: > Tests ran through, no visible regressions so far. Thanks Thomas! As per my email response to Dan I made a further correction to the ARM code: http://cr.openjdk.java.net/~dholmes/8234372/webrev.v3/ Thanks, David > Cheers, Thomas > > On Tue, Feb 11, 2020 at 9:35 AM Thomas St?fe > wrote: > > Hi David, > > I did not find anything wrong in your patch. Nice cleanup, and great > archaeological work :) > > Only small nits and some bike shedding: > > --- > > So, if I get this right: > > is_in_stack -> is in live stack (base ... sp] > on_local_stack ?-> is in (base...start] includes guard pages > is_in_usable_stack -> is in (base...start - guard] excludes guard pages > > The naming is confusing but I saw you recommended renaming the > functions in the JBS comments, and I like all your suggestions > better than what we have now. > > ----- > > http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/src/hotspot/cpu/aarch64/frame_aarch64.cpp.udiff.html > > - ?if (locals > thread->stack_base() || locals < (address) fp()) > return false; > + ?if (locals >= thread->stack_base() || locals < (address) fp()) > return false; > > This would be easier to read as a negated positive (also applies to > all other frame_xxx.cpp). > > Just an idea, maybe we could add a function > Thread::is_in_stack_limited_by(ptr, arbitrary_end_ptr) which could > compare that ptr is between (base .. arbitrary_end_ptr] and based on > that we could implement the other three stack functions. > > For cases like this we could then write: > if (!thread->is_in_stack_limited_by(locals, fp()) > > But I am unsure, maybe I overthink things. > > ------- > > http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/src/hotspot/os/linux/os_linux.cpp.udiff.html > > - ?if (addr < ?t->stack_base() && addr >= > t->stack_reserved_zone_base()) { > + ?if (t->is_in_usable_stack(addr)) { > > First confused me but then I read Fredericks comment in JBS so I > think it is okay. > > But it would be nice to be able to remove this > manually-expand-stack-coding altogether :) > > ------- > > Cheers, Thomas > > On Tue, Feb 11, 2020 at 6:40 AM David Holmes > > wrote: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8234372 > webrev: http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/ > > Following on from JDK-8215355 I checked all uses of > Thread::stack_base() > to watch for range tests that should be exclusive but are > inclusive, and > vice-versa. And in addition clarified and streamlined the > various "in > stack" checks that are made. > > Summary of changes: > > src/hotspot/cpu/aarch64/frame_aarch64.cpp > src/hotspot/cpu/arm/frame_arm.cpp > src/hotspot/cpu/ppc/frame_ppc.cpp > src/hotspot/cpu/s390/frame_s390.cpp > src/hotspot/cpu/sparc/frame_sparc.cpp > src/hotspot/cpu/x86/frame_x86.cpp > > In terms of actual bugs the implementation of > frame::safe_for_sender on > all platforms except x86 and aarch64 was using the wrong range > test in a > number of cases, so these are all now correct and consistent. > > All platforms had an incorrect range check in relation to the > "locals". > > All platforms now use is_in_usable_stack to check for a valid > sp, rather > than duplicating (sometimes incorrectly) that logic. > > -- > > src/hotspot/os/linux/os_linux.cpp > src/hotspot/os/windows/os_windows.cpp > > Replaced explicit range check with is_in_usable_stack > > src/hotspot/os_cpu/linux_arm/os_linux_arm.cpp > src/hotspot/os_cpu/linux_s390/thread_linux_s390.cpp > > Replaced explicit range check with on_local_stack. > > --- > > src/hotspot/share/runtime/thread.?pp > > Moved is_in_usable_stack from Thread to JavaThread (guard > regions are > only relevant for JavaThreads). > > Clarified functionality and use of the three "in stack" variants. > > Removed redundant check from is_in_stack: > > !? ?// Allow non Java threads to call this without stack_base > !? ?if (_stack_base == NULL) return true; > > As this is executed by the current thread, and the very first > thing a > thread does is set its stack base and size, it is impossible to > find a > NULL stack_base (which is already asserted inside stack_base()). [I > tested this extensively just as a sanity check: tiers 1-5 plus > hotspot > runtime/serviceability/gc.] > > Misc cleanup to use stack_end() rather than recalulate it. > > --- > > There are some further possible cleanups here but I didn't want > to go > too far with things that would obscure the functional changes > too much. > As mentioned in the bug report the three "in stack" functions would > benefit from some minor renamings so that their relationship is > clearer. > But I can leave that to a follow on RFE. Further, it may be > possible to > replace a lot of the remaining uses of stack_base() with a more > constrained "in stack" function that takes a limit. For example, > rather > than something like: > > if (thread->stack_base() > fp && fp >= sp) > > have: > > if (thread->is_in_stack_range(fp /* addr*/, sp /*limit*/)) > > which checks the given addr against stack_base and the limit, > and checks > the limit against stack_end(). The difficultly may lie in > determining > whether checking against the limit should be a > or >= test, as > it will > be dependent on the context. Again this seems like something for a > second RFE. > > --- > > Testing: > > Thanks to Andrew Haley for taking the frame changes for a spin on > ARM/PPC/Aarch64/S390(?). > > I also ran our tier 1 to 3 testing on x86 and sparc. > > Thanks, > David > From christoph.langer at sap.com Wed Feb 12 13:26:00 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 12 Feb 2020 13:26:00 +0000 Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images In-Reply-To: <9cd2165f-64f0-a096-9b99-aa8bc0798b34@oracle.com> References: <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com> <2b92559a-6a1a-119a-1e62-eed6f9d3bfc8@oracle.com> <1989082f-cddf-9d4f-262b-5f9072f46cf7@oracle.com> <9cd2165f-64f0-a096-9b99-aa8bc0798b34@oracle.com> Message-ID: Hi Magnus, Erik, I started off with Matthias? patch and tried to address your concerns. Especially I explored adding the stripped pdbs to the jmods as well. Here is what I came up with: http://cr.openjdk.java.net/~clanger/webrevs/8237192.0/ It?s a bit hacky in that it?ll copy support/modules_libs to support/modules_libs_stripped and fix the pdbs to ship in there. The same goes for modules_cmds. The problem with that approach is that probably not all dependencies are placed correctly and also there?s a bit more of duplication of binaries in the build directories. I think, it could be repaired eventually by refactoring, e.g. have a support/modules_dbgsymbols folder where the real debug symbol files get placed and used from there. There?s also two additional caveats, one is that jimage.pdb and jpackage.pdb exist twice. Once for the libs and once for the launchers with the same name. This will cause failures when jlinking. I decided to keep the pdbs for the libs. And I also had to take care of the classlist generation, to have the resulting classlist placed in support/modules_libs_stripped as well. I furthermore updated the naming of options and variables, hopefully to your like. And I made the debug output logInfo. I tested (on Windows), both, with --enable-public-debug-symbols and without. Without the option, everything seems as before. With the option enabled, the stripped debug symbols will be installed in the bundles and also in the jmods. ?? Please let me know what you think. Thanks & Best regards Christoph From: Magnus Ihse Bursie Sent: Freitag, 7. Februar 2020 14:09 To: Baesken, Matthias ; Erik Joelsson ; Langer, Christoph ; David Holmes ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images On 2020-02-07 09:50, Baesken, Matthias wrote: Hello, here is a slightly changed new webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.4/ In Bundles.gmk, this line: $(ECHO) found stripped pdb file $$$${f}, we rename it to: $$$${f%stripped.pdb}pdb; \ It looks almost like left-over debug output. If you want to keep it, please rephrase to something more terse, that fits better with the existing style of build messages. Also, it should probably be on the LOG=info level, so add a $(LOG_INFO). In NativeCompilation.gmk: Why not just a simple, ifeq ($(ENABLE_STRIPPED_PDBS), true) $1_EXTRA_LDFLAGS += "-pdbstripped:$$($1_SYMBOLS_DIR)/$$($1_NOSUFFIX).stripped.pdb" endif ? I see no reason to duplicate code. In jdk-options.m4: I'm not 100% sure about the name of the new option. --enable-stripped-pdb does not fully convey the fact that we do not strip the *existing* pdb:s, but instead also add a new type. Maybe --enable-bundle-stripped-pdb? /Magnus (adjusted $(JRE_STRIPPED_PDB_FILES) in Bundles.gmk, that was in the wrong place ) . I think it needs to be a separate option as it's all orthogonal to the main debug symbols file creation. Yes it is a separate option I agree that?s better . One has to set --enable-stripped-pdbs=yes . Best regards , Matthias On 2020-02-06 04:48, Magnus Ihse Bursie wrote: On 2020-02-06 12:36, Langer, Christoph wrote: Hi, let me chime in to this discussion at this point. So, first of all, Matthias, thanks for your work so far. Erik, I fully understand your points and I agree that it's probably a good idea to refactor this whole process of creating these different types of bundles. Our idea is to provide functionality in the build system to add "public" or stripped debug files to the delivery image of the JDK. This would provide better information in case of crashes and would hence allow for better supportability. That's something we'd probably want to enable in SapMachine binary distributions. I very much support the idea of using these stripped pdb files. It has been a long standing issue in hotspot on Windows to not have access to stacktraces. So, can we get to add a configure option like the one proposed by Matthias into the current code base? The option should be turned off by default. If it is switched on, the images/jdk folder in the build directory will have both, the *stripped.pdb files and the standard *.pdb files. However, having *stripped.pdb files around should not matter in terms of functionality (for developers and testing) as they'd not be used in the presence of the "real" pdb files anyway. The actual JDK bundle for delivery would then contain the *stripped.pdb files, renamed to *.pdb. And the debug symbols bundle would have the full *.pdb files only. Both could then be overlaid as needed. I think you raised two concerns. One is that you'd rather like to refactor bundling for several reasons. But I guess, should you eventually get to your refactoring, it shouldn't be a problem to take the functionality of this new option along. The other was regarding JMODs. I believe it's correct, that JMODs have never carried external debug information. For platforms other than MS Windows that's actually not a big problem because debug information can be internalized. And jlink has gotten several additions to set flags for stripping that data to the right level. I assume if jlinked images on Windows should ever be enabled to carry debug information, inclusion of external debug files would have to be added to JMODs and jlink. But that's definitely out of scope here. The argument "jmods have never carried external debug information" just doesn't apply here. Neither has the distribution bundles, for the exact same reason. You really should not compare these new stripped pdb files to the existing debug symbol files, they are different files with different purposes. One is meant to be shipped to customers, the other is not. You say you want to ship these new stripped pdb files with your distribution so that customers have them present when they use your distribution. If you then omit these new files from the jmods, any customer created jlinked image will not have these new stripped pdb files, which IMO is a very weird and unexpected behavior from a customer point of view. Jlinking new images is an integral and promoted way of using a JDK, so any mismatch between the original JDK distribution and what you are able to jlink is a serious discrepancy. So, what do you think? What speaks against adding this option (that is off by default)? My main objective is that you introduce further discrepancies between the original distribution JDK image and what's possible to generate using jlink from that distribution JDK image. My second objective is that the already convoluted bundles creation logic becomes even more convoluted. I believe there is a better possible solution in the build but it will require a lot more work to figure out. All that said, if you still wish to continue, I will stop standing in the way. While Erik will need to comment on this himself, from my POV this looks okay. The conditions are: * This is controlled by a separate option (or perhaps even better as a new argument to --with-native-debug-symbols), so without this, nothing will change. I think it needs to be a separate option as it's all orthogonal to the main debug symbols file creation. * The code need to make sure to separate stripped.pdb and normal pdb files, when enabled. * And there must not be a heavy penalty in added code complexity. /Erik /Magnus Thanks Christoph -----Original Message----- From: build-dev On Behalf Of Erik Joelsson Sent: Donnerstag, 23. Januar 2020 18:49 To: Baesken, Matthias; David Holmes ; 'build-dev at openjdk.java.net' >; 'hotspot-dev at openjdk.java.net' > Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images On 2020-01-23 00:03, Baesken, Matthias wrote: Hi Erik, yes true sorry for answering your comments a bit late . If a user runs jlink and includes all the jmods we ship with the JDK, the result should be essentially equivalent to the original JDK image. The way the stripped pdb files are included in the bundles sort of at the last second of the build here breaks this property. I think we should address this in a separate bug/CR . Maybe. I realize that my proposal below is quite a big task. But on the other hand, I don't think breaking the relationship between the jmods and the distribution bundles is on the table really. Looking for example into a Linux build, I see a lot of debuginfo files in the jdk image (more or less for every shared lib) . But when looking into the jmods of that jdk image , no debuginfo files are in there ( I checked the java.base jmod). So putting the files with debug information into the jmods seems to be something that was not done so far cross platform (or is there some build switch for it that I did not check?) . Maybe to keep the jmods as small as possible . No, we do not put the debuginfo files in the jmods nor the bundles because those are not intended to be shipped to customers. We are currently overlaying them into images/jdk in the build output to make them available for local debugging. (This is convoluted and I would very much like to get away from this practice at some point so that there is a 1-1 mapping between images/jdk and bundles/jdk*-bin.tar.gz.) The stripped pdb files you are proposing are on the contrary intended for shipping to customers (as I understand your proposal) so comparing them with the debuginfo files is not relevant. Now if MS had been kind enough to define a separate file type for the stripped pdbs, so that they could live alongside the full pdbs, we wouldn't have this issue. The heart of the problem is that only one set of files (either stripped or full) can be present and usable in images/jdk at a time. We have 2 main uses for images/jdk. 1. Developer running and debugging locally 2. Serve as the source for generating the distribution bundles We currently have one image serving both of these purposes, which is already creating complicated and convoluted build steps. To properly solve this I would argue for splitting these two apart into two different images for the two different purposes. The build procedure would then be, first build the images for distribution, only containing what should go into each bundle. Then create the developer jdk image by copying files from the distribution images. On Windows, the first image would contain the stripped pdbs and when building the second, those would get overwritten with the full pdbs. Now that I figured out a working model that would solve a bunch of other problems as well, I would love to implement it, but I doubt I will have time in the near future. /Erik To properly implement this, care will need to be taken to juggle the two sets of pdb files around, making sure each build and test use case has the correct one in place where and when it's needed. Quite possibly, we cannot cover all use cases with one build configuration. Developers needing the full debug symbols when debugging locally would likely need to disable the stripped symbols so they get the full symbols everywhere. Possibly this would need to be the default for debug builds and configurable for release builds. From my limited experience , the developers do not work with the bundles (that would contain now after my patch the stripped pds) but with a "normal" jdk image that is created my make all. Best regards, Matthias This still does not address anything in my objection. /Erik On 2020-01-22 07:46, Baesken, Matthias wrote: Hello, here is an updated version : http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/ this one supports a configure switch "--enable-stripped-pdbs" to enable the feature . Best regards, Matthias -----Original Message----- From: Baesken, Matthias Sent: Dienstag, 21. Januar 2020 11:03 To: 'David Holmes'; 'build- dev at openjdk.java.net'; 'hotspot- dev at openjdk.java.net' Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images Hi David , yes I think it makes sense to have a configure option for this . Not everyone would like to have a larger JDK (even it is only a bit larger). From erik.gahlin at oracle.com Wed Feb 12 13:29:24 2020 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Wed, 12 Feb 2020 14:29:24 +0100 Subject: Copyrights issue GPLV2 with no classpath exception in OpenJDK14 : Implement JFR Event Streaming In-Reply-To: References: Message-ID: <961d1f0e-ebf2-8a15-e591-c79b48586399@oracle.com> Hi, Thanks for noticing this. I have filed a bug on this. https://bugs.openjdk.java.net/browse/JDK-8238959 Erik On 2020-02-12 13:04, Archana Nogriya wrote: > Hi, > > We have noticed that, > src/jdk.jfr/share/classes/jdk/jfr/internal/consumer/ConstantLookup.java > src/jdk.jfr/share/classes/jdk/jfr/internal/consumer/FileAccess.java, > files > has > been added couple of month back, and in new copyright added with GPLV2 > but there is no mention of Classpath exception. > > Since this code is part of release and distributed please can I > request you to add Classpath exception in the copyrights. > > JEP: _https://bugs.openjdk.java.net/browse/JDK-8226511_ > Changeset : _https://hg.openjdk.java.net/jdk/jdk/rev/c16ac7a2eba4_ > > > Many Thanks & Regards > Archana Nogriya > Java Runtimes Development and Project Manager > IBM Hursley > IBM United Kingdom Ltd > internet email: archana.nogriya at uk.ibm.com > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with > number 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From stefan.johansson at oracle.com Wed Feb 12 15:32:14 2020 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 12 Feb 2020 16:32:14 +0100 Subject: RFR[XS]: 8238272: Eliminate cast_from_oop to narrowOop* In-Reply-To: References: <267AF7FC-B6EB-4232-A51F-035A7217BCC6@oracle.com> <44ad6bd5-b66a-3306-0cb9-f518d1b8e2c4@redhat.com> <805C4111-CF21-4EAF-AC78-742045A3DD73@oracle.com> <0F754E34-462D-4183-8CE0-F2933D4F310F@oracle.com> Message-ID: <92B4C857-5B01-450C-BAFD-0EA5AF054BFD@oracle.com> > 9 feb. 2020 kl. 23:26 skrev Kim Barrett : > >> On Jan 31, 2020, at 5:12 PM, Kim Barrett wrote: >> >> [?] >> Here's a new webrev: >> >> https://cr.openjdk.java.net/~kbarrett/8238272/open.01/ Looks good, Stefan >> >> I didn't bother with an incremental webrev, since the new version is just >> a one-line change replacing the previous one-line change. >> >> Testing: tier1 > > Ping? Looking for a second reviewer. > From glaubitz at physik.fu-berlin.de Wed Feb 12 16:14:13 2020 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Wed, 12 Feb 2020 17:14:13 +0100 Subject: RFR: 8199138: Add RISC-V support to Zero Message-ID: <7f93b41f-9979-0302-6fe8-7bd12e75f39a@physik.fu-berlin.de> Hi! This is an updated RFR to add basic RISC-V support to Zero. This patch is being used for the riscv64 port in Debian. Please review the changes in [1]. Thanks, Adrian > [1] http://cr.openjdk.java.net/~glaubitz/8199138/webrev.00/ -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From erik.joelsson at oracle.com Wed Feb 12 16:42:59 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 12 Feb 2020 08:42:59 -0800 Subject: RFR: 8199138: Add RISC-V support to Zero In-Reply-To: <7f93b41f-9979-0302-6fe8-7bd12e75f39a@physik.fu-berlin.de> References: <7f93b41f-9979-0302-6fe8-7bd12e75f39a@physik.fu-berlin.de> Message-ID: <83e1c8cf-25c8-df10-72cb-50930cb11a66@oracle.com> Build changes look ok to me. Someone from hotspot should review the cpp file. /Erik On 2020-02-12 08:14, John Paul Adrian Glaubitz wrote: > Hi! > > This is an updated RFR to add basic RISC-V support to Zero. > > This patch is being used for the riscv64 port in Debian. > > Please review the changes in [1]. > > Thanks, > Adrian > >> [1] http://cr.openjdk.java.net/~glaubitz/8199138/webrev.00/ From shade at redhat.com Wed Feb 12 16:51:32 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 12 Feb 2020 17:51:32 +0100 Subject: RFR: 8199138: Add RISC-V support to Zero In-Reply-To: <7f93b41f-9979-0302-6fe8-7bd12e75f39a@physik.fu-berlin.de> References: <7f93b41f-9979-0302-6fe8-7bd12e75f39a@physik.fu-berlin.de> Message-ID: <552d208d-2317-4b93-8232-9439d9a0accb@redhat.com> On 2/12/20 5:14 PM, John Paul Adrian Glaubitz wrote: >> [1] http://cr.openjdk.java.net/~glaubitz/8199138/webrev.00/ Neat. Looks good to me. Minor nits in os_linux.cpp: *) Can you move the comment to the #define line, as it is done in the similar blocks in the same file? 1854 #ifndef EM_RISCV /* RISCV */ 1855 #define EM_RISCV 243 1856 #endif *) I believe this one is sorted alphabetically, so RISCV should be between __powerpc64__ and S390? - AARCH64, ALPHA, ARM, AMD64, IA32, IA64, M68K, MIPS, MIPSEL, PARISC, __powerpc__, __powerpc64__, S390, SH, __sparc + AARCH64, ALPHA, ARM, AMD64, IA32, IA64, M68K, MIPS, MIPSEL, PARISC, __powerpc__, __powerpc64__, S390, SH, __sparc, RISCV -- Thanks, -Aleksey From glaubitz at physik.fu-berlin.de Wed Feb 12 16:54:28 2020 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Wed, 12 Feb 2020 17:54:28 +0100 Subject: RFR: 8199138: Add RISC-V support to Zero In-Reply-To: <552d208d-2317-4b93-8232-9439d9a0accb@redhat.com> References: <7f93b41f-9979-0302-6fe8-7bd12e75f39a@physik.fu-berlin.de> <552d208d-2317-4b93-8232-9439d9a0accb@redhat.com> Message-ID: <30763728-3b0d-1109-e99f-050d74302a07@physik.fu-berlin.de> Hi! On 2/12/20 5:51 PM, Aleksey Shipilev wrote: > On 2/12/20 5:14 PM, John Paul Adrian Glaubitz wrote: >>> [1] http://cr.openjdk.java.net/~glaubitz/8199138/webrev.00/ > > Neat. Looks good to me. > > Minor nits in os_linux.cpp: > > *) Can you move the comment to the #define line, as it is done in the similar blocks in the same file? > > 1854 #ifndef EM_RISCV /* RISCV */ > 1855 #define EM_RISCV 243 > 1856 #endif Yes. > *) I believe this one is sorted alphabetically, so RISCV should be between __powerpc64__ and S390? > > - AARCH64, ALPHA, ARM, AMD64, IA32, IA64, M68K, MIPS, MIPSEL, PARISC, __powerpc__, > __powerpc64__, S390, SH, __sparc > + AARCH64, ALPHA, ARM, AMD64, IA32, IA64, M68K, MIPS, MIPSEL, PARISC, __powerpc__, > __powerpc64__, S390, SH, __sparc, RISCV Sure. I assume I can push with those changes and mark it as Reviewed-by: erikj, shade? Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From shade at redhat.com Wed Feb 12 16:59:48 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 12 Feb 2020 17:59:48 +0100 Subject: RFR: 8199138: Add RISC-V support to Zero In-Reply-To: <30763728-3b0d-1109-e99f-050d74302a07@physik.fu-berlin.de> References: <7f93b41f-9979-0302-6fe8-7bd12e75f39a@physik.fu-berlin.de> <552d208d-2317-4b93-8232-9439d9a0accb@redhat.com> <30763728-3b0d-1109-e99f-050d74302a07@physik.fu-berlin.de> Message-ID: <82096ac0-c8ec-eb01-2f2f-e81a4530913e@redhat.com> On 2/12/20 5:54 PM, John Paul Adrian Glaubitz wrote: > I assume I can push with those changes and mark it as Reviewed-by: erikj, shade? Mark it, yes. I believe non-trivial (yet exceedingly simple) things like these require waiting for 24 hours to anyone else to chime in with comments. There seem to be no rush to get it in, right? -- Thanks, -Aleksey From glaubitz at physik.fu-berlin.de Wed Feb 12 17:00:55 2020 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Wed, 12 Feb 2020 18:00:55 +0100 Subject: RFR: 8199138: Add RISC-V support to Zero In-Reply-To: <82096ac0-c8ec-eb01-2f2f-e81a4530913e@redhat.com> References: <7f93b41f-9979-0302-6fe8-7bd12e75f39a@physik.fu-berlin.de> <552d208d-2317-4b93-8232-9439d9a0accb@redhat.com> <30763728-3b0d-1109-e99f-050d74302a07@physik.fu-berlin.de> <82096ac0-c8ec-eb01-2f2f-e81a4530913e@redhat.com> Message-ID: <077415d2-eac0-b742-0335-9f0010bb70db@physik.fu-berlin.de> On 2/12/20 5:59 PM, Aleksey Shipilev wrote: > On 2/12/20 5:54 PM, John Paul Adrian Glaubitz wrote: >> I assume I can push with those changes and mark it as Reviewed-by: erikj, shade? > > Mark it, yes. I believe non-trivial (yet exceedingly simple) things like these require waiting for > 24 hours to anyone else to chime in with comments. There seem to be no rush to get it in, right? No rush, no. Just wanted to make sure that it's fine to push then :). Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From glaubitz at physik.fu-berlin.de Wed Feb 12 17:08:10 2020 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Wed, 12 Feb 2020 18:08:10 +0100 Subject: RFR: 8199138: Add RISC-V support to Zero In-Reply-To: <552d208d-2317-4b93-8232-9439d9a0accb@redhat.com> References: <7f93b41f-9979-0302-6fe8-7bd12e75f39a@physik.fu-berlin.de> <552d208d-2317-4b93-8232-9439d9a0accb@redhat.com> Message-ID: <2f225c34-b731-81e9-8e2b-16d7496d7161@physik.fu-berlin.de> Hi! On 2/12/20 5:51 PM, Aleksey Shipilev wrote: > Neat. Looks good to me. > > Minor nits in os_linux.cpp: > > *) Can you move the comment to the #define line, as it is done in the similar blocks in the same file? > > 1854 #ifndef EM_RISCV /* RISCV */ > 1855 #define EM_RISCV 243 > 1856 #endif > > *) I believe this one is sorted alphabetically, so RISCV should be between __powerpc64__ and S390? > > - AARCH64, ALPHA, ARM, AMD64, IA32, IA64, M68K, MIPS, MIPSEL, PARISC, __powerpc__, > __powerpc64__, S390, SH, __sparc > + AARCH64, ALPHA, ARM, AMD64, IA32, IA64, M68K, MIPS, MIPSEL, PARISC, __powerpc__, > __powerpc64__, S390, SH, __sparc, RISCV I have done that now. Updated RFR in [1]. Adrian > [1] http://cr.openjdk.java.net/~glaubitz/8199138/webrev.01/ -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From shade at redhat.com Wed Feb 12 17:13:26 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 12 Feb 2020 18:13:26 +0100 Subject: RFR: 8199138: Add RISC-V support to Zero In-Reply-To: <077415d2-eac0-b742-0335-9f0010bb70db@physik.fu-berlin.de> References: <7f93b41f-9979-0302-6fe8-7bd12e75f39a@physik.fu-berlin.de> <552d208d-2317-4b93-8232-9439d9a0accb@redhat.com> <30763728-3b0d-1109-e99f-050d74302a07@physik.fu-berlin.de> <82096ac0-c8ec-eb01-2f2f-e81a4530913e@redhat.com> <077415d2-eac0-b742-0335-9f0010bb70db@physik.fu-berlin.de> Message-ID: <231ef5a1-8e7d-385c-5604-43f0cb1e2c7c@redhat.com> On 2/12/20 6:00 PM, John Paul Adrian Glaubitz wrote: > On 2/12/20 5:59 PM, Aleksey Shipilev wrote: >> On 2/12/20 5:54 PM, John Paul Adrian Glaubitz wrote: >>> I assume I can push with those changes and mark it as Reviewed-by: erikj, shade? >> >> Mark it, yes. I believe non-trivial (yet exceedingly simple) things like these require waiting for >> 24 hours to anyone else to chime in with comments. There seem to be no rush to get it in, right? > > No rush, no. Just wanted to make sure that it's fine to push then :). Yes, I believe it would be fine to push then. -Aleksey From kim.barrett at oracle.com Wed Feb 12 18:43:23 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 12 Feb 2020 13:43:23 -0500 Subject: RFR[XS]: 8238272: Eliminate cast_from_oop to narrowOop* In-Reply-To: <92B4C857-5B01-450C-BAFD-0EA5AF054BFD@oracle.com> References: <267AF7FC-B6EB-4232-A51F-035A7217BCC6@oracle.com> <44ad6bd5-b66a-3306-0cb9-f518d1b8e2c4@redhat.com> <805C4111-CF21-4EAF-AC78-742045A3DD73@oracle.com> <0F754E34-462D-4183-8CE0-F2933D4F310F@oracle.com> <92B4C857-5B01-450C-BAFD-0EA5AF054BFD@oracle.com> Message-ID: <0B965F11-B174-4C36-AB94-8441FE5960D7@oracle.com> > On Feb 12, 2020, at 10:32 AM, Stefan Johansson wrote: > > > >> 9 feb. 2020 kl. 23:26 skrev Kim Barrett : >> >>> On Jan 31, 2020, at 5:12 PM, Kim Barrett wrote: >>> >>> [?] >>> Here's a new webrev: >>> >>> https://cr.openjdk.java.net/~kbarrett/8238272/open.01/ > Looks good, > Stefan Thanks. From shade at redhat.com Wed Feb 12 18:45:30 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 12 Feb 2020 19:45:30 +0100 Subject: RFR (XS) 8238909: x86_32 fails gtest:power_of_2 Message-ID: <36e47275-651c-1bb5-55c9-a7063c62c4be@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8238909 Fix: https://cr.openjdk.java.net/~shade/8238909/webrev.01/ The problem here is is_power_of_2(intptr_t), which truncates int64_t on x86_32, and replies "false" on large power-of-two values. is_power_of_2_long(jlong) is awkward, but alternatives seem worse: *) "LP64_ONLY(is_power_of_2(value)) NOT_LP64(is_power_of_2_long(value))" looks worse; *) Overloading is_power_of_2(int64_t) quickly runs afoul of ambiguities on the platforms I tried; *) Specializing the whole thing for 8-byte types seems to invite even more template madness; Testing: affected test on {x86_64, x86_32}; jdk-submit -- Thanks, -Aleksey From stefan.karlsson at oracle.com Wed Feb 12 18:54:09 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 12 Feb 2020 19:54:09 +0100 Subject: RFR (XS) 8238909: x86_32 fails gtest:power_of_2 In-Reply-To: <36e47275-651c-1bb5-55c9-a7063c62c4be@redhat.com> References: <36e47275-651c-1bb5-55c9-a7063c62c4be@redhat.com> Message-ID: <894ae817-7617-e601-adac-9c9d78fa1138@oracle.com> On 2020-02-12 19:45, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8238909 > > Fix: > https://cr.openjdk.java.net/~shade/8238909/webrev.01/ > > The problem here is is_power_of_2(intptr_t), which truncates int64_t on x86_32, and replies "false" > on large power-of-two values. is_power_of_2_long(jlong) is awkward, but alternatives seem worse: > *) "LP64_ONLY(is_power_of_2(value)) NOT_LP64(is_power_of_2_long(value))" looks worse; > *) Overloading is_power_of_2(int64_t) quickly runs afoul of ambiguities on the platforms I tried; > *) Specializing the whole thing for 8-byte types seems to invite even more template madness; FWIW, we have a template version in align.hpp: // Temporary declaration until this file has been restructured. template bool is_power_of_2_t(T x) { ? return (x != T(0)) && ((x & (x - 1)) == T(0)); } Would that version work in this situation? StefanK > > Testing: affected test on {x86_64, x86_32}; jdk-submit > From shade at redhat.com Wed Feb 12 19:15:03 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 12 Feb 2020 20:15:03 +0100 Subject: RFR (XS) 8238909: x86_32 fails gtest:power_of_2 In-Reply-To: <894ae817-7617-e601-adac-9c9d78fa1138@oracle.com> References: <36e47275-651c-1bb5-55c9-a7063c62c4be@redhat.com> <894ae817-7617-e601-adac-9c9d78fa1138@oracle.com> Message-ID: <864b1418-e4d4-0b8a-26a0-68c567ec46b7@redhat.com> On 2/12/20 7:54 PM, Stefan Karlsson wrote: > On 2020-02-12 19:45, Aleksey Shipilev wrote: >> Fix: >> https://cr.openjdk.java.net/~shade/8238909/webrev.01/ >> > FWIW, we have a template version in align.hpp: > > // Temporary declaration until this file has been restructured. > template > bool is_power_of_2_t(T x) { > ? return (x != T(0)) && ((x & (x - 1)) == T(0)); > } > > Would that version work in this situation? Oh! Ohhh! It does seem to work. Testing... Why isn't it in globalDefinitions.hpp? -- Thanks, -Aleksey From stefan.karlsson at oracle.com Wed Feb 12 19:23:02 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 12 Feb 2020 20:23:02 +0100 Subject: RFR (XS) 8238909: x86_32 fails gtest:power_of_2 In-Reply-To: <864b1418-e4d4-0b8a-26a0-68c567ec46b7@redhat.com> References: <36e47275-651c-1bb5-55c9-a7063c62c4be@redhat.com> <894ae817-7617-e601-adac-9c9d78fa1138@oracle.com> <864b1418-e4d4-0b8a-26a0-68c567ec46b7@redhat.com> Message-ID: <7e5905bd-3b66-5b8f-7f3e-57ec9e1a81e5@oracle.com> On 2020-02-12 20:15, Aleksey Shipilev wrote: > On 2/12/20 7:54 PM, Stefan Karlsson wrote: >> On 2020-02-12 19:45, Aleksey Shipilev wrote: >>> Fix: >>> https://cr.openjdk.java.net/~shade/8238909/webrev.01/ >>> >> FWIW, we have a template version in align.hpp: >> >> // Temporary declaration until this file has been restructured. >> template >> bool is_power_of_2_t(T x) { >> ? return (x != T(0)) && ((x & (x - 1)) == T(0)); >> } >> >> Would that version work in this situation? > Oh! Ohhh! It does seem to work. Testing... > > Why isn't it in globalDefinitions.hpp? IIRC, there used to be a circular dependency between globalDefinitions.hpp and align.hpp. I created an RFE some time ago, that I never got to: https://bugs.openjdk.java.net/browse/JDK-8183574 I also wonder why the is_power_of_2* functions are not in the new utilities/powerOfTwo.hpp file. StefanK > From shade at redhat.com Wed Feb 12 20:30:02 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 12 Feb 2020 21:30:02 +0100 Subject: RFR (XS) 8238909: x86_32 fails gtest:power_of_2 In-Reply-To: <36e47275-651c-1bb5-55c9-a7063c62c4be@redhat.com> References: <36e47275-651c-1bb5-55c9-a7063c62c4be@redhat.com> Message-ID: On 2/12/20 7:45 PM, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8238909 > > Fix: > https://cr.openjdk.java.net/~shade/8238909/webrev.01/ Scratch that, Stefan suggested a cleaner version: https://cr.openjdk.java.net/~shade/8238909/webrev.02/ Testing: affected test {x86_64, x86_32}; jdk-submit (clean) -- Thanks, -Aleksey From thomas.stuefe at gmail.com Wed Feb 12 20:34:53 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 12 Feb 2020 21:34:53 +0100 Subject: RFR: 8234372: Investigate use of Thread::stack_base() and queries for "in stack" In-Reply-To: References: <67b3bb43-b53c-0151-33ca-2f58202c29af@oracle.com> Message-ID: Hi David, still looks good (same nits apply, but if you want to push this version its fine by me). Cheers, Thomas On Wed, Feb 12, 2020 at 2:15 PM David Holmes wrote: > On 12/02/2020 5:23 pm, Thomas St?fe wrote: > > Tests ran through, no visible regressions so far. > > Thanks Thomas! > > As per my email response to Dan I made a further correction to the ARM > code: > > http://cr.openjdk.java.net/~dholmes/8234372/webrev.v3/ > > Thanks, > David > > > > Cheers, Thomas > > > > On Tue, Feb 11, 2020 at 9:35 AM Thomas St?fe > > wrote: > > > > Hi David, > > > > I did not find anything wrong in your patch. Nice cleanup, and great > > archaeological work :) > > > > Only small nits and some bike shedding: > > > > --- > > > > So, if I get this right: > > > > is_in_stack -> is in live stack (base ... sp] > > on_local_stack -> is in (base...start] includes guard pages > > is_in_usable_stack -> is in (base...start - guard] excludes guard > pages > > > > The naming is confusing but I saw you recommended renaming the > > functions in the JBS comments, and I like all your suggestions > > better than what we have now. > > > > ----- > > > > > http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/src/hotspot/cpu/aarch64/frame_aarch64.cpp.udiff.html > > > > - if (locals > thread->stack_base() || locals < (address) fp()) > > return false; > > + if (locals >= thread->stack_base() || locals < (address) fp()) > > return false; > > > > This would be easier to read as a negated positive (also applies to > > all other frame_xxx.cpp). > > > > Just an idea, maybe we could add a function > > Thread::is_in_stack_limited_by(ptr, arbitrary_end_ptr) which could > > compare that ptr is between (base .. arbitrary_end_ptr] and based on > > that we could implement the other three stack functions. > > > > For cases like this we could then write: > > if (!thread->is_in_stack_limited_by(locals, fp()) > > > > But I am unsure, maybe I overthink things. > > > > ------- > > > > > http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/src/hotspot/os/linux/os_linux.cpp.udiff.html > > > > - if (addr < t->stack_base() && addr >= > > t->stack_reserved_zone_base()) { > > + if (t->is_in_usable_stack(addr)) { > > > > First confused me but then I read Fredericks comment in JBS so I > > think it is okay. > > > > But it would be nice to be able to remove this > > manually-expand-stack-coding altogether :) > > > > ------- > > > > Cheers, Thomas > > > > On Tue, Feb 11, 2020 at 6:40 AM David Holmes > > > wrote: > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8234372 > > webrev: http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/ > > > > Following on from JDK-8215355 I checked all uses of > > Thread::stack_base() > > to watch for range tests that should be exclusive but are > > inclusive, and > > vice-versa. And in addition clarified and streamlined the > > various "in > > stack" checks that are made. > > > > Summary of changes: > > > > src/hotspot/cpu/aarch64/frame_aarch64.cpp > > src/hotspot/cpu/arm/frame_arm.cpp > > src/hotspot/cpu/ppc/frame_ppc.cpp > > src/hotspot/cpu/s390/frame_s390.cpp > > src/hotspot/cpu/sparc/frame_sparc.cpp > > src/hotspot/cpu/x86/frame_x86.cpp > > > > In terms of actual bugs the implementation of > > frame::safe_for_sender on > > all platforms except x86 and aarch64 was using the wrong range > > test in a > > number of cases, so these are all now correct and consistent. > > > > All platforms had an incorrect range check in relation to the > > "locals". > > > > All platforms now use is_in_usable_stack to check for a valid > > sp, rather > > than duplicating (sometimes incorrectly) that logic. > > > > -- > > > > src/hotspot/os/linux/os_linux.cpp > > src/hotspot/os/windows/os_windows.cpp > > > > Replaced explicit range check with is_in_usable_stack > > > > src/hotspot/os_cpu/linux_arm/os_linux_arm.cpp > > src/hotspot/os_cpu/linux_s390/thread_linux_s390.cpp > > > > Replaced explicit range check with on_local_stack. > > > > --- > > > > src/hotspot/share/runtime/thread.?pp > > > > Moved is_in_usable_stack from Thread to JavaThread (guard > > regions are > > only relevant for JavaThreads). > > > > Clarified functionality and use of the three "in stack" variants. > > > > Removed redundant check from is_in_stack: > > > > ! // Allow non Java threads to call this without stack_base > > ! if (_stack_base == NULL) return true; > > > > As this is executed by the current thread, and the very first > > thing a > > thread does is set its stack base and size, it is impossible to > > find a > > NULL stack_base (which is already asserted inside stack_base()). > [I > > tested this extensively just as a sanity check: tiers 1-5 plus > > hotspot > > runtime/serviceability/gc.] > > > > Misc cleanup to use stack_end() rather than recalulate it. > > > > --- > > > > There are some further possible cleanups here but I didn't want > > to go > > too far with things that would obscure the functional changes > > too much. > > As mentioned in the bug report the three "in stack" functions > would > > benefit from some minor renamings so that their relationship is > > clearer. > > But I can leave that to a follow on RFE. Further, it may be > > possible to > > replace a lot of the remaining uses of stack_base() with a more > > constrained "in stack" function that takes a limit. For example, > > rather > > than something like: > > > > if (thread->stack_base() > fp && fp >= sp) > > > > have: > > > > if (thread->is_in_stack_range(fp /* addr*/, sp /*limit*/)) > > > > which checks the given addr against stack_base and the limit, > > and checks > > the limit against stack_end(). The difficultly may lie in > > determining > > whether checking against the limit should be a > or >= test, as > > it will > > be dependent on the context. Again this seems like something for > a > > second RFE. > > > > --- > > > > Testing: > > > > Thanks to Andrew Haley for taking the frame changes for a spin on > > ARM/PPC/Aarch64/S390(?). > > > > I also ran our tier 1 to 3 testing on x86 and sparc. > > > > Thanks, > > David > > > From david.holmes at oracle.com Wed Feb 12 22:06:45 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Feb 2020 08:06:45 +1000 Subject: RFR: 8199138: Add RISC-V support to Zero In-Reply-To: <2f225c34-b731-81e9-8e2b-16d7496d7161@physik.fu-berlin.de> References: <7f93b41f-9979-0302-6fe8-7bd12e75f39a@physik.fu-berlin.de> <552d208d-2317-4b93-8232-9439d9a0accb@redhat.com> <2f225c34-b731-81e9-8e2b-16d7496d7161@physik.fu-berlin.de> Message-ID: <44746c5b-b4d0-d272-a02d-089d24aae43f@oracle.com> This seems fine to me too. Thanks, David On 13/02/2020 3:08 am, John Paul Adrian Glaubitz wrote: > Hi! > > On 2/12/20 5:51 PM, Aleksey Shipilev wrote: >> Neat. Looks good to me. >> >> Minor nits in os_linux.cpp: >> >> *) Can you move the comment to the #define line, as it is done in the similar blocks in the same file? >> >> 1854 #ifndef EM_RISCV /* RISCV */ >> 1855 #define EM_RISCV 243 >> 1856 #endif >> >> *) I believe this one is sorted alphabetically, so RISCV should be between __powerpc64__ and S390? >> >> - AARCH64, ALPHA, ARM, AMD64, IA32, IA64, M68K, MIPS, MIPSEL, PARISC, __powerpc__, >> __powerpc64__, S390, SH, __sparc >> + AARCH64, ALPHA, ARM, AMD64, IA32, IA64, M68K, MIPS, MIPSEL, PARISC, __powerpc__, >> __powerpc64__, S390, SH, __sparc, RISCV > > I have done that now. Updated RFR in [1]. > > Adrian > >> [1] http://cr.openjdk.java.net/~glaubitz/8199138/webrev.01/ > From david.holmes at oracle.com Wed Feb 12 22:07:33 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Feb 2020 08:07:33 +1000 Subject: RFR: 8234372: Investigate use of Thread::stack_base() and queries for "in stack" In-Reply-To: References: <67b3bb43-b53c-0151-33ca-2f58202c29af@oracle.com> Message-ID: <2e170b8c-d59d-351f-9c7a-6c8a3cf7ab43@oracle.com> On 13/02/2020 6:34 am, Thomas St?fe wrote: > Hi David, > > still looks good (same nits apply, but if you want to push this version > its fine by me). Thanks Thomas! David ----- > Cheers, Thomas > > On Wed, Feb 12, 2020 at 2:15 PM David Holmes > wrote: > > On 12/02/2020 5:23 pm, Thomas St?fe wrote: > > Tests ran through, no visible regressions so far. > > Thanks Thomas! > > As per my email response to Dan I made a further correction to the > ARM code: > > http://cr.openjdk.java.net/~dholmes/8234372/webrev.v3/ > > Thanks, > David > > > > Cheers, Thomas > > > > On Tue, Feb 11, 2020 at 9:35 AM Thomas St?fe > > > >> wrote: > > > >? ? ?Hi David, > > > >? ? ?I did not find anything wrong in your patch. Nice cleanup, > and great > >? ? ?archaeological work :) > > > >? ? ?Only small nits and some bike shedding: > > > >? ? ?--- > > > >? ? ?So, if I get this right: > > > >? ? ?is_in_stack -> is in live stack (base ... sp] > >? ? ?on_local_stack ?-> is in (base...start] includes guard pages > >? ? ?is_in_usable_stack -> is in (base...start - guard] excludes > guard pages > > > >? ? ?The naming is confusing but I saw you recommended renaming the > >? ? ?functions in the JBS comments, and I like all your suggestions > >? ? ?better than what we have now. > > > >? ? ?----- > > > > > http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/src/hotspot/cpu/aarch64/frame_aarch64.cpp.udiff.html > > > >? ? ?- ?if (locals > thread->stack_base() || locals < (address) fp()) > >? ? ?return false; > >? ? ?+ ?if (locals >= thread->stack_base() || locals < (address) fp()) > >? ? ?return false; > > > >? ? ?This would be easier to read as a negated positive (also > applies to > >? ? ?all other frame_xxx.cpp). > > > >? ? ?Just an idea, maybe we could add a function > >? ? ?Thread::is_in_stack_limited_by(ptr, arbitrary_end_ptr) which > could > >? ? ?compare that ptr is between (base .. arbitrary_end_ptr] and > based on > >? ? ?that we could implement the other three stack functions. > > > >? ? ?For cases like this we could then write: > >? ? ?if (!thread->is_in_stack_limited_by(locals, fp()) > > > >? ? ?But I am unsure, maybe I overthink things. > > > >? ? ?------- > > > > > http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/src/hotspot/os/linux/os_linux.cpp.udiff.html > > > >? ? ?- ?if (addr < ?t->stack_base() && addr >= > >? ? ?t->stack_reserved_zone_base()) { > >? ? ?+ ?if (t->is_in_usable_stack(addr)) { > > > >? ? ?First confused me but then I read Fredericks comment in JBS so I > >? ? ?think it is okay. > > > >? ? ?But it would be nice to be able to remove this > >? ? ?manually-expand-stack-coding altogether :) > > > >? ? ?------- > > > >? ? ?Cheers, Thomas > > > >? ? ?On Tue, Feb 11, 2020 at 6:40 AM David Holmes > >? ? ? > >> > wrote: > > > >? ? ? ? ?Bug: https://bugs.openjdk.java.net/browse/JDK-8234372 > >? ? ? ? ?webrev: > http://cr.openjdk.java.net/~dholmes/8234372/webrev.v2/ > > > >? ? ? ? ?Following on from JDK-8215355 I checked all uses of > >? ? ? ? ?Thread::stack_base() > >? ? ? ? ?to watch for range tests that should be exclusive but are > >? ? ? ? ?inclusive, and > >? ? ? ? ?vice-versa. And in addition clarified and streamlined the > >? ? ? ? ?various "in > >? ? ? ? ?stack" checks that are made. > > > >? ? ? ? ?Summary of changes: > > > >? ? ? ? ?src/hotspot/cpu/aarch64/frame_aarch64.cpp > >? ? ? ? ?src/hotspot/cpu/arm/frame_arm.cpp > >? ? ? ? ?src/hotspot/cpu/ppc/frame_ppc.cpp > >? ? ? ? ?src/hotspot/cpu/s390/frame_s390.cpp > >? ? ? ? ?src/hotspot/cpu/sparc/frame_sparc.cpp > >? ? ? ? ?src/hotspot/cpu/x86/frame_x86.cpp > > > >? ? ? ? ?In terms of actual bugs the implementation of > >? ? ? ? ?frame::safe_for_sender on > >? ? ? ? ?all platforms except x86 and aarch64 was using the wrong > range > >? ? ? ? ?test in a > >? ? ? ? ?number of cases, so these are all now correct and consistent. > > > >? ? ? ? ?All platforms had an incorrect range check in relation to the > >? ? ? ? ?"locals". > > > >? ? ? ? ?All platforms now use is_in_usable_stack to check for a valid > >? ? ? ? ?sp, rather > >? ? ? ? ?than duplicating (sometimes incorrectly) that logic. > > > >? ? ? ? ?-- > > > >? ? ? ? ?src/hotspot/os/linux/os_linux.cpp > >? ? ? ? ?src/hotspot/os/windows/os_windows.cpp > > > >? ? ? ? ?Replaced explicit range check with is_in_usable_stack > > > >? ? ? ? ?src/hotspot/os_cpu/linux_arm/os_linux_arm.cpp > >? ? ? ? ?src/hotspot/os_cpu/linux_s390/thread_linux_s390.cpp > > > >? ? ? ? ?Replaced explicit range check with on_local_stack. > > > >? ? ? ? ?--- > > > >? ? ? ? ?src/hotspot/share/runtime/thread.?pp > > > >? ? ? ? ?Moved is_in_usable_stack from Thread to JavaThread (guard > >? ? ? ? ?regions are > >? ? ? ? ?only relevant for JavaThreads). > > > >? ? ? ? ?Clarified functionality and use of the three "in stack" > variants. > > > >? ? ? ? ?Removed redundant check from is_in_stack: > > > >? ? ? ? ?!? ?// Allow non Java threads to call this without stack_base > >? ? ? ? ?!? ?if (_stack_base == NULL) return true; > > > >? ? ? ? ?As this is executed by the current thread, and the very first > >? ? ? ? ?thing a > >? ? ? ? ?thread does is set its stack base and size, it is > impossible to > >? ? ? ? ?find a > >? ? ? ? ?NULL stack_base (which is already asserted inside > stack_base()). [I > >? ? ? ? ?tested this extensively just as a sanity check: tiers 1-5 > plus > >? ? ? ? ?hotspot > >? ? ? ? ?runtime/serviceability/gc.] > > > >? ? ? ? ?Misc cleanup to use stack_end() rather than recalulate it. > > > >? ? ? ? ?--- > > > >? ? ? ? ?There are some further possible cleanups here but I > didn't want > >? ? ? ? ?to go > >? ? ? ? ?too far with things that would obscure the functional changes > >? ? ? ? ?too much. > >? ? ? ? ?As mentioned in the bug report the three "in stack" > functions would > >? ? ? ? ?benefit from some minor renamings so that their > relationship is > >? ? ? ? ?clearer. > >? ? ? ? ?But I can leave that to a follow on RFE. Further, it may be > >? ? ? ? ?possible to > >? ? ? ? ?replace a lot of the remaining uses of stack_base() with > a more > >? ? ? ? ?constrained "in stack" function that takes a limit. For > example, > >? ? ? ? ?rather > >? ? ? ? ?than something like: > > > >? ? ? ? ?if (thread->stack_base() > fp && fp >= sp) > > > >? ? ? ? ?have: > > > >? ? ? ? ?if (thread->is_in_stack_range(fp /* addr*/, sp /*limit*/)) > > > >? ? ? ? ?which checks the given addr against stack_base and the limit, > >? ? ? ? ?and checks > >? ? ? ? ?the limit against stack_end(). The difficultly may lie in > >? ? ? ? ?determining > >? ? ? ? ?whether checking against the limit should be a > or >= > test, as > >? ? ? ? ?it will > >? ? ? ? ?be dependent on the context. Again this seems like > something for a > >? ? ? ? ?second RFE. > > > >? ? ? ? ?--- > > > >? ? ? ? ?Testing: > > > >? ? ? ? ?Thanks to Andrew Haley for taking the frame changes for a > spin on > >? ? ? ? ?ARM/PPC/Aarch64/S390(?). > > > >? ? ? ? ?I also ran our tier 1 to 3 testing on x86 and sparc. > > > >? ? ? ? ?Thanks, > >? ? ? ? ?David > > > From erik.joelsson at oracle.com Wed Feb 12 22:16:50 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 12 Feb 2020 14:16:50 -0800 Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images In-Reply-To: References: <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com> <2b92559a-6a1a-119a-1e62-eed6f9d3bfc8@oracle.com> <1989082f-cddf-9d4f-262b-5f9072f46cf7@oracle.com> <9cd2165f-64f0-a096-9b99-aa8bc0798b34@oracle.com> Message-ID: Hello Christoph, This patch certainly looks better to me, though I agree it's a bit hackish to have to filter and rename the stripped.pdb files twice, once for jmods and again for bundles. I think I'm ok with it for now though. The future improvement I would like to make would be to create two sets of jdk images, one that contains debug symbols and demos, which we continue to use for testing, and another which only contains exactly what we bundle up, including the correctly named top dir. The latter would be created first and used as input for the former. I think lots of things would be cleaner then, especially Bundles.gmk. But, that can wait for later. With your current proposal, my proposed future change will apply seamlessly in regard to the stripped pdb files and our distribution bundles. The clash between launchers and libs is annoying, and we also have it for java.exe and java.dll already, though the build is explicitly handling that one somewhere else. Now to review, there are some details I will nitpick about. CreateJmods.gmk: 174, 185: Rule declarations should be indented like any other line inside conditionals. But, I have a problem with these rules as the target is the directory. This will not work well with incremental builds. I would recommend doing a SetupCopyFiles construct instead so you get individual rules for each file. It would look something like this: rename-stripped-pdb = \ ??? $(patsubst %.stripped.pdb,%.pdb,$1) $(eval $(call SetupCopyFiles, COPY_STRIPPED_LIBS, \ ??? SRC := $(LIBS_DIR), \ ??? DEST := $(LIBS_DIR_STRIPPED), \ ??? FILES := $(call FindFiles, $(LIBS_DIR)), \ ??? NAME_MACRO := rename-stripped-pdb, \ )) DEPS += $(COPY_STRIPPED_LIBS) For the corresponding CMD_DIR, the removal of jimage and friends can be done with $(filter ) around The FindFiles call. GenerateLinkOptData.gmk: Please indent inside ifeq block. I would prefer having the TARGETS += inside the conditional block. Seems you also left a commented out endif there. NativeCompilation.gmk 919: You changed the continuation indent from 4 to 2, please revert. /Erik On 2020-02-12 05:26, Langer, Christoph wrote: > > Hi Magnus, Erik, > > I started off with Matthias? patch and tried to address your concerns. > Especially I explored adding the stripped pdbs to the jmods as well. > Here is what I came up with: > > http://cr.openjdk.java.net/~clanger/webrevs/8237192.0/ > > It?s a bit hacky in that it?ll copy support/modules_libs to > support/modules_libs_stripped and fix the pdbs to ship in there. The > same goes for modules_cmds. The problem with that approach is that > probably not all dependencies are placed correctly and also there?s a > bit more of duplication of binaries in the build directories. I think, > it could be repaired eventually by refactoring, e.g. have a > support/modules_dbgsymbols folder where the real debug symbol files > get placed and used from there. > > There?s also two additional caveats, one is that jimage.pdb and > jpackage.pdb exist twice. Once for the libs and once for the launchers > with the same name. This will cause failures when jlinking. I decided > to keep the pdbs for the libs. And I also had to take care of the > classlist generation, to have the resulting classlist placed in > support/modules_libs_stripped as well. > > I furthermore updated the naming of options and variables, hopefully > to your like. And I made the debug output logInfo. > > I tested (on Windows), both, with --enable-public-debug-symbols and > without. Without the option, everything seems as before. With the > option enabled, the stripped debug symbols will be installed in the > bundles and also in the jmods. ?? > > Please let me know what you think. > > Thanks & Best regards > > Christoph > > *From:*Magnus Ihse Bursie > *Sent:* Freitag, 7. Februar 2020 14:09 > *To:* Baesken, Matthias ; Erik Joelsson > ; Langer, Christoph > ; David Holmes ; > 'build-dev at openjdk.java.net' ; > 'hotspot-dev at openjdk.java.net' > *Subject:* Re: RFR: 8237192: Generate stripped/public pdbs on Windows > for jdk images > > On 2020-02-07 09:50, Baesken, Matthias wrote: > > Hello, here is? a slightly changed? new? webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.4/ > > In Bundles.gmk, this line: > > $(ECHO) found stripped pdb file $$$${f}, we rename it to: > $$$${f%stripped.pdb}pdb; \ > > It looks almost like left-over debug output. If you want to keep it, > please rephrase to something more terse, that fits better with the > existing style of build messages. Also, it should probably be on the > LOG=info level, so add a $(LOG_INFO). > > In NativeCompilation.gmk: > Why not just a simple, > ????????? ifeq ($(ENABLE_STRIPPED_PDBS), true) > ??????????? $1_EXTRA_LDFLAGS += > "-pdbstripped:$$($1_SYMBOLS_DIR)/$$($1_NOSUFFIX).stripped.pdb" > ????????? endif > ? > I see no reason to duplicate code. > > In jdk-options.m4: > > I'm not 100% sure about the name of the new option. > --enable-stripped-pdb does not fully convey the fact that we do not > strip the *existing* pdb:s, but instead also add a new type. Maybe > --enable-bundle-stripped-pdb? > > /Magnus > > (adjusted $(JRE_STRIPPED_PDB_FILES)? in Bundles.gmk, that was in the wrong place )? . > > I think it needs to be a separate option as it's all orthogonal to the > > main debug symbols file creation. > > Yes it is a separate option I agree that?s better .? One has to set? --enable-stripped-pdbs=yes? . > > Best regards , Matthias > > On 2020-02-06 04:48, Magnus Ihse Bursie wrote: > > On 2020-02-06 12:36, Langer, Christoph wrote: > > Hi, > > let me chime in to this discussion at this point. So, first of all, Matthias, > > thanks for your work so far. > > Erik, I fully understand your points and I agree that it's probably a good > > idea to refactor this whole process of creating these different types of > > bundles. > > Our idea is to provide functionality in the build system to add "public" or > > stripped debug files to the delivery image of the JDK. This would provide > > better information in case of crashes and would hence allow for better > > supportability. That's something we'd probably want to enable in > > SapMachine binary distributions. > > I very much support the idea of using these stripped pdb files. It has > > been a long standing issue in hotspot on Windows to not have access to > > stacktraces. > > So, can we get to add a configure option like the one proposed by > > Matthias into the current code base? > > The option should be turned off by default. If it is switched on, the > > images/jdk folder in the build directory will have both, the *stripped.pdb > > files and the standard *.pdb files. However, having *stripped.pdb files > > around should not matter in terms of functionality (for developers and > > testing) as they'd not be used in the presence of the "real" pdb files anyway. > > The actual JDK bundle for delivery would then contain the *stripped.pdb > > files, renamed to *.pdb. And the debug symbols bundle would have the full > > *.pdb files only. Both could then be overlaid as needed. > > I think you raised two concerns. > > One is that you'd rather like to refactor bundling for several reasons. But I > > guess, should you eventually get to your refactoring, it shouldn't be a > > problem to take the functionality of this new option along. > > The other was regarding JMODs. I believe it's correct, that JMODs have > > never carried external debug information. For platforms other than MS > > Windows that's actually not a big problem because debug information can be > > internalized. And jlink has gotten several additions to set flags for stripping > > that data to the right level. I assume if jlinked images on Windows should > > ever be enabled to carry debug information, inclusion of external debug files > > would have to be added to JMODs and jlink. But that's definitely out of scope > > here. > > The argument "jmods have never carried external debug information" just > > doesn't apply here. Neither has the distribution bundles, for the exact > > same reason. You really should not compare these new stripped pdb files > > to the existing debug symbol files, they are different files with > > different purposes. One is meant to be shipped to customers, the other > > is not. You say you want to ship these new stripped pdb files with your > > distribution so that customers have them present when they use your > > distribution. If you then omit these new files from the jmods, any > > customer created jlinked image will not have these new stripped pdb > > files, which IMO is a very weird and unexpected behavior from a customer > > point of view. Jlinking new images is an integral and promoted way of > > using a JDK, so any mismatch between the original JDK distribution and > > what you are able to jlink is a serious discrepancy. > > So, what do you think? What speaks against adding this option (that is off > > by default)? > > My main objective is that you introduce further discrepancies between > > the original distribution JDK image and what's possible to generate > > using jlink from that distribution JDK image. My second objective is > > that the already convoluted bundles creation logic becomes even more > > convoluted. I believe there is a better possible solution in the build > > but it will require a lot more work to figure out. > > All that said, if you still wish to continue, I will stop standing in > > the way. > > While Erik will need to comment on this himself, from my POV this > > looks okay. The conditions are: > > * This is controlled by a separate option (or perhaps even better as a > > new argument to --with-native-debug-symbols), so without this, nothing > > will change. > > I think it needs to be a separate option as it's all orthogonal to the > > main debug symbols file creation. > > * The code need to make sure to separate stripped.pdb and normal pdb > > files, when enabled. > > * And there must not be a heavy penalty in added code complexity. > > /Erik > > /Magnus > > Thanks > > Christoph > > -----Original Message----- > > From: build-dev ? On Behalf Of > > Erik > > Joelsson > > Sent: Donnerstag, 23. Januar 2020 18:49 > > To: Baesken, Matthias ; David Holmes > > ; 'build-dev at openjdk.java.net ' > dev at openjdk.java.net >; 'hotspot-dev at openjdk.java.net ' > dev at openjdk.java.net > > > Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows > > for > > jdk images > > On 2020-01-23 00:03, Baesken, Matthias wrote: > > Hi Erik,? yes true sorry for answering? your comments a bit late . > > If a user runs jlink and includes all the jmods we ship with the JDK, the > > result > > should be essentially equivalent to the original JDK image. The way > > the > > stripped pdb files are included in the bundles sort of at the last > > second of the build here breaks this property. > > I think we should address this in a separate bug/CR . > > Maybe. I realize that my proposal below is quite a big task. But on the > > other hand, I don't think breaking the relationship between the jmods > > and the distribution bundles is on the table really. > > Looking for example? into a Linux? build, I see? a lot of debuginfo files? in > > the > > jdk image (more or less for every shared lib)? . > > But when looking into the jmods? of that jdk image ,? no debuginfo files > > are > > in there ( I checked the java.base jmod). > > So? putting? the? files with debug information into the jmods? seems to > > be > > something that was not done so far? cross platform? (or is there some > > build > > switch? for it that I did not check?) . > > Maybe to keep the jmods as small as possible . > > No, we do not put the debuginfo files in the jmods nor the bundles > > because those are not intended to be shipped to customers. We are > > currently overlaying them into images/jdk in the build output to make > > them available for local debugging. (This is convoluted and I would very > > much like to get away from this practice at some point so that there is > > a 1-1 mapping between images/jdk and bundles/jdk*-bin.tar.gz.) The > > stripped pdb files you are proposing are on the contrary intended for > > shipping to customers (as I understand your proposal) so comparing > > them > > with the debuginfo files is not relevant. > > Now if MS had been kind enough to define a separate file type for the > > stripped pdbs, so that they could live alongside the full pdbs, we > > wouldn't have this issue. The heart of the problem is that only one set > > of files (either stripped or full) can be present and usable in > > images/jdk at a time. We have 2 main uses for images/jdk. > > 1. Developer running and debugging locally > > 2. Serve as the source for generating the distribution bundles > > We currently have one image serving both of these purposes, which is > > already creating complicated and convoluted build steps. To properly > > solve this I would argue for splitting these two apart into two > > different images for the two different purposes. The build procedure > > would then be, first build the images for distribution, only containing > > what should go into each bundle. Then create the developer jdk image > > by > > copying files from the distribution images. On Windows, the first image > > would contain the stripped pdbs and when building the second, those > > would get overwritten with the full pdbs. > > Now that I figured out a working model that would solve a bunch of > > other > > problems as well, I would love to implement it, but I doubt I will have > > time in the near future. > > /Erik > > To properly implement this, care will need to be taken to juggle the > > two > > sets of pdb files around, making sure each build and test use case has > > the correct one in place where and when it's needed. Quite possibly, > > we > > cannot cover all use cases with one build configuration. Developers > > needing the full debug symbols when debugging locally would likely > > need > > to disable the stripped symbols so they get the full symbols > > everywhere. > > Possibly this would need to be the default for debug builds and > > configurable for release builds. > > ? From my limited experience , the developers? do not work with the > > bundles (that? would contain now after my patch the stripped pds)? but > > with > > a "normal" jdk image that? is created my make all. > > Best regards, Matthias > > This still does not address anything in my objection. > > /Erik > > On 2020-01-22 07:46, Baesken, Matthias wrote: > > Hello,? here is an updated version? : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/ > > this one supports a configure switch? "--enable-stripped-pdbs"??? to > > enable > > the feature . > > Best regards, Matthias > > -----Original Message----- > > From: Baesken, Matthias > > Sent: Dienstag, 21. Januar 2020 11:03 > > To: 'David Holmes' ; 'build- > > dev at openjdk.java.net ' ; 'hotspot- > > dev at openjdk.java.net ' > > Subject: RE: RFR: 8237192: Generate stripped/public pdbs on > > Windows > > for > > jdk images > > Hi David ,? yes I think it makes sense to have a configure option for > > this . > > Not everyone would like to have a larger JDK (even it is only a bit > > larger). > From david.holmes at oracle.com Thu Feb 13 04:23:01 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Feb 2020 14:23:01 +1000 Subject: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux In-Reply-To: References: Message-ID: On 12/02/2020 6:31 pm, Baesken, Matthias wrote: > New webrev? : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.3/ > > ???* this time ,? only touching? os_aix.cpp? ( I leave os_bsd.cpp as it > is , might > > ???? make the discussion?? more ?simple ) > > ?? * Added handling of? failing O_CLOEXEC?? ( O_CLOEXEC_is_known_to_work > > ???? == -1? ? case ) Seems okay. One nit: ! int oflag_with_o_cloexec = oflag; ! oflag_with_o_cloexec |= O_CLOEXEC; could just be: ! int oflag_with_o_cloexec = oflag | O_CLOEXEC; Thanks, David > ?Best regards, Matthias > > *From:*Baesken, Matthias > *Sent:* Donnerstag, 6. Februar 2020 08:55 > *To:* 'Thomas St?fe' > *Cc:* David Holmes ; hotspot-dev at openjdk.java.net > *Subject:* RE: RFR: 8237830: support O_CLOEXEC in os::open on other OS > than Linux > > Hi Thomas , the coding was borrowed from linux, see : > > https://hg.openjdk.java.net/jdk/jdk/file/7b57401deb0c/src/hotspot/os/linux/os_linux.cpp#l5616 > > I would be fine with ?always calling fcntl (your suggestion 1) ,? do you > think? it was done otherwise ?back then?? in os_linux because of > performance issues ? > > Probably os::open in os_linux? should be adjusted then? to the? new > ?always call fcntl? approach too, right ? > > Best regards, Matthias > > Hi Matthias: > > + // Validate that the use of the O_CLOEXEC flag on open above worked. > > +? static sig_atomic_t O_CLOEXEC_is_known_to_work = 0; > > +? if (!O_CLOEXEC_is_known_to_work) { > > ???? int flags = ::fcntl(fd, F_GETFD); > > -??? if (flags != -1) > > +??? if (flags != -1) { > > +????? if ((flags & FD_CLOEXEC) != 0) { > > +??????? O_CLOEXEC_is_known_to_work = 1; > > +????? } else { > > ?????? ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); > > ?? } > > -#endif > > +??? } > > +? } > > This is either too complicated or not complicated enough :) > > > if we know for a fact that open(..O_CLOEXEC) did work at least once, we > never again check. > if we know for a fact that open(..O_CLOEXEC) failed to work, we recheck > again and again. Do we expect it to suddenly start working? If yes, > could it also suddenly start failing? Should we then not check always? > > I propose the following change: > > 1) If open(..O_CLOEXEC) could sometimes fail and sometimes not, or we > are just not sure, I would do the fcntl test every time. Or, even just > always set the flag again with fcntl. So, just remove the > O_CLOEXEC_is_known_to_work conditional variable. > > Which would reduce the code to a one liner: > > + ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); > > 2) If you are sure that it either always works or fails, you need a > conditional with three states: unknown, works, does not work. And only > check if unknown. > > I personally would prefer (1) because it is way simpler and fcntl is not > such an expensive operation that we have to save time (also, how often > do we open files anyway?). > > ---- > > Otherwise looks fine. This is a useful fix. > > > Cheers, Thomas > > On Tue, Jan 28, 2020 at 12:32 PM Baesken, Matthias > > wrote: > > New webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.1/ > > (one 'c' got lost somehow in the older webrev) > > Best regards, Matthias > From stefan.karlsson at oracle.com Thu Feb 13 07:59:45 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 13 Feb 2020 08:59:45 +0100 Subject: RFR (XS) 8238909: x86_32 fails gtest:power_of_2 In-Reply-To: References: <36e47275-651c-1bb5-55c9-a7063c62c4be@redhat.com> Message-ID: <3f9c5dee-8b60-4fe4-78cf-6cf7a61e4034@oracle.com> Looks good. StefanK On 2020-02-12 21:30, Aleksey Shipilev wrote: > On 2/12/20 7:45 PM, Aleksey Shipilev wrote: >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8238909 >> >> Fix: >> https://cr.openjdk.java.net/~shade/8238909/webrev.01/ > > Scratch that, Stefan suggested a cleaner version: > https://cr.openjdk.java.net/~shade/8238909/webrev.02/ > > Testing: affected test {x86_64, x86_32}; jdk-submit (clean) > From matthias.baesken at sap.com Thu Feb 13 08:56:11 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 13 Feb 2020 08:56:11 +0000 Subject: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux In-Reply-To: References: Message-ID: Hi David, thanks for the review . I'll follow your " One nit:" advice ?? ! Best Regards, Matthias > > On 12/02/2020 6:31 pm, Baesken, Matthias wrote: > > New webrev? : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.3/ > > > > ???* this time ,? only touching? os_aix.cpp? ( I leave os_bsd.cpp as it > > is , might > > > > ???? make the discussion?? more ?simple ) > > > > ?? * Added handling of? failing O_CLOEXEC?? ( > O_CLOEXEC_is_known_to_work > > > > ???? == -1? ? case ) > > Seems okay. > > One nit: > > ! int oflag_with_o_cloexec = oflag; > ! oflag_with_o_cloexec |= O_CLOEXEC; > > could just be: > > ! int oflag_with_o_cloexec = oflag | O_CLOEXEC; > > Thanks, > David > From claes.redestad at oracle.com Thu Feb 13 11:04:23 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 13 Feb 2020 12:04:23 +0100 Subject: RFR (XS) 8238909: x86_32 fails gtest:power_of_2 In-Reply-To: References: <36e47275-651c-1bb5-55c9-a7063c62c4be@redhat.com> Message-ID: On 2020-02-12 21:30, Aleksey Shipilev wrote: > On 2/12/20 7:45 PM, Aleksey Shipilev wrote: >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8238909 >> >> Fix: >> https://cr.openjdk.java.net/~shade/8238909/webrev.01/ > > Scratch that, Stefan suggested a cleaner version: > https://cr.openjdk.java.net/~shade/8238909/webrev.02/ Looks good, and nice catch. The use of is_power_of_2 was carried forward from a z-only helper method that is 64-bit only, so we all missed that it might be problematic on 32-bit. As a follow-up RFE (that I can volunteer to take care of) I'd suggest moving is_power_of_2_t from align.hpp to powerOfTwo.hpp and invert the dependency. The methods in globalDefinitions are hard to disentangle. And my gut feeling is that we should move away from any operations that uses intptr_t as a type for arithmetic operations in favor of template functions. /Claes From matthias.baesken at sap.com Thu Feb 13 11:17:09 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 13 Feb 2020 11:17:09 +0000 Subject: RFR [XS]: 8239000: handle ContendedPaddingWidth in vm_version_ppc Message-ID: Hello, please review this small adjustment to vm_version_ppc . The ContendedPaddingWidth parameter should be handled in vm_version_ppc like it is done on other architectures (x86/s390). See vm_version_sparc/s390/x86 .cpp for comparision. Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8239000 http://cr.openjdk.java.net/~mbaesken/webrevs/8239000.0/ Best regards, Matthias From stefan.karlsson at oracle.com Thu Feb 13 11:18:24 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 13 Feb 2020 12:18:24 +0100 Subject: RFR (XS) 8238909: x86_32 fails gtest:power_of_2 In-Reply-To: References: <36e47275-651c-1bb5-55c9-a7063c62c4be@redhat.com> Message-ID: On 2020-02-13 12:04, Claes Redestad wrote: > > > On 2020-02-12 21:30, Aleksey Shipilev wrote: >> On 2/12/20 7:45 PM, Aleksey Shipilev wrote: >>> Bug: >>> ?? https://bugs.openjdk.java.net/browse/JDK-8238909 >>> >>> Fix: >>> ?? https://cr.openjdk.java.net/~shade/8238909/webrev.01/ >> >> Scratch that, Stefan suggested a cleaner version: >> ?? https://cr.openjdk.java.net/~shade/8238909/webrev.02/ > > Looks good, and nice catch. The use of is_power_of_2 was carried > forward from a z-only helper method that is 64-bit only, so we all > missed that it might be problematic on 32-bit. > > As a follow-up RFE (that I can volunteer to take care of) I'd suggest > moving is_power_of_2_t from align.hpp to powerOfTwo.hpp and invert the > dependency. I have a patch with this that I'm testing right now. > > The methods in globalDefinitions are hard to disentangle. And my gut > feeling is that we should move away from any operations that uses > intptr_t as a type for arithmetic operations in favor of template > functions. My proposal is to move exact_log2 and exact_log2_log to powerOfTwo.hpp. Their semantics is "log2 of an is_power_of_2 value", so I think it sort-of makes sense to group them together with the other power-of-2 functions. StefanK > > /Claes From claes.redestad at oracle.com Thu Feb 13 11:31:40 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 13 Feb 2020 12:31:40 +0100 Subject: RFR (XS) 8238909: x86_32 fails gtest:power_of_2 In-Reply-To: References: <36e47275-651c-1bb5-55c9-a7063c62c4be@redhat.com> Message-ID: <8a4d7900-4ad7-ef25-eb66-9061cbeceac5@oracle.com> On 2020-02-13 12:18, Stefan Karlsson wrote: >> > > My proposal is to move exact_log2 and exact_log2_log to powerOfTwo.hpp. > Their semantics is "log2 of an is_power_of_2 value", so I think it > sort-of makes sense to group them together with the other power-of-2 > functions. +1 Hopefully there won't be any (or at least not too many) tricky circularity issues due powerOfTwo.hpp depending on various metaprogramming utilities (and count_leading_zeros). /Claes From Alan.Bateman at oracle.com Thu Feb 13 12:24:41 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 13 Feb 2020 12:24:41 +0000 Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images In-Reply-To: References: <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com> <2b92559a-6a1a-119a-1e62-eed6f9d3bfc8@oracle.com> <1989082f-cddf-9d4f-262b-5f9072f46cf7@oracle.com> <9cd2165f-64f0-a096-9b99-aa8bc0798b34@oracle.com> Message-ID: On 12/02/2020 22:16, Erik Joelsson wrote: > Hello Christoph, > > This patch certainly looks better to me, though I agree it's a bit > hackish to have to filter and rename the stripped.pdb files twice, > once for jmods and again for bundles. I think I'm ok with it for now > though. The future improvement I would like to make would be to create > two sets of jdk images, one that contains debug symbols and demos, > which we continue to use for testing, and another which only contains > exactly what we bundle up, including the correctly named top dir. The > latter would be created first and used as input for the former. I > think lots of things would be cleaner then, especially Bundles.gmk. There may be a role for a jlink plugin here. On Linux, jlink --strip-native-debug-symbols runs the objdump tool and sub-options can be used to configure the debuginfo files to keep or exclude. In time I'm sure there will be other plugins like this. -Alan. From glaubitz at physik.fu-berlin.de Thu Feb 13 12:26:55 2020 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Thu, 13 Feb 2020 13:26:55 +0100 Subject: RFR: 8239001: Hotspot build broken on linux-sparc after 8238281 Message-ID: Hello! Hotspot fails to build on linux-sparc after 8238281 due to the redefinition of offset_of() in src/hotspot/share/prims/jvm.cpp [1]: /home/glaubitz/jdk/src/hotspot/cpu/sparc/macroAssembler_sparc.hpp: In static member function 'static int RegistersForDebugging::i_offset(int)': /home/glaubitz/jdk/src/hotspot/cpu/sparc/macroAssembler_sparc.hpp:488:74: error: 'j' cannot appear in a constant-expression 488 | static int i_offset(int j) { return offset_of(RegistersForDebugging, i[j]); } | ^ Since offsetof() can only be used with constant expressions, I have reused the old definition of offset_of() in src/hotspot/cpu/sparc/macroAssembler_sparc.hpp to fix the issue. I couldn't come up with a more elegant solution, but I'm open for suggestions. Please review my change in [2]. Thanks, Adrian > [1] https://bugs.openjdk.java.net/browse/JDK-8239001 > [2] http://cr.openjdk.java.net/~glaubitz/8239001/webrev.00/ -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From stefan.karlsson at oracle.com Thu Feb 13 14:47:11 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 13 Feb 2020 15:47:11 +0100 Subject: RFR: 8183574: Unify the is_power_of_2 functions Message-ID: <12a20d19-5458-c422-2c8b-87a82d6b4f67@oracle.com> Hi all, Please review this patch to consolidate all is_power_of_2 functions into one template function and place it in the existing powerOfTwo.hpp file. To resolve a circular dependency between globalDefinitions.hpp and powerOfTwo.hpp, I also moved exact_log2 and exact_log2_long. These functions take a power-of-two value and returns the log2 value, so I think it makes sense to move them to powerOfTwo.hpp. https://cr.openjdk.java.net/~stefank/8183574/webrev.01 https://bugs.openjdk.java.net/browse/JDK-8183574 Tested with tier1,2,3 Also built locally with {fastdebug,release}x{minimal,zero,aarch64,ppc,s390,arm32,shenandoah} Thanks, StefanK From frederic.parain at oracle.com Thu Feb 13 14:54:28 2020 From: frederic.parain at oracle.com (Frederic Parain) Date: Thu, 13 Feb 2020 09:54:28 -0500 Subject: RFR[XS]: 8238961 Assertion failure in new field layout code when ContendedPaddingWidth == 0. Message-ID: <756AB5B4-B054-447D-9D0A-A8FAB7946745@oracle.com> Please review this very small fix in field layout code. The issue is that option ContendedPaddingWidth can be set to zero, causing an assertion failure when trying to create a padding block of size zero. One site inserting padding blocks was protected against zero padding width, but another site was not. This fix makes the second site robust against this case. CR: https://bugs.openjdk.java.net/browse/JDK-8238961 Webrev: http://cr.openjdk.java.net/~fparain/8238961/webrev.00/index.html Testing: - tests test/hotspot/jtreg/runtime/contended manually run with ContendedPaddingWidth forced to zero (tests fails because they are not designed to support the case where the padding width is zero, but the VM doesn?t crash anymore because of the assert) - tier 1-3 Thank you, Fred From claes.redestad at oracle.com Thu Feb 13 15:08:36 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 13 Feb 2020 16:08:36 +0100 Subject: RFR: 8183574: Unify the is_power_of_2 functions In-Reply-To: <12a20d19-5458-c422-2c8b-87a82d6b4f67@oracle.com> References: <12a20d19-5458-c422-2c8b-87a82d6b4f67@oracle.com> Message-ID: <208c743b-b66b-b564-a961-4f3b9bed6234@oracle.com> On 2020-02-13 15:47, Stefan Karlsson wrote: > Hi all, > > Please review this patch to consolidate all is_power_of_2 functions into > one template function and place it in the existing powerOfTwo.hpp file. > > To resolve a circular dependency between globalDefinitions.hpp and > powerOfTwo.hpp, I also moved exact_log2 and exact_log2_long. These > functions take a power-of-two value and returns the log2 value, so I > think it makes sense to move them to powerOfTwo.hpp. > > https://cr.openjdk.java.net/~stefank/8183574/webrev.01 Looks good! It seems exact_log2 and exact_log2_long could also be replaced by a single template function, but that might be better suited as a follow- up. Thanks! /Claes From harold.seigel at oracle.com Thu Feb 13 15:25:16 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Thu, 13 Feb 2020 10:25:16 -0500 Subject: RFR[XS]: 8238961 Assertion failure in new field layout code when ContendedPaddingWidth == 0. In-Reply-To: <756AB5B4-B054-447D-9D0A-A8FAB7946745@oracle.com> References: <756AB5B4-B054-447D-9D0A-A8FAB7946745@oracle.com> Message-ID: <4afd62ec-f068-cce8-e68a-1789d2ef0c8d@oracle.com> Looks good! Harold On 2/13/2020 9:54 AM, Frederic Parain wrote: > Please review this very small fix in field layout code. > > The issue is that option ContendedPaddingWidth can be set to zero, > causing an assertion failure when trying to create a padding block > of size zero. One site inserting padding blocks was protected against > zero padding width, but another site was not. This fix makes the > second site robust against this case. > > CR: https://bugs.openjdk.java.net/browse/JDK-8238961 > Webrev: http://cr.openjdk.java.net/~fparain/8238961/webrev.00/index.html > > Testing: > - tests test/hotspot/jtreg/runtime/contended manually run with ContendedPaddingWidth > forced to zero (tests fails because they are not designed to support the case > where the padding width is zero, but the VM doesn?t crash anymore because of the > assert) > - tier 1-3 > > Thank you, > > Fred > From frederic.parain at oracle.com Thu Feb 13 15:26:46 2020 From: frederic.parain at oracle.com (Frederic Parain) Date: Thu, 13 Feb 2020 10:26:46 -0500 Subject: RFR[XS]: 8238961 Assertion failure in new field layout code when ContendedPaddingWidth == 0. In-Reply-To: <4afd62ec-f068-cce8-e68a-1789d2ef0c8d@oracle.com> References: <756AB5B4-B054-447D-9D0A-A8FAB7946745@oracle.com> <4afd62ec-f068-cce8-e68a-1789d2ef0c8d@oracle.com> Message-ID: Thank you Harold. Fred > On Feb 13, 2020, at 10:25, Harold Seigel wrote: > > Looks good! > > Harold > > On 2/13/2020 9:54 AM, Frederic Parain wrote: >> Please review this very small fix in field layout code. >> >> The issue is that option ContendedPaddingWidth can be set to zero, >> causing an assertion failure when trying to create a padding block >> of size zero. One site inserting padding blocks was protected against >> zero padding width, but another site was not. This fix makes the >> second site robust against this case. >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8238961 >> Webrev: http://cr.openjdk.java.net/~fparain/8238961/webrev.00/index.html >> >> Testing: >> - tests test/hotspot/jtreg/runtime/contended manually run with ContendedPaddingWidth >> forced to zero (tests fails because they are not designed to support the case >> where the padding width is zero, but the VM doesn?t crash anymore because of the >> assert) >> - tier 1-3 >> >> Thank you, >> >> Fred >> From coleen.phillimore at oracle.com Thu Feb 13 15:39:53 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 13 Feb 2020 10:39:53 -0500 Subject: RFR[XS]: 8238961 Assertion failure in new field layout code when ContendedPaddingWidth == 0. In-Reply-To: <756AB5B4-B054-447D-9D0A-A8FAB7946745@oracle.com> References: <756AB5B4-B054-447D-9D0A-A8FAB7946745@oracle.com> Message-ID: <334c0601-bddc-b8ae-e7f3-9466aa0688b0@oracle.com> Looks good! Coleen On 2/13/20 9:54 AM, Frederic Parain wrote: > Please review this very small fix in field layout code. > > The issue is that option ContendedPaddingWidth can be set to zero, > causing an assertion failure when trying to create a padding block > of size zero. One site inserting padding blocks was protected against > zero padding width, but another site was not. This fix makes the > second site robust against this case. > > CR: https://bugs.openjdk.java.net/browse/JDK-8238961 > Webrev: http://cr.openjdk.java.net/~fparain/8238961/webrev.00/index.html > > Testing: > - tests test/hotspot/jtreg/runtime/contended manually run with ContendedPaddingWidth > forced to zero (tests fails because they are not designed to support the case > where the padding width is zero, but the VM doesn?t crash anymore because of the > assert) > - tier 1-3 > > Thank you, > > Fred > From frederic.parain at oracle.com Thu Feb 13 15:40:51 2020 From: frederic.parain at oracle.com (Frederic Parain) Date: Thu, 13 Feb 2020 10:40:51 -0500 Subject: RFR[XS]: 8238961 Assertion failure in new field layout code when ContendedPaddingWidth == 0. In-Reply-To: <334c0601-bddc-b8ae-e7f3-9466aa0688b0@oracle.com> References: <756AB5B4-B054-447D-9D0A-A8FAB7946745@oracle.com> <334c0601-bddc-b8ae-e7f3-9466aa0688b0@oracle.com> Message-ID: <7E66E525-0EA2-4BB1-9C28-B81290967F17@oracle.com> Thank you Coleen. Fred > On Feb 13, 2020, at 10:39, coleen.phillimore at oracle.com wrote: > > Looks good! > Coleen > > On 2/13/20 9:54 AM, Frederic Parain wrote: >> Please review this very small fix in field layout code. >> >> The issue is that option ContendedPaddingWidth can be set to zero, >> causing an assertion failure when trying to create a padding block >> of size zero. One site inserting padding blocks was protected against >> zero padding width, but another site was not. This fix makes the >> second site robust against this case. >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8238961 >> Webrev: http://cr.openjdk.java.net/~fparain/8238961/webrev.00/index.html >> >> Testing: >> - tests test/hotspot/jtreg/runtime/contended manually run with ContendedPaddingWidth >> forced to zero (tests fails because they are not designed to support the case >> where the padding width is zero, but the VM doesn?t crash anymore because of the >> assert) >> - tier 1-3 >> >> Thank you, >> >> Fred >> > From stefan.karlsson at oracle.com Thu Feb 13 15:55:04 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 13 Feb 2020 16:55:04 +0100 Subject: RFR: 8183574: Unify the is_power_of_2 functions In-Reply-To: <208c743b-b66b-b564-a961-4f3b9bed6234@oracle.com> References: <12a20d19-5458-c422-2c8b-87a82d6b4f67@oracle.com> <208c743b-b66b-b564-a961-4f3b9bed6234@oracle.com> Message-ID: <148be190-a3bb-41e7-0429-80a6b7dbdc4b@oracle.com> On 2020-02-13 16:08, Claes Redestad wrote: > On 2020-02-13 15:47, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to consolidate all is_power_of_2 functions >> into one template function and place it in the existing >> powerOfTwo.hpp file. >> >> To resolve a circular dependency between globalDefinitions.hpp and >> powerOfTwo.hpp, I also moved exact_log2 and exact_log2_long. These >> functions take a power-of-two value and returns the log2 value, so I >> think it makes sense to move them to powerOfTwo.hpp. >> >> https://cr.openjdk.java.net/~stefank/8183574/webrev.01 > > Looks good! Thanks! > > It seems exact_log2 and exact_log2_long could also be replaced by a > single template function, but that might be better suited as a follow- > up. Right. The format specifier needs to be different depending on parameter type, so I skipped trying to unify them for now. StefanK > Thanks! > > /Claes From shade at redhat.com Thu Feb 13 16:07:12 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 13 Feb 2020 17:07:12 +0100 Subject: RFR (XS) 8238909: x86_32 fails gtest:power_of_2 In-Reply-To: References: <36e47275-651c-1bb5-55c9-a7063c62c4be@redhat.com> Message-ID: <7c4388cb-caa1-71a1-fef9-9f31d7a393ea@redhat.com> On 2/13/20 12:04 PM, Claes Redestad wrote: > On 2020-02-12 21:30, Aleksey Shipilev wrote: >> On 2/12/20 7:45 PM, Aleksey Shipilev wrote: >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8238909 >>> >>> Fix: >>> https://cr.openjdk.java.net/~shade/8238909/webrev.01/ >> >> Scratch that, Stefan suggested a cleaner version: >> https://cr.openjdk.java.net/~shade/8238909/webrev.02/ > > Looks good, and nice catch. The use of is_power_of_2 was carried > forward from a z-only helper method that is 64-bit only, so we all > missed that it might be problematic on 32-bit. Thanks, pushed. -- Thanks, -Aleksey From richard.reingruber at sap.com Thu Feb 13 16:45:47 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Thu, 13 Feb 2020 16:45:47 +0000 Subject: RFR(XS) 8239005: [TESTBUG] test/hotspot/jtreg/runtime/StackGuardPages/TestStackGuardPages.java: exeinvoke.c: must initialize static state before calling do_overflow() Message-ID: Hi, please review this small test fix Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239005/webrev.0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8239005 On one of our linuxppc64le boxes the test was failing due to the missing reset of the static variable _rec_count before the first call to do_overflow() [1] in run_native_overflow() [2]. Consequently the recursion limit set at [3] is too high. On our linuxppc64le box we then get a SIGSEGV with SEGV_ACCERR which is reported as test failure. On my linuxx86_64 box I get an SIGSEGV with SEGV_MAPERR which is not reported as error. With the fix we don't get a second SIGSEGV. Maybe a second SIGSEGV should be reported as failure...? I tested on linuxx86_64 and linuxppc64le Many more details to be found in the bug. Thanks, Richard. [1] definition of do_overflow() http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c#l128 [2] _rec_count is not reset to 0 before call to do_overflow() http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c#l168 [3] recursion limit is set too high http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c#l191 From igor.ignatyev at oracle.com Thu Feb 13 16:50:14 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 13 Feb 2020 08:50:14 -0800 Subject: RFR(S) : 8238943: switch to jtreg 5.0 Message-ID: <02927010-0642-47C0-8749-1EAC90CC90FB@oracle.com> http://cr.openjdk.java.net/~iignatyev//8238943/webrev.00 > 10 lines changed: 1 ins; 0 del; 9 mod; Hi all, could you please review the patch which changes jtreg version used in jdk/jdk to the latest and greatest -- jtreg 5.0? and as (recently became) usually, this patch also bumps requiredVersion in all the jtreg test suites in order to reduce possible discrepancy in results. JBS: https://bugs.openjdk.java.net/browse/JDK-8238943 webrev: http://cr.openjdk.java.net/~iignatyev/8238943/webrev.00 testing: tier1-4 Thanks, -- Igor From jonathan.gibbons at oracle.com Thu Feb 13 17:04:12 2020 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 13 Feb 2020 09:04:12 -0800 (PST) Subject: RFR(S) : 8238943: switch to jtreg 5.0 In-Reply-To: <02927010-0642-47C0-8749-1EAC90CC90FB@oracle.com> References: <02927010-0642-47C0-8749-1EAC90CC90FB@oracle.com> Message-ID: Igor, The change to langtools/TEST.ROOT is OK. That being said, there are some old entries there which could also be cleaned up (separately?)? These lines probably date from the development of JDK 9 and the evolution of Project Jigsaw. > 20 # Use new module options > 21 useNewOptions=true > 22 > 23 # Use --patch-module instead of -Xmodule: > 24 useNewPatchModule=true -- Jon On 2/13/20 8:50 AM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8238943/webrev.00 >> 10 lines changed: 1 ins; 0 del; 9 mod; > Hi all, > > could you please review the patch which changes jtreg version used in jdk/jdk to the latest and greatest -- jtreg 5.0? and as (recently became) usually, this patch also bumps requiredVersion in all the jtreg test suites in order to reduce possible discrepancy in results. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8238943 > webrev: http://cr.openjdk.java.net/~iignatyev/8238943/webrev.00 > testing: tier1-4 > > Thanks, > -- Igor > From erik.joelsson at oracle.com Thu Feb 13 17:23:28 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 13 Feb 2020 09:23:28 -0800 Subject: RFR(S) : 8238943: switch to jtreg 5.0 In-Reply-To: <02927010-0642-47C0-8749-1EAC90CC90FB@oracle.com> References: <02927010-0642-47C0-8749-1EAC90CC90FB@oracle.com> Message-ID: <4b74eb1e-e634-93a7-4b3d-e534aca985bc@oracle.com> Looks good, but could you change the "version" field to "5.0", it should work now. /Erik On 2020-02-13 08:50, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8238943/webrev.00 >> 10 lines changed: 1 ins; 0 del; 9 mod; > Hi all, > > could you please review the patch which changes jtreg version used in jdk/jdk to the latest and greatest -- jtreg 5.0? and as (recently became) usually, this patch also bumps requiredVersion in all the jtreg test suites in order to reduce possible discrepancy in results. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8238943 > webrev: http://cr.openjdk.java.net/~iignatyev/8238943/webrev.00 > testing: tier1-4 > > Thanks, > -- Igor > From patricio.chilano.mateo at oracle.com Thu Feb 13 17:47:08 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Thu, 13 Feb 2020 14:47:08 -0300 Subject: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant In-Reply-To: References: Message-ID: <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com> Hi Richard, I?m only commenting on the handshake changes. I see that operation VM_EnterInterpOnlyMode can be called inside operation VM_SetFramePop which also allows nested operations. Here is a comment in VM_SetFramePop definition: // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. So if we change VM_EnterInterpOnlyMode to be a handshake, then now we could have a handshake inside a safepoint operation. The issue I see there is that at the end of the handshake the polling page of the target thread could be disarmed. So if the target thread happens to be in a blocked state just transiently and wakes up then it will not stop for the ongoing safepoint. Maybe I can file an RFE to assert that the polling page is armed at the beginning of disarm_safepoint(). I think one option could be to remove SafepointMechanism::disarm_if_needed() in HandshakeState::clear_handshake() and let each JavaThread disarm itself for the handshake case. Alternatively I think you could do something similar to what we do in Deoptimization::deoptimize_all_marked(): ? EnterInterpOnlyModeClosure hs; ? if (SafepointSynchronize::is_at_safepoint()) { ??? hs.do_thread(state->get_thread()); ? } else { ??? Handshake::execute(&hs, state->get_thread()); ? } (you could pass ?EnterInterpOnlyModeClosure? directly to the HandshakeClosure() constructor) I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is always called in a nested operation or just sometimes. Thanks, Patricio On 2/12/20 7:23 AM, Reingruber, Richard wrote: > // Repost including hotspot runtime and gc lists. > // Dean Long suggested to do so, because the enhancement replaces a vm operation > // with a handshake. > // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html > > Hi, > > could I please get reviews for this small enhancement in hotspot's jvmti implementation: > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8238585 > > The change avoids making all compiled methods on stack not_entrant when switching a java thread to > interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack. > > Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations. > > Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. > > Thanks, Richard. > > See also my question if anyone knows a reason for making the compiled methods not_entrant: > http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html From igor.ignatyev at oracle.com Thu Feb 13 18:08:54 2020 From: igor.ignatyev at oracle.com (Igor Ignatev) Date: Thu, 13 Feb 2020 10:08:54 -0800 Subject: RFR(S) : 8238943: switch to jtreg 5.0 In-Reply-To: <4b74eb1e-e634-93a7-4b3d-e534aca985bc@oracle.com> References: <4b74eb1e-e634-93a7-4b3d-e534aca985bc@oracle.com> Message-ID: Oh, I?m sorry I actually changed it to 5.0 when were (re)doing testing, and apparently forgot to replace the webrev, the right is http://cr.openjdk.java.net/~iignatyev//8238943/webrev.01 ; with version field value being the only difference b/w .00 and .01 Thanks, ? Igor > On Feb 13, 2020, at 9:23 AM, Erik Joelsson wrote: > > ?Looks good, but could you change the "version" field to "5.0", it should work now. > > /Erik > >>> On 2020-02-13 08:50, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8238943/webrev.00 >>> 10 lines changed: 1 ins; 0 del; 9 mod; >> Hi all, >> could you please review the patch which changes jtreg version used in jdk/jdk to the latest and greatest -- jtreg 5.0? and as (recently became) usually, this patch also bumps requiredVersion in all the jtreg test suites in order to reduce possible discrepancy in results. >> JBS: https://bugs.openjdk.java.net/browse/JDK-8238943 >> webrev: http://cr.openjdk.java.net/~iignatyev/8238943/webrev.00 >> testing: tier1-4 >> Thanks, >> -- Igor From huizhe.wang at oracle.com Thu Feb 13 18:19:04 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 13 Feb 2020 10:19:04 -0800 Subject: RFR(S) : 8238943: switch to jtreg 5.0 In-Reply-To: References: <4b74eb1e-e634-93a7-4b3d-e534aca985bc@oracle.com> Message-ID: +1 for the change to test/jaxp/TEST.ROOT. Best, Joe On 2/13/20 10:08 AM, Igor Ignatev wrote: > Oh, I?m sorry I actually changed it to 5.0 when were (re)doing testing, and apparently forgot to replace the webrev, the right is http://cr.openjdk.java.net/~iignatyev//8238943/webrev.01 ; with version field value being the only difference b/w .00 and .01 > > Thanks, > ? Igor > >> On Feb 13, 2020, at 9:23 AM, Erik Joelsson wrote: >> >> ?Looks good, but could you change the "version" field to "5.0", it should work now. >> >> /Erik >> >>>> On 2020-02-13 08:50, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev//8238943/webrev.00 >>>> 10 lines changed: 1 ins; 0 del; 9 mod; >>> Hi all, >>> could you please review the patch which changes jtreg version used in jdk/jdk to the latest and greatest -- jtreg 5.0? and as (recently became) usually, this patch also bumps requiredVersion in all the jtreg test suites in order to reduce possible discrepancy in results. >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238943 >>> webrev: http://cr.openjdk.java.net/~iignatyev/8238943/webrev.00 >>> testing: tier1-4 >>> Thanks, >>> -- Igor From erik.joelsson at oracle.com Thu Feb 13 18:29:01 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 13 Feb 2020 10:29:01 -0800 Subject: RFR(S) : 8238943: switch to jtreg 5.0 In-Reply-To: References: <4b74eb1e-e634-93a7-4b3d-e534aca985bc@oracle.com> Message-ID: Looks good. /Erik On 2020-02-13 10:08, Igor Ignatev wrote: > Oh, I?m sorry I actually changed it to 5.0 when were (re)doing testing, and apparently forgot to replace the webrev, the right is http://cr.openjdk.java.net/~iignatyev//8238943/webrev.01 ; with version field value being the only difference b/w .00 and .01 > > Thanks, > ? Igor > >> On Feb 13, 2020, at 9:23 AM, Erik Joelsson wrote: >> >> ?Looks good, but could you change the "version" field to "5.0", it should work now. >> >> /Erik >> >>>> On 2020-02-13 08:50, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev//8238943/webrev.00 >>>> 10 lines changed: 1 ins; 0 del; 9 mod; >>> Hi all, >>> could you please review the patch which changes jtreg version used in jdk/jdk to the latest and greatest -- jtreg 5.0? and as (recently became) usually, this patch also bumps requiredVersion in all the jtreg test suites in order to reduce possible discrepancy in results. >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238943 >>> webrev: http://cr.openjdk.java.net/~iignatyev/8238943/webrev.00 >>> testing: tier1-4 >>> Thanks, >>> -- Igor From kim.barrett at oracle.com Thu Feb 13 19:35:19 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 13 Feb 2020 14:35:19 -0500 Subject: RFR: 8239001: Hotspot build broken on linux-sparc after 8238281 In-Reply-To: References: Message-ID: > On Feb 13, 2020, at 7:26 AM, John Paul Adrian Glaubitz wrote: > > Hello! > > Hotspot fails to build on linux-sparc after 8238281 due to the redefinition > of offset_of() in src/hotspot/share/prims/jvm.cpp [1]: > > /home/glaubitz/jdk/src/hotspot/cpu/sparc/macroAssembler_sparc.hpp: In static member function 'static int RegistersForDebugging::i_offset(int)': > /home/glaubitz/jdk/src/hotspot/cpu/sparc/macroAssembler_sparc.hpp:488:74: error: 'j' cannot appear in a constant-expression > 488 | static int i_offset(int j) { return offset_of(RegistersForDebugging, i[j]); } > | ^ > > Since offsetof() can only be used with constant expressions, I have reused > the old definition of offset_of() in src/hotspot/cpu/sparc/macroAssembler_sparc.hpp > to fix the issue. > > I couldn't come up with a more elegant solution, but I'm open for suggestions. > > Please review my change in [2]. > > Thanks, > Adrian > >> [1] https://bugs.openjdk.java.net/browse/JDK-8239001 >> [2] http://cr.openjdk.java.net/~glaubitz/8239001/webrev.00/ I don't think this is the right way to address this problem. I think the JDK-8238281 change to offset_of and the associated addition of -Wno-invalid-offsetof to the build configuration was a mistake, and should be reverted. Those changes seem unrelated to the purpose of JDK-8238281, which was to raise the minimum acceptable gcc version. From frederic.parain at oracle.com Thu Feb 13 20:28:24 2020 From: frederic.parain at oracle.com (Frederic Parain) Date: Thu, 13 Feb 2020 15:28:24 -0500 Subject: RFR[S]: 8239014 -XX:-UseEmptySlotsInSupers sometime fails to reproduce the layout of the old code Message-ID: <4C24883A-9EED-45E2-94C3-4AD91ACBAEC7@oracle.com> Please review this small change that fix the behavior of-XX: -UseEmptySlotsInSupers. CR: https://bugs.openjdk.java.net/browse/JDK-8239014 Webrev: http://cr.openjdk.java.net/~fparain/8239014/webrev.00/index.html Thank you, Fred From vladimir.x.ivanov at oracle.com Thu Feb 13 21:44:36 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 14 Feb 2020 00:44:36 +0300 Subject: [15] RFR (S): 8238696: x86: Enumerate all detected CPU features in VM_Version feature string In-Reply-To: References: <98506aca-4a47-a337-a707-19c2a424ca01@oracle.com> <3b882679-b8e1-8e2c-fd8e-261d48224001@oracle.com> <47bf9452-fb75-0293-0a54-3df9e3a3a716@oracle.com> <2267f17c-2f81-eec3-d31f-4e031e34b615@oracle.com> Message-ID: <274053d4-bdef-438f-742f-b5ef1ac2e65a@oracle.com> >> ?? http://cr.openjdk.java.net/~vlivanov/8238696/webrev.03 > > Functional part seems fine. Thanks, David. >> Considering some tests have hard-coded feature names, I decided to >> enhance the new test and check that all reported features are known >> (at least, on x86). > > Not sure about the test - I don't know enough about which features are > "always" presents on x86/x64. I'd suggest using @requires x86 rather > than the internal Platform checks. The test does a different check: the full list of known CPU features is hard-coded into the test and the test checks that all features reported by CPUInfo.getFeatures() are present in the list. So, if a new feature detection is added into the JVM, the test has to be adjusted as well. The test does look a bit artificial, but I found it convenient when playing with the functionality. Regarding @requires, I'd like to keep Platform.is* checks: (1) the test has cross-platform case; 73 for (String feature : features) { 74 assertTrue(CPUInfo.hasFeature(feature), feature); 75 } (2) it's easy to extend the test and enumerate known features for other platforms Best regards, Vladimir Ivanov >>>> On 10.02.2020 17:12, Vladimir Ivanov wrote: >>>>> Thanks for taking a look, David. >>>>> >>>>> Updated webrev: >>>>> ?? http://cr.openjdk.java.net/~vlivanov/8238696/webrev.01/ >>>>> >>>>>>> http://cr.openjdk.java.net/~vlivanov/8238696/webrev.00/ >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8238696 >>>>>>> >>>>>>> Abstract_VM_Version::features_string() doesn't contain all the >>>>>>> CPU features detected by the JVM on x86 (for example, for AVX-512 >>>>>>> only AVX-512F is included as "evex"). >>>>>>> >>>>>>> The fix adds missing features to the list [1]. >>>>>> >>>>>> The function naming that you have changed for >>>>>> supports_avx512_vpclmulqdq was explicitly requested by Vladimir K.: >>>>>> >>>>>> http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-December/036326.html >>>>>> >>>>>> >>>>>> If anything I would suggest supports_vpopcntdq() should have the >>>>>> avx512 part added. >>>>> >>>>> Makes sense. Also, decided to rename VNNI, VAES, and VBMI2. They >>>>> are listed in the manual as part of AVX512. >>>>> >>>>>> The actual expansion of the feature string seems okay. It does >>>>>> make me wonder why these items have been left out previously. >>>>> >>>>> I believe it was just overlooked. Added a comment near cpu feature >>>>> constant declarations. >>>>> >>>>> Also, introduced a check to ensure the temporary on-stack buffer is >>>>> large enough. >>>>> >>>>> Best regards, >>>>> Vladimir Ivanov >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> Testing: manual (-Xlog:os+cpu=trace output on different hardware), >>>>>>> ????????? hs-precheckin-comp, hs-tier1, hs-tier2 >>>>>>> >>>>>>> Best regards, >>>>>>> Vladimir Ivanov >>>>>>> >>>>>>> [1] $ java -Xlog:os+cpu=trace -XX:UseAVX=3 -version >>>>>>> >>>>>>> ... >>>>>>> [0.010s][info][os,cpu] UseSSE=4? UseAVX=3? UseAES=1 >>>>>>> MaxVectorSize=64 >>>>>>> ... >>>>>>> [0.011s][info][os,cpu] CPU:total 8 (initial active 8) (4 cores >>>>>>> per cpu, 2 threads per core) family 6 model 85 stepping 4, cmov, >>>>>>> cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, >>>>>>> vzeroupper, avx, avx2, aes, clmul, erms, rtm, 3dnowpref, lzcnt, >>>>>>> ht, tsc, bmi1, bmi2, adx, avx512f, avx512dq, avx512cd, avx512bw, >>>>>>> avx512vl, fma, clflush, clflushopt, clwb >>>>>>> ... >>>>>>> [0.011s][info][os,cpu] flags??????? : fpu vme de pse tsc msr pae >>>>>>> mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse >>>>>>> sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon >>>>>>> rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx >>>>>>> ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt >>>>>>> tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm >>>>>>> abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb >>>>>>> stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust >>>>>>> bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq >>>>>>> rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl >>>>>>> xsaveopt xsavec xgetbv1 xsaves arat umip pku ospke md_clear >>>>>>> ... From kim.barrett at oracle.com Thu Feb 13 21:54:52 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 13 Feb 2020 16:54:52 -0500 Subject: RFR: 8183574: Unify the is_power_of_2 functions In-Reply-To: <12a20d19-5458-c422-2c8b-87a82d6b4f67@oracle.com> References: <12a20d19-5458-c422-2c8b-87a82d6b4f67@oracle.com> Message-ID: <67BFC4CB-C428-4F9A-9764-CB9FFF0ABB73@oracle.com> > On Feb 13, 2020, at 9:47 AM, Stefan Karlsson wrote: > > Hi all, > > Please review this patch to consolidate all is_power_of_2 functions into one template function and place it in the existing powerOfTwo.hpp file. > > To resolve a circular dependency between globalDefinitions.hpp and powerOfTwo.hpp, I also moved exact_log2 and exact_log2_long. These functions take a power-of-two value and returns the log2 value, so I think it makes sense to move them to powerOfTwo.hpp. > > https://cr.openjdk.java.net/~stefank/8183574/webrev.01 > https://bugs.openjdk.java.net/browse/JDK-8183574 > > Tested with tier1,2,3 > Also built locally with {fastdebug,release}x{minimal,zero,aarch64,ppc,s390,arm32,shenandoah} > > Thanks, > StefanK That was a lot(!) of files to click through. Looks good. ------------------------------------------------------------------------------ src/hotspot/share/code/vtableStubs.cpp 132 assert(is_power_of_2(int(N)), "N must be a power of 2"); Conversion of N to int is because it's defined via enum (because that's what we used to do). Better would be to no longer define N that way. (Also the related mask.) This could be a followup cleanup if that's more convenient. And I don't need a new webrev if you change this now. ------------------------------------------------------------------------------ I also noticed some probably unnecessary casts here and there. I think those are better dealt with separately (perhaps as the code is touched for other reasons), rather than adding clutter to this change. From magnus.ihse.bursie at oracle.com Thu Feb 13 22:21:19 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 13 Feb 2020 23:21:19 +0100 Subject: RFR: 8239001: Hotspot build broken on linux-sparc after 8238281 In-Reply-To: References: Message-ID: <7EC523AF-BD5E-4DE3-A480-4C88FD823E47@oracle.com> I searched the code base for mentions of old gcc workarounds, and tried to remedy them now that they were not needed. If it turns out that they still were needed, by all means, revert that part of the patch. /Magnus 13 feb. 2020 kl. 20:35 skrev Kim Barrett : >> On Feb 13, 2020, at 7:26 AM, John Paul Adrian Glaubitz wrote: >> >> Hello! >> >> Hotspot fails to build on linux-sparc after 8238281 due to the redefinition >> of offset_of() in src/hotspot/share/prims/jvm.cpp [1]: >> >> /home/glaubitz/jdk/src/hotspot/cpu/sparc/macroAssembler_sparc.hpp: In static member function 'static int RegistersForDebugging::i_offset(int)': >> /home/glaubitz/jdk/src/hotspot/cpu/sparc/macroAssembler_sparc.hpp:488:74: error: 'j' cannot appear in a constant-expression >> 488 | static int i_offset(int j) { return offset_of(RegistersForDebugging, i[j]); } >> | ^ >> >> Since offsetof() can only be used with constant expressions, I have reused >> the old definition of offset_of() in src/hotspot/cpu/sparc/macroAssembler_sparc.hpp >> to fix the issue. >> >> I couldn't come up with a more elegant solution, but I'm open for suggestions. >> >> Please review my change in [2]. >> >> Thanks, >> Adrian >> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8239001 >>> [2] http://cr.openjdk.java.net/~glaubitz/8239001/webrev.00/ > > I don't think this is the right way to address this problem. > > I think the JDK-8238281 change to offset_of and the associated > addition of -Wno-invalid-offsetof to the build configuration was a > mistake, and should be reverted. Those changes seem unrelated to the > purpose of JDK-8238281, which was to raise the minimum acceptable gcc > version. > From stefan.karlsson at oracle.com Thu Feb 13 22:24:08 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 13 Feb 2020 23:24:08 +0100 Subject: RFR: 8183574: Unify the is_power_of_2 functions In-Reply-To: <67BFC4CB-C428-4F9A-9764-CB9FFF0ABB73@oracle.com> References: <12a20d19-5458-c422-2c8b-87a82d6b4f67@oracle.com> <67BFC4CB-C428-4F9A-9764-CB9FFF0ABB73@oracle.com> Message-ID: <4c4409f9-abf4-cf65-1652-86e3297bedb7@oracle.com> On 2020-02-13 22:54, Kim Barrett wrote: >> On Feb 13, 2020, at 9:47 AM, Stefan Karlsson wrote: >> >> Hi all, >> >> Please review this patch to consolidate all is_power_of_2 functions into one template function and place it in the existing powerOfTwo.hpp file. >> >> To resolve a circular dependency between globalDefinitions.hpp and powerOfTwo.hpp, I also moved exact_log2 and exact_log2_long. These functions take a power-of-two value and returns the log2 value, so I think it makes sense to move them to powerOfTwo.hpp. >> >> https://cr.openjdk.java.net/~stefank/8183574/webrev.01 >> https://bugs.openjdk.java.net/browse/JDK-8183574 >> >> Tested with tier1,2,3 >> Also built locally with {fastdebug,release}x{minimal,zero,aarch64,ppc,s390,arm32,shenandoah} >> >> Thanks, >> StefanK > That was a lot(!) of files to click through. > > Looks good. Thanks. > > ------------------------------------------------------------------------------ > src/hotspot/share/code/vtableStubs.cpp > 132 assert(is_power_of_2(int(N)), "N must be a power of 2"); > > Conversion of N to int is because it's defined via enum (because > that's what we used to do). Better would be to no longer define N > that way. (Also the related mask.) I found references that using enums here would be valid in C++11. > > This could be a followup cleanup if that's more convenient. And I > don't need a new webrev if you change this now. OK. I'd prefer to leave this as is, because it's the minimal patch needed to get this to compile. > > ------------------------------------------------------------------------------ > > I also noticed some probably unnecessary casts here and there. I > think those are better dealt with separately (perhaps as the code is > touched for other reasons), rather than adding clutter to this change. OK. Thanks for the review, StefanK > > From david.holmes at oracle.com Thu Feb 13 22:43:32 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 14 Feb 2020 08:43:32 +1000 Subject: RFR(XS) 8239005: [TESTBUG] test/hotspot/jtreg/runtime/StackGuardPages/TestStackGuardPages.java: exeinvoke.c: must initialize static state before calling do_overflow() In-Reply-To: References: Message-ID: <6fc2b655-e7f5-9f15-7bb1-46625492fa20@oracle.com> Hi Richard, This seems fine to me. Good catch! Thanks, David On 14/02/2020 2:45 am, Reingruber, Richard wrote: > Hi, > > please review this small test fix > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239005/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8239005 > > On one of our linuxppc64le boxes the test was failing due to the missing reset of the static > variable _rec_count before the first call to do_overflow() [1] in run_native_overflow() [2]. > Consequently the recursion limit set at [3] is too high. On our linuxppc64le box we then get a > SIGSEGV with SEGV_ACCERR which is reported as test failure. > > On my linuxx86_64 box I get an SIGSEGV with SEGV_MAPERR which is not reported as error. > > With the fix we don't get a second SIGSEGV. Maybe a second SIGSEGV should be reported as failure...? > > I tested on linuxx86_64 and linuxppc64le > > Many more details to be found in the bug. > > Thanks, Richard. > > [1] definition of do_overflow() > http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c#l128 > > [2] _rec_count is not reset to 0 before call to do_overflow() > http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c#l168 > > [3] recursion limit is set too high > http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c#l191 > From david.holmes at oracle.com Thu Feb 13 22:53:15 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 14 Feb 2020 08:53:15 +1000 Subject: RFR[S]: 8239014 -XX:-UseEmptySlotsInSupers sometime fails to reproduce the layout of the old code In-Reply-To: <4C24883A-9EED-45E2-94C3-4AD91ACBAEC7@oracle.com> References: <4C24883A-9EED-45E2-94C3-4AD91ACBAEC7@oracle.com> Message-ID: <3d807131-157c-b6ff-0b64-a2bba8e03175@oracle.com> Hi Fred, Can you add a testcase for this? I can't really evaluate the change just by looking at the code. Thanks, David On 14/02/2020 6:28 am, Frederic Parain wrote: > Please review this small change that fix the behavior of-XX: -UseEmptySlotsInSupers. > > CR: https://bugs.openjdk.java.net/browse/JDK-8239014 > Webrev: http://cr.openjdk.java.net/~fparain/8239014/webrev.00/index.html > > Thank you, > > Fred > From vladimir.kozlov at oracle.com Thu Feb 13 23:36:00 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 13 Feb 2020 15:36:00 -0800 Subject: [15] RFR (S): 8238696: x86: Enumerate all detected CPU features in VM_Version feature string In-Reply-To: <2267f17c-2f81-eec3-d31f-4e031e34b615@oracle.com> References: <98506aca-4a47-a337-a707-19c2a424ca01@oracle.com> <3b882679-b8e1-8e2c-fd8e-261d48224001@oracle.com> <47bf9452-fb75-0293-0a54-3df9e3a3a716@oracle.com> <2267f17c-2f81-eec3-d31f-4e031e34b615@oracle.com> Message-ID: <958b4c49-54ca-8243-4baa-bb2deab5e24f@oracle.com> Changes looks good to me. I wish we have a definition of CPU features which specify name and enumeration so that it will be printed automatically, if supported, by VM_Version::get_processor_features(). Counting by hand how many %s I have to add is bug prone. We also need to check that we have enough bits for enumeration. It could be a starter RFE. Thanks, Vladimir K On 2/11/20 5:58 AM, Vladimir Ivanov wrote: > >> Now I'm wondering about using the dot as a terminator. Doesn't the return value from jio_snprintf >> tell you whether you exceeded the buffer? > > Yes, very good point. How about the following? > > ? http://cr.openjdk.java.net/~vlivanov/8238696/webrev.03 > > Considering some tests have hard-coded feature names, I decided to enhance the new test and check > that all reported features are known (at least, on x86). > > Best regards, > Vladimir Ivanov > >>> On 10.02.2020 17:12, Vladimir Ivanov wrote: >>>> Thanks for taking a look, David. >>>> >>>> Updated webrev: >>>> ?? http://cr.openjdk.java.net/~vlivanov/8238696/webrev.01/ >>>> >>>>>> http://cr.openjdk.java.net/~vlivanov/8238696/webrev.00/ >>>>>> https://bugs.openjdk.java.net/browse/JDK-8238696 >>>>>> >>>>>> Abstract_VM_Version::features_string() doesn't contain all the CPU features detected by the >>>>>> JVM on x86 (for example, for AVX-512 only AVX-512F is included as "evex"). >>>>>> >>>>>> The fix adds missing features to the list [1]. >>>>> >>>>> The function naming that you have changed for supports_avx512_vpclmulqdq was explicitly >>>>> requested by Vladimir K.: >>>>> >>>>> http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-December/036326.html >>>>> >>>>> If anything I would suggest supports_vpopcntdq() should have the avx512 part added. >>>> >>>> Makes sense. Also, decided to rename VNNI, VAES, and VBMI2. They are listed in the manual as >>>> part of AVX512. >>>> >>>>> The actual expansion of the feature string seems okay. It does make me wonder why these items >>>>> have been left out previously. >>>> >>>> I believe it was just overlooked. Added a comment near cpu feature constant declarations. >>>> >>>> Also, introduced a check to ensure the temporary on-stack buffer is large enough. >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> Testing: manual (-Xlog:os+cpu=trace output on different hardware), >>>>>> ????????? hs-precheckin-comp, hs-tier1, hs-tier2 >>>>>> >>>>>> Best regards, >>>>>> Vladimir Ivanov >>>>>> >>>>>> [1] $ java -Xlog:os+cpu=trace -XX:UseAVX=3 -version >>>>>> >>>>>> ... >>>>>> [0.010s][info][os,cpu] UseSSE=4? UseAVX=3? UseAES=1? MaxVectorSize=64 >>>>>> ... >>>>>> [0.011s][info][os,cpu] CPU:total 8 (initial active 8) (4 cores per cpu, 2 threads per core) >>>>>> family 6 model 85 stepping 4, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, >>>>>> popcnt, vzeroupper, avx, avx2, aes, clmul, erms, rtm, 3dnowpref, lzcnt, ht, tsc, bmi1, bmi2, >>>>>> adx, avx512f, avx512dq, avx512cd, avx512bw, avx512vl, fma, clflush, clflushopt, clwb >>>>>> ... >>>>>> [0.011s][info][os,cpu] flags??????? : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca >>>>>> cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc >>>>>> arch_perfmon rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 >>>>>> pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor >>>>>> lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi >>>>>> flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f >>>>>> avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 >>>>>> xsaves arat umip pku ospke md_clear >>>>>> ... From vladimir.kozlov at oracle.com Fri Feb 14 00:56:26 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 13 Feb 2020 16:56:26 -0800 Subject: [15] RFR (M): 8238681: Make -XX:UseSSE flag x86-specific In-Reply-To: References: Message-ID: http://cr.openjdk.java.net/~vlivanov/8238681/webrev.01/ gives me "404 - Not Found" Looking on webrev.00. compiler/c1/Test6579789.java and compiler/c1/Test6855215.java use UseSSE flag without @requires. Graal/JVMCI supports only 64 bit x86 and access UseSSE only in 'AMD64' specific files. So it is fine. Otherwise changes seems good. Thanks, Vladimir On 2/10/20 6:19 AM, Vladimir Ivanov wrote: > http://cr.openjdk.java.net/~vlivanov/8238681/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8238681 > > Turn UseSSE into x86-specific flag since it doesn't have any meaning on non-x86 platforms. > > As a cleanup, I made all the code which assumes UseSSE<2 x86-32 specific. (x86-64 ABI requires SSE2.) > > Also, as a precaution, I made UseSSE obsolete on non-x86 platforms. > > Testing: hs-precheckin-comp, tier1-5, linux x86-32 fastdebug build. > > Thanks! > > Best regards, > Vladimir Ivanov From kim.barrett at oracle.com Fri Feb 14 01:10:42 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 13 Feb 2020 20:10:42 -0500 Subject: RFR: 8239001: Hotspot build broken on linux-sparc after 8238281 In-Reply-To: <7EC523AF-BD5E-4DE3-A480-4C88FD823E47@oracle.com> References: <7EC523AF-BD5E-4DE3-A480-4C88FD823E47@oracle.com> Message-ID: > On Feb 13, 2020, at 5:21 PM, Magnus Ihse Bursie wrote: > > I searched the code base for mentions of old gcc workarounds, and tried to remedy them now that they were not needed. > > If it turns out that they still were needed, by all means, revert that part of the patch. I think the offset_of definition in question was not an "old gcc workaround". offsetof with a non-POD type is undefined behavior in C++98. C++11 relaxed the restriction to allow standard-layout types. C++17 changed offsetof with a non-standard-layout type to be "conditionally supported", which is more or less effectively UB for portable code. Contrary to what was said in the RFR for JDK-8238281, we should not be considering eliminating offset_of and just using offsetof directly. offset_of is the HotSpot portability shim to let us knowingly do something that is potentially problematic but we claim we know what we're doing. The fact that gcc warns about problematic offsetof uses (unless -Wno-invalid-offsetof) is a good thing, because it helps keep us honest about using that portability shim. (This is similar in concept to JDK-8214976.) (There are a couple of direct uses of offsetof in os_perf_solaris.cpp.) Then there's the separate issue that the failing code in the RegistersForDebugging class is using offset_of / offsetof in an invalid way, since the result is supposed to be a constant expression and that's impossible because these uses involve the non-constant "j" argument. I think that can be fixed by something like /* Add this in RegistersForDebugging */ private: static const RegistersForDebugging& _dummy; // not ODR-used so not defined /* Change offset functions based on this model. */ static int i_offset(int j) { return offset_of(RegistersForDebugging, i) + j * (sizeof _dummy.i[0]); } Of course, reverting to the gcc implementation of offset_of also "fixes" this problem, because that version of offset_of happens to "accidentally" be more lenient. From christoph.langer at sap.com Fri Feb 14 09:12:34 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 14 Feb 2020 09:12:34 +0000 Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images In-Reply-To: References: <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com> <2b92559a-6a1a-119a-1e62-eed6f9d3bfc8@oracle.com> <1989082f-cddf-9d4f-262b-5f9072f46cf7@oracle.com> <9cd2165f-64f0-a096-9b99-aa8bc0798b34@oracle.com> Message-ID: Hi Alan, > On 12/02/2020 22:16, Erik Joelsson wrote: > > Hello Christoph, > > > > This patch certainly looks better to me, though I agree it's a bit > > hackish to have to filter and rename the stripped.pdb files twice, > > once for jmods and again for bundles. I think I'm ok with it for now > > though. The future improvement I would like to make would be to create > > two sets of jdk images, one that contains debug symbols and demos, > > which we continue to use for testing, and another which only contains > > exactly what we bundle up, including the correctly named top dir. The > > latter would be created first and used as input for the former. I > > think lots of things would be cleaner then, especially Bundles.gmk. > There may be a role for a jlink plugin here. On Linux, jlink > --strip-native-debug-symbols runs the objdump tool and sub-options can > be used to configure the debuginfo files to keep or exclude. In time I'm > sure there will be other plugins like this. Yes, something similar should be doable (and desirable) for Windows. In fact, that makes me think whether we should generalize this option to something like --with-external-debug-symbols-in-product-bundles=. So, that way one could decide which PDBs to ship and it could eventually also be made available for other platforms and external debug symbols. And jlink plugins could then handle this and remove/strip this information. Best regards Christoph From richard.reingruber at sap.com Fri Feb 14 12:58:41 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 14 Feb 2020 12:58:41 +0000 Subject: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant In-Reply-To: <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com> References: <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com> Message-ID: Hi Patricio, thanks for having a look. > I?m only commenting on the handshake changes. > I see that operation VM_EnterInterpOnlyMode can be called inside > operation VM_SetFramePop which also allows nested operations. Here is a > comment in VM_SetFramePop definition: > > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. > > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we > could have a handshake inside a safepoint operation. The issue I see > there is that at the end of the handshake the polling page of the target > thread could be disarmed. So if the target thread happens to be in a > blocked state just transiently and wakes up then it will not stop for > the ongoing safepoint. Maybe I can file an RFE to assert that the > polling page is armed at the beginning of disarm_safepoint(). I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a handshake cannot be nested in a vm operation. Maybe it should be asserted in the Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? > Alternatively I think you could do something similar to what we do in > Deoptimization::deoptimize_all_marked(): > > EnterInterpOnlyModeClosure hs; > if (SafepointSynchronize::is_at_safepoint()) { > hs.do_thread(state->get_thread()); > } else { > Handshake::execute(&hs, state->get_thread()); > } > (you could pass ?EnterInterpOnlyModeClosure? directly to the > HandshakeClosure() constructor) Maybe this could be used also in the Handshake::execute() methods as general solution? > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is > always called in a nested operation or just sometimes. At least one execution path without vm operation exists: JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong JvmtiEventControllerPrivate::recompute_enabled() : void JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further encouraged to do it with a handshake :) Thanks again, Richard. -----Original Message----- From: Patricio Chilano Sent: Donnerstag, 13. Februar 2020 18:47 To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant Hi Richard, I?m only commenting on the handshake changes. I see that operation VM_EnterInterpOnlyMode can be called inside operation VM_SetFramePop which also allows nested operations. Here is a comment in VM_SetFramePop definition: // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. So if we change VM_EnterInterpOnlyMode to be a handshake, then now we could have a handshake inside a safepoint operation. The issue I see there is that at the end of the handshake the polling page of the target thread could be disarmed. So if the target thread happens to be in a blocked state just transiently and wakes up then it will not stop for the ongoing safepoint. Maybe I can file an RFE to assert that the polling page is armed at the beginning of disarm_safepoint(). I think one option could be to remove SafepointMechanism::disarm_if_needed() in HandshakeState::clear_handshake() and let each JavaThread disarm itself for the handshake case. Alternatively I think you could do something similar to what we do in Deoptimization::deoptimize_all_marked(): ? EnterInterpOnlyModeClosure hs; ? if (SafepointSynchronize::is_at_safepoint()) { ??? hs.do_thread(state->get_thread()); ? } else { ??? Handshake::execute(&hs, state->get_thread()); ? } (you could pass ?EnterInterpOnlyModeClosure? directly to the HandshakeClosure() constructor) I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is always called in a nested operation or just sometimes. Thanks, Patricio On 2/12/20 7:23 AM, Reingruber, Richard wrote: > // Repost including hotspot runtime and gc lists. > // Dean Long suggested to do so, because the enhancement replaces a vm operation > // with a handshake. > // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html > > Hi, > > could I please get reviews for this small enhancement in hotspot's jvmti implementation: > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8238585 > > The change avoids making all compiled methods on stack not_entrant when switching a java thread to > interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack. > > Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations. > > Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. > > Thanks, Richard. > > See also my question if anyone knows a reason for making the compiled methods not_entrant: > http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html From richard.reingruber at sap.com Fri Feb 14 13:02:10 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 14 Feb 2020 13:02:10 +0000 Subject: RFR(XS) 8239005: [TESTBUG] test/hotspot/jtreg/runtime/StackGuardPages/TestStackGuardPages.java: exeinvoke.c: must initialize static state before calling do_overflow() In-Reply-To: <6fc2b655-e7f5-9f15-7bb1-46625492fa20@oracle.com> References: <6fc2b655-e7f5-9f15-7bb1-46625492fa20@oracle.com> Message-ID: Hi David, Thanks for reviewing. Richard. -----Original Message----- From: David Holmes Sent: Donnerstag, 13. Februar 2020 23:44 To: Reingruber, Richard ; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(XS) 8239005: [TESTBUG] test/hotspot/jtreg/runtime/StackGuardPages/TestStackGuardPages.java: exeinvoke.c: must initialize static state before calling do_overflow() Hi Richard, This seems fine to me. Good catch! Thanks, David On 14/02/2020 2:45 am, Reingruber, Richard wrote: > Hi, > > please review this small test fix > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239005/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8239005 > > On one of our linuxppc64le boxes the test was failing due to the missing reset of the static > variable _rec_count before the first call to do_overflow() [1] in run_native_overflow() [2]. > Consequently the recursion limit set at [3] is too high. On our linuxppc64le box we then get a > SIGSEGV with SEGV_ACCERR which is reported as test failure. > > On my linuxx86_64 box I get an SIGSEGV with SEGV_MAPERR which is not reported as error. > > With the fix we don't get a second SIGSEGV. Maybe a second SIGSEGV should be reported as failure...? > > I tested on linuxx86_64 and linuxppc64le > > Many more details to be found in the bug. > > Thanks, Richard. > > [1] definition of do_overflow() > http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c#l128 > > [2] _rec_count is not reset to 0 before call to do_overflow() > http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c#l168 > > [3] recursion limit is set too high > http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/runtime/StackGuardPages/exeinvoke.c#l191 > From patricio.chilano.mateo at oracle.com Fri Feb 14 14:53:52 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Fri, 14 Feb 2020 11:53:52 -0300 Subject: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant In-Reply-To: References: <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com> Message-ID: <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com> Hi Richard, On 2/14/20 9:58 AM, Reingruber, Richard wrote: > Hi Patricio, > > thanks for having a look. > > > I?m only commenting on the handshake changes. > > I see that operation VM_EnterInterpOnlyMode can be called inside > > operation VM_SetFramePop which also allows nested operations. Here is a > > comment in VM_SetFramePop definition: > > > > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is > > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. > > > > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we > > could have a handshake inside a safepoint operation. The issue I see > > there is that at the end of the handshake the polling page of the target > > thread could be disarmed. So if the target thread happens to be in a > > blocked state just transiently and wakes up then it will not stop for > > the ongoing safepoint. Maybe I can file an RFE to assert that the > > polling page is armed at the beginning of disarm_safepoint(). > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a > handshake cannot be nested in a vm operation. Maybe it should be asserted in the > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? > > > Alternatively I think you could do something similar to what we do in > > Deoptimization::deoptimize_all_marked(): > > > > EnterInterpOnlyModeClosure hs; > > if (SafepointSynchronize::is_at_safepoint()) { > > hs.do_thread(state->get_thread()); > > } else { > > Handshake::execute(&hs, state->get_thread()); > > } > > (you could pass ?EnterInterpOnlyModeClosure? directly to the > > HandshakeClosure() constructor) > > Maybe this could be used also in the Handshake::execute() methods as general solution? Right, we could also do that. Avoiding to clear the polling page in HandshakeState::clear_handshake() should be enough to fix this issue and execute a handshake inside a safepoint, but adding that "if" statement in Hanshake::execute() sounds good to avoid all the extra code that we go through when executing a handshake. I filed 8239084 to make that change. > > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is > > always called in a nested operation or just sometimes. > > At least one execution path without vm operation exists: > > JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void > JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong > JvmtiEventControllerPrivate::recompute_enabled() : void > JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) > JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void > JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError > jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further > encouraged to do it with a handshake :) Ah! I think you can still do it with a handshake with the Deoptimization::deoptimize_all_marked() like solution. I can change the if-else statement with just the Handshake::execute() call in 8239084. But up to you.? : ) Thanks, Patricio > Thanks again, > Richard. > > -----Original Message----- > From: Patricio Chilano > Sent: Donnerstag, 13. Februar 2020 18:47 > To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net > Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant > > Hi Richard, > > I?m only commenting on the handshake changes. > I see that operation VM_EnterInterpOnlyMode can be called inside > operation VM_SetFramePop which also allows nested operations. Here is a > comment in VM_SetFramePop definition: > > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. > > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we > could have a handshake inside a safepoint operation. The issue I see > there is that at the end of the handshake the polling page of the target > thread could be disarmed. So if the target thread happens to be in a > blocked state just transiently and wakes up then it will not stop for > the ongoing safepoint. Maybe I can file an RFE to assert that the > polling page is armed at the beginning of disarm_safepoint(). > > I think one option could be to remove > SafepointMechanism::disarm_if_needed() in > HandshakeState::clear_handshake() and let each JavaThread disarm itself > for the handshake case. > > Alternatively I think you could do something similar to what we do in > Deoptimization::deoptimize_all_marked(): > > ? EnterInterpOnlyModeClosure hs; > ? if (SafepointSynchronize::is_at_safepoint()) { > ??? hs.do_thread(state->get_thread()); > ? } else { > ??? Handshake::execute(&hs, state->get_thread()); > ? } > (you could pass ?EnterInterpOnlyModeClosure? directly to the > HandshakeClosure() constructor) > > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is > always called in a nested operation or just sometimes. > > Thanks, > Patricio > > On 2/12/20 7:23 AM, Reingruber, Richard wrote: >> // Repost including hotspot runtime and gc lists. >> // Dean Long suggested to do so, because the enhancement replaces a vm operation >> // with a handshake. >> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html >> >> Hi, >> >> could I please get reviews for this small enhancement in hotspot's jvmti implementation: >> >> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8238585 >> >> The change avoids making all compiled methods on stack not_entrant when switching a java thread to >> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack. >> >> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations. >> >> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. >> >> Thanks, Richard. >> >> See also my question if anyone knows a reason for making the compiled methods not_entrant: >> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html From vladimir.x.ivanov at oracle.com Fri Feb 14 16:10:30 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 14 Feb 2020 19:10:30 +0300 Subject: [15] RFR (M): 8238681: Make -XX:UseSSE flag x86-specific In-Reply-To: References: Message-ID: <070c12dc-bf7b-c5ca-76d7-9ea6127cb4e0@oracle.com> Thanks for the feedback, Vladimir. > compiler/c1/Test6579789.java and compiler/c1/Test6855215.java use UseSSE > flag without @requires. Good catch. But I'm not sure @requires [1] is the best option here: * -XX:UseSSE=0 has any effect only on x86-32; * both bugs were reported against x86-32, but at least 6579789 is not platform-specific. So it makes sense to run the tests on other platforms. So, I'd prefer to use -XX:+IgnoreUnrecognizedVMOptions instead. What's your take on it? Best regards, Vladimir Ivanov [1] diff --git a/test/hotspot/jtreg/compiler/c1/Test6579789.java b/test/hotspot/jtreg/compiler/c1/Test6579789.java --- a/test/hotspot/jtreg/compiler/c1/Test6579789.java +++ b/test/hotspot/jtreg/compiler/c1/Test6579789.java @@ -26,6 +26,8 @@ * @bug 6579789 * @summary Internal error "c1_LinearScan.cpp:1429 Error: assert(false,"")" in debuggee with fastdebug VM * + * @requires os.arch=="i386" + * * @run main/othervm -Xcomp -XX:UseSSE=0 * -XX:CompileCommand=compileonly,compiler.c1.Test6579789::bug * compiler.c1.Test6579789 diff --git a/test/hotspot/jtreg/compiler/c1/Test6855215.java b/test/hotspot/jtreg/compiler/c1/Test6855215.java --- a/test/hotspot/jtreg/compiler/c1/Test6855215.java +++ b/test/hotspot/jtreg/compiler/c1/Test6855215.java @@ -26,6 +26,8 @@ * @bug 6855215 * @summary Calculation error (NaN) after about 1500 calculations * + * @requires os.arch=="i386" + * * @run main/othervm -Xbatch -XX:UseSSE=0 compiler.c1.Test6855215 */ > > Otherwise changes seems good. > > Thanks, > Vladimir > > On 2/10/20 6:19 AM, Vladimir Ivanov wrote: >> http://cr.openjdk.java.net/~vlivanov/8238681/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8238681 >> >> Turn UseSSE into x86-specific flag since it doesn't have any meaning >> on non-x86 platforms. >> >> As a cleanup, I made all the code which assumes UseSSE<2 x86-32 >> specific. (x86-64 ABI requires SSE2.) >> >> Also, as a precaution, I made UseSSE obsolete on non-x86 platforms. >> >> Testing: hs-precheckin-comp, tier1-5, linux x86-32 fastdebug build. >> >> Thanks! >> >> Best regards, >> Vladimir Ivanov From vladimir.x.ivanov at oracle.com Fri Feb 14 16:16:31 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 14 Feb 2020 19:16:31 +0300 Subject: [15] RFR (S): 8238696: x86: Enumerate all detected CPU features in VM_Version feature string In-Reply-To: <958b4c49-54ca-8243-4baa-bb2deab5e24f@oracle.com> References: <98506aca-4a47-a337-a707-19c2a424ca01@oracle.com> <3b882679-b8e1-8e2c-fd8e-261d48224001@oracle.com> <47bf9452-fb75-0293-0a54-3df9e3a3a716@oracle.com> <2267f17c-2f81-eec3-d31f-4e031e34b615@oracle.com> <958b4c49-54ca-8243-4baa-bb2deab5e24f@oracle.com> Message-ID: <4b00fe64-68a9-334c-42b2-14eb0ddd25cf@oracle.com> > Changes looks good to me. Thanks, Vladimir. > I wish we have a definition of CPU features which specify name and > enumeration so that it will be printed automatically, if supported, by > VM_Version::get_processor_features(). Counting by hand how many %s I > have to add is bug prone. We also need to check that we have enough bits > for enumeration. > > It could be a starter RFE. Yes, it would be nice. Filed JDK-8239090 [1]. Best regards, Vladimir Ivanov [1] https://bugs.openjdk.java.net/browse/JDK-8239090 > On 2/11/20 5:58 AM, Vladimir Ivanov wrote: >> >>> Now I'm wondering about using the dot as a terminator. Doesn't the >>> return value from jio_snprintf tell you whether you exceeded the buffer? >> >> Yes, very good point. How about the following? >> >> ?? http://cr.openjdk.java.net/~vlivanov/8238696/webrev.03 >> >> Considering some tests have hard-coded feature names, I decided to >> enhance the new test and check that all reported features are known >> (at least, on x86). >> >> Best regards, >> Vladimir Ivanov >> >>>> On 10.02.2020 17:12, Vladimir Ivanov wrote: >>>>> Thanks for taking a look, David. >>>>> >>>>> Updated webrev: >>>>> ?? http://cr.openjdk.java.net/~vlivanov/8238696/webrev.01/ >>>>> >>>>>>> http://cr.openjdk.java.net/~vlivanov/8238696/webrev.00/ >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8238696 >>>>>>> >>>>>>> Abstract_VM_Version::features_string() doesn't contain all the >>>>>>> CPU features detected by the JVM on x86 (for example, for AVX-512 >>>>>>> only AVX-512F is included as "evex"). >>>>>>> >>>>>>> The fix adds missing features to the list [1]. >>>>>> >>>>>> The function naming that you have changed for >>>>>> supports_avx512_vpclmulqdq was explicitly requested by Vladimir K.: >>>>>> >>>>>> http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-December/036326.html >>>>>> >>>>>> >>>>>> If anything I would suggest supports_vpopcntdq() should have the >>>>>> avx512 part added. >>>>> >>>>> Makes sense. Also, decided to rename VNNI, VAES, and VBMI2. They >>>>> are listed in the manual as part of AVX512. >>>>> >>>>>> The actual expansion of the feature string seems okay. It does >>>>>> make me wonder why these items have been left out previously. >>>>> >>>>> I believe it was just overlooked. Added a comment near cpu feature >>>>> constant declarations. >>>>> >>>>> Also, introduced a check to ensure the temporary on-stack buffer is >>>>> large enough. >>>>> >>>>> Best regards, >>>>> Vladimir Ivanov >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> Testing: manual (-Xlog:os+cpu=trace output on different hardware), >>>>>>> ????????? hs-precheckin-comp, hs-tier1, hs-tier2 >>>>>>> >>>>>>> Best regards, >>>>>>> Vladimir Ivanov >>>>>>> >>>>>>> [1] $ java -Xlog:os+cpu=trace -XX:UseAVX=3 -version >>>>>>> >>>>>>> ... >>>>>>> [0.010s][info][os,cpu] UseSSE=4? UseAVX=3? UseAES=1 >>>>>>> MaxVectorSize=64 >>>>>>> ... >>>>>>> [0.011s][info][os,cpu] CPU:total 8 (initial active 8) (4 cores >>>>>>> per cpu, 2 threads per core) family 6 model 85 stepping 4, cmov, >>>>>>> cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, >>>>>>> vzeroupper, avx, avx2, aes, clmul, erms, rtm, 3dnowpref, lzcnt, >>>>>>> ht, tsc, bmi1, bmi2, adx, avx512f, avx512dq, avx512cd, avx512bw, >>>>>>> avx512vl, fma, clflush, clflushopt, clwb >>>>>>> ... >>>>>>> [0.011s][info][os,cpu] flags??????? : fpu vme de pse tsc msr pae >>>>>>> mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse >>>>>>> sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon >>>>>>> rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx >>>>>>> ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt >>>>>>> tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm >>>>>>> abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb >>>>>>> stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust >>>>>>> bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq >>>>>>> rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl >>>>>>> xsaveopt xsavec xgetbv1 xsaves arat umip pku ospke md_clear >>>>>>> ... From vladimir.kozlov at oracle.com Fri Feb 14 17:44:04 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 14 Feb 2020 09:44:04 -0800 Subject: [15] RFR (M): 8238681: Make -XX:UseSSE flag x86-specific In-Reply-To: <070c12dc-bf7b-c5ca-76d7-9ea6127cb4e0@oracle.com> References: <070c12dc-bf7b-c5ca-76d7-9ea6127cb4e0@oracle.com> Message-ID: <6eb9085e-fe71-c922-085c-3b4dd3d86f1d@oracle.com> I am fine with -XX:+IgnoreUnrecognizedVMOptions. I am for testing on more platforms ;) Thanks, Vladimir K On 2/14/20 8:10 AM, Vladimir Ivanov wrote: > Thanks for the feedback, Vladimir. > >> compiler/c1/Test6579789.java and compiler/c1/Test6855215.java use UseSSE flag without @requires. > > Good catch. But I'm not sure @requires [1] is the best option here: > > ?? * -XX:UseSSE=0 has any effect only on x86-32; > > ?? * both bugs were reported against x86-32, but at least 6579789 is not platform-specific. So it > makes sense to run the tests on other platforms. > > So, I'd prefer to use -XX:+IgnoreUnrecognizedVMOptions instead. What's your take on it? > > Best regards, > Vladimir Ivanov > > [1] > diff --git a/test/hotspot/jtreg/compiler/c1/Test6579789.java > b/test/hotspot/jtreg/compiler/c1/Test6579789.java > --- a/test/hotspot/jtreg/compiler/c1/Test6579789.java > +++ b/test/hotspot/jtreg/compiler/c1/Test6579789.java > @@ -26,6 +26,8 @@ > ? * @bug 6579789 > ? * @summary Internal error "c1_LinearScan.cpp:1429 Error: assert(false,"")" in debuggee with > fastdebug VM > ? * > + * @requires os.arch=="i386" > + * > ? * @run main/othervm -Xcomp -XX:UseSSE=0 > ? *????? -XX:CompileCommand=compileonly,compiler.c1.Test6579789::bug > ? *????? compiler.c1.Test6579789 > diff --git a/test/hotspot/jtreg/compiler/c1/Test6855215.java > b/test/hotspot/jtreg/compiler/c1/Test6855215.java > --- a/test/hotspot/jtreg/compiler/c1/Test6855215.java > +++ b/test/hotspot/jtreg/compiler/c1/Test6855215.java > @@ -26,6 +26,8 @@ > ? * @bug 6855215 > ? * @summary Calculation error (NaN) after about 1500 calculations > ? * > + * @requires os.arch=="i386" > + * > ? * @run main/othervm -Xbatch -XX:UseSSE=0 compiler.c1.Test6855215 > ? */ > > >> >> Otherwise changes seems good. >> >> Thanks, >> Vladimir >> >> On 2/10/20 6:19 AM, Vladimir Ivanov wrote: >>> http://cr.openjdk.java.net/~vlivanov/8238681/webrev.00/ >>> https://bugs.openjdk.java.net/browse/JDK-8238681 >>> >>> Turn UseSSE into x86-specific flag since it doesn't have any meaning on non-x86 platforms. >>> >>> As a cleanup, I made all the code which assumes UseSSE<2 x86-32 specific. (x86-64 ABI requires >>> SSE2.) >>> >>> Also, as a precaution, I made UseSSE obsolete on non-x86 platforms. >>> >>> Testing: hs-precheckin-comp, tier1-5, linux x86-32 fastdebug build. >>> >>> Thanks! >>> >>> Best regards, >>> Vladimir Ivanov From frederic.parain at oracle.com Fri Feb 14 18:34:19 2020 From: frederic.parain at oracle.com (Frederic Parain) Date: Fri, 14 Feb 2020 13:34:19 -0500 Subject: RFR[S]: 8239014 -XX:-UseEmptySlotsInSupers sometime fails to reproduce the layout of the old code In-Reply-To: <3d807131-157c-b6ff-0b64-a2bba8e03175@oracle.com> References: <4C24883A-9EED-45E2-94C3-4AD91ACBAEC7@oracle.com> <3d807131-157c-b6ff-0b64-a2bba8e03175@oracle.com> Message-ID: <1E2C4D7D-AF8D-4642-90A0-7FC1A53B6F3A@oracle.com> Hi David, Here?s a new webrev with a test case: http://cr.openjdk.java.net/~fparain/8239014/webrev.01/ When -XX:-UseEmptySlotsInSupers is used, the field layout code emulates the behavior of the old code by appending fields after the rounded up end of the super class, adding primitive fields by decreasing size, then oops. The issue was that this strategy doesn?t fill the gap after the header when the class pointer is 4 bytes and the first field is either a long or a double. There was a special case in the old code to handle this situation, but it was missing in the emulation mode of the new code. Thank you, Fred > On Feb 13, 2020, at 17:53, David Holmes wrote: > > Hi Fred, > > Can you add a testcase for this? I can't really evaluate the change just by looking at the code. > > Thanks, > David > > On 14/02/2020 6:28 am, Frederic Parain wrote: >> Please review this small change that fix the behavior of-XX: -UseEmptySlotsInSupers. >> CR: https://bugs.openjdk.java.net/browse/JDK-8239014 >> Webrev: http://cr.openjdk.java.net/~fparain/8239014/webrev.00/index.html >> Thank you, >> Fred From richard.reingruber at sap.com Fri Feb 14 18:47:20 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 14 Feb 2020 18:47:20 +0000 Subject: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant In-Reply-To: <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com> References: <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com> <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com> Message-ID: Hi Patricio, > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? > > > > > Alternatively I think you could do something similar to what we do in > > > Deoptimization::deoptimize_all_marked(): > > > > > > EnterInterpOnlyModeClosure hs; > > > if (SafepointSynchronize::is_at_safepoint()) { > > > hs.do_thread(state->get_thread()); > > > } else { > > > Handshake::execute(&hs, state->get_thread()); > > > } > > > (you could pass ?EnterInterpOnlyModeClosure? directly to the > > > HandshakeClosure() constructor) > > > > Maybe this could be used also in the Handshake::execute() methods as general solution? > Right, we could also do that. Avoiding to clear the polling page in > HandshakeState::clear_handshake() should be enough to fix this issue and > execute a handshake inside a safepoint, but adding that "if" statement > in Hanshake::execute() sounds good to avoid all the extra code that we > go through when executing a handshake. I filed 8239084 to make that change. Thanks for taking care of this and creating the RFE. > > > > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is > > > always called in a nested operation or just sometimes. > > > > At least one execution path without vm operation exists: > > > > JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void > > JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong > > JvmtiEventControllerPrivate::recompute_enabled() : void > > JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) > > JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void > > JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError > > jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError > > > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further > > encouraged to do it with a handshake :) > Ah! I think you can still do it with a handshake with the > Deoptimization::deoptimize_all_marked() like solution. I can change the > if-else statement with just the Handshake::execute() call in 8239084. > But up to you. : ) Well, I think that's enough encouragement :) I'll wait for 8239084 and try then again. (no urgency and all) Thanks, Richard. -----Original Message----- From: Patricio Chilano Sent: Freitag, 14. Februar 2020 15:54 To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant Hi Richard, On 2/14/20 9:58 AM, Reingruber, Richard wrote: > Hi Patricio, > > thanks for having a look. > > > I?m only commenting on the handshake changes. > > I see that operation VM_EnterInterpOnlyMode can be called inside > > operation VM_SetFramePop which also allows nested operations. Here is a > > comment in VM_SetFramePop definition: > > > > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is > > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. > > > > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we > > could have a handshake inside a safepoint operation. The issue I see > > there is that at the end of the handshake the polling page of the target > > thread could be disarmed. So if the target thread happens to be in a > > blocked state just transiently and wakes up then it will not stop for > > the ongoing safepoint. Maybe I can file an RFE to assert that the > > polling page is armed at the beginning of disarm_safepoint(). > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a > handshake cannot be nested in a vm operation. Maybe it should be asserted in the > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation? > > > Alternatively I think you could do something similar to what we do in > > Deoptimization::deoptimize_all_marked(): > > > > EnterInterpOnlyModeClosure hs; > > if (SafepointSynchronize::is_at_safepoint()) { > > hs.do_thread(state->get_thread()); > > } else { > > Handshake::execute(&hs, state->get_thread()); > > } > > (you could pass ?EnterInterpOnlyModeClosure? directly to the > > HandshakeClosure() constructor) > > Maybe this could be used also in the Handshake::execute() methods as general solution? Right, we could also do that. Avoiding to clear the polling page in HandshakeState::clear_handshake() should be enough to fix this issue and execute a handshake inside a safepoint, but adding that "if" statement in Hanshake::execute() sounds good to avoid all the extra code that we go through when executing a handshake. I filed 8239084 to make that change. > > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is > > always called in a nested operation or just sometimes. > > At least one execution path without vm operation exists: > > JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void > JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong > JvmtiEventControllerPrivate::recompute_enabled() : void > JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches) > JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void > JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError > jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further > encouraged to do it with a handshake :) Ah! I think you can still do it with a handshake with the Deoptimization::deoptimize_all_marked() like solution. I can change the if-else statement with just the Handshake::execute() call in 8239084. But up to you.? : ) Thanks, Patricio > Thanks again, > Richard. > > -----Original Message----- > From: Patricio Chilano > Sent: Donnerstag, 13. Februar 2020 18:47 > To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net > Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant > > Hi Richard, > > I?m only commenting on the handshake changes. > I see that operation VM_EnterInterpOnlyMode can be called inside > operation VM_SetFramePop which also allows nested operations. Here is a > comment in VM_SetFramePop definition: > > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled. > > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we > could have a handshake inside a safepoint operation. The issue I see > there is that at the end of the handshake the polling page of the target > thread could be disarmed. So if the target thread happens to be in a > blocked state just transiently and wakes up then it will not stop for > the ongoing safepoint. Maybe I can file an RFE to assert that the > polling page is armed at the beginning of disarm_safepoint(). > > I think one option could be to remove > SafepointMechanism::disarm_if_needed() in > HandshakeState::clear_handshake() and let each JavaThread disarm itself > for the handshake case. > > Alternatively I think you could do something similar to what we do in > Deoptimization::deoptimize_all_marked(): > > ? EnterInterpOnlyModeClosure hs; > ? if (SafepointSynchronize::is_at_safepoint()) { > ??? hs.do_thread(state->get_thread()); > ? } else { > ??? Handshake::execute(&hs, state->get_thread()); > ? } > (you could pass ?EnterInterpOnlyModeClosure? directly to the > HandshakeClosure() constructor) > > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is > always called in a nested operation or just sometimes. > > Thanks, > Patricio > > On 2/12/20 7:23 AM, Reingruber, Richard wrote: >> // Repost including hotspot runtime and gc lists. >> // Dean Long suggested to do so, because the enhancement replaces a vm operation >> // with a handshake. >> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html >> >> Hi, >> >> could I please get reviews for this small enhancement in hotspot's jvmti implementation: >> >> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8238585 >> >> The change avoids making all compiled methods on stack not_entrant when switching a java thread to >> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack. >> >> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations. >> >> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms. >> >> Thanks, Richard. >> >> See also my question if anyone knows a reason for making the compiled methods not_entrant: >> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html From glaubitz at physik.fu-berlin.de Sat Feb 15 18:06:38 2020 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Sat, 15 Feb 2020 19:06:38 +0100 Subject: RFR: 8239001: Hotspot build broken on linux-sparc after 8238281 In-Reply-To: References: Message-ID: <64164050-0a8c-5bb5-572e-0ca04180ad0b@physik.fu-berlin.de> On 2/13/20 8:35 PM, Kim Barrett wrote: > I don't think this is the right way to address this problem. > > I think the JDK-8238281 change to offset_of and the associated > addition of -Wno-invalid-offsetof to the build configuration was a > mistake, and should be reverted. Those changes seem unrelated to the > purpose of JDK-8238281, which was to raise the minimum acceptable gcc > version. Alternatively, this works as well: glaubitz at gcc202:~/jdk-hg$ hg diff diff -r 274a0bcce99d src/hotspot/cpu/sparc/macroAssembler_sparc.hpp --- a/src/hotspot/cpu/sparc/macroAssembler_sparc.hpp Sat Feb 15 17:35:57 2020 +0800 +++ b/src/hotspot/cpu/sparc/macroAssembler_sparc.hpp Sat Feb 15 21:05:10 2020 +0300 @@ -485,12 +485,12 @@ void print(outputStream* s); - static int i_offset(int j) { return offset_of(RegistersForDebugging, i[j]); } - static int l_offset(int j) { return offset_of(RegistersForDebugging, l[j]); } - static int o_offset(int j) { return offset_of(RegistersForDebugging, o[j]); } - static int g_offset(int j) { return offset_of(RegistersForDebugging, g[j]); } - static int f_offset(int j) { return offset_of(RegistersForDebugging, f[j]); } - static int d_offset(int j) { return offset_of(RegistersForDebugging, d[j / 2]); } + static int i_offset(int j) { return offset_of(RegistersForDebugging, i) + j * sizeof(((RegistersForDebugging*)16)->i[0]); } + static int l_offset(int j) { return offset_of(RegistersForDebugging, l) + j * sizeof(((RegistersForDebugging*)16)->l[0]); } + static int o_offset(int j) { return offset_of(RegistersForDebugging, o) + j * sizeof(((RegistersForDebugging*)16)->o[0]); } + static int g_offset(int j) { return offset_of(RegistersForDebugging, g) + j * sizeof(((RegistersForDebugging*)16)->g[0]); } + static int f_offset(int j) { return offset_of(RegistersForDebugging, f) + j * sizeof(((RegistersForDebugging*)16)->f[0]); } + static int d_offset(int j) { return offset_of(RegistersForDebugging, d) + (j / 2) * sizeof(((RegistersForDebugging*)16)->d[0]); } // gen asm code to save regs static void save_registers(MacroAssembler* a); What about this approach? Alternatively, with C++11, this should work as well: #include static int i_offset(int j) { return offset_of(RegistersForDebugging, i) + j * sizeof(std::declval().i[0]); } Comments? Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From glaubitz at physik.fu-berlin.de Sat Feb 15 18:33:59 2020 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Sat, 15 Feb 2020 19:33:59 +0100 Subject: RFR: 8239001: Hotspot build broken on linux-sparc after 8238281 Message-ID: <277220fd-c8aa-ac58-5c98-15b195b7b9a2@physik.fu-berlin.de> Hi! Here is another approach for fixing 8238281 [1]. Please review. Adrian > [1] http://cr.openjdk.java.net/~glaubitz/8239001/webrev.01/ -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From kim.barrett at oracle.com Sat Feb 15 21:30:54 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 15 Feb 2020 16:30:54 -0500 Subject: RFR: 8239001: Hotspot build broken on linux-sparc after 8238281 In-Reply-To: <64164050-0a8c-5bb5-572e-0ca04180ad0b@physik.fu-berlin.de> References: <64164050-0a8c-5bb5-572e-0ca04180ad0b@physik.fu-berlin.de> Message-ID: <52DE2202-9671-4BF0-947C-C3164A274D7F@oracle.com> > On Feb 15, 2020, at 1:06 PM, John Paul Adrian Glaubitz wrote: > > On 2/13/20 8:35 PM, Kim Barrett wrote: >> I don't think this is the right way to address this problem. >> >> I think the JDK-8238281 change to offset_of and the associated >> addition of -Wno-invalid-offsetof to the build configuration was a >> mistake, and should be reverted. Those changes seem unrelated to the >> purpose of JDK-8238281, which was to raise the minimum acceptable gcc >> version. > > Alternatively, this works as well: > > glaubitz at gcc202:~/jdk-hg$ hg diff > diff -r 274a0bcce99d src/hotspot/cpu/sparc/macroAssembler_sparc.hpp > --- a/src/hotspot/cpu/sparc/macroAssembler_sparc.hpp Sat Feb 15 17:35:57 2020 +0800 > +++ b/src/hotspot/cpu/sparc/macroAssembler_sparc.hpp Sat Feb 15 21:05:10 2020 +0300 > @@ -485,12 +485,12 @@ > > void print(outputStream* s); > > - static int i_offset(int j) { return offset_of(RegistersForDebugging, i[j]); } > - static int l_offset(int j) { return offset_of(RegistersForDebugging, l[j]); } > - static int o_offset(int j) { return offset_of(RegistersForDebugging, o[j]); } > - static int g_offset(int j) { return offset_of(RegistersForDebugging, g[j]); } > - static int f_offset(int j) { return offset_of(RegistersForDebugging, f[j]); } > - static int d_offset(int j) { return offset_of(RegistersForDebugging, d[j / 2]); } > + static int i_offset(int j) { return offset_of(RegistersForDebugging, i) + j * sizeof(((RegistersForDebugging*)16)->i[0]); } > + static int l_offset(int j) { return offset_of(RegistersForDebugging, l) + j * sizeof(((RegistersForDebugging*)16)->l[0]); } > + static int o_offset(int j) { return offset_of(RegistersForDebugging, o) + j * sizeof(((RegistersForDebugging*)16)->o[0]); } > + static int g_offset(int j) { return offset_of(RegistersForDebugging, g) + j * sizeof(((RegistersForDebugging*)16)->g[0]); } > + static int f_offset(int j) { return offset_of(RegistersForDebugging, f) + j * sizeof(((RegistersForDebugging*)16)->f[0]); } > + static int d_offset(int j) { return offset_of(RegistersForDebugging, d) + (j / 2) * sizeof(((RegistersForDebugging*)16)->d[0]); } > > // gen asm code to save regs > static void save_registers(MacroAssembler* a); > > What about this approach? > > Alternatively, with C++11, this should work as well: > > #include > > static int i_offset(int j) { return offset_of(RegistersForDebugging, i) + j * sizeof(std::declval().i[0]); } > > Comments? I prefer the approach using a non-ODR-used dummy to the approach of casting of some random number to an address. Non-ODR-used declarations are well supported by the standard (C++03 3.2) and widely used; that's the basis for the "sizeof trick" metaprogramming idiom, for example. Using std::declval is morally equivalent, but we haven't turned on C++11 yet. From aph at redhat.com Sun Feb 16 11:07:24 2020 From: aph at redhat.com (Andrew Haley) Date: Sun, 16 Feb 2020 11:07:24 +0000 Subject: RFR: 8239001: Hotspot build broken on linux-sparc after 8238281 In-Reply-To: <277220fd-c8aa-ac58-5c98-15b195b7b9a2@physik.fu-berlin.de> References: <277220fd-c8aa-ac58-5c98-15b195b7b9a2@physik.fu-berlin.de> Message-ID: <5e754287-6cd3-168b-3274-3a00be6aa45b@redhat.com> On 2/15/20 6:33 PM, John Paul Adrian Glaubitz wrote: > > Here is another approach for fixing 8238281 [1]. Please review. > >> [1] http://cr.openjdk.java.net/~glaubitz/8239001/webrev.01/ Can you give us a bit more explanation about why 8238281 broke linux-sparc, and why your proposal is valid? This doesn't look valid to me: (((RegistersForDebugging*)16)->f[0]) -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Sun Feb 16 11:14:01 2020 From: aph at redhat.com (Andrew Haley) Date: Sun, 16 Feb 2020 11:14:01 +0000 Subject: RFR: 8239001: Hotspot build broken on linux-sparc after 8238281 In-Reply-To: <5e754287-6cd3-168b-3274-3a00be6aa45b@redhat.com> References: <277220fd-c8aa-ac58-5c98-15b195b7b9a2@physik.fu-berlin.de> <5e754287-6cd3-168b-3274-3a00be6aa45b@redhat.com> Message-ID: <9300560f-551a-1014-f642-1922e3f38df2@redhat.com> On 2/16/20 11:07 AM, Andrew Haley wrote: > Can you give us a bit more explanation about why 8238281 broke linux-sparc, > and why your proposal is valid? This doesn't look valid to me: > > (((RegistersForDebugging*)16)->f[0]) Sorry, I didn't see the other thread. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From glaubitz at physik.fu-berlin.de Sun Feb 16 11:19:38 2020 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Sun, 16 Feb 2020 12:19:38 +0100 Subject: RFR: 8239001: Hotspot build broken on linux-sparc after 8238281 In-Reply-To: <5e754287-6cd3-168b-3274-3a00be6aa45b@redhat.com> References: <277220fd-c8aa-ac58-5c98-15b195b7b9a2@physik.fu-berlin.de> <5e754287-6cd3-168b-3274-3a00be6aa45b@redhat.com> Message-ID: <92cc500a-23fa-9861-2f47-347d3fe8fc46@physik.fu-berlin.de> Hi Andrew! On 2/16/20 12:07 PM, Andrew Haley wrote: > On 2/15/20 6:33 PM, John Paul Adrian Glaubitz wrote: >> >> Here is another approach for fixing 8238281 [1]. Please review. >> >>> [1] http://cr.openjdk.java.net/~glaubitz/8239001/webrev.01/ > > Can you give us a bit more explanation about why 8238281 broke linux-sparc, > and why your proposal is valid? This doesn't look valid to me: The problem is that i[j] is a non-const expression which is not allowed when using offsetof(). I have included the GCC error message in the bug report [1]. > (((RegistersForDebugging*)16)->f[0]) > This basically just calculates the offset within the array f with the offset j, hence the factor j. I have to admit though that I haven't fully understood Kim's non-odr-used approach yet. Thanks, Adrian > [1] https://bugs.openjdk.java.net/browse/JDK-8239001 -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From glaubitz at physik.fu-berlin.de Sun Feb 16 19:58:56 2020 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Sun, 16 Feb 2020 20:58:56 +0100 Subject: RFR: 8239001: Hotspot build broken on linux-sparc after 8238281 In-Reply-To: <52DE2202-9671-4BF0-947C-C3164A274D7F@oracle.com> References: <64164050-0a8c-5bb5-572e-0ca04180ad0b@physik.fu-berlin.de> <52DE2202-9671-4BF0-947C-C3164A274D7F@oracle.com> Message-ID: Hi Kim! On 2/15/20 10:30 PM, Kim Barrett wrote: > I prefer the approach using a non-ODR-used dummy to the approach of > casting of some random number to an address. Non-ODR-used > declarations are well supported by the standard (C++03 3.2) and widely > used; that's the basis for the "sizeof trick" metaprogramming idiom, > for example. I've now understood that approach and implemented it. Please see my patch in [1]. FWIW, I have tried to put the declaration of RegistersForDebugging outside the X_offset() functions but that doesn't work. Hope the change is now okay as is :). Adrian > [1] http://cr.openjdk.java.net/~glaubitz/8239001/webrev.02/ -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From augustnagro at gmail.com Mon Feb 17 05:04:07 2020 From: augustnagro at gmail.com (August Nagro) Date: Sun, 16 Feb 2020 23:04:07 -0600 Subject: JEP 303 Sanity Check Message-ID: I think I have a cool use-case for the java.lang.constant API and JEP 303, Intrinsics for LDC and InvokeDynamic. Imagine that you?re writing a deserialization library that for simplicity has a single method T readValue(byte[] data, Class cls). The JSON libraries like Jackson have a similar API. Let's also assume that one of the methods is annotated with @Deserializer, and could be a constructor, static method, etc. Jackson currently uses Reflection to find and invoke the @Deserializer method. This works, but direct bytecode generation with the Afterburner module yields ~70% performance improvement. Creating MethodHandles ourselves will have the same performance as Reflection, since we cannot make them static final. But the MethodHandles we?re after are already in the constant pool. So what if instead, we: 1. Use reflection to find the method, generate a DynamicConstantDesc for the method, and cache in a Map, DynamicConstantDesc> for future readValue()s. 2. Get the MethodHandle with Intrinsics.ldc(), which compiles down to an actual ldc bytecode 3. invoke(). I watched Brian?s talk on JEP 303, and saw the example for loading int.class from the constant pool: DynamicConstantDesc> dcd1 = DynamicConstantDesc.ofNamed(BSM_PRIMITIVE_CLASS, "I", CD_Class); // would use Intrinsics.ldc here instead of the reflective resolution when it becomes available. Class intCls = dcd1.resolveConstantDesc(lookup); System.out.println(intCls); // prints ?int' But I?m struggling to construct a minimal example that loads a MethodHandle.. especially on the NameAndType part. Anyway, is this a viable usecase? And any advice on using the new constant API would be helpful. Regards, August Nagro From shade at redhat.com Mon Feb 17 08:07:57 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 17 Feb 2020 09:07:57 +0100 Subject: RFR: 8199138: Add RISC-V support to Zero In-Reply-To: <231ef5a1-8e7d-385c-5604-43f0cb1e2c7c@redhat.com> References: <7f93b41f-9979-0302-6fe8-7bd12e75f39a@physik.fu-berlin.de> <552d208d-2317-4b93-8232-9439d9a0accb@redhat.com> <30763728-3b0d-1109-e99f-050d74302a07@physik.fu-berlin.de> <82096ac0-c8ec-eb01-2f2f-e81a4530913e@redhat.com> <077415d2-eac0-b742-0335-9f0010bb70db@physik.fu-berlin.de> <231ef5a1-8e7d-385c-5604-43f0cb1e2c7c@redhat.com> Message-ID: <9078de18-afc0-b632-e058-9b63593f5758@redhat.com> On 2/12/20 6:13 PM, Aleksey Shipilev wrote: > On 2/12/20 6:00 PM, John Paul Adrian Glaubitz wrote: >> On 2/12/20 5:59 PM, Aleksey Shipilev wrote: >>> On 2/12/20 5:54 PM, John Paul Adrian Glaubitz wrote: >>>> I assume I can push with those changes and mark it as Reviewed-by: erikj, shade? >>> >>> Mark it, yes. I believe non-trivial (yet exceedingly simple) things like these require waiting for >>> 24 hours to anyone else to chime in with comments. There seem to be no rush to get it in, right? >> >> No rush, no. Just wanted to make sure that it's fine to push then :). > > Yes, I believe it would be fine to push then. I think it is fine to push now ;) -- Thanks, -Aleksey From glaubitz at physik.fu-berlin.de Mon Feb 17 08:49:51 2020 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Mon, 17 Feb 2020 09:49:51 +0100 Subject: RFR: 8199138: Add RISC-V support to Zero In-Reply-To: <9078de18-afc0-b632-e058-9b63593f5758@redhat.com> References: <7f93b41f-9979-0302-6fe8-7bd12e75f39a@physik.fu-berlin.de> <552d208d-2317-4b93-8232-9439d9a0accb@redhat.com> <30763728-3b0d-1109-e99f-050d74302a07@physik.fu-berlin.de> <82096ac0-c8ec-eb01-2f2f-e81a4530913e@redhat.com> <077415d2-eac0-b742-0335-9f0010bb70db@physik.fu-berlin.de> <231ef5a1-8e7d-385c-5604-43f0cb1e2c7c@redhat.com> <9078de18-afc0-b632-e058-9b63593f5758@redhat.com> Message-ID: Hi Aleksey! On 2/17/20 9:07 AM, Aleksey Shipilev wrote: > On 2/12/20 6:13 PM, Aleksey Shipilev wrote: >> On 2/12/20 6:00 PM, John Paul Adrian Glaubitz wrote: >>> On 2/12/20 5:59 PM, Aleksey Shipilev wrote: >>>> On 2/12/20 5:54 PM, John Paul Adrian Glaubitz wrote: >>>>> I assume I can push with those changes and mark it as Reviewed-by: erikj, shade? >>>> >>>> Mark it, yes. I believe non-trivial (yet exceedingly simple) things like these require waiting for >>>> 24 hours to anyone else to chime in with comments. There seem to be no rush to get it in, right? >>> >>> No rush, no. Just wanted to make sure that it's fine to push then :). >> >> Yes, I believe it would be fine to push then. > > I think it is fine to push now ;) I have come up with a better and cleaner approach now which I will post later this week. But I want to get the fix for JDK-8239001 out first. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From lutz.schmidt at sap.com Mon Feb 17 09:09:16 2020 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Mon, 17 Feb 2020 09:09:16 +0000 Subject: RFR [XS]: 8239000: handle ContendedPaddingWidth in vm_version_ppc In-Reply-To: References: Message-ID: <1CB543E3-CBC7-44AA-BF0F-DE0B7C999C06@sap.com> Hi Matthias, your change looks good to me. Please note: I'm not a reviewer! I have one question, though. Is there a reason why you only make ppc "look like the other architectures"? aarch64 is missing that code as well. Thanks for considering! Lutz ?On 13.02.20, 12:18, "hotspot-dev on behalf of Baesken, Matthias" wrote: Hello, please review this small adjustment to vm_version_ppc . The ContendedPaddingWidth parameter should be handled in vm_version_ppc like it is done on other architectures (x86/s390). See vm_version_sparc/s390/x86 .cpp for comparision. Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8239000 http://cr.openjdk.java.net/~mbaesken/webrevs/8239000.0/ Best regards, Matthias From christoph.langer at sap.com Mon Feb 17 09:16:38 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 17 Feb 2020 09:16:38 +0000 Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images In-Reply-To: References: <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com> <2b92559a-6a1a-119a-1e62-eed6f9d3bfc8@oracle.com> <1989082f-cddf-9d4f-262b-5f9072f46cf7@oracle.com> <9cd2165f-64f0-a096-9b99-aa8bc0798b34@oracle.com> Message-ID: Hi Erik, thanks for your review. While I?ve addressed all your points (thanks for the great hints regarding usage of SetupCopyFiles), I also enhanced the configure option a little bit. What you can now pass is --with-external-symbols-in-bundles=. Default setting is none. It?ll require --with-native-debug-symbols=external obviously and is currently only implemented for Windows. This is checked in configure. Depending on the value used, the bundles and jmods will contain either no pdbs, public pdbs or the full pdbs. This configure option could easily be enhanced to work for Linux/Unix/Mac as well ? but I didn?t want to go too far with my change now. What also comes with my change is debug files for executables (launchers, cmds) in images/ which weren?t copied so far in Images.gmk. It?s however needed because bundles are created from these images and with the current bundle logic, stripped pdb files need to exist in images/? to get bundled for option ?public?. I also removed the special handling of the pdb file for java.exe in Launcher-java.base.gmk, as the filtering to make sure it?s not overwriting the pdb for java.dll has to be done in CreateJmods.gmk and Images.gmk anyway. Eventually, when this is all refactored, one should probably generate the pdb files to ship into support/modules_libs and support/modules_cmds and only in the case of public pdbs redirect generation of the full pdbs to something like support/modules_libs_full_debug_info or something like that. Then, from support/modules_libs and support/modules_cmds, one should generate the base image via jlink which can simply be packed into the shipment bundle. And the other image with all debug/demos can be created by copying the base image and then applying the stuff from support/ modules_libs_full_debug_info etc. Here's the new webrev: http://cr.openjdk.java.net/~clanger/webrevs/8237192.1/ Best regards Christoph From: Erik Joelsson Sent: Mittwoch, 12. Februar 2020 23:17 To: Langer, Christoph ; Magnus Ihse Bursie ; 'build-dev at openjdk.java.net' Cc: 'hotspot-dev at openjdk.java.net' ; Baesken, Matthias Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images Hello Christoph, This patch certainly looks better to me, though I agree it's a bit hackish to have to filter and rename the stripped.pdb files twice, once for jmods and again for bundles. I think I'm ok with it for now though. The future improvement I would like to make would be to create two sets of jdk images, one that contains debug symbols and demos, which we continue to use for testing, and another which only contains exactly what we bundle up, including the correctly named top dir. The latter would be created first and used as input for the former. I think lots of things would be cleaner then, especially Bundles.gmk. But, that can wait for later. With your current proposal, my proposed future change will apply seamlessly in regard to the stripped pdb files and our distribution bundles. The clash between launchers and libs is annoying, and we also have it for java.exe and java.dll already, though the build is explicitly handling that one somewhere else. Now to review, there are some details I will nitpick about. CreateJmods.gmk: 174, 185: Rule declarations should be indented like any other line inside conditionals. But, I have a problem with these rules as the target is the directory. This will not work well with incremental builds. I would recommend doing a SetupCopyFiles construct instead so you get individual rules for each file. It would look something like this: rename-stripped-pdb = \ $(patsubst %.stripped.pdb,%.pdb,$1) $(eval $(call SetupCopyFiles, COPY_STRIPPED_LIBS, \ SRC := $(LIBS_DIR), \ DEST := $(LIBS_DIR_STRIPPED), \ FILES := $(call FindFiles, $(LIBS_DIR)), \ NAME_MACRO := rename-stripped-pdb, \ )) DEPS += $(COPY_STRIPPED_LIBS) For the corresponding CMD_DIR, the removal of jimage and friends can be done with $(filter ) around The FindFiles call. GenerateLinkOptData.gmk: Please indent inside ifeq block. I would prefer having the TARGETS += inside the conditional block. Seems you also left a commented out endif there. NativeCompilation.gmk 919: You changed the continuation indent from 4 to 2, please revert. /Erik On 2020-02-12 05:26, Langer, Christoph wrote: Hi Magnus, Erik, I started off with Matthias? patch and tried to address your concerns. Especially I explored adding the stripped pdbs to the jmods as well. Here is what I came up with: http://cr.openjdk.java.net/~clanger/webrevs/8237192.0/ It?s a bit hacky in that it?ll copy support/modules_libs to support/modules_libs_stripped and fix the pdbs to ship in there. The same goes for modules_cmds. The problem with that approach is that probably not all dependencies are placed correctly and also there?s a bit more of duplication of binaries in the build directories. I think, it could be repaired eventually by refactoring, e.g. have a support/modules_dbgsymbols folder where the real debug symbol files get placed and used from there. There?s also two additional caveats, one is that jimage.pdb and jpackage.pdb exist twice. Once for the libs and once for the launchers with the same name. This will cause failures when jlinking. I decided to keep the pdbs for the libs. And I also had to take care of the classlist generation, to have the resulting classlist placed in support/modules_libs_stripped as well. I furthermore updated the naming of options and variables, hopefully to your like. And I made the debug output logInfo. I tested (on Windows), both, with --enable-public-debug-symbols and without. Without the option, everything seems as before. With the option enabled, the stripped debug symbols will be installed in the bundles and also in the jmods. ?? Please let me know what you think. Thanks & Best regards Christoph From: Magnus Ihse Bursie Sent: Freitag, 7. Februar 2020 14:09 To: Baesken, Matthias ; Erik Joelsson ; Langer, Christoph ; David Holmes ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images On 2020-02-07 09:50, Baesken, Matthias wrote: Hello, here is a slightly changed new webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.4/ In Bundles.gmk, this line: $(ECHO) found stripped pdb file $$$${f}, we rename it to: $$$${f%stripped.pdb}pdb; \ It looks almost like left-over debug output. If you want to keep it, please rephrase to something more terse, that fits better with the existing style of build messages. Also, it should probably be on the LOG=info level, so add a $(LOG_INFO). In NativeCompilation.gmk: Why not just a simple, ifeq ($(ENABLE_STRIPPED_PDBS), true) $1_EXTRA_LDFLAGS += "-pdbstripped:$$($1_SYMBOLS_DIR)/$$($1_NOSUFFIX).stripped.pdb" endif ? I see no reason to duplicate code. In jdk-options.m4: I'm not 100% sure about the name of the new option. --enable-stripped-pdb does not fully convey the fact that we do not strip the *existing* pdb:s, but instead also add a new type. Maybe --enable-bundle-stripped-pdb? /Magnus (adjusted $(JRE_STRIPPED_PDB_FILES) in Bundles.gmk, that was in the wrong place ) . I think it needs to be a separate option as it's all orthogonal to the main debug symbols file creation. Yes it is a separate option I agree that?s better . One has to set --enable-stripped-pdbs=yes . Best regards , Matthias On 2020-02-06 04:48, Magnus Ihse Bursie wrote: On 2020-02-06 12:36, Langer, Christoph wrote: Hi, let me chime in to this discussion at this point. So, first of all, Matthias, thanks for your work so far. Erik, I fully understand your points and I agree that it's probably a good idea to refactor this whole process of creating these different types of bundles. Our idea is to provide functionality in the build system to add "public" or stripped debug files to the delivery image of the JDK. This would provide better information in case of crashes and would hence allow for better supportability. That's something we'd probably want to enable in SapMachine binary distributions. I very much support the idea of using these stripped pdb files. It has been a long standing issue in hotspot on Windows to not have access to stacktraces. So, can we get to add a configure option like the one proposed by Matthias into the current code base? The option should be turned off by default. If it is switched on, the images/jdk folder in the build directory will have both, the *stripped.pdb files and the standard *.pdb files. However, having *stripped.pdb files around should not matter in terms of functionality (for developers and testing) as they'd not be used in the presence of the "real" pdb files anyway. The actual JDK bundle for delivery would then contain the *stripped.pdb files, renamed to *.pdb. And the debug symbols bundle would have the full *.pdb files only. Both could then be overlaid as needed. I think you raised two concerns. One is that you'd rather like to refactor bundling for several reasons. But I guess, should you eventually get to your refactoring, it shouldn't be a problem to take the functionality of this new option along. The other was regarding JMODs. I believe it's correct, that JMODs have never carried external debug information. For platforms other than MS Windows that's actually not a big problem because debug information can be internalized. And jlink has gotten several additions to set flags for stripping that data to the right level. I assume if jlinked images on Windows should ever be enabled to carry debug information, inclusion of external debug files would have to be added to JMODs and jlink. But that's definitely out of scope here. The argument "jmods have never carried external debug information" just doesn't apply here. Neither has the distribution bundles, for the exact same reason. You really should not compare these new stripped pdb files to the existing debug symbol files, they are different files with different purposes. One is meant to be shipped to customers, the other is not. You say you want to ship these new stripped pdb files with your distribution so that customers have them present when they use your distribution. If you then omit these new files from the jmods, any customer created jlinked image will not have these new stripped pdb files, which IMO is a very weird and unexpected behavior from a customer point of view. Jlinking new images is an integral and promoted way of using a JDK, so any mismatch between the original JDK distribution and what you are able to jlink is a serious discrepancy. So, what do you think? What speaks against adding this option (that is off by default)? My main objective is that you introduce further discrepancies between the original distribution JDK image and what's possible to generate using jlink from that distribution JDK image. My second objective is that the already convoluted bundles creation logic becomes even more convoluted. I believe there is a better possible solution in the build but it will require a lot more work to figure out. All that said, if you still wish to continue, I will stop standing in the way. While Erik will need to comment on this himself, from my POV this looks okay. The conditions are: * This is controlled by a separate option (or perhaps even better as a new argument to --with-native-debug-symbols), so without this, nothing will change. I think it needs to be a separate option as it's all orthogonal to the main debug symbols file creation. * The code need to make sure to separate stripped.pdb and normal pdb files, when enabled. * And there must not be a heavy penalty in added code complexity. /Erik /Magnus Thanks Christoph -----Original Message----- From: build-dev On Behalf Of Erik Joelsson Sent: Donnerstag, 23. Januar 2020 18:49 To: Baesken, Matthias; David Holmes ; 'build-dev at openjdk.java.net' >; 'hotspot-dev at openjdk.java.net' > Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images On 2020-01-23 00:03, Baesken, Matthias wrote: Hi Erik, yes true sorry for answering your comments a bit late . If a user runs jlink and includes all the jmods we ship with the JDK, the result should be essentially equivalent to the original JDK image. The way the stripped pdb files are included in the bundles sort of at the last second of the build here breaks this property. I think we should address this in a separate bug/CR . Maybe. I realize that my proposal below is quite a big task. But on the other hand, I don't think breaking the relationship between the jmods and the distribution bundles is on the table really. Looking for example into a Linux build, I see a lot of debuginfo files in the jdk image (more or less for every shared lib) . But when looking into the jmods of that jdk image , no debuginfo files are in there ( I checked the java.base jmod). So putting the files with debug information into the jmods seems to be something that was not done so far cross platform (or is there some build switch for it that I did not check?) . Maybe to keep the jmods as small as possible . No, we do not put the debuginfo files in the jmods nor the bundles because those are not intended to be shipped to customers. We are currently overlaying them into images/jdk in the build output to make them available for local debugging. (This is convoluted and I would very much like to get away from this practice at some point so that there is a 1-1 mapping between images/jdk and bundles/jdk*-bin.tar.gz.) The stripped pdb files you are proposing are on the contrary intended for shipping to customers (as I understand your proposal) so comparing them with the debuginfo files is not relevant. Now if MS had been kind enough to define a separate file type for the stripped pdbs, so that they could live alongside the full pdbs, we wouldn't have this issue. The heart of the problem is that only one set of files (either stripped or full) can be present and usable in images/jdk at a time. We have 2 main uses for images/jdk. 1. Developer running and debugging locally 2. Serve as the source for generating the distribution bundles We currently have one image serving both of these purposes, which is already creating complicated and convoluted build steps. To properly solve this I would argue for splitting these two apart into two different images for the two different purposes. The build procedure would then be, first build the images for distribution, only containing what should go into each bundle. Then create the developer jdk image by copying files from the distribution images. On Windows, the first image would contain the stripped pdbs and when building the second, those would get overwritten with the full pdbs. Now that I figured out a working model that would solve a bunch of other problems as well, I would love to implement it, but I doubt I will have time in the near future. /Erik To properly implement this, care will need to be taken to juggle the two sets of pdb files around, making sure each build and test use case has the correct one in place where and when it's needed. Quite possibly, we cannot cover all use cases with one build configuration. Developers needing the full debug symbols when debugging locally would likely need to disable the stripped symbols so they get the full symbols everywhere. Possibly this would need to be the default for debug builds and configurable for release builds. From my limited experience , the developers do not work with the bundles (that would contain now after my patch the stripped pds) but with a "normal" jdk image that is created my make all. Best regards, Matthias This still does not address anything in my objection. /Erik On 2020-01-22 07:46, Baesken, Matthias wrote: Hello, here is an updated version : http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/ this one supports a configure switch "--enable-stripped-pdbs" to enable the feature . Best regards, Matthias -----Original Message----- From: Baesken, Matthias Sent: Dienstag, 21. Januar 2020 11:03 To: 'David Holmes'; 'build- dev at openjdk.java.net'; 'hotspot- dev at openjdk.java.net' Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images Hi David , yes I think it makes sense to have a configure option for this . Not everyone would like to have a larger JDK (even it is only a bit larger). From matthias.baesken at sap.com Mon Feb 17 10:00:17 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 17 Feb 2020 10:00:17 +0000 Subject: RFR [XS]: 8239000: handle ContendedPaddingWidth in vm_version_ppc In-Reply-To: <1CB543E3-CBC7-44AA-BF0F-DE0B7C999C06@sap.com> References: <1CB543E3-CBC7-44AA-BF0F-DE0B7C999C06@sap.com> Message-ID: Hi Lutz, > > Hi Matthias, > > your change looks good to me. Please note: I'm not a reviewer! > > I have one question, though. Is there a reason why you only make ppc "look > like the other architectures"? aarch64 is missing that code as well. > The cache line size situation seems to be a little bit complicated on different aarch HW, see for instance the discussion here : https://github.com/littlekernel/lk/issues/172 So I just changed ppc for now . Best regards, Matthias > Thanks for considering! > Lutz > > ?On 13.02.20, 12:18, "hotspot-dev on behalf of Baesken, Matthias" dev-bounces at openjdk.java.net on behalf of matthias.baesken at sap.com> > wrote: > > Hello, please review this small adjustment to vm_version_ppc . > > The ContendedPaddingWidth parameter should be handled in > vm_version_ppc like it is done on other architectures (x86/s390). See > vm_version_sparc/s390/x86 .cpp for comparision. > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8239000 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8239000.0/ > > > Best regards, Matthias > From christoph.langer at sap.com Mon Feb 17 10:05:30 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 17 Feb 2020 10:05:30 +0000 Subject: RFR [XS]: 8239000: handle ContendedPaddingWidth in vm_version_ppc In-Reply-To: References: Message-ID: Hi Matthias, I think this is good. Best regards Christoph > -----Original Message----- > From: hotspot-dev On Behalf Of > Baesken, Matthias > Sent: Donnerstag, 13. Februar 2020 12:17 > To: 'hotspot-dev at openjdk.java.net' > Subject: RFR [XS]: 8239000: handle ContendedPaddingWidth in > vm_version_ppc > > Hello, please review this small adjustment to vm_version_ppc . > > The ContendedPaddingWidth parameter should be handled in > vm_version_ppc like it is done on other architectures (x86/s390). See > vm_version_sparc/s390/x86 .cpp for comparision. > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8239000 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8239000.0/ > > > Best regards, Matthias From christoph.langer at sap.com Mon Feb 17 10:13:47 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 17 Feb 2020 10:13:47 +0000 Subject: RFR(XS) 8239005: [TESTBUG] test/hotspot/jtreg/runtime/StackGuardPages/TestStackGuardPages.java: exeinvoke.c: must initialize static state before calling do_overflow() In-Reply-To: References: Message-ID: Hi Richard, +1 /Christoph > -----Original Message----- > From: hotspot-dev On Behalf Of > Reingruber, Richard > Sent: Donnerstag, 13. Februar 2020 17:46 > To: hotspot-dev at openjdk.java.net; hotspot-runtime- > dev at openjdk.java.net > Subject: RFR(XS) 8239005: [TESTBUG] > test/hotspot/jtreg/runtime/StackGuardPages/TestStackGuardPages.java: > exeinvoke.c: must initialize static state before calling do_overflow() > > Hi, > > please review this small test fix > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239005/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8239005 > > On one of our linuxppc64le boxes the test was failing due to the missing > reset of the static > variable _rec_count before the first call to do_overflow() [1] in > run_native_overflow() [2]. > Consequently the recursion limit set at [3] is too high. On our linuxppc64le > box we then get a > SIGSEGV with SEGV_ACCERR which is reported as test failure. > > On my linuxx86_64 box I get an SIGSEGV with SEGV_MAPERR which is not > reported as error. > > With the fix we don't get a second SIGSEGV. Maybe a second SIGSEGV > should be reported as failure...? > > I tested on linuxx86_64 and linuxppc64le > > Many more details to be found in the bug. > > Thanks, Richard. > > [1] definition of do_overflow() > > http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/ru > ntime/StackGuardPages/exeinvoke.c#l128 > > [2] _rec_count is not reset to 0 before call to do_overflow() > > http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/ru > ntime/StackGuardPages/exeinvoke.c#l168 > > [3] recursion limit is set too high > > http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/ru > ntime/StackGuardPages/exeinvoke.c#l191 From richard.reingruber at sap.com Mon Feb 17 10:25:22 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Mon, 17 Feb 2020 10:25:22 +0000 Subject: RFR(XS) 8239005: [TESTBUG] test/hotspot/jtreg/runtime/StackGuardPages/TestStackGuardPages.java: exeinvoke.c: must initialize static state before calling do_overflow() In-Reply-To: References: Message-ID: Hi Christoph, thanks for reviewing! Cheers, Richard. -----Original Message----- From: Langer, Christoph Sent: Montag, 17. Februar 2020 11:14 To: Reingruber, Richard ; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: RE: RFR(XS) 8239005: [TESTBUG] test/hotspot/jtreg/runtime/StackGuardPages/TestStackGuardPages.java: exeinvoke.c: must initialize static state before calling do_overflow() Hi Richard, +1 /Christoph > -----Original Message----- > From: hotspot-dev On Behalf Of > Reingruber, Richard > Sent: Donnerstag, 13. Februar 2020 17:46 > To: hotspot-dev at openjdk.java.net; hotspot-runtime- > dev at openjdk.java.net > Subject: RFR(XS) 8239005: [TESTBUG] > test/hotspot/jtreg/runtime/StackGuardPages/TestStackGuardPages.java: > exeinvoke.c: must initialize static state before calling do_overflow() > > Hi, > > please review this small test fix > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239005/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8239005 > > On one of our linuxppc64le boxes the test was failing due to the missing > reset of the static > variable _rec_count before the first call to do_overflow() [1] in > run_native_overflow() [2]. > Consequently the recursion limit set at [3] is too high. On our linuxppc64le > box we then get a > SIGSEGV with SEGV_ACCERR which is reported as test failure. > > On my linuxx86_64 box I get an SIGSEGV with SEGV_MAPERR which is not > reported as error. > > With the fix we don't get a second SIGSEGV. Maybe a second SIGSEGV > should be reported as failure...? > > I tested on linuxx86_64 and linuxppc64le > > Many more details to be found in the bug. > > Thanks, Richard. > > [1] definition of do_overflow() > > http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/ru > ntime/StackGuardPages/exeinvoke.c#l128 > > [2] _rec_count is not reset to 0 before call to do_overflow() > > http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/ru > ntime/StackGuardPages/exeinvoke.c#l168 > > [3] recursion limit is set too high > > http://hg.openjdk.java.net/jdk/jdk/file/c27f595b4dd0/test/hotspot/jtreg/ru > ntime/StackGuardPages/exeinvoke.c#l191 From lutz.schmidt at sap.com Mon Feb 17 10:34:07 2020 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Mon, 17 Feb 2020 10:34:07 +0000 Subject: RFR [XS]: 8239000: handle ContendedPaddingWidth in vm_version_ppc In-Reply-To: References: <1CB543E3-CBC7-44AA-BF0F-DE0B7C999C06@sap.com> Message-ID: OK, Matthias. I wasn't aware of this variability. Looks like the ARM gurus will have to take care of that, should it become necessary. Thanks, Lutz ?On 17.02.20, 11:00, "Baesken, Matthias" wrote: Hi Lutz, > > Hi Matthias, > > your change looks good to me. Please note: I'm not a reviewer! > > I have one question, though. Is there a reason why you only make ppc "look > like the other architectures"? aarch64 is missing that code as well. > The cache line size situation seems to be a little bit complicated on different aarch HW, see for instance the discussion here : https://github.com/littlekernel/lk/issues/172 So I just changed ppc for now . Best regards, Matthias > Thanks for considering! > Lutz > > On 13.02.20, 12:18, "hotspot-dev on behalf of Baesken, Matthias" dev-bounces at openjdk.java.net on behalf of matthias.baesken at sap.com> > wrote: > > Hello, please review this small adjustment to vm_version_ppc . > > The ContendedPaddingWidth parameter should be handled in > vm_version_ppc like it is done on other architectures (x86/s390). See > vm_version_sparc/s390/x86 .cpp for comparision. > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8239000 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8239000.0/ > > > Best regards, Matthias > From matthias.baesken at sap.com Mon Feb 17 13:10:15 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 17 Feb 2020 13:10:15 +0000 Subject: RFR [XS] : 8239224: libproc_impl.c previous_thr may be used uninitialized warning Message-ID: Hello, please review this small fix. When building a product build with gcc7 on linux x86_64, and trying out different optimization options, I run into this error when optimizing for space (-Os) : /open_jdk/jdk_5/jdk/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.c: In function 'delete_thread_info': /open_jdk/jdk_5/jdk/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.c:285:26: error: 'previous_thr' may be used uninitialized in this function [-Werror=maybe-uninitialized] previous_thr->next = current_thr->next; ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~ We should probably better initialize previous_thr to avoid this warning. (it does not show up however with the default optimization options ). Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8239224 http://cr.openjdk.java.net/~mbaesken/webrevs/8239224.0/ Thanks, Matthias From david.holmes at oracle.com Mon Feb 17 13:21:05 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 17 Feb 2020 23:21:05 +1000 Subject: RFR [XS] : 8239224: libproc_impl.c previous_thr may be used uninitialized warning In-Reply-To: References: Message-ID: <36f1495f-b77e-7104-2a26-5bb3cc7a1cf4@oracle.com> Hi Matthias, On 17/02/2020 11:10 pm, Baesken, Matthias wrote: > Hello, please review this small fix. > > When building a product build with gcc7 on linux x86_64, and trying out different optimization options, I run into this error when > optimizing for space (-Os) : > > > /open_jdk/jdk_5/jdk/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.c: In function 'delete_thread_info': > /open_jdk/jdk_5/jdk/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_impl.c:285:26: error: 'previous_thr' may be used uninitialized in this function [-Werror=maybe-uninitialized] > previous_thr->next = current_thr->next; > ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~ > > We should probably better initialize previous_thr to avoid this warning. > > (it does not show up however with the default optimization options ). Seems like a spurious warning to me but initializing to NULL is harmless. I don't see the need for the NULL check however as previous_thr can't be NULL if we reach that part of the code. Thanks, David > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8239224 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8239224.0/ > > > Thanks, Matthias > From matthias.baesken at sap.com Mon Feb 17 13:54:09 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 17 Feb 2020 13:54:09 +0000 Subject: RFR [XS] : 8239224: libproc_impl.c previous_thr may be used uninitialized warning In-Reply-To: <36f1495f-b77e-7104-2a26-5bb3cc7a1cf4@oracle.com> References: <36f1495f-b77e-7104-2a26-5bb3cc7a1cf4@oracle.com> Message-ID: Hi David, building with gcc / -Os sometimes gives strange (+different) warnings when comparing to the other O-modes. > I don't see the need for the NULL check however as previous_thr can't be > NULL if we reach that part of the code. What about current_thr != NULL, but current_thr == thr_to_be_removed ? The we would not enter the body of the while-loop , would not return early in 283 , but we would access "previous_thr->next = current_thr->next;" ? 277 while (current_thr && current_thr != thr_to_be_removed) { 278 previous_thr = current_thr; 279 current_thr = current_thr->next; 280 } 281 if (current_thr == NULL) { 282 print_error("Could not find the thread to be removed"); 283 return; 284 } 285 if (previous_thr != NULL) { 286 previous_thr->next = current_thr->next; 287 } Best regards, Matthias > > On 17/02/2020 11:10 pm, Baesken, Matthias wrote: > > Hello, please review this small fix. > > > > When building a product build with gcc7 on linux x86_64, and trying out > different optimization options, I run into this error when > > optimizing for space (-Os) : > > > > > > > /open_jdk/jdk_5/jdk/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_i > mpl.c: In function 'delete_thread_info': > > > /open_jdk/jdk_5/jdk/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_i > mpl.c:285:26: error: 'previous_thr' may be used uninitialized in this function [- > Werror=maybe-uninitialized] > > previous_thr->next = current_thr->next; > > ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~ > > > > We should probably better initialize previous_thr to avoid this warning. > > > > (it does not show up however with the default optimization options ). > > Seems like a spurious warning to me but initializing to NULL is harmless. > > I don't see the need for the NULL check however as previous_thr can't be > NULL if we reach that part of the code. > > Thanks, > David > > > > > Bug/webrev : > > > > https://bugs.openjdk.java.net/browse/JDK-8239224 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8239224.0/ > > > > > > Thanks, Matthias > > From zgu at redhat.com Mon Feb 17 14:51:39 2020 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 17 Feb 2020 09:51:39 -0500 Subject: [15] RFR 8238633: JVMTI heap walk should consult GC for marking oops In-Reply-To: <8ea20a15-bdb9-27b0-c306-154f539a3674@oracle.com> References: <8ea20a15-bdb9-27b0-c306-154f539a3674@oracle.com> Message-ID: <3a5d60fd-04d5-da96-3d79-242d43fdec79@redhat.com> Hi Stefan, Thanks for the review and suggestions, updated accordingly: http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/ > > --- > Previously, the calls to 'mark' and 'visited' were inlineable, but now > every GC has to take a virtual call when marking the objects. My guess > is that this code is slow anyway, and that it doesn't matter too much, > but did you measure the effect of that change with, for example, G1? > I did rough measurement, timing vmTestbase/nsk/jvmti/unit/heap/HeapWalkTests/TestDescription.java test. If you know any tests/benchmarks I should measure, please let me know. Thanks, -Zhengyu > Thanks, > StefanK > >> Test: >> ?? hotspot_gc >> ?? vmTestbase_nsk_jdi >> ?? vmTestbase_nsk_jvmti >> >> Thanks, >> >> -Zhengyu >> >> > From david.holmes at oracle.com Mon Feb 17 21:55:16 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Feb 2020 07:55:16 +1000 Subject: RFR [XS] : 8239224: libproc_impl.c previous_thr may be used uninitialized warning In-Reply-To: References: <36f1495f-b77e-7104-2a26-5bb3cc7a1cf4@oracle.com> Message-ID: <12479df5-561d-3432-27d9-939475b1edb7@oracle.com> On 17/02/2020 11:54 pm, Baesken, Matthias wrote: > Hi David, building with gcc / -Os sometimes gives strange (+different) warnings when comparing to the other O-modes. > >> I don't see the need for the NULL check however as previous_thr can't be >> NULL if we reach that part of the code. > > What about current_thr != NULL, but current_thr == thr_to_be_removed ? > The we would not enter the body of the while-loop , would not return early in 283 , but we would access "previous_thr->next = current_thr->next;" ? If we reach the end of the loop, or we find the target thread in the loop, then previous_thr will be pointing to the last non-NULL element in the linked list. David ----- > > 277 while (current_thr && current_thr != thr_to_be_removed) { > 278 previous_thr = current_thr; > 279 current_thr = current_thr->next; > 280 } > 281 if (current_thr == NULL) { > 282 print_error("Could not find the thread to be removed"); > 283 return; > 284 } > 285 if (previous_thr != NULL) { > 286 previous_thr->next = current_thr->next; > 287 } > > > Best regards, Matthias > > >> >> On 17/02/2020 11:10 pm, Baesken, Matthias wrote: >>> Hello, please review this small fix. >>> >>> When building a product build with gcc7 on linux x86_64, and trying out >> different optimization options, I run into this error when >>> optimizing for space (-Os) : >>> >>> >>> >> /open_jdk/jdk_5/jdk/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_i >> mpl.c: In function 'delete_thread_info': >>> >> /open_jdk/jdk_5/jdk/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_i >> mpl.c:285:26: error: 'previous_thr' may be used uninitialized in this function [- >> Werror=maybe-uninitialized] >>> previous_thr->next = current_thr->next; >>> ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~ >>> >>> We should probably better initialize previous_thr to avoid this warning. >>> >>> (it does not show up however with the default optimization options ). >> >> Seems like a spurious warning to me but initializing to NULL is harmless. >> >> I don't see the need for the NULL check however as previous_thr can't be >> NULL if we reach that part of the code. >> >> Thanks, >> David >> >>> >>> Bug/webrev : >>> >>> https://bugs.openjdk.java.net/browse/JDK-8239224 >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8239224.0/ >>> >>> >>> Thanks, Matthias >>> From glaubitz at physik.fu-berlin.de Tue Feb 18 10:49:44 2020 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Tue, 18 Feb 2020 11:49:44 +0100 Subject: RFR: 8239001: Hotspot build broken on linux-sparc after 8238281 Message-ID: <83670766-525a-f0ab-32bb-20ec0b030cfb@physik.fu-berlin.de> Hi! I'm still looking for feedback on my third revision to address 8239001 yet [1] which fixes Hotspot on linux-sparc [2] after 8238281. Could someone have a look, please? Thanks, Adrian > [1] http://cr.openjdk.java.net/~glaubitz/8239001/webrev.02/ > [2] https://bugs.openjdk.java.net/browse/JDK-8239001 -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From christoph.langer at sap.com Tue Feb 18 11:12:47 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 18 Feb 2020 11:12:47 +0000 Subject: RFR [XS] : 8239224: libproc_impl.c previous_thr may be used uninitialized warning In-Reply-To: <12479df5-561d-3432-27d9-939475b1edb7@oracle.com> References: <36f1495f-b77e-7104-2a26-5bb3cc7a1cf4@oracle.com> <12479df5-561d-3432-27d9-939475b1edb7@oracle.com> Message-ID: Hi, while I agree with David and I think we can't reach line 285 with "previous_thr" uninitialized, I think your code change is benign. And when it quiesces the compiler, it should probably be good to go. Cheers Christoph > -----Original Message----- > From: hotspot-dev On Behalf Of > David Holmes > Sent: Montag, 17. Februar 2020 22:55 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Subject: Re: RFR [XS] : 8239224: libproc_impl.c previous_thr may be used > uninitialized warning > > On 17/02/2020 11:54 pm, Baesken, Matthias wrote: > > Hi David, building with gcc / -Os sometimes gives strange (+different) > warnings when comparing to the other O-modes. > > > >> I don't see the need for the NULL check however as previous_thr can't be > >> NULL if we reach that part of the code. > > > > What about current_thr != NULL, but current_thr == > thr_to_be_removed ? > > The we would not enter the body of the while-loop , would not return > early in 283 , but we would access "previous_thr->next = current_thr- > >next;" ? > > If we reach the end of the loop, or we find the target thread in the > loop, then previous_thr will be pointing to the last non-NULL element in > the linked list. > > David > ----- > > > > > 277 while (current_thr && current_thr != thr_to_be_removed) { > > 278 previous_thr = current_thr; > > 279 current_thr = current_thr->next; > > 280 } > > 281 if (current_thr == NULL) { > > 282 print_error("Could not find the thread to be removed"); > > 283 return; > > 284 } > > 285 if (previous_thr != NULL) { > > 286 previous_thr->next = current_thr->next; > > 287 } > > > > > > Best regards, Matthias > > > > > >> > >> On 17/02/2020 11:10 pm, Baesken, Matthias wrote: > >>> Hello, please review this small fix. > >>> > >>> When building a product build with gcc7 on linux x86_64, and trying out > >> different optimization options, I run into this error when > >>> optimizing for space (-Os) : > >>> > >>> > >>> > >> > /open_jdk/jdk_5/jdk/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_i > >> mpl.c: In function 'delete_thread_info': > >>> > >> > /open_jdk/jdk_5/jdk/src/jdk.hotspot.agent/linux/native/libsaproc/libproc_i > >> mpl.c:285:26: error: 'previous_thr' may be used uninitialized in this > function [- > >> Werror=maybe-uninitialized] > >>> previous_thr->next = current_thr->next; > >>> ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~ > >>> > >>> We should probably better initialize previous_thr to avoid this warning. > >>> > >>> (it does not show up however with the default optimization options ). > >> > >> Seems like a spurious warning to me but initializing to NULL is harmless. > >> > >> I don't see the need for the NULL check however as previous_thr can't be > >> NULL if we reach that part of the code. > >> > >> Thanks, > >> David > >> > >>> > >>> Bug/webrev : > >>> > >>> https://bugs.openjdk.java.net/browse/JDK-8239224 > >>> > >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8239224.0/ > >>> > >>> > >>> Thanks, Matthias > >>> From david.holmes at oracle.com Tue Feb 18 12:39:57 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Feb 2020 22:39:57 +1000 Subject: [15] RFR (M): 8238681: Make -XX:UseSSE flag x86-specific In-Reply-To: <6eb9085e-fe71-c922-085c-3b4dd3d86f1d@oracle.com> References: <070c12dc-bf7b-c5ca-76d7-9ea6127cb4e0@oracle.com> <6eb9085e-fe71-c922-085c-3b4dd3d86f1d@oracle.com> Message-ID: +1 Thanks, David On 15/02/2020 3:44 am, Vladimir Kozlov wrote: > I am fine with -XX:+IgnoreUnrecognizedVMOptions. I am for testing on > more platforms ;) > > Thanks, > Vladimir K > > On 2/14/20 8:10 AM, Vladimir Ivanov wrote: >> Thanks for the feedback, Vladimir. >> >>> compiler/c1/Test6579789.java and compiler/c1/Test6855215.java use >>> UseSSE flag without @requires. >> >> Good catch. But I'm not sure @requires [1] is the best option here: >> >> ??? * -XX:UseSSE=0 has any effect only on x86-32; >> >> ??? * both bugs were reported against x86-32, but at least 6579789 is >> not platform-specific. So it makes sense to run the tests on other >> platforms. >> >> So, I'd prefer to use -XX:+IgnoreUnrecognizedVMOptions instead. What's >> your take on it? >> >> Best regards, >> Vladimir Ivanov >> >> [1] >> diff --git a/test/hotspot/jtreg/compiler/c1/Test6579789.java >> b/test/hotspot/jtreg/compiler/c1/Test6579789.java >> --- a/test/hotspot/jtreg/compiler/c1/Test6579789.java >> +++ b/test/hotspot/jtreg/compiler/c1/Test6579789.java >> @@ -26,6 +26,8 @@ >> ?? * @bug 6579789 >> ?? * @summary Internal error "c1_LinearScan.cpp:1429 Error: >> assert(false,"")" in debuggee with fastdebug VM >> ?? * >> + * @requires os.arch=="i386" >> + * >> ?? * @run main/othervm -Xcomp -XX:UseSSE=0 >> ?? *????? -XX:CompileCommand=compileonly,compiler.c1.Test6579789::bug >> ?? *????? compiler.c1.Test6579789 >> diff --git a/test/hotspot/jtreg/compiler/c1/Test6855215.java >> b/test/hotspot/jtreg/compiler/c1/Test6855215.java >> --- a/test/hotspot/jtreg/compiler/c1/Test6855215.java >> +++ b/test/hotspot/jtreg/compiler/c1/Test6855215.java >> @@ -26,6 +26,8 @@ >> ?? * @bug 6855215 >> ?? * @summary Calculation error (NaN) after about 1500 calculations >> ?? * >> + * @requires os.arch=="i386" >> + * >> ?? * @run main/othervm -Xbatch -XX:UseSSE=0 compiler.c1.Test6855215 >> ?? */ >> >> >>> >>> Otherwise changes seems good. >>> >>> Thanks, >>> Vladimir >>> >>> On 2/10/20 6:19 AM, Vladimir Ivanov wrote: >>>> http://cr.openjdk.java.net/~vlivanov/8238681/webrev.00/ >>>> https://bugs.openjdk.java.net/browse/JDK-8238681 >>>> >>>> Turn UseSSE into x86-specific flag since it doesn't have any meaning >>>> on non-x86 platforms. >>>> >>>> As a cleanup, I made all the code which assumes UseSSE<2 x86-32 >>>> specific. (x86-64 ABI requires SSE2.) >>>> >>>> Also, as a precaution, I made UseSSE obsolete on non-x86 platforms. >>>> >>>> Testing: hs-precheckin-comp, tier1-5, linux x86-32 fastdebug build. >>>> >>>> Thanks! >>>> >>>> Best regards, >>>> Vladimir Ivanov From vladimir.x.ivanov at oracle.com Tue Feb 18 12:40:46 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 18 Feb 2020 15:40:46 +0300 Subject: [15] RFR (M): 8238681: Make -XX:UseSSE flag x86-specific In-Reply-To: References: <070c12dc-bf7b-c5ca-76d7-9ea6127cb4e0@oracle.com> <6eb9085e-fe71-c922-085c-3b4dd3d86f1d@oracle.com> Message-ID: <9eecf8a9-90bf-2d75-a960-0eba1fc90d3c@oracle.com> Thanks for the reviews, David and Vladimir. Best regards, Vladimir Ivanov On 18.02.2020 15:39, David Holmes wrote: > +1 > > Thanks, > David > > On 15/02/2020 3:44 am, Vladimir Kozlov wrote: >> I am fine with -XX:+IgnoreUnrecognizedVMOptions. I am for testing on >> more platforms ;) >> >> Thanks, >> Vladimir K >> >> On 2/14/20 8:10 AM, Vladimir Ivanov wrote: >>> Thanks for the feedback, Vladimir. >>> >>>> compiler/c1/Test6579789.java and compiler/c1/Test6855215.java use >>>> UseSSE flag without @requires. >>> >>> Good catch. But I'm not sure @requires [1] is the best option here: >>> >>> ??? * -XX:UseSSE=0 has any effect only on x86-32; >>> >>> ??? * both bugs were reported against x86-32, but at least 6579789 is >>> not platform-specific. So it makes sense to run the tests on other >>> platforms. >>> >>> So, I'd prefer to use -XX:+IgnoreUnrecognizedVMOptions instead. >>> What's your take on it? >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> [1] >>> diff --git a/test/hotspot/jtreg/compiler/c1/Test6579789.java >>> b/test/hotspot/jtreg/compiler/c1/Test6579789.java >>> --- a/test/hotspot/jtreg/compiler/c1/Test6579789.java >>> +++ b/test/hotspot/jtreg/compiler/c1/Test6579789.java >>> @@ -26,6 +26,8 @@ >>> ?? * @bug 6579789 >>> ?? * @summary Internal error "c1_LinearScan.cpp:1429 Error: >>> assert(false,"")" in debuggee with fastdebug VM >>> ?? * >>> + * @requires os.arch=="i386" >>> + * >>> ?? * @run main/othervm -Xcomp -XX:UseSSE=0 >>> ?? *????? -XX:CompileCommand=compileonly,compiler.c1.Test6579789::bug >>> ?? *????? compiler.c1.Test6579789 >>> diff --git a/test/hotspot/jtreg/compiler/c1/Test6855215.java >>> b/test/hotspot/jtreg/compiler/c1/Test6855215.java >>> --- a/test/hotspot/jtreg/compiler/c1/Test6855215.java >>> +++ b/test/hotspot/jtreg/compiler/c1/Test6855215.java >>> @@ -26,6 +26,8 @@ >>> ?? * @bug 6855215 >>> ?? * @summary Calculation error (NaN) after about 1500 calculations >>> ?? * >>> + * @requires os.arch=="i386" >>> + * >>> ?? * @run main/othervm -Xbatch -XX:UseSSE=0 compiler.c1.Test6855215 >>> ?? */ >>> >>> >>>> >>>> Otherwise changes seems good. >>>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 2/10/20 6:19 AM, Vladimir Ivanov wrote: >>>>> http://cr.openjdk.java.net/~vlivanov/8238681/webrev.00/ >>>>> https://bugs.openjdk.java.net/browse/JDK-8238681 >>>>> >>>>> Turn UseSSE into x86-specific flag since it doesn't have any >>>>> meaning on non-x86 platforms. >>>>> >>>>> As a cleanup, I made all the code which assumes UseSSE<2 x86-32 >>>>> specific. (x86-64 ABI requires SSE2.) >>>>> >>>>> Also, as a precaution, I made UseSSE obsolete on non-x86 platforms. >>>>> >>>>> Testing: hs-precheckin-comp, tier1-5, linux x86-32 fastdebug build. >>>>> >>>>> Thanks! >>>>> >>>>> Best regards, >>>>> Vladimir Ivanov From christoph.langer at sap.com Tue Feb 18 14:47:09 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 18 Feb 2020 14:47:09 +0000 Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images In-Reply-To: References: <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com> <2b92559a-6a1a-119a-1e62-eed6f9d3bfc8@oracle.com> <1989082f-cddf-9d4f-262b-5f9072f46cf7@oracle.com> <9cd2165f-64f0-a096-9b99-aa8bc0798b34@oracle.com> Message-ID: Hi, I had to update my change a little bit ? I forgot to exclude jimage.map, java.map and jpackage.map when copying the cmd debug symbols in Images.gmk. Furthermore, I needed to modify tests jdk/tools/launcher/TestHelper.java and jdk/tools/launcher/VersionCheck.java to cope with debug symbol files in images/jdk/bin on non Windows platforms. http://cr.openjdk.java.net/~clanger/webrevs/8237192.2/ Cheers Christoph From: Langer, Christoph Sent: Montag, 17. Februar 2020 10:17 To: Erik Joelsson ; Magnus Ihse Bursie ; 'build-dev at openjdk.java.net' Cc: 'hotspot-dev at openjdk.java.net' ; Baesken, Matthias Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images Hi Erik, thanks for your review. While I?ve addressed all your points (thanks for the great hints regarding usage of SetupCopyFiles), I also enhanced the configure option a little bit. What you can now pass is --with-external-symbols-in-bundles=. Default setting is none. It?ll require --with-native-debug-symbols=external obviously and is currently only implemented for Windows. This is checked in configure. Depending on the value used, the bundles and jmods will contain either no pdbs, public pdbs or the full pdbs. This configure option could easily be enhanced to work for Linux/Unix/Mac as well ? but I didn?t want to go too far with my change now. What also comes with my change is debug files for executables (launchers, cmds) in images/ which weren?t copied so far in Images.gmk. It?s however needed because bundles are created from these images and with the current bundle logic, stripped pdb files need to exist in images/? to get bundled for option ?public?. I also removed the special handling of the pdb file for java.exe in Launcher-java.base.gmk, as the filtering to make sure it?s not overwriting the pdb for java.dll has to be done in CreateJmods.gmk and Images.gmk anyway. Eventually, when this is all refactored, one should probably generate the pdb files to ship into support/modules_libs and support/modules_cmds and only in the case of public pdbs redirect generation of the full pdbs to something like support/modules_libs_full_debug_info or something like that. Then, from support/modules_libs and support/modules_cmds, one should generate the base image via jlink which can simply be packed into the shipment bundle. And the other image with all debug/demos can be created by copying the base image and then applying the stuff from support/ modules_libs_full_debug_info etc. Here's the new webrev: http://cr.openjdk.java.net/~clanger/webrevs/8237192.1/ Best regards Christoph From: Erik Joelsson > Sent: Mittwoch, 12. Februar 2020 23:17 To: Langer, Christoph >; Magnus Ihse Bursie >; 'build-dev at openjdk.java.net' > Cc: 'hotspot-dev at openjdk.java.net' >; Baesken, Matthias > Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images Hello Christoph, This patch certainly looks better to me, though I agree it's a bit hackish to have to filter and rename the stripped.pdb files twice, once for jmods and again for bundles. I think I'm ok with it for now though. The future improvement I would like to make would be to create two sets of jdk images, one that contains debug symbols and demos, which we continue to use for testing, and another which only contains exactly what we bundle up, including the correctly named top dir. The latter would be created first and used as input for the former. I think lots of things would be cleaner then, especially Bundles.gmk. But, that can wait for later. With your current proposal, my proposed future change will apply seamlessly in regard to the stripped pdb files and our distribution bundles. The clash between launchers and libs is annoying, and we also have it for java.exe and java.dll already, though the build is explicitly handling that one somewhere else. Now to review, there are some details I will nitpick about. CreateJmods.gmk: 174, 185: Rule declarations should be indented like any other line inside conditionals. But, I have a problem with these rules as the target is the directory. This will not work well with incremental builds. I would recommend doing a SetupCopyFiles construct instead so you get individual rules for each file. It would look something like this: rename-stripped-pdb = \ $(patsubst %.stripped.pdb,%.pdb,$1) $(eval $(call SetupCopyFiles, COPY_STRIPPED_LIBS, \ SRC := $(LIBS_DIR), \ DEST := $(LIBS_DIR_STRIPPED), \ FILES := $(call FindFiles, $(LIBS_DIR)), \ NAME_MACRO := rename-stripped-pdb, \ )) DEPS += $(COPY_STRIPPED_LIBS) For the corresponding CMD_DIR, the removal of jimage and friends can be done with $(filter ) around The FindFiles call. GenerateLinkOptData.gmk: Please indent inside ifeq block. I would prefer having the TARGETS += inside the conditional block. Seems you also left a commented out endif there. NativeCompilation.gmk 919: You changed the continuation indent from 4 to 2, please revert. /Erik On 2020-02-12 05:26, Langer, Christoph wrote: Hi Magnus, Erik, I started off with Matthias? patch and tried to address your concerns. Especially I explored adding the stripped pdbs to the jmods as well. Here is what I came up with: http://cr.openjdk.java.net/~clanger/webrevs/8237192.0/ It?s a bit hacky in that it?ll copy support/modules_libs to support/modules_libs_stripped and fix the pdbs to ship in there. The same goes for modules_cmds. The problem with that approach is that probably not all dependencies are placed correctly and also there?s a bit more of duplication of binaries in the build directories. I think, it could be repaired eventually by refactoring, e.g. have a support/modules_dbgsymbols folder where the real debug symbol files get placed and used from there. There?s also two additional caveats, one is that jimage.pdb and jpackage.pdb exist twice. Once for the libs and once for the launchers with the same name. This will cause failures when jlinking. I decided to keep the pdbs for the libs. And I also had to take care of the classlist generation, to have the resulting classlist placed in support/modules_libs_stripped as well. I furthermore updated the naming of options and variables, hopefully to your like. And I made the debug output logInfo. I tested (on Windows), both, with --enable-public-debug-symbols and without. Without the option, everything seems as before. With the option enabled, the stripped debug symbols will be installed in the bundles and also in the jmods. ?? Please let me know what you think. Thanks & Best regards Christoph From: Magnus Ihse Bursie Sent: Freitag, 7. Februar 2020 14:09 To: Baesken, Matthias ; Erik Joelsson ; Langer, Christoph ; David Holmes ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images On 2020-02-07 09:50, Baesken, Matthias wrote: Hello, here is a slightly changed new webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.4/ In Bundles.gmk, this line: $(ECHO) found stripped pdb file $$$${f}, we rename it to: $$$${f%stripped.pdb}pdb; \ It looks almost like left-over debug output. If you want to keep it, please rephrase to something more terse, that fits better with the existing style of build messages. Also, it should probably be on the LOG=info level, so add a $(LOG_INFO). In NativeCompilation.gmk: Why not just a simple, ifeq ($(ENABLE_STRIPPED_PDBS), true) $1_EXTRA_LDFLAGS += "-pdbstripped:$$($1_SYMBOLS_DIR)/$$($1_NOSUFFIX).stripped.pdb" endif ? I see no reason to duplicate code. In jdk-options.m4: I'm not 100% sure about the name of the new option. --enable-stripped-pdb does not fully convey the fact that we do not strip the *existing* pdb:s, but instead also add a new type. Maybe --enable-bundle-stripped-pdb? /Magnus (adjusted $(JRE_STRIPPED_PDB_FILES) in Bundles.gmk, that was in the wrong place ) . I think it needs to be a separate option as it's all orthogonal to the main debug symbols file creation. Yes it is a separate option I agree that?s better . One has to set --enable-stripped-pdbs=yes . Best regards , Matthias On 2020-02-06 04:48, Magnus Ihse Bursie wrote: On 2020-02-06 12:36, Langer, Christoph wrote: Hi, let me chime in to this discussion at this point. So, first of all, Matthias, thanks for your work so far. Erik, I fully understand your points and I agree that it's probably a good idea to refactor this whole process of creating these different types of bundles. Our idea is to provide functionality in the build system to add "public" or stripped debug files to the delivery image of the JDK. This would provide better information in case of crashes and would hence allow for better supportability. That's something we'd probably want to enable in SapMachine binary distributions. I very much support the idea of using these stripped pdb files. It has been a long standing issue in hotspot on Windows to not have access to stacktraces. So, can we get to add a configure option like the one proposed by Matthias into the current code base? The option should be turned off by default. If it is switched on, the images/jdk folder in the build directory will have both, the *stripped.pdb files and the standard *.pdb files. However, having *stripped.pdb files around should not matter in terms of functionality (for developers and testing) as they'd not be used in the presence of the "real" pdb files anyway. The actual JDK bundle for delivery would then contain the *stripped.pdb files, renamed to *.pdb. And the debug symbols bundle would have the full *.pdb files only. Both could then be overlaid as needed. I think you raised two concerns. One is that you'd rather like to refactor bundling for several reasons. But I guess, should you eventually get to your refactoring, it shouldn't be a problem to take the functionality of this new option along. The other was regarding JMODs. I believe it's correct, that JMODs have never carried external debug information. For platforms other than MS Windows that's actually not a big problem because debug information can be internalized. And jlink has gotten several additions to set flags for stripping that data to the right level. I assume if jlinked images on Windows should ever be enabled to carry debug information, inclusion of external debug files would have to be added to JMODs and jlink. But that's definitely out of scope here. The argument "jmods have never carried external debug information" just doesn't apply here. Neither has the distribution bundles, for the exact same reason. You really should not compare these new stripped pdb files to the existing debug symbol files, they are different files with different purposes. One is meant to be shipped to customers, the other is not. You say you want to ship these new stripped pdb files with your distribution so that customers have them present when they use your distribution. If you then omit these new files from the jmods, any customer created jlinked image will not have these new stripped pdb files, which IMO is a very weird and unexpected behavior from a customer point of view. Jlinking new images is an integral and promoted way of using a JDK, so any mismatch between the original JDK distribution and what you are able to jlink is a serious discrepancy. So, what do you think? What speaks against adding this option (that is off by default)? My main objective is that you introduce further discrepancies between the original distribution JDK image and what's possible to generate using jlink from that distribution JDK image. My second objective is that the already convoluted bundles creation logic becomes even more convoluted. I believe there is a better possible solution in the build but it will require a lot more work to figure out. All that said, if you still wish to continue, I will stop standing in the way. While Erik will need to comment on this himself, from my POV this looks okay. The conditions are: * This is controlled by a separate option (or perhaps even better as a new argument to --with-native-debug-symbols), so without this, nothing will change. I think it needs to be a separate option as it's all orthogonal to the main debug symbols file creation. * The code need to make sure to separate stripped.pdb and normal pdb files, when enabled. * And there must not be a heavy penalty in added code complexity. /Erik /Magnus Thanks Christoph -----Original Message----- From: build-dev On Behalf Of Erik Joelsson Sent: Donnerstag, 23. Januar 2020 18:49 To: Baesken, Matthias; David Holmes ; 'build-dev at openjdk.java.net' >; 'hotspot-dev at openjdk.java.net' > Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images On 2020-01-23 00:03, Baesken, Matthias wrote: Hi Erik, yes true sorry for answering your comments a bit late . If a user runs jlink and includes all the jmods we ship with the JDK, the result should be essentially equivalent to the original JDK image. The way the stripped pdb files are included in the bundles sort of at the last second of the build here breaks this property. I think we should address this in a separate bug/CR . Maybe. I realize that my proposal below is quite a big task. But on the other hand, I don't think breaking the relationship between the jmods and the distribution bundles is on the table really. Looking for example into a Linux build, I see a lot of debuginfo files in the jdk image (more or less for every shared lib) . But when looking into the jmods of that jdk image , no debuginfo files are in there ( I checked the java.base jmod). So putting the files with debug information into the jmods seems to be something that was not done so far cross platform (or is there some build switch for it that I did not check?) . Maybe to keep the jmods as small as possible . No, we do not put the debuginfo files in the jmods nor the bundles because those are not intended to be shipped to customers. We are currently overlaying them into images/jdk in the build output to make them available for local debugging. (This is convoluted and I would very much like to get away from this practice at some point so that there is a 1-1 mapping between images/jdk and bundles/jdk*-bin.tar.gz.) The stripped pdb files you are proposing are on the contrary intended for shipping to customers (as I understand your proposal) so comparing them with the debuginfo files is not relevant. Now if MS had been kind enough to define a separate file type for the stripped pdbs, so that they could live alongside the full pdbs, we wouldn't have this issue. The heart of the problem is that only one set of files (either stripped or full) can be present and usable in images/jdk at a time. We have 2 main uses for images/jdk. 1. Developer running and debugging locally 2. Serve as the source for generating the distribution bundles We currently have one image serving both of these purposes, which is already creating complicated and convoluted build steps. To properly solve this I would argue for splitting these two apart into two different images for the two different purposes. The build procedure would then be, first build the images for distribution, only containing what should go into each bundle. Then create the developer jdk image by copying files from the distribution images. On Windows, the first image would contain the stripped pdbs and when building the second, those would get overwritten with the full pdbs. Now that I figured out a working model that would solve a bunch of other problems as well, I would love to implement it, but I doubt I will have time in the near future. /Erik To properly implement this, care will need to be taken to juggle the two sets of pdb files around, making sure each build and test use case has the correct one in place where and when it's needed. Quite possibly, we cannot cover all use cases with one build configuration. Developers needing the full debug symbols when debugging locally would likely need to disable the stripped symbols so they get the full symbols everywhere. Possibly this would need to be the default for debug builds and configurable for release builds. From my limited experience , the developers do not work with the bundles (that would contain now after my patch the stripped pds) but with a "normal" jdk image that is created my make all. Best regards, Matthias This still does not address anything in my objection. /Erik On 2020-01-22 07:46, Baesken, Matthias wrote: Hello, here is an updated version : http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/ this one supports a configure switch "--enable-stripped-pdbs" to enable the feature . Best regards, Matthias -----Original Message----- From: Baesken, Matthias Sent: Dienstag, 21. Januar 2020 11:03 To: 'David Holmes'; 'build- dev at openjdk.java.net'; 'hotspot- dev at openjdk.java.net' Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images Hi David , yes I think it makes sense to have a configure option for this . Not everyone would like to have a larger JDK (even it is only a bit larger). From martinrb at google.com Tue Feb 18 15:13:07 2020 From: martinrb at google.com (Martin Buchholz) Date: Tue, 18 Feb 2020 07:13:07 -0800 Subject: Question about JDK-8043780 (was: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux) In-Reply-To: References: Message-ID: These flags have been part of POSIX since 2001, so it could be time to assume they work unconditionally, but I like maximal portability. Just because O_CLOEXEC is defined at compile time doesn't mean it will work at runtime. I checked both. It seems like a serious bug in AIX if using O_CLOEXEC could return EINVAL for some file systems but not others. If that's true then you would need a fallback using fcntl always. But maybe that's a good idea. Could we write a shared implementation of this for all posix platforms? Perhaps. Try to use O_CLOEXEC. first time it fails with EINVAL, remember that and from then on, fall back to fcntl with FD_CLOEXEC. Humans are not good at getting this sort of code correct! On Tue, Feb 11, 2020 at 6:40 AM Baesken, Matthias wrote: > Hello, new webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.3/ > > > > - Only touching os_aix.cpp ( I leave os_bsd.cpp as it is , might > make the discussion more simple ) > - Added handling of failing O_CLOEXEC ( O_CLOEXEC_is_known_to_work > == -1 ? case ) > > > > Best regards, Matthias > > > > > > *From:* Baesken, Matthias > *Sent:* Dienstag, 11. Februar 2020 14:06 > *To:* 'Thomas St?fe' ; 'Martin Buchholz' < > martinrb at google.com> > *Cc:* 'David Holmes' ; ' > hotspot-dev at openjdk.java.net' > *Subject:* RE: Question about JDK-8043780 (was: RFR: 8237830: support > O_CLOEXEC in os::open on other OS than Linux) > > > > > > >I wonder whether we could shorten the coding to a simple, unconditional > > > > > >::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); > > > > > > Looks like we would need to call first fcntl with F_GETFD AND then F_SETFD > (or where would flags come from?) : > > > > int flags = ::fcntl(fd, F_GETFD); > > ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); > > > > That?s 2 calls that are not really needed . > > > > > > Best regards, Matthias > > > > > > *From:* Baesken, Matthias > *Sent:* Dienstag, 11. Februar 2020 10:09 > *To:* 'Thomas St?fe' ; Martin Buchholz < > martinrb at google.com> > *Cc:* David Holmes ; hotspot-dev at openjdk.java.net > *Subject:* RE: Question about JDK-8043780 (was: RFR: 8237830: support > O_CLOEXEC in os::open on other OS than Linux) > > > > Ping ? any comments on this ? > > > > https://bugs.openjdk.java.net/browse/JDK-8043780 > > > > mentions : ?Backward compatibility needs to be provided via #ifdef and > checking (once) whether O_CLOEXEC works.? > > > > So it was intentional that just ***once*** it is checked whether O_CLOEXEC > works (in the linux version) . > > > > > > Best regards, Matthias > > > > > > *From:* Thomas St?fe > *Sent:* Donnerstag, 6. Februar 2020 11:30 > *To:* Martin Buchholz > *Cc:* David Holmes ; hotspot-dev at openjdk.java.net; > Baesken, Matthias > *Subject:* Question about JDK-8043780 (was: RFR: 8237830: support > O_CLOEXEC in os::open on other OS than Linux) > > > > Hi Martin, > > > > do you remember https://bugs.openjdk.java.net/browse/JDK-8043780 ? > > > > Some questions came up about your patch: > > > > Your original coding does this > check-O_CLOEXEC-after-open-and-set-if-missing-dance, but if it worked ar > least once it will stop doing that. However, if it fails to work it will > retest over and over again. Was this intentional? Do we expect it to start > working at some point? If yes, could it also stop working at some point? > > > > I wonder whether we could shorten the coding to a simple, unconditional > > > > ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); > > > > to be done after open. > > > > If open(CLOEXEC) worked, this would be a unnecessary noop. If it did not, > we set it now. We'd always pay for a fcntl call but I think it is cheap and > open is not called that often anyway. > > > > -- > > > > But then, I wonder whether we can scratch that coding altogether. > O_CLOEXEC has been in the kernel since 2.6.23. This is 13 years old. 2.6 > line topped out at 2.6.39 in 2011, and I would expect anyone still running > on 2.6 to have at least a recent kernel. > > > > What do you think? > > > > Thank you! > > > > Cheers, Thomas > > > > > > On Thu, Feb 6, 2020 at 8:55 AM Baesken, Matthias > wrote: > > Hi Thomas , the coding was borrowed from linux, see : > > > > > https://hg.openjdk.java.net/jdk/jdk/file/7b57401deb0c/src/hotspot/os/linux/os_linux.cpp#l5616 > > > > I would be fine with always calling fcntl (your suggestion 1) , do you > think it was done otherwise back then in os_linux because of > performance issues ? > > Probably os::open in os_linux should be adjusted then to the new > ?always call fcntl? approach too, right ? > > > > > > Best regards, Matthias > > > > > > > > Hi Matthias: > > > > + // Validate that the use of the O_CLOEXEC flag on open above worked. > > + static sig_atomic_t O_CLOEXEC_is_known_to_work = 0; > > + if (!O_CLOEXEC_is_known_to_work) { > > int flags = ::fcntl(fd, F_GETFD); > > - if (flags != -1) > > + if (flags != -1) { > > + if ((flags & FD_CLOEXEC) != 0) { > > + O_CLOEXEC_is_known_to_work = 1; > > + } else { > > ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); > > } > > -#endif > > + } > > + } > > > > This is either too complicated or not complicated enough :) > > > if we know for a fact that open(..O_CLOEXEC) did work at least once, we > never again check. > if we know for a fact that open(..O_CLOEXEC) failed to work, we recheck > again and again. Do we expect it to suddenly start working? If yes, could > it also suddenly start failing? Should we then not check always? > > I propose the following change: > > 1) If open(..O_CLOEXEC) could sometimes fail and sometimes not, or we are > just not sure, I would do the fcntl test every time. Or, even just always > set the flag again with fcntl. So, just remove the > O_CLOEXEC_is_known_to_work conditional variable. > > Which would reduce the code to a one liner: > > + ::fcntl(fd, F_SETFD, flags | FD_CLOEXEC); > > 2) If you are sure that it either always works or fails, you need a > conditional with three states: unknown, works, does not work. And only > check if unknown. > > I personally would prefer (1) because it is way simpler and fcntl is not > such an expensive operation that we have to save time (also, how often do > we open files anyway?). > > ---- > > > > Otherwise looks fine. This is a useful fix. > > > Cheers, Thomas > > > > > > > > > > On Tue, Jan 28, 2020 at 12:32 PM Baesken, Matthias < > matthias.baesken at sap.com> wrote: > > New webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.1/ > > (one 'c' got lost somehow in the older webrev) > > Best regards, Matthias > > From kim.barrett at oracle.com Tue Feb 18 15:55:12 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 18 Feb 2020 10:55:12 -0500 Subject: RFR: 8239001: Hotspot build broken on linux-sparc after 8238281 In-Reply-To: References: <64164050-0a8c-5bb5-572e-0ca04180ad0b@physik.fu-berlin.de> <52DE2202-9671-4BF0-947C-C3164A274D7F@oracle.com> Message-ID: <334DC25E-C123-49F6-BB2A-2693599C3D8F@oracle.com> > On Feb 16, 2020, at 2:58 PM, John Paul Adrian Glaubitz wrote: > > Hi Kim! > > On 2/15/20 10:30 PM, Kim Barrett wrote: >> I prefer the approach using a non-ODR-used dummy to the approach of >> casting of some random number to an address. Non-ODR-used >> declarations are well supported by the standard (C++03 3.2) and widely >> used; that's the basis for the "sizeof trick" metaprogramming idiom, >> for example. > > I've now understood that approach and implemented it. Please see my patch in [1]. > > FWIW, I have tried to put the declaration of RegistersForDebugging outside > the X_offset() functions but that doesn't work. Hope the change is now > okay as is :). > > Adrian > >> [1] http://cr.openjdk.java.net/~glaubitz/8239001/webrev.02/ Nope. That unnecessarily pollutes the global namespace with ?dummy_regs?. The dummy declaration should be in the RegistersForDebugging class as I suggested earlier. But you say that ?doesn?t work?. I thought perhaps that meant I?d made was a mistake in the code snippet I?d sent, but that seems to work for me. So what do you mean by ?doesn?t work?? From glaubitz at physik.fu-berlin.de Tue Feb 18 16:07:16 2020 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Tue, 18 Feb 2020 17:07:16 +0100 Subject: RFR: 8239001: Hotspot build broken on linux-sparc after 8238281 In-Reply-To: <334DC25E-C123-49F6-BB2A-2693599C3D8F@oracle.com> References: <64164050-0a8c-5bb5-572e-0ca04180ad0b@physik.fu-berlin.de> <52DE2202-9671-4BF0-947C-C3164A274D7F@oracle.com> <334DC25E-C123-49F6-BB2A-2693599C3D8F@oracle.com> Message-ID: <958bee2c-ba4d-2029-9152-5f59d32d03e8@physik.fu-berlin.de> On 2/18/20 4:55 PM, Kim Barrett wrote: > Nope. That unnecessarily pollutes the global namespace with ?dummy_regs?. > > The dummy declaration should be in the RegistersForDebugging class as I suggested earlier. > But you say that ?doesn?t work?. I thought perhaps that meant I?d made was a mistake in the > code snippet I?d sent, but that seems to work for me. So what do you mean by ?doesn?t work?? Ah, I completely missed that mail and the code snippet. Sorry. I forgot to use a reference type when I tried the definition within the class which is why it failed for me. I will test the proper approach and post a patch in a few minutes. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From glaubitz at physik.fu-berlin.de Tue Feb 18 16:24:39 2020 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Tue, 18 Feb 2020 17:24:39 +0100 Subject: RFR: 8239001: Hotspot build broken on linux-sparc after 8238281 In-Reply-To: <958bee2c-ba4d-2029-9152-5f59d32d03e8@physik.fu-berlin.de> References: <64164050-0a8c-5bb5-572e-0ca04180ad0b@physik.fu-berlin.de> <52DE2202-9671-4BF0-947C-C3164A274D7F@oracle.com> <334DC25E-C123-49F6-BB2A-2693599C3D8F@oracle.com> <958bee2c-ba4d-2029-9152-5f59d32d03e8@physik.fu-berlin.de> Message-ID: Hi Kim! On 2/18/20 5:07 PM, John Paul Adrian Glaubitz wrote: > I forgot to use a reference type when I tried the definition within the class > which is why it failed for me. I will test the proper approach and post a > patch in a few minutes. Here's the patch with the changes as you suggested [1]. Adrian > [1] http://cr.openjdk.java.net/~glaubitz/8239001/webrev.03/ -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From martin.doerr at sap.com Tue Feb 18 17:05:27 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 18 Feb 2020 17:05:27 +0000 Subject: RFR(XS): 8239363: PPC64: Wrong code generation after JDK-8183574 Message-ID: Hi, JDK-8183574: "Unify the is_power_of_2 functions" broke a piece of PPC64 code. Issue with more details: https://bugs.openjdk.java.net/browse/JDK-8239363 We need to restore usage of log2_long. Webrev: http://cr.openjdk.java.net/~mdoerr/8239363_ppc64_andi/webrev.00/ Please review. Best regards, Martin From erik.joelsson at oracle.com Tue Feb 18 17:08:31 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 18 Feb 2020 09:08:31 -0800 Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images In-Reply-To: References: <2b92559a-6a1a-119a-1e62-eed6f9d3bfc8@oracle.com> <1989082f-cddf-9d4f-262b-5f9072f46cf7@oracle.com> <9cd2165f-64f0-a096-9b99-aa8bc0798b34@oracle.com> Message-ID: <913bd89d-19ad-aeb4-f560-39e278aceacf@oracle.com> Hello Christoph, Thanks for hanging in there, this is now looking good to me. Nice to see a more general solution to the java.pdb problem. /Erik On 2020-02-18 06:47, Langer, Christoph wrote: > > Hi, > > I had to update my change a little bit ? I forgot to exclude > jimage.map, java.map and jpackage.map when copying the cmd debug > symbols in Images.gmk. > > Furthermore, I needed to modify tests > jdk/tools/launcher/TestHelper.java and > jdk/tools/launcher/VersionCheck.java to cope with debug symbol files > in images/jdk/bin on non Windows platforms. > > http://cr.openjdk.java.net/~clanger/webrevs/8237192.2/ > > Cheers > > Christoph > > *From:*Langer, Christoph > *Sent:* Montag, 17. Februar 2020 10:17 > *To:* Erik Joelsson ; Magnus Ihse Bursie > ; 'build-dev at openjdk.java.net' > > *Cc:* 'hotspot-dev at openjdk.java.net' ; > Baesken, Matthias > *Subject:* RE: RFR: 8237192: Generate stripped/public pdbs on Windows > for jdk images > > Hi Erik, > > thanks for your review. While I?ve addressed all your points (thanks > for the great hints regarding usage of SetupCopyFiles), I also > enhanced the configure option a little bit. > > What you can now pass is > --with-external-symbols-in-bundles=. Default setting > is none. It?ll require --with-native-debug-symbols=external obviously > and is currently only implemented for Windows. This is checked in > configure. > > Depending on the value used, the bundles and jmods will contain either > no pdbs, public pdbs or the full pdbs. This configure option could > easily be enhanced to work for Linux/Unix/Mac as well ? but I didn?t > want to go too far with my change now. > > What also comes with my change is debug files for executables > (launchers, cmds) in images/ which weren?t copied so far in > Images.gmk. It?s however needed because bundles are created from these > images and with the current bundle logic, stripped pdb files need to > exist in images/? to get bundled for option ?public?. > > I also removed the special handling of the pdb file for java.exe in > Launcher-java.base.gmk, as the filtering to make sure it?s not > overwriting the pdb for java.dll has to be done in CreateJmods.gmk and > Images.gmk anyway. > > Eventually, when this is all refactored, one should probably generate > the pdb files to ship into support/modules_libs and > support/modules_cmds and only in the case of public pdbs redirect > generation of the full pdbs to something like > support/modules_libs_full_debug_info or something like that. Then, > from support/modules_libs and support/modules_cmds, one should > generate the base image via jlink which can simply be packed into the > shipment bundle. And the other image with all debug/demos can be > created by copying the base image and then applying the stuff from > support/ modules_libs_full_debug_info etc. > > Here's the new webrev: > http://cr.openjdk.java.net/~clanger/webrevs/8237192.1/ > > > Best regards > > Christoph > > *From:*Erik Joelsson > > *Sent:* Mittwoch, 12. Februar 2020 23:17 > *To:* Langer, Christoph >; Magnus Ihse Bursie > >; 'build-dev at openjdk.java.net' > > > *Cc:* 'hotspot-dev at openjdk.java.net' >; Baesken, Matthias > > > *Subject:* Re: RFR: 8237192: Generate stripped/public pdbs on Windows > for jdk images > > Hello Christoph, > > This patch certainly looks better to me, though I agree it's a bit > hackish to have to filter and rename the stripped.pdb files twice, > once for jmods and again for bundles. I think I'm ok with it for now > though. The future improvement I would like to make would be to create > two sets of jdk images, one that contains debug symbols and demos, > which we continue to use for testing, and another which only contains > exactly what we bundle up, including the correctly named top dir. The > latter would be created first and used as input for the former. I > think lots of things would be cleaner then, especially Bundles.gmk. > > But, that can wait for later. With your current proposal, my proposed > future change will apply seamlessly in regard to the stripped pdb > files and our distribution bundles. > > The clash between launchers and libs is annoying, and we also have it > for java.exe and java.dll already, though the build is explicitly > handling that one somewhere else. > > Now to review, there are some details I will nitpick about. > > CreateJmods.gmk: > > 174, 185: Rule declarations should be indented like any other line > inside conditionals. But, I have a problem with these rules as the > target is the directory. This will not work well with incremental > builds. I would recommend doing a SetupCopyFiles construct instead so > you get individual rules for each file. It would look something like this: > > rename-stripped-pdb = \ > ??? $(patsubst %.stripped.pdb,%.pdb,$1) > $(eval $(call SetupCopyFiles, COPY_STRIPPED_LIBS, \ > ??? SRC := $(LIBS_DIR), \ > ??? DEST := $(LIBS_DIR_STRIPPED), \ > ??? FILES := $(call FindFiles, $(LIBS_DIR)), \ > ??? NAME_MACRO := rename-stripped-pdb, \ > )) > > DEPS += $(COPY_STRIPPED_LIBS) > > For the corresponding CMD_DIR, the removal of jimage and friends can > be done with $(filter ) around The FindFiles call. > > GenerateLinkOptData.gmk: > > Please indent inside ifeq block. I would prefer having the TARGETS += > inside the conditional block. Seems you also left a commented out > endif there. > > NativeCompilation.gmk > > 919: You changed the continuation indent from 4 to 2, please revert. > > /Erik > > On 2020-02-12 05:26, Langer, Christoph wrote: > > Hi Magnus, Erik, > > I started off with Matthias? patch and tried to address your concerns. > Especially I explored adding the stripped pdbs to the jmods as well. > Here is what I came up with: > > http://cr.openjdk.java.net/~clanger/webrevs/8237192.0/ > > It?s a bit hacky in that it?ll copy support/modules_libs to > support/modules_libs_stripped and fix the pdbs to ship in there. The > same goes for modules_cmds. The problem with that approach is that > probably not all dependencies are placed correctly and also there?s a > bit more of duplication of binaries in the build directories. I think, > it could be repaired eventually by refactoring, e.g. have a > support/modules_dbgsymbols folder where the real debug symbol files > get placed and used from there. > > There?s also two additional caveats, one is that jimage.pdb and > jpackage.pdb exist twice. Once for the libs and once for the launchers > with the same name. This will cause failures when jlinking. I decided > to keep the pdbs for the libs. And I also had to take care of the > classlist generation, to have the resulting classlist placed in > support/modules_libs_stripped as well. > > I furthermore updated the naming of options and variables, hopefully > to your like. And I made the debug output logInfo. > > I tested (on Windows), both, with --enable-public-debug-symbols and > without. Without the option, everything seems as before. With the > option enabled, the stripped debug symbols will be installed in the > bundles and also in the jmods. ?? > > Please let me know what you think. > > Thanks & Best regards > > Christoph > > *From:*Magnus Ihse Bursie > > *Sent:* Freitag, 7. Februar 2020 14:09 > *To:* Baesken, Matthias > ; Erik Joelsson > ; Langer, > Christoph > ; David Holmes > ; > 'build-dev at openjdk.java.net ' > ; > 'hotspot-dev at openjdk.java.net ' > > *Subject:* Re: RFR: 8237192: Generate stripped/public pdbs on Windows > for jdk images > > On 2020-02-07 09:50, Baesken, Matthias wrote: > > > > Hello, here is? a slightly changed? new? webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.4/ > > In Bundles.gmk, this line: > > $(ECHO) found stripped pdb file $$$${f}, we rename it to: > $$$${f%stripped.pdb}pdb; \ > > It looks almost like left-over debug output. If you want to keep it, > please rephrase to something more terse, that fits better with the > existing style of build messages. Also, it should probably be on the > LOG=info level, so add a $(LOG_INFO). > > In NativeCompilation.gmk: > Why not just a simple, > ????????? ifeq ($(ENABLE_STRIPPED_PDBS), true) > ??????????? $1_EXTRA_LDFLAGS += > "-pdbstripped:$$($1_SYMBOLS_DIR)/$$($1_NOSUFFIX).stripped.pdb" > ????????? endif > ? > I see no reason to duplicate code. > > In jdk-options.m4: > > I'm not 100% sure about the name of the new option. > --enable-stripped-pdb does not fully convey the fact that we do not > strip the *existing* pdb:s, but instead also add a new type. Maybe > --enable-bundle-stripped-pdb? > > /Magnus > > > > > > (adjusted $(JRE_STRIPPED_PDB_FILES)? in Bundles.gmk, that was in the wrong place )? . > > > > I think it needs to be a separate option as it's all orthogonal to the > > main debug symbols file creation. > > > > Yes it is a separate option I agree that?s better .? One has to set? --enable-stripped-pdbs=yes? . > > > > Best regards , Matthias > > > > > > > > On 2020-02-06 04:48, Magnus Ihse Bursie wrote: > > On 2020-02-06 12:36, Langer, Christoph wrote: > > Hi, > > > > let me chime in to this discussion at this point. So, first of all, Matthias, > > thanks for your work so far. > > > > Erik, I fully understand your points and I agree that it's probably a good > > idea to refactor this whole process of creating these different types of > > bundles. > > > > Our idea is to provide functionality in the build system to add "public" or > > stripped debug files to the delivery image of the JDK. This would provide > > better information in case of crashes and would hence allow for better > > supportability. That's something we'd probably want to enable in > > SapMachine binary distributions. > > I very much support the idea of using these stripped pdb files. It has > > been a long standing issue in hotspot on Windows to not have access to > > stacktraces. > > So, can we get to add a configure option like the one proposed by > > Matthias into the current code base? > > The option should be turned off by default. If it is switched on, the > > images/jdk folder in the build directory will have both, the *stripped.pdb > > files and the standard *.pdb files. However, having *stripped.pdb files > > around should not matter in terms of functionality (for developers and > > testing) as they'd not be used in the presence of the "real" pdb files anyway. > > The actual JDK bundle for delivery would then contain the *stripped.pdb > > files, renamed to *.pdb. And the debug symbols bundle would have the full > > *.pdb files only. Both could then be overlaid as needed. > > > > I think you raised two concerns. > > One is that you'd rather like to refactor bundling for several reasons. But I > > guess, should you eventually get to your refactoring, it shouldn't be a > > problem to take the functionality of this new option along. > > The other was regarding JMODs. I believe it's correct, that JMODs have > > never carried external debug information. For platforms other than MS > > Windows that's actually not a big problem because debug information can be > > internalized. And jlink has gotten several additions to set flags for stripping > > that data to the right level. I assume if jlinked images on Windows should > > ever be enabled to carry debug information, inclusion of external debug files > > would have to be added to JMODs and jlink. But that's definitely out of scope > > here. > > The argument "jmods have never carried external debug information" just > > doesn't apply here. Neither has the distribution bundles, for the exact > > same reason. You really should not compare these new stripped pdb files > > to the existing debug symbol files, they are different files with > > different purposes. One is meant to be shipped to customers, the other > > is not. You say you want to ship these new stripped pdb files with your > > distribution so that customers have them present when they use your > > distribution. If you then omit these new files from the jmods, any > > customer created jlinked image will not have these new stripped pdb > > files, which IMO is a very weird and unexpected behavior from a customer > > point of view. Jlinking new images is an integral and promoted way of > > using a JDK, so any mismatch between the original JDK distribution and > > what you are able to jlink is a serious discrepancy. > > So, what do you think? What speaks against adding this option (that is off > > by default)? > > > > My main objective is that you introduce further discrepancies between > > the original distribution JDK image and what's possible to generate > > using jlink from that distribution JDK image. My second objective is > > that the already convoluted bundles creation logic becomes even more > > convoluted. I believe there is a better possible solution in the build > > but it will require a lot more work to figure out. > > > > All that said, if you still wish to continue, I will stop standing in > > the way. > > > > While Erik will need to comment on this himself, from my POV this > > looks okay. The conditions are: > > > > * This is controlled by a separate option (or perhaps even better as a > > new argument to --with-native-debug-symbols), so without this, nothing > > will change. > > > > I think it needs to be a separate option as it's all orthogonal to the > > main debug symbols file creation. > > * The code need to make sure to separate stripped.pdb and normal pdb > > files, when enabled. > > > > * And there must not be a heavy penalty in added code complexity. > > > > /Erik > > /Magnus > > > > Thanks > > Christoph > > > > -----Original Message----- > > From: build-dev ? On Behalf Of > > Erik > > Joelsson > > Sent: Donnerstag, 23. Januar 2020 18:49 > > To: Baesken, Matthias ; David Holmes > > ; 'build-dev at openjdk.java.net ' > dev at openjdk.java.net >; 'hotspot-dev at openjdk.java.net ' > dev at openjdk.java.net > > > Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows > > for > > jdk images > > > > > > On 2020-01-23 00:03, Baesken, Matthias wrote: > > Hi Erik,? yes true sorry for answering? your comments a bit late . > > > > If a user runs jlink and includes all the jmods we ship with the JDK, the > > result > > should be essentially equivalent to the original JDK image. The way > > the > > stripped pdb files are included in the bundles sort of at the last > > second of the build here breaks this property. > > I think we should address this in a separate bug/CR . > > Maybe. I realize that my proposal below is quite a big task. But on the > > other hand, I don't think breaking the relationship between the jmods > > and the distribution bundles is on the table really. > > Looking for example? into a Linux? build, I see? a lot of debuginfo files? in > > the > > jdk image (more or less for every shared lib)? . > > But when looking into the jmods? of that jdk image ,? no debuginfo files > > are > > in there ( I checked the java.base jmod). > > So? putting? the? files with debug information into the jmods? seems to > > be > > something that was not done so far? cross platform? (or is there some > > build > > switch? for it that I did not check?) . > > Maybe to keep the jmods as small as possible . > > No, we do not put the debuginfo files in the jmods nor the bundles > > because those are not intended to be shipped to customers. We are > > currently overlaying them into images/jdk in the build output to make > > them available for local debugging. (This is convoluted and I would very > > much like to get away from this practice at some point so that there is > > a 1-1 mapping between images/jdk and bundles/jdk*-bin.tar.gz.) The > > stripped pdb files you are proposing are on the contrary intended for > > shipping to customers (as I understand your proposal) so comparing > > them > > with the debuginfo files is not relevant. > > > > Now if MS had been kind enough to define a separate file type for the > > stripped pdbs, so that they could live alongside the full pdbs, we > > wouldn't have this issue. The heart of the problem is that only one set > > of files (either stripped or full) can be present and usable in > > images/jdk at a time. We have 2 main uses for images/jdk. > > > > 1. Developer running and debugging locally > > > > 2. Serve as the source for generating the distribution bundles > > > > We currently have one image serving both of these purposes, which is > > already creating complicated and convoluted build steps. To properly > > solve this I would argue for splitting these two apart into two > > different images for the two different purposes. The build procedure > > would then be, first build the images for distribution, only containing > > what should go into each bundle. Then create the developer jdk image > > by > > copying files from the distribution images. On Windows, the first image > > would contain the stripped pdbs and when building the second, those > > would get overwritten with the full pdbs. > > > > Now that I figured out a working model that would solve a bunch of > > other > > problems as well, I would love to implement it, but I doubt I will have > > time in the near future. > > > > /Erik > > > > To properly implement this, care will need to be taken to juggle the > > two > > sets of pdb files around, making sure each build and test use case has > > the correct one in place where and when it's needed. Quite possibly, > > we > > cannot cover all use cases with one build configuration. Developers > > needing the full debug symbols when debugging locally would likely > > need > > to disable the stripped symbols so they get the full symbols > > everywhere. > > Possibly this would need to be the default for debug builds and > > configurable for release builds. > > ? From my limited experience , the developers? do not work with the > > bundles (that? would contain now after my patch the stripped pds)? but > > with > > a "normal" jdk image that? is created my make all. > > Best regards, Matthias > > > > > > > > This still does not address anything in my objection. > > > > /Erik > > > > On 2020-01-22 07:46, Baesken, Matthias wrote: > > Hello,? here is an updated version? : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/ > > > > this one supports a configure switch? "--enable-stripped-pdbs"??? to > > enable > > the feature . > > Best regards, Matthias > > > > > > -----Original Message----- > > From: Baesken, Matthias > > Sent: Dienstag, 21. Januar 2020 11:03 > > To: 'David Holmes' ; 'build- > > dev at openjdk.java.net ' ; 'hotspot- > > dev at openjdk.java.net ' > > Subject: RE: RFR: 8237192: Generate stripped/public pdbs on > > Windows > > for > > jdk images > > > > > > Hi David ,? yes I think it makes sense to have a configure option for > > this . > > Not everyone would like to have a larger JDK (even it is only a bit > > larger). > > > From thomas.stuefe at gmail.com Tue Feb 18 19:33:48 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 18 Feb 2020 20:33:48 +0100 Subject: RFR(XS): 8239363: PPC64: Wrong code generation after JDK-8183574 In-Reply-To: References: Message-ID: Looks good Martin. Brackets look a bit excessive but I guess one never has enough brackets :) Cheers, Thomas On Tue, Feb 18, 2020 at 6:06 PM Doerr, Martin wrote: > Hi, > > JDK-8183574: "Unify the is_power_of_2 functions" broke a piece of PPC64 > code. > > Issue with more details: > https://bugs.openjdk.java.net/browse/JDK-8239363 > > We need to restore usage of log2_long. > Webrev: > http://cr.openjdk.java.net/~mdoerr/8239363_ppc64_andi/webrev.00/ > > Please review. > > Best regards, > Martin > > From stefan.karlsson at oracle.com Tue Feb 18 20:05:17 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 18 Feb 2020 21:05:17 +0100 Subject: RFR(XS): 8239363: PPC64: Wrong code generation after JDK-8183574 In-Reply-To: References: Message-ID: Looks good. That was an unintentional change. Thanks, StefanK On 2020-02-18 18:05, Doerr, Martin wrote: > > Hi, > > JDK-8183574: ?Unify the is_power_of_2 functions? broke a piece of > PPC64 code. > > Issue with more details: > > https://bugs.openjdk.java.net/browse/JDK-8239363 > > We need to restore usage of log2_long. > > Webrev: > > http://cr.openjdk.java.net/~mdoerr/8239363_ppc64_andi/webrev.00/ > > Please review. > > Best regards, > > Martin > From kim.barrett at oracle.com Tue Feb 18 20:26:28 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 18 Feb 2020 15:26:28 -0500 Subject: RFR: 8239001: Hotspot build broken on linux-sparc after 8238281 In-Reply-To: References: <64164050-0a8c-5bb5-572e-0ca04180ad0b@physik.fu-berlin.de> <52DE2202-9671-4BF0-947C-C3164A274D7F@oracle.com> <334DC25E-C123-49F6-BB2A-2693599C3D8F@oracle.com> <958bee2c-ba4d-2029-9152-5f59d32d03e8@physik.fu-berlin.de> Message-ID: <53D98EE7-9FD1-4E49-95A2-17A782CC733E@oracle.com> > On Feb 18, 2020, at 11:24 AM, John Paul Adrian Glaubitz wrote: > > Hi Kim! > > On 2/18/20 5:07 PM, John Paul Adrian Glaubitz wrote: >> I forgot to use a reference type when I tried the definition within the class >> which is why it failed for me. I will test the proper approach and post a >> patch in a few minutes. > > Here's the patch with the changes as you suggested [1]. > > Adrian > >> [1] http://cr.openjdk.java.net/~glaubitz/8239001/webrev.03/ Looks good. > > -- > .''`. John Paul Adrian Glaubitz > : :' : Debian Developer - glaubitz at debian.org > `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de > `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From serguei.spitsyn at oracle.com Tue Feb 18 20:53:38 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 18 Feb 2020 12:53:38 -0800 Subject: [15] RFR 8238633: JVMTI heap walk should consult GC for marking oops In-Reply-To: <3a5d60fd-04d5-da96-3d79-242d43fdec79@redhat.com> References: <8ea20a15-bdb9-27b0-c306-154f539a3674@oracle.com> <3a5d60fd-04d5-da96-3d79-242d43fdec79@redhat.com> Message-ID: <723567f2-a61b-302f-0853-32de05cb562d@oracle.com> Hi Zhengyu, It looks okay to me. The testing you do looks enough for verification. But I'm not sure about performance testing though. Thanks, Serguei On 2/17/20 6:51 AM, Zhengyu Gu wrote: > Hi Stefan, > > Thanks for the review and suggestions, updated accordingly: > > http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/ > >> >> --- >> Previously, the calls to 'mark' and 'visited' were inlineable, but >> now every GC has to take a virtual call when marking the objects. My >> guess is that this code is slow anyway, and that it doesn't matter >> too much, but did you measure the effect of that change with, for >> example, G1? >> > I did rough measurement, timing > vmTestbase/nsk/jvmti/unit/heap/HeapWalkTests/TestDescription.java test. > > If you know any tests/benchmarks I should measure, please let me know. > > Thanks, > > -Zhengyu > > >> Thanks, >> StefanK >> >>> Test: >>> ?? hotspot_gc >>> ?? vmTestbase_nsk_jdi >>> ?? vmTestbase_nsk_jvmti >>> >>> Thanks, >>> >>> -Zhengyu >>> >>> >> > From lutz.schmidt at sap.com Tue Feb 18 20:58:58 2020 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Tue, 18 Feb 2020 20:58:58 +0000 Subject: RFR(XS): 8239363: PPC64: Wrong code generation after JDK-8183574 In-Reply-To: References: Message-ID: <767C414C-B395-4FF9-97D1-70A1FAFCBD0A@sap.com> Hi Martin, your change looks good. Thanks for quickly detecting and fixing this. Please note: I'm not a reviewer! Regards, Lutz ?On 18.02.20, 18:08, "hotspot-dev on behalf of Doerr, Martin" wrote: Hi, JDK-8183574: "Unify the is_power_of_2 functions" broke a piece of PPC64 code. Issue with more details: https://bugs.openjdk.java.net/browse/JDK-8239363 We need to restore usage of log2_long. Webrev: http://cr.openjdk.java.net/~mdoerr/8239363_ppc64_andi/webrev.00/ Please review. Best regards, Martin From zgu at redhat.com Tue Feb 18 21:18:38 2020 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 18 Feb 2020 16:18:38 -0500 Subject: [15] RFR 8238633: JVMTI heap walk should consult GC for marking oops In-Reply-To: <723567f2-a61b-302f-0853-32de05cb562d@oracle.com> References: <8ea20a15-bdb9-27b0-c306-154f539a3674@oracle.com> <3a5d60fd-04d5-da96-3d79-242d43fdec79@redhat.com> <723567f2-a61b-302f-0853-32de05cb562d@oracle.com> Message-ID: Hi Serguei, On 2/18/20 3:53 PM, serguei.spitsyn at oracle.com wrote: > Hi Zhengyu, > > It looks okay to me. > The testing you do looks enough for verification. > But I'm not sure about performance testing though. Thanks for reviewing. I asked around my colleagues, if they knew any benchmarks for JVMTI heap walk, the answer was 'no'. As Stefan mentioned, this is a slow piece of code, I doubt if there is any benchmarks for it. I would appreciate it if any performance people can chip in. Thanks, -Zhengyu > > Thanks, > Serguei > > > On 2/17/20 6:51 AM, Zhengyu Gu wrote: >> Hi Stefan, >> >> Thanks for the review and suggestions, updated accordingly: >> >> http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/ >> >>> >>> --- >>> Previously, the calls to 'mark' and 'visited' were inlineable, but >>> now every GC has to take a virtual call when marking the objects. My >>> guess is that this code is slow anyway, and that it doesn't matter >>> too much, but did you measure the effect of that change with, for >>> example, G1? >>> >> I did rough measurement, timing >> vmTestbase/nsk/jvmti/unit/heap/HeapWalkTests/TestDescription.java test. >> >> If you know any tests/benchmarks I should measure, please let me know. >> >> Thanks, >> >> -Zhengyu >> >> >>> Thanks, >>> StefanK >>> >>>> Test: >>>> ?? hotspot_gc >>>> ?? vmTestbase_nsk_jdi >>>> ?? vmTestbase_nsk_jvmti >>>> >>>> Thanks, >>>> >>>> -Zhengyu >>>> >>>> >>> >> > From coleen.phillimore at oracle.com Tue Feb 18 23:46:58 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 18 Feb 2020 18:46:58 -0500 Subject: RFR[S]: 8239014 -XX:-UseEmptySlotsInSupers sometime fails to reproduce the layout of the old code In-Reply-To: <1E2C4D7D-AF8D-4642-90A0-7FC1A53B6F3A@oracle.com> References: <4C24883A-9EED-45E2-94C3-4AD91ACBAEC7@oracle.com> <3d807131-157c-b6ff-0b64-a2bba8e03175@oracle.com> <1E2C4D7D-AF8D-4642-90A0-7FC1A53B6F3A@oracle.com> Message-ID: <9322553e-f3dd-0ebc-eba3-49c8deee7fb7@oracle.com> This looks good.? It makes sense, we pack small fields into the gap between _klass and a long/double in the old layout. Coleen On 2/14/20 1:34 PM, Frederic Parain wrote: > Hi David, > > Here?s a new webrev with a test case: > http://cr.openjdk.java.net/~fparain/8239014/webrev.01/ > > When -XX:-UseEmptySlotsInSupers is used, the field layout code > emulates the behavior of the old code by appending fields after > the rounded up end of the super class, adding primitive fields > by decreasing size, then oops. > > The issue was that this strategy doesn?t fill the gap after the > header when the class pointer is 4 bytes and the first field is > either a long or a double. There was a special case in the old > code to handle this situation, but it was missing in the > emulation mode of the new code. > > Thank you, > > Fred > > >> On Feb 13, 2020, at 17:53, David Holmes wrote: >> >> Hi Fred, >> >> Can you add a testcase for this? I can't really evaluate the change just by looking at the code. >> >> Thanks, >> David >> >> On 14/02/2020 6:28 am, Frederic Parain wrote: >>> Please review this small change that fix the behavior of-XX: -UseEmptySlotsInSupers. >>> CR: https://bugs.openjdk.java.net/browse/JDK-8239014 >>> Webrev: http://cr.openjdk.java.net/~fparain/8239014/webrev.00/index.html >>> Thank you, >>> Fred From david.holmes at oracle.com Wed Feb 19 03:27:16 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Feb 2020 13:27:16 +1000 Subject: RFR: 8239001: Hotspot build broken on linux-sparc after 8238281 In-Reply-To: <53D98EE7-9FD1-4E49-95A2-17A782CC733E@oracle.com> References: <64164050-0a8c-5bb5-572e-0ca04180ad0b@physik.fu-berlin.de> <52DE2202-9671-4BF0-947C-C3164A274D7F@oracle.com> <334DC25E-C123-49F6-BB2A-2693599C3D8F@oracle.com> <958bee2c-ba4d-2029-9152-5f59d32d03e8@physik.fu-berlin.de> <53D98EE7-9FD1-4E49-95A2-17A782CC733E@oracle.com> Message-ID: <69af486f-4272-0207-9de1-204a0223ed13@oracle.com> Looks good to me too. David On 19/02/2020 6:26 am, Kim Barrett wrote: >> On Feb 18, 2020, at 11:24 AM, John Paul Adrian Glaubitz wrote: >> >> Hi Kim! >> >> On 2/18/20 5:07 PM, John Paul Adrian Glaubitz wrote: >>> I forgot to use a reference type when I tried the definition within the class >>> which is why it failed for me. I will test the proper approach and post a >>> patch in a few minutes. >> >> Here's the patch with the changes as you suggested [1]. >> >> Adrian >> >>> [1] http://cr.openjdk.java.net/~glaubitz/8239001/webrev.03/ > > Looks good. > >> >> -- >> .''`. John Paul Adrian Glaubitz >> : :' : Debian Developer - glaubitz at debian.org >> `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de >> `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 > > From david.holmes at oracle.com Wed Feb 19 03:41:09 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Feb 2020 13:41:09 +1000 Subject: RFR[S]: 8239014 -XX:-UseEmptySlotsInSupers sometime fails to reproduce the layout of the old code In-Reply-To: <1E2C4D7D-AF8D-4642-90A0-7FC1A53B6F3A@oracle.com> References: <4C24883A-9EED-45E2-94C3-4AD91ACBAEC7@oracle.com> <3d807131-157c-b6ff-0b64-a2bba8e03175@oracle.com> <1E2C4D7D-AF8D-4642-90A0-7FC1A53B6F3A@oracle.com> Message-ID: Hi Fred, Sorry for the delay getting back to this. On 15/02/2020 4:34 am, Frederic Parain wrote: > Hi David, > > Here?s a new webrev with a test case: > http://cr.openjdk.java.net/~fparain/8239014/webrev.01/ > > When -XX:-UseEmptySlotsInSupers is used, the field layout code > emulates the behavior of the old code by appending fields after > the rounded up end of the super class, adding primitive fields > by decreasing size, then oops. > > The issue was that this strategy doesn?t fill the gap after the > header when the class pointer is 4 bytes and the first field is > either a long or a double. There was a special case in the old > code to handle this situation, but it was missing in the > emulation mode of the new code. Thanks for clarifying. Changes look good. David > Thank you, > > Fred > > >> On Feb 13, 2020, at 17:53, David Holmes wrote: >> >> Hi Fred, >> >> Can you add a testcase for this? I can't really evaluate the change just by looking at the code. >> >> Thanks, >> David >> >> On 14/02/2020 6:28 am, Frederic Parain wrote: >>> Please review this small change that fix the behavior of-XX: -UseEmptySlotsInSupers. >>> CR: https://bugs.openjdk.java.net/browse/JDK-8239014 >>> Webrev: http://cr.openjdk.java.net/~fparain/8239014/webrev.00/index.html >>> Thank you, >>> Fred > From aoqi at loongson.cn Wed Feb 19 06:20:25 2020 From: aoqi at loongson.cn (Ao Qi) Date: Wed, 19 Feb 2020 14:20:25 +0800 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation Message-ID: Hi all, Could you please review this patch? JBS: https://bugs.openjdk.java.net/browse/JDK-8239423 webrev: http://cr.openjdk.java.net/~aoqi/8239423/webrev.00/ (22 lines changed: 19 ins; 0 del; 3 mod; 153 unchg) Thanks, Ao Qi From david.holmes at oracle.com Wed Feb 19 07:01:54 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Feb 2020 17:01:54 +1000 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: References: Message-ID: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> Hi, On 19/02/2020 4:20 pm, Ao Qi wrote: > Hi all, > > Could you please review this patch? > > JBS: https://bugs.openjdk.java.net/browse/JDK-8239423 > webrev: http://cr.openjdk.java.net/~aoqi/8239423/webrev.00/ (22 lines > changed: 19 ins; 0 del; 3 mod; 153 unchg) I'm not seeing the connection between the reported bug and the fix. It seems to me that passing -XX:-TieredCompilation to a test that explicitly sets -XX:TieredStopAtLevel=n is simply not something that can work. In fact I'm surprised it doesn't trigger an error when validating the command-line args. You fix seems more related to whether C1 and/or C2 are available. ?? Cheers, David > Thanks, > Ao Qi > From aoqi at loongson.cn Wed Feb 19 07:26:20 2020 From: aoqi at loongson.cn (Ao Qi) Date: Wed, 19 Feb 2020 15:26:20 +0800 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> Message-ID: On Wed, Feb 19, 2020 at 3:01 PM David Holmes wrote: > > Hi, > > On 19/02/2020 4:20 pm, Ao Qi wrote: > > Hi all, > > > > Could you please review this patch? > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8239423 > > webrev: http://cr.openjdk.java.net/~aoqi/8239423/webrev.00/ (22 lines > > changed: 19 ins; 0 del; 3 mod; 153 unchg) > > I'm not seeing the connection between the reported bug and the fix. It > seems to me that passing -XX:-TieredCompilation to a test that > explicitly sets -XX:TieredStopAtLevel=n is simply not something that can > work. In fact I'm surprised it doesn't trigger an error when validating > the command-line args. "-XX:-TieredCompilation" in the title is a way to trigger the bug, not the cause. Sorry for the confusion. The bug also can be triggered by the jdk configured with --with-jvm-features=-compiler1. > > You fix seems more related to whether C1 and/or C2 are available. Yes. Cheers, Ao Qi > > ?? > > Cheers, > David > > > Thanks, > > Ao Qi > > From david.holmes at oracle.com Wed Feb 19 07:40:12 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Feb 2020 17:40:12 +1000 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> Message-ID: <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> On 19/02/2020 5:26 pm, Ao Qi wrote: > On Wed, Feb 19, 2020 at 3:01 PM David Holmes wrote: >> >> Hi, >> >> On 19/02/2020 4:20 pm, Ao Qi wrote: >>> Hi all, >>> >>> Could you please review this patch? >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8239423 >>> webrev: http://cr.openjdk.java.net/~aoqi/8239423/webrev.00/ (22 lines >>> changed: 19 ins; 0 del; 3 mod; 153 unchg) >> >> I'm not seeing the connection between the reported bug and the fix. It >> seems to me that passing -XX:-TieredCompilation to a test that >> explicitly sets -XX:TieredStopAtLevel=n is simply not something that can >> work. In fact I'm surprised it doesn't trigger an error when validating >> the command-line args. > > "-XX:-TieredCompilation" in the title is a way to trigger the bug, not > the cause. Sorry for the confusion. The bug also can be triggered by > the jdk configured with --with-jvm-features=-compiler1. > >> >> You fix seems more related to whether C1 and/or C2 are available. > > Yes. I've had a look at this and I'm seeing the problem but I don't yet think this is the right fix. I see some general issues with the ability to use the compiler flags in a way that may not make sense (to me anyway); then I see an issue with the way the test determines the available levels. Looking at the test it determines the available levels as follows: - If tiered is enabled - use TieredStopAtLevel=n to set the maximum level and thus the available levels - else if server - only level is 4 - else // client, minimal, emulated client - only level is 1 So with -TieredCompilation or a server only build the level is set to 4. But the WhiteBox::compile_method rejects that as an invalid level for the configuration. I don't think the fix is to require C2, in fact I don't see how that fixes the problem anyway if you have a C2 only build without tiered because the test will still use level=4 which the WB rejects: STDOUT: WB error: invalid compilation level 4 The test in WB is: if (comp_level > MIN2((CompLevel) TieredStopAtLevel, CompLevel_highest_tier)) but I don't know how either of those values are set when (a) tiered is disabled; or (b) you have a C1 or C2 only build (which should imply tiered is disabled as well). Cheers, David > Cheers, > Ao Qi > >> >> ?? >> >> Cheers, >> David >> >>> Thanks, >>> Ao Qi >>> > From martin.doerr at sap.com Wed Feb 19 09:11:02 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 19 Feb 2020 09:11:02 +0000 Subject: RFR(XS): 8239363: PPC64: Wrong code generation after JDK-8183574 In-Reply-To: <767C414C-B395-4FF9-97D1-70A1FAFCBD0A@sap.com> References: <767C414C-B395-4FF9-97D1-70A1FAFCBD0A@sap.com> Message-ID: Thanks for reviewing! Pushed. Best regards, Martin > -----Original Message----- > From: Schmidt, Lutz > Sent: Dienstag, 18. Februar 2020 21:59 > To: Doerr, Martin ; hotspot-dev developers > (hotspot-dev at openjdk.java.net) ; Stefan > Karlsson > Subject: Re: RFR(XS): 8239363: PPC64: Wrong code generation after JDK- > 8183574 > > Hi Martin, > > your change looks good. Thanks for quickly detecting and fixing this. Please > note: I'm not a reviewer! > > Regards, > Lutz > > ?On 18.02.20, 18:08, "hotspot-dev on behalf of Doerr, Martin" bounces at openjdk.java.net on behalf of martin.doerr at sap.com> wrote: > > Hi, > > JDK-8183574: "Unify the is_power_of_2 functions" broke a piece of PPC64 > code. > > Issue with more details: > https://bugs.openjdk.java.net/browse/JDK-8239363 > > We need to restore usage of log2_long. > Webrev: > http://cr.openjdk.java.net/~mdoerr/8239363_ppc64_andi/webrev.00/ > > Please review. > > Best regards, > Martin > > From richard.reingruber at sap.com Wed Feb 19 10:44:29 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Wed, 19 Feb 2020 10:44:29 +0000 Subject: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS Message-ID: Hi, please review this small test fix Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239449/webrev.0/ Bug: https://bugs.openjdk.java.net/browse/JDK-8239449 On linux systems with older glibc versions (e.g. 2.11) the option -XX:+AdjustStackSizeForTLS has no effect, because the private function __pthread_get_minstack is not provided, and the test fails. With the fix the test is skipped instead. Thanks, Richard. From david.holmes at oracle.com Wed Feb 19 12:32:08 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Feb 2020 22:32:08 +1000 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> Message-ID: <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> On 19/02/2020 5:40 pm, David Holmes wrote: > On 19/02/2020 5:26 pm, Ao Qi wrote: >> On Wed, Feb 19, 2020 at 3:01 PM David Holmes >> wrote: >>> >>> Hi, >>> >>> On 19/02/2020 4:20 pm, Ao Qi wrote: >>>> Hi all, >>>> >>>> Could you please review this patch? >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8239423 >>>> webrev: http://cr.openjdk.java.net/~aoqi/8239423/webrev.00/ (22 lines >>>> changed: 19 ins; 0 del; 3 mod; 153 unchg) >>> >>> I'm not seeing the connection between the reported bug and the fix. It >>> seems to me that passing -XX:-TieredCompilation to a test that >>> explicitly sets -XX:TieredStopAtLevel=n is simply not something that can >>> work. In fact I'm surprised it doesn't trigger an error when validating >>> the command-line args. >> >> "-XX:-TieredCompilation" in the title is a way to trigger the bug, not >> the cause. Sorry for the confusion. The bug also can be triggered by >> the jdk configured with --with-jvm-features=-compiler1. >> >>> >>> You fix seems more related to whether C1 and/or C2 are available. >> >> Yes. > > I've had a look at this and I'm seeing the problem but I don't yet think > this is the right fix. I see some general issues with the ability to use > the compiler flags in a way that may not make sense (to me anyway); then > I see an issue with the way the test determines the available levels. > > Looking at the test it determines the available levels as follows: > - If tiered is enabled > ??? - use TieredStopAtLevel=n to set the maximum level and thus the > available levels > - else if server > ? - only level is 4 > - else // client, minimal, emulated client > ? - only level is 1 > > So with -TieredCompilation or a server only build the level is set to 4. > But the WhiteBox::compile_method rejects that as an invalid level for > the configuration. > > I don't think the fix is to require C2, in fact I don't see how that > fixes the problem anyway if you have a C2 only build without tiered > because the test will still use level=4 which the WB rejects: > > STDOUT: > WB error: invalid compilation level 4 > > The test in WB is: > > ? if (comp_level > MIN2((CompLevel) TieredStopAtLevel, > CompLevel_highest_tier)) > > but I don't know how either of those values are set when (a) tiered is > disabled; or (b) you have a C1 or C2 only build (which should imply > tiered is disabled as well). Digging deeper if C2 exists then CompLevel_highest_tier should be set as: #if defined(COMPILER2) CompLevel CompLevel_highest_tier = CompLevel_full_optimization; where: CompLevel_full_optimization = 4 and from the command-line: -XX:TieredStopAtLevel=4 and so I don't understand how the above failure can trigger. One of those fields must not be what is is expected to be. David > Cheers, > David > >> Cheers, >> Ao Qi >> >>> >>> ?? >>> >>> Cheers, >>> David >>> >>>> Thanks, >>>> Ao Qi >>>> >> From david.holmes at oracle.com Wed Feb 19 13:15:23 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 19 Feb 2020 23:15:23 +1000 Subject: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS In-Reply-To: References: Message-ID: <6f818b3d-96d7-eb48-71f0-5b666844ba42@oracle.com> Hi Richard, On 19/02/2020 8:44 pm, Reingruber, Richard wrote: > Hi, > > please review this small test fix > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239449/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8239449 > > On linux systems with older glibc versions (e.g. 2.11) the option -XX:+AdjustStackSizeForTLS has no effect, because the private function __pthread_get_minstack is not provided, and the test fails. With the fix the test is skipped instead. How old is glibc 2.11? Is it not simpler for the test to try to dlsym the function itself to see if it exists? Thanks, David > Thanks, Richard. > From aoqi at loongson.cn Wed Feb 19 13:30:39 2020 From: aoqi at loongson.cn (Ao Qi) Date: Wed, 19 Feb 2020 21:30:39 +0800 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> Message-ID: On Wed, Feb 19, 2020 at 8:32 PM David Holmes wrote: > > On 19/02/2020 5:40 pm, David Holmes wrote: > > On 19/02/2020 5:26 pm, Ao Qi wrote: > >> On Wed, Feb 19, 2020 at 3:01 PM David Holmes > >> wrote: > >>> > >>> Hi, > >>> > >>> On 19/02/2020 4:20 pm, Ao Qi wrote: > >>>> Hi all, > >>>> > >>>> Could you please review this patch? > >>>> > >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8239423 > >>>> webrev: http://cr.openjdk.java.net/~aoqi/8239423/webrev.00/ (22 lines > >>>> changed: 19 ins; 0 del; 3 mod; 153 unchg) > >>> > >>> I'm not seeing the connection between the reported bug and the fix. It > >>> seems to me that passing -XX:-TieredCompilation to a test that > >>> explicitly sets -XX:TieredStopAtLevel=n is simply not something that can > >>> work. In fact I'm surprised it doesn't trigger an error when validating > >>> the command-line args. > >> > >> "-XX:-TieredCompilation" in the title is a way to trigger the bug, not > >> the cause. Sorry for the confusion. The bug also can be triggered by > >> the jdk configured with --with-jvm-features=-compiler1. > >> > >>> > >>> You fix seems more related to whether C1 and/or C2 are available. > >> > >> Yes. > > > > I've had a look at this and I'm seeing the problem but I don't yet think > > this is the right fix. I see some general issues with the ability to use > > the compiler flags in a way that may not make sense (to me anyway); then > > I see an issue with the way the test determines the available levels. > > > > Looking at the test it determines the available levels as follows: > > - If tiered is enabled > > - use TieredStopAtLevel=n to set the maximum level and thus the > > available levels > > - else if server > > - only level is 4 > > - else // client, minimal, emulated client > > - only level is 1 > > > > So with -TieredCompilation or a server only build the level is set to 4. > > But the WhiteBox::compile_method rejects that as an invalid level for > > the configuration. > > > > I don't think the fix is to require C2, in fact I don't see how that > > fixes the problem anyway if you have a C2 only build without tiered > > because the test will still use level=4 which the WB rejects: > > > > STDOUT: > > WB error: invalid compilation level 4 > > > > The test in WB is: > > > > if (comp_level > MIN2((CompLevel) TieredStopAtLevel, > > CompLevel_highest_tier)) > > > > but I don't know how either of those values are set when (a) tiered is > > disabled; or (b) you have a C1 or C2 only build (which should imply > > tiered is disabled as well). Thanks very much for your hint, David. > > Digging deeper if C2 exists then CompLevel_highest_tier should be set as: > > #if defined(COMPILER2) > CompLevel CompLevel_highest_tier = CompLevel_full_optimization; > > where: > > CompLevel_full_optimization = 4 > > and from the command-line: > > -XX:TieredStopAtLevel=4 > > and so I don't understand how the above failure can trigger. One of > those fields must not be what is is expected to be. I think it is the "-XX:TieredStopAtLevel=1" that trigger the failure. I was also seeing this issue. Here is a table: |mode |cl|tsl|cht|>| |+tiered(default)|4 |4 |4 |false| |+tiered(default)|1 |1 |4 |false| |-tiered |4 |4 |4 |false| |-tiered |4 |1 |4 |*true*| |c2 only |4 |4 |4 |false| |c2 only |4 |1 |4 |*true*| |c1 only |1 |4 |1 |false| |c1 only |1 |1 |1 |false| cl: comp_level(level to set) tsl: TieredStopAtLevel cht: CompLevel_highest_tier ">": comp_level > min(TieredStopAtLevel, CompLevel_highest_tier) If ">" is true, the test failed. I think the combination of "c2 only(comp_level=4/CompLevel_highest_tier=4) and TieredStopAtLevel=1" should be skipped. I am not sure if we should skip the combination of "-tiered(comp_level=4/CompLevel_highest_tier=4) and TieredStopAtLevel=1" or we should set comp_level=1 when -tiered is used. I need more time to follow up on this issue. Cheers, Ao Qi > > David > > > > Cheers, > > David > > > >> Cheers, > >> Ao Qi > >> > >>> > >>> ?? > >>> > >>> Cheers, > >>> David > >>> > >>>> Thanks, > >>>> Ao Qi > >>>> > >> From frederic.parain at oracle.com Wed Feb 19 13:37:22 2020 From: frederic.parain at oracle.com (Frederic Parain) Date: Wed, 19 Feb 2020 08:37:22 -0500 Subject: RFR[S]: 8239014 -XX:-UseEmptySlotsInSupers sometime fails to reproduce the layout of the old code In-Reply-To: References: <4C24883A-9EED-45E2-94C3-4AD91ACBAEC7@oracle.com> <3d807131-157c-b6ff-0b64-a2bba8e03175@oracle.com> <1E2C4D7D-AF8D-4642-90A0-7FC1A53B6F3A@oracle.com> Message-ID: <1C4B043C-AAC2-40E7-8123-E827967C97A7@oracle.com> Thank you David. Fred > On Feb 18, 2020, at 22:41, David Holmes wrote: > > Hi Fred, > > Sorry for the delay getting back to this. > > On 15/02/2020 4:34 am, Frederic Parain wrote: >> Hi David, >> Here?s a new webrev with a test case: >> http://cr.openjdk.java.net/~fparain/8239014/webrev.01/ >> When -XX:-UseEmptySlotsInSupers is used, the field layout code >> emulates the behavior of the old code by appending fields after >> the rounded up end of the super class, adding primitive fields >> by decreasing size, then oops. >> The issue was that this strategy doesn?t fill the gap after the >> header when the class pointer is 4 bytes and the first field is >> either a long or a double. There was a special case in the old >> code to handle this situation, but it was missing in the >> emulation mode of the new code. > > Thanks for clarifying. > > Changes look good. > > David > >> Thank you, >> Fred >>> On Feb 13, 2020, at 17:53, David Holmes wrote: >>> >>> Hi Fred, >>> >>> Can you add a testcase for this? I can't really evaluate the change just by looking at the code. >>> >>> Thanks, >>> David >>> >>> On 14/02/2020 6:28 am, Frederic Parain wrote: >>>> Please review this small change that fix the behavior of-XX: -UseEmptySlotsInSupers. >>>> CR: https://bugs.openjdk.java.net/browse/JDK-8239014 >>>> Webrev: http://cr.openjdk.java.net/~fparain/8239014/webrev.00/index.html >>>> Thank you, >>>> Fred From frederic.parain at oracle.com Wed Feb 19 13:37:34 2020 From: frederic.parain at oracle.com (Frederic Parain) Date: Wed, 19 Feb 2020 08:37:34 -0500 Subject: RFR[S]: 8239014 -XX:-UseEmptySlotsInSupers sometime fails to reproduce the layout of the old code In-Reply-To: <9322553e-f3dd-0ebc-eba3-49c8deee7fb7@oracle.com> References: <4C24883A-9EED-45E2-94C3-4AD91ACBAEC7@oracle.com> <3d807131-157c-b6ff-0b64-a2bba8e03175@oracle.com> <1E2C4D7D-AF8D-4642-90A0-7FC1A53B6F3A@oracle.com> <9322553e-f3dd-0ebc-eba3-49c8deee7fb7@oracle.com> Message-ID: <31174B3E-0AD0-49F0-91E6-384D187F35DD@oracle.com> Thank you Coleen, Fred > On Feb 18, 2020, at 18:46, coleen.phillimore at oracle.com wrote: > > This looks good. It makes sense, we pack small fields into the gap between _klass and a long/double in the old layout. > Coleen > > On 2/14/20 1:34 PM, Frederic Parain wrote: >> Hi David, >> >> Here?s a new webrev with a test case: >> http://cr.openjdk.java.net/~fparain/8239014/webrev.01/ >> >> When -XX:-UseEmptySlotsInSupers is used, the field layout code >> emulates the behavior of the old code by appending fields after >> the rounded up end of the super class, adding primitive fields >> by decreasing size, then oops. >> >> The issue was that this strategy doesn?t fill the gap after the >> header when the class pointer is 4 bytes and the first field is >> either a long or a double. There was a special case in the old >> code to handle this situation, but it was missing in the >> emulation mode of the new code. >> >> Thank you, >> >> Fred >> >> >>> On Feb 13, 2020, at 17:53, David Holmes wrote: >>> >>> Hi Fred, >>> >>> Can you add a testcase for this? I can't really evaluate the change just by looking at the code. >>> >>> Thanks, >>> David >>> >>> On 14/02/2020 6:28 am, Frederic Parain wrote: >>>> Please review this small change that fix the behavior of-XX: -UseEmptySlotsInSupers. >>>> CR: https://bugs.openjdk.java.net/browse/JDK-8239014 >>>> Webrev: http://cr.openjdk.java.net/~fparain/8239014/webrev.00/index.html >>>> Thank you, >>>> Fred > From richard.reingruber at sap.com Wed Feb 19 13:45:26 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Wed, 19 Feb 2020 13:45:26 +0000 Subject: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS In-Reply-To: <6f818b3d-96d7-eb48-71f0-5b666844ba42@oracle.com> References: <6f818b3d-96d7-eb48-71f0-5b666844ba42@oracle.com> Message-ID: Hi David, > > > > On linux systems with older glibc versions (e.g. 2.11) the option -XX:+AdjustStackSizeForTLS > > has no effect, because the private function __pthread_get_minstack is not provided, and the > > test fails. With the fix the test is skipped instead. > > How old is glibc 2.11? > About 10y. Seems it was released 2009-11-03 [1]. It is used in SLES 11 SP4. Unfortunately I don't know, when __pthread_get_minstack was added to glibc. > Is it not simpler for the test to try to dlsym the function itself to > see if it exists? > I've done something similar before. I checked for the phrase "Lookup of __pthread_get_minstack failed", but that has the disadvantage that the test would not fail if __pthread_get_minstack gets removed in a future glibc version. Thanks, Richard. [1] https://sourceware.org/glibc/wiki/Glibc%20Timeline -----Original Message----- From: David Holmes Sent: Mittwoch, 19. Februar 2020 14:15 To: Reingruber, Richard ; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS Hi Richard, On 19/02/2020 8:44 pm, Reingruber, Richard wrote: > Hi, > > please review this small test fix > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239449/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8239449 > > On linux systems with older glibc versions (e.g. 2.11) the option -XX:+AdjustStackSizeForTLS has no effect, because the private function __pthread_get_minstack is not provided, and the test fails. With the fix the test is skipped instead. How old is glibc 2.11? Is it not simpler for the test to try to dlsym the function itself to see if it exists? Thanks, David > Thanks, Richard. > From matthias.baesken at sap.com Wed Feb 19 14:21:55 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 19 Feb 2020 14:21:55 +0000 Subject: RFR [XS]: 8239462: jdk.hotspot.agent misses some ReleaseStringUTFChars calls in case of early returns Message-ID: Hello, please review this small change . We miss at a few places ReleaseStringUTFChars calls in the native jdk.hotspot.agent coding. This happens in case of early returns . Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8239462 http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.0/ Thanks, Matthias From christoph.langer at sap.com Wed Feb 19 15:42:22 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 19 Feb 2020 15:42:22 +0000 Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images In-Reply-To: <913bd89d-19ad-aeb4-f560-39e278aceacf@oracle.com> References: <2b92559a-6a1a-119a-1e62-eed6f9d3bfc8@oracle.com> <1989082f-cddf-9d4f-262b-5f9072f46cf7@oracle.com> <9cd2165f-64f0-a096-9b99-aa8bc0798b34@oracle.com> <913bd89d-19ad-aeb4-f560-39e278aceacf@oracle.com> Message-ID: Hi Erik, thanks for the review. Seems like it?s looking good now in our CI. I?ll run it through jdk-submit before pushing. Best regards Christoph From: Erik Joelsson Sent: Dienstag, 18. Februar 2020 18:09 To: Langer, Christoph ; Magnus Ihse Bursie ; 'build-dev at openjdk.java.net' Cc: 'hotspot-dev at openjdk.java.net' ; Baesken, Matthias Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images Hello Christoph, Thanks for hanging in there, this is now looking good to me. Nice to see a more general solution to the java.pdb problem. /Erik On 2020-02-18 06:47, Langer, Christoph wrote: Hi, I had to update my change a little bit ? I forgot to exclude jimage.map, java.map and jpackage.map when copying the cmd debug symbols in Images.gmk. Furthermore, I needed to modify tests jdk/tools/launcher/TestHelper.java and jdk/tools/launcher/VersionCheck.java to cope with debug symbol files in images/jdk/bin on non Windows platforms. http://cr.openjdk.java.net/~clanger/webrevs/8237192.2/ Cheers Christoph From: Langer, Christoph Sent: Montag, 17. Februar 2020 10:17 To: Erik Joelsson ; Magnus Ihse Bursie ; 'build-dev at openjdk.java.net' Cc: 'hotspot-dev at openjdk.java.net' ; Baesken, Matthias Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images Hi Erik, thanks for your review. While I?ve addressed all your points (thanks for the great hints regarding usage of SetupCopyFiles), I also enhanced the configure option a little bit. What you can now pass is --with-external-symbols-in-bundles=. Default setting is none. It?ll require --with-native-debug-symbols=external obviously and is currently only implemented for Windows. This is checked in configure. Depending on the value used, the bundles and jmods will contain either no pdbs, public pdbs or the full pdbs. This configure option could easily be enhanced to work for Linux/Unix/Mac as well ? but I didn?t want to go too far with my change now. What also comes with my change is debug files for executables (launchers, cmds) in images/ which weren?t copied so far in Images.gmk. It?s however needed because bundles are created from these images and with the current bundle logic, stripped pdb files need to exist in images/? to get bundled for option ?public?. I also removed the special handling of the pdb file for java.exe in Launcher-java.base.gmk, as the filtering to make sure it?s not overwriting the pdb for java.dll has to be done in CreateJmods.gmk and Images.gmk anyway. Eventually, when this is all refactored, one should probably generate the pdb files to ship into support/modules_libs and support/modules_cmds and only in the case of public pdbs redirect generation of the full pdbs to something like support/modules_libs_full_debug_info or something like that. Then, from support/modules_libs and support/modules_cmds, one should generate the base image via jlink which can simply be packed into the shipment bundle. And the other image with all debug/demos can be created by copying the base image and then applying the stuff from support/ modules_libs_full_debug_info etc. Here's the new webrev: http://cr.openjdk.java.net/~clanger/webrevs/8237192.1/ Best regards Christoph From: Erik Joelsson > Sent: Mittwoch, 12. Februar 2020 23:17 To: Langer, Christoph >; Magnus Ihse Bursie >; 'build-dev at openjdk.java.net' > Cc: 'hotspot-dev at openjdk.java.net' >; Baesken, Matthias > Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images Hello Christoph, This patch certainly looks better to me, though I agree it's a bit hackish to have to filter and rename the stripped.pdb files twice, once for jmods and again for bundles. I think I'm ok with it for now though. The future improvement I would like to make would be to create two sets of jdk images, one that contains debug symbols and demos, which we continue to use for testing, and another which only contains exactly what we bundle up, including the correctly named top dir. The latter would be created first and used as input for the former. I think lots of things would be cleaner then, especially Bundles.gmk. But, that can wait for later. With your current proposal, my proposed future change will apply seamlessly in regard to the stripped pdb files and our distribution bundles. The clash between launchers and libs is annoying, and we also have it for java.exe and java.dll already, though the build is explicitly handling that one somewhere else. Now to review, there are some details I will nitpick about. CreateJmods.gmk: 174, 185: Rule declarations should be indented like any other line inside conditionals. But, I have a problem with these rules as the target is the directory. This will not work well with incremental builds. I would recommend doing a SetupCopyFiles construct instead so you get individual rules for each file. It would look something like this: rename-stripped-pdb = \ $(patsubst %.stripped.pdb,%.pdb,$1) $(eval $(call SetupCopyFiles, COPY_STRIPPED_LIBS, \ SRC := $(LIBS_DIR), \ DEST := $(LIBS_DIR_STRIPPED), \ FILES := $(call FindFiles, $(LIBS_DIR)), \ NAME_MACRO := rename-stripped-pdb, \ )) DEPS += $(COPY_STRIPPED_LIBS) For the corresponding CMD_DIR, the removal of jimage and friends can be done with $(filter ) around The FindFiles call. GenerateLinkOptData.gmk: Please indent inside ifeq block. I would prefer having the TARGETS += inside the conditional block. Seems you also left a commented out endif there. NativeCompilation.gmk 919: You changed the continuation indent from 4 to 2, please revert. /Erik On 2020-02-12 05:26, Langer, Christoph wrote: Hi Magnus, Erik, I started off with Matthias? patch and tried to address your concerns. Especially I explored adding the stripped pdbs to the jmods as well. Here is what I came up with: http://cr.openjdk.java.net/~clanger/webrevs/8237192.0/ It?s a bit hacky in that it?ll copy support/modules_libs to support/modules_libs_stripped and fix the pdbs to ship in there. The same goes for modules_cmds. The problem with that approach is that probably not all dependencies are placed correctly and also there?s a bit more of duplication of binaries in the build directories. I think, it could be repaired eventually by refactoring, e.g. have a support/modules_dbgsymbols folder where the real debug symbol files get placed and used from there. There?s also two additional caveats, one is that jimage.pdb and jpackage.pdb exist twice. Once for the libs and once for the launchers with the same name. This will cause failures when jlinking. I decided to keep the pdbs for the libs. And I also had to take care of the classlist generation, to have the resulting classlist placed in support/modules_libs_stripped as well. I furthermore updated the naming of options and variables, hopefully to your like. And I made the debug output logInfo. I tested (on Windows), both, with --enable-public-debug-symbols and without. Without the option, everything seems as before. With the option enabled, the stripped debug symbols will be installed in the bundles and also in the jmods. ?? Please let me know what you think. Thanks & Best regards Christoph From: Magnus Ihse Bursie Sent: Freitag, 7. Februar 2020 14:09 To: Baesken, Matthias ; Erik Joelsson ; Langer, Christoph ; David Holmes ; 'build-dev at openjdk.java.net' ; 'hotspot-dev at openjdk.java.net' Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images On 2020-02-07 09:50, Baesken, Matthias wrote: Hello, here is a slightly changed new webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.4/ In Bundles.gmk, this line: $(ECHO) found stripped pdb file $$$${f}, we rename it to: $$$${f%stripped.pdb}pdb; \ It looks almost like left-over debug output. If you want to keep it, please rephrase to something more terse, that fits better with the existing style of build messages. Also, it should probably be on the LOG=info level, so add a $(LOG_INFO). In NativeCompilation.gmk: Why not just a simple, ifeq ($(ENABLE_STRIPPED_PDBS), true) $1_EXTRA_LDFLAGS += "-pdbstripped:$$($1_SYMBOLS_DIR)/$$($1_NOSUFFIX).stripped.pdb" endif ? I see no reason to duplicate code. In jdk-options.m4: I'm not 100% sure about the name of the new option. --enable-stripped-pdb does not fully convey the fact that we do not strip the *existing* pdb:s, but instead also add a new type. Maybe --enable-bundle-stripped-pdb? /Magnus (adjusted $(JRE_STRIPPED_PDB_FILES) in Bundles.gmk, that was in the wrong place ) . I think it needs to be a separate option as it's all orthogonal to the main debug symbols file creation. Yes it is a separate option I agree that?s better . One has to set --enable-stripped-pdbs=yes . Best regards , Matthias On 2020-02-06 04:48, Magnus Ihse Bursie wrote: On 2020-02-06 12:36, Langer, Christoph wrote: Hi, let me chime in to this discussion at this point. So, first of all, Matthias, thanks for your work so far. Erik, I fully understand your points and I agree that it's probably a good idea to refactor this whole process of creating these different types of bundles. Our idea is to provide functionality in the build system to add "public" or stripped debug files to the delivery image of the JDK. This would provide better information in case of crashes and would hence allow for better supportability. That's something we'd probably want to enable in SapMachine binary distributions. I very much support the idea of using these stripped pdb files. It has been a long standing issue in hotspot on Windows to not have access to stacktraces. So, can we get to add a configure option like the one proposed by Matthias into the current code base? The option should be turned off by default. If it is switched on, the images/jdk folder in the build directory will have both, the *stripped.pdb files and the standard *.pdb files. However, having *stripped.pdb files around should not matter in terms of functionality (for developers and testing) as they'd not be used in the presence of the "real" pdb files anyway. The actual JDK bundle for delivery would then contain the *stripped.pdb files, renamed to *.pdb. And the debug symbols bundle would have the full *.pdb files only. Both could then be overlaid as needed. I think you raised two concerns. One is that you'd rather like to refactor bundling for several reasons. But I guess, should you eventually get to your refactoring, it shouldn't be a problem to take the functionality of this new option along. The other was regarding JMODs. I believe it's correct, that JMODs have never carried external debug information. For platforms other than MS Windows that's actually not a big problem because debug information can be internalized. And jlink has gotten several additions to set flags for stripping that data to the right level. I assume if jlinked images on Windows should ever be enabled to carry debug information, inclusion of external debug files would have to be added to JMODs and jlink. But that's definitely out of scope here. The argument "jmods have never carried external debug information" just doesn't apply here. Neither has the distribution bundles, for the exact same reason. You really should not compare these new stripped pdb files to the existing debug symbol files, they are different files with different purposes. One is meant to be shipped to customers, the other is not. You say you want to ship these new stripped pdb files with your distribution so that customers have them present when they use your distribution. If you then omit these new files from the jmods, any customer created jlinked image will not have these new stripped pdb files, which IMO is a very weird and unexpected behavior from a customer point of view. Jlinking new images is an integral and promoted way of using a JDK, so any mismatch between the original JDK distribution and what you are able to jlink is a serious discrepancy. So, what do you think? What speaks against adding this option (that is off by default)? My main objective is that you introduce further discrepancies between the original distribution JDK image and what's possible to generate using jlink from that distribution JDK image. My second objective is that the already convoluted bundles creation logic becomes even more convoluted. I believe there is a better possible solution in the build but it will require a lot more work to figure out. All that said, if you still wish to continue, I will stop standing in the way. While Erik will need to comment on this himself, from my POV this looks okay. The conditions are: * This is controlled by a separate option (or perhaps even better as a new argument to --with-native-debug-symbols), so without this, nothing will change. I think it needs to be a separate option as it's all orthogonal to the main debug symbols file creation. * The code need to make sure to separate stripped.pdb and normal pdb files, when enabled. * And there must not be a heavy penalty in added code complexity. /Erik /Magnus Thanks Christoph -----Original Message----- From: build-dev On Behalf Of Erik Joelsson Sent: Donnerstag, 23. Januar 2020 18:49 To: Baesken, Matthias; David Holmes ; 'build-dev at openjdk.java.net' >; 'hotspot-dev at openjdk.java.net' > Subject: Re: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images On 2020-01-23 00:03, Baesken, Matthias wrote: Hi Erik, yes true sorry for answering your comments a bit late . If a user runs jlink and includes all the jmods we ship with the JDK, the result should be essentially equivalent to the original JDK image. The way the stripped pdb files are included in the bundles sort of at the last second of the build here breaks this property. I think we should address this in a separate bug/CR . Maybe. I realize that my proposal below is quite a big task. But on the other hand, I don't think breaking the relationship between the jmods and the distribution bundles is on the table really. Looking for example into a Linux build, I see a lot of debuginfo files in the jdk image (more or less for every shared lib) . But when looking into the jmods of that jdk image , no debuginfo files are in there ( I checked the java.base jmod). So putting the files with debug information into the jmods seems to be something that was not done so far cross platform (or is there some build switch for it that I did not check?) . Maybe to keep the jmods as small as possible . No, we do not put the debuginfo files in the jmods nor the bundles because those are not intended to be shipped to customers. We are currently overlaying them into images/jdk in the build output to make them available for local debugging. (This is convoluted and I would very much like to get away from this practice at some point so that there is a 1-1 mapping between images/jdk and bundles/jdk*-bin.tar.gz.) The stripped pdb files you are proposing are on the contrary intended for shipping to customers (as I understand your proposal) so comparing them with the debuginfo files is not relevant. Now if MS had been kind enough to define a separate file type for the stripped pdbs, so that they could live alongside the full pdbs, we wouldn't have this issue. The heart of the problem is that only one set of files (either stripped or full) can be present and usable in images/jdk at a time. We have 2 main uses for images/jdk. 1. Developer running and debugging locally 2. Serve as the source for generating the distribution bundles We currently have one image serving both of these purposes, which is already creating complicated and convoluted build steps. To properly solve this I would argue for splitting these two apart into two different images for the two different purposes. The build procedure would then be, first build the images for distribution, only containing what should go into each bundle. Then create the developer jdk image by copying files from the distribution images. On Windows, the first image would contain the stripped pdbs and when building the second, those would get overwritten with the full pdbs. Now that I figured out a working model that would solve a bunch of other problems as well, I would love to implement it, but I doubt I will have time in the near future. /Erik To properly implement this, care will need to be taken to juggle the two sets of pdb files around, making sure each build and test use case has the correct one in place where and when it's needed. Quite possibly, we cannot cover all use cases with one build configuration. Developers needing the full debug symbols when debugging locally would likely need to disable the stripped symbols so they get the full symbols everywhere. Possibly this would need to be the default for debug builds and configurable for release builds. From my limited experience , the developers do not work with the bundles (that would contain now after my patch the stripped pds) but with a "normal" jdk image that is created my make all. Best regards, Matthias This still does not address anything in my objection. /Erik On 2020-01-22 07:46, Baesken, Matthias wrote: Hello, here is an updated version : http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/ this one supports a configure switch "--enable-stripped-pdbs" to enable the feature . Best regards, Matthias -----Original Message----- From: Baesken, Matthias Sent: Dienstag, 21. Januar 2020 11:03 To: 'David Holmes'; 'build- dev at openjdk.java.net'; 'hotspot- dev at openjdk.java.net' Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk images Hi David , yes I think it makes sense to have a configure option for this . Not everyone would like to have a larger JDK (even it is only a bit larger). From jianglizhou at google.com Wed Feb 19 18:23:57 2020 From: jianglizhou at google.com (Jiangli Zhou) Date: Wed, 19 Feb 2020 10:23:57 -0800 Subject: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS In-Reply-To: References: Message-ID: Hi Richard, David's suggestion of using dlsym seems to be more reliable. Regarding the stability of __pthread_get_minstack in future glibc versions, Florian has added comment to __pthread_get_minstack about external users to ensure it's not removed until TCB is separately allocated. Please see: http://51.15.138.76/patch/18989/ and https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. Best regards, Jiangli On Wed, Feb 19, 2020 at 2:44 AM Reingruber, Richard wrote: > > Hi, > > please review this small test fix > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239449/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8239449 > > On linux systems with older glibc versions (e.g. 2.11) the option -XX:+AdjustStackSizeForTLS has no effect, because the private function __pthread_get_minstack is not provided, and the test fails. With the fix the test is skipped instead. > > Thanks, Richard. From christoph.langer at sap.com Wed Feb 19 21:20:19 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 19 Feb 2020 21:20:19 +0000 Subject: RFR [XS]: 8239462: jdk.hotspot.agent misses some ReleaseStringUTFChars calls in case of early returns In-Reply-To: References: Message-ID: Hi Matthias, I think this is good. In line 1187 of src/jdk.hotspot.agent/solaris/native/libsaproc/saproc.cpp, there?s a space missing after the if. I guess you should insert it before pushing ?? Cheers Christoph From: serviceability-dev On Behalf Of Baesken, Matthias Sent: Mittwoch, 19. Februar 2020 15:22 To: 'hotspot-dev at openjdk.java.net' Cc: serviceability-dev at openjdk.java.net Subject: [CAUTION] RFR [XS]: 8239462: jdk.hotspot.agent misses some ReleaseStringUTFChars calls in case of early returns Hello, please review this small change . We miss at a few places ReleaseStringUTFChars calls in the native jdk.hotspot.agent coding. This happens in case of early returns . Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8239462 http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.0/ Thanks, Matthias From alexey.menkov at oracle.com Thu Feb 20 00:16:23 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Wed, 19 Feb 2020 16:16:23 -0800 Subject: RFR [XS]: 8239462: jdk.hotspot.agent misses some ReleaseStringUTFChars calls in case of early returns In-Reply-To: References: Message-ID: <6ed0d004-3e8c-4b27-f583-06dbacf45173@oracle.com> Looks like src/jdk.hotspot.agent/linux/native/libsaproc/LinuxDebuggerLocal.cpp has similar issues. It would be nice to fix them as well. --alex On 02/19/2020 06:21, Baesken, Matthias wrote: > Hello, please review this small change . > We miss at a few places ReleaseStringUTFChars calls in the native > jdk.hotspot.agent coding. > This happens in case of early returns . > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8239462 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.0/ > > > Thanks, Matthias > From david.holmes at oracle.com Thu Feb 20 03:47:37 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Feb 2020 13:47:37 +1000 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> Message-ID: <4ac31ff3-df28-c3c4-5b63-9c2849716d00@oracle.com> On 19/02/2020 11:30 pm, Ao Qi wrote: > On Wed, Feb 19, 2020 at 8:32 PM David Holmes wrote: >> >> On 19/02/2020 5:40 pm, David Holmes wrote: >>> On 19/02/2020 5:26 pm, Ao Qi wrote: >>>> On Wed, Feb 19, 2020 at 3:01 PM David Holmes >>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> On 19/02/2020 4:20 pm, Ao Qi wrote: >>>>>> Hi all, >>>>>> >>>>>> Could you please review this patch? >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8239423 >>>>>> webrev: http://cr.openjdk.java.net/~aoqi/8239423/webrev.00/ (22 lines >>>>>> changed: 19 ins; 0 del; 3 mod; 153 unchg) >>>>> >>>>> I'm not seeing the connection between the reported bug and the fix. It >>>>> seems to me that passing -XX:-TieredCompilation to a test that >>>>> explicitly sets -XX:TieredStopAtLevel=n is simply not something that can >>>>> work. In fact I'm surprised it doesn't trigger an error when validating >>>>> the command-line args. >>>> >>>> "-XX:-TieredCompilation" in the title is a way to trigger the bug, not >>>> the cause. Sorry for the confusion. The bug also can be triggered by >>>> the jdk configured with --with-jvm-features=-compiler1. >>>> >>>>> >>>>> You fix seems more related to whether C1 and/or C2 are available. >>>> >>>> Yes. >>> >>> I've had a look at this and I'm seeing the problem but I don't yet think >>> this is the right fix. I see some general issues with the ability to use >>> the compiler flags in a way that may not make sense (to me anyway); then >>> I see an issue with the way the test determines the available levels. >>> >>> Looking at the test it determines the available levels as follows: >>> - If tiered is enabled >>> - use TieredStopAtLevel=n to set the maximum level and thus the >>> available levels >>> - else if server >>> - only level is 4 >>> - else // client, minimal, emulated client >>> - only level is 1 >>> >>> So with -TieredCompilation or a server only build the level is set to 4. >>> But the WhiteBox::compile_method rejects that as an invalid level for >>> the configuration. >>> >>> I don't think the fix is to require C2, in fact I don't see how that >>> fixes the problem anyway if you have a C2 only build without tiered >>> because the test will still use level=4 which the WB rejects: >>> >>> STDOUT: >>> WB error: invalid compilation level 4 >>> >>> The test in WB is: >>> >>> if (comp_level > MIN2((CompLevel) TieredStopAtLevel, >>> CompLevel_highest_tier)) >>> >>> but I don't know how either of those values are set when (a) tiered is >>> disabled; or (b) you have a C1 or C2 only build (which should imply >>> tiered is disabled as well). > > Thanks very much for your hint, David. > >> >> Digging deeper if C2 exists then CompLevel_highest_tier should be set as: >> >> #if defined(COMPILER2) >> CompLevel CompLevel_highest_tier = CompLevel_full_optimization; >> >> where: >> >> CompLevel_full_optimization = 4 >> >> and from the command-line: >> >> -XX:TieredStopAtLevel=4 >> >> and so I don't understand how the above failure can trigger. One of >> those fields must not be what is is expected to be. > > I think it is the "-XX:TieredStopAtLevel=1" that trigger the failure. Ah! Now I see the problem. In WhiteBox we have: if (comp_level > MIN2((CompLevel) TieredStopAtLevel, CompLevel_highest_tier)) { tty->print_cr("WB error: invalid compilation level %d", comp_level); return false; } but TieredStopAtLevel should be ignored when tiered compilation is disabled, or not present. I actually think it is a bug that the VM allows you to set TieredStopAtLevel even though it is ignored on those configurations. The WB code should be changed to something like: #ifdef TIERED CompLevel cl_min = MIN2((CompLevel) TieredStopAtLevel, CompLevel_highest_tier); #else CompLevel cl_min = CompLevel_highest_tier; #endif if (comp_level > cl_min) { tty->print_cr("WB error: invalid compilation level %d", comp_level); return false; } There are other places in WhiteBox that need similar fixes. Cheers, David > I was also seeing this issue. Here is a table: > > |mode |cl|tsl|cht|>| > |+tiered(default)|4 |4 |4 |false| > |+tiered(default)|1 |1 |4 |false| > |-tiered |4 |4 |4 |false| > |-tiered |4 |1 |4 |*true*| > |c2 only |4 |4 |4 |false| > |c2 only |4 |1 |4 |*true*| > |c1 only |1 |4 |1 |false| > |c1 only |1 |1 |1 |false| > > cl: comp_level(level to set) > tsl: TieredStopAtLevel > cht: CompLevel_highest_tier > ">": comp_level > min(TieredStopAtLevel, CompLevel_highest_tier) > > If ">" is true, the test failed. I think the combination of "c2 > only(comp_level=4/CompLevel_highest_tier=4) and TieredStopAtLevel=1" > should be skipped. I am not sure if we should skip the combination of > "-tiered(comp_level=4/CompLevel_highest_tier=4) and > TieredStopAtLevel=1" or we should set comp_level=1 when -tiered is > used. > > I need more time to follow up on this issue. > > Cheers, > Ao Qi > >> >> David >> >> >>> Cheers, >>> David >>> >>>> Cheers, >>>> Ao Qi >>>> >>>>> >>>>> ?? >>>>> >>>>> Cheers, >>>>> David >>>>> >>>>>> Thanks, >>>>>> Ao Qi >>>>>> >>>> > From christoph.langer at sap.com Thu Feb 20 08:06:54 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 20 Feb 2020 08:06:54 +0000 Subject: Mach5 test failure when testing JDK-8237192 Message-ID: Hi, I tested my change for JDK-8237192 in the submit repo. I got this back. Can anybody from Oracle please have a look whether the failures could be related to my patch? At first sight and from the information I can see here, I don?t see the relation? Thanks Christoph From: do-not-reply at oracle.com Sent: Mittwoch, 19. Februar 2020 21:16 To: Langer, Christoph Subject: [Mach5] mach5-one-clanger-JDK-8237192-20200219-1906-8861422: FAILED, Failed tests: 1 Job: mach5-one-clanger-JDK-8237192-20200219-1906-8861422 BuildId: 2020-02-19-1905201.christoph.langer.source Failed tests: showing 1 out of 1 Test Tier Platform Description serviceability/sa/ClhsdbCDSJstackPrintAll.java tier1 macosx-x64-debug Exception: java.io.IOException: LingeredApp terminated with non-zero exit code ... Tasks Summary * UNABLE_TO_RUN: 1 * PASSED: 76 * FAILED: 1 * NA: 0 * NOTHING_TO_RUN: 0 * EXECUTED_WITH_FAILURE: 1 * HARNESS_ERROR: 0 * KILLED: 0 Build 1 Failed * solaris-sparcv9-open-debug-solaris-sparcv9-build-11 SOURCE_MASTER REASON_INVALID_...d68-25dad266ef13-O20132041 is no longer valid Test 1 Unable to run * tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-open-debug-62 Dependency task failed: mach5...s-sparcv9-open-debug-solaris-sparcv9-build-11 1 Executed with failure * tier1-debug-open_test_hotspot_jtreg_tier1_serviceability-macosx-x64-debug-59 Results: total: 44, passed: 43, failed: 1, skipped: 0 From david.holmes at oracle.com Thu Feb 20 08:28:25 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Feb 2020 18:28:25 +1000 Subject: Mach5 test failure when testing JDK-8237192 In-Reply-To: References: Message-ID: Hi Christoph, The Solaris failure looks like an infra issue. The test failure is a crash - info below. I don't see any open, or recently fixed, bugs for the same crash. David ----- # # A fatal error has been detected by the Java Runtime Environment: # # Internal Error (/scratch/mesos/slaves/90726e33-be99-4e27-9d68-25dad266ef13-S3967/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/9c4085d3-922a-46c2-b44a-9835e74ddb36/runs/ecd258eb-38e6-4966-b233-0cdfa582f713/workspace/open/src/hotspot/share/opto/gcm.cpp:276), pid=34522, tid=43267 # assert(false) failed: unscheduable graph # # JRE version: Java(TM) SE Runtime Environment (15.0) (fastdebug build 15-internal+0-2020-02-19-1905201.christoph.langer.source) # Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 15-internal+0-2020-02-19-1905201.christoph.langer.source, mixed mode, sharing, tiered, compressed oops, g1 gc, bsd-amd64) # Core dump will be written. Default location: core.34522 # # If you would like to submit a bug report, please visit: # https://bugreport.java.com/bugreport/crash.jsp # --------------- S U M M A R Y ------------ Command Line: -Denv.class.path=/scratch/mesos/slaves/90726e33-be99-4e27-9d68-25dad266ef13-S80/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/fb2f58a7-7e16-4340-8783-4207c1d11742/runs/3ca5d907-2207-49d1-a673-f3617d83173d/testoutput/test-support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/1/serviceability/sa/ClhsdbCDSJstackPrintAll.d:/scratch/mesos/jib-master/install/2020-02-19-1905201.christoph.langer.source/src.full/open/test/hotspot/jtreg/serviceability/sa:/scratch/mesos/slaves/90726e33-be99-4e27-9d68-25dad266ef13-S80/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd-0196/executors/fb2f58a7-7e16-4340-8783-4207c1d11742/runs/3ca5d907-2207-49d1-a673-f3617d83173d/testoutput/test-support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/1/test/lib:/scratch/mesos/jib-master/install/2020-02-19-1905201.christoph.langer.source/src.full/open/test/lib:/scratch/mesos/jib-master/install/java/re/jtreg/4.2/promoted/all/b16/bundles/jtreg_bin-4.2.zip/jtreg/lib/javatest.jar:/scratch/mesos/jib-master/install/java/re/jtreg/4.2/promoted/all/b16/bundles/jtreg_bin-4.2.zip/jtreg/lib/jtreg.jar -Dapplication.home=/scratch/mesos/jib-master/install/2020-02-19-1905201.christoph.langer.source/macosx-x64-debug.jdk/jdk-15/fastdebug -Xms8m -Djdk.module.main=jdk.hotspot.agent jdk.hotspot.agent/sun.jvm.hotspot.SALauncher clhsdb --pid=34506 Host: scaaa915.us.oracle.com, MacPro6,1 x86_64 3700 MHz, 8 cores, 16G, Darwin 17.5.0 Time: Wed Feb 19 19:49:38 2020 GMT elapsed time: 5 seconds (0d 0h 0m 5s) --------------- T H R E A D --------------- Current thread (0x00007f9158008800): JavaThread "C2 CompilerThread0" daemon [_thread_in_native, id=43267, stack(0x0000700007ca8000,0x0000700007da8000)] Current CompileTask: C2: 5392 726 ! 4 jdk.internal.reflect.GeneratedConstructorAccessor3::newInstance (53 bytes) Stack: [0x0000700007ca8000,0x0000700007da8000], sp=0x0000700007da3b70, free space=1006k Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.dylib+0xb48133] VMError::report_and_die(int, char const*, char const*, __va_list_tag*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned long)+0x6e5 V [libjvm.dylib+0xb4884f] VMError::report_and_die(Thread*, void*, char const*, int, char const*, char const*, __va_list_tag*)+0x47 V [libjvm.dylib+0x338454] report_vm_error(char const*, int, char const*, char const*, ...)+0x145 V [libjvm.dylib+0x49c0fb] assert_dom(Block*, Block*, Node*, PhaseCFG const*)+0x15d V [libjvm.dylib+0x497588] PhaseCFG::schedule_early(VectorSet&, Node_Stack&)+0x29a V [libjvm.dylib+0x499c49] PhaseCFG::global_code_motion()+0x1ad V [libjvm.dylib+0x49a211] PhaseCFG::do_global_code_motion()+0x41 V [libjvm.dylib+0x304254] Compile::Code_Gen()+0x224 V [libjvm.dylib+0x301f3b] Compile::Compile(ciEnv*, C2Compiler*, ciMethod*, int, bool, bool, bool, DirectiveSet*)+0xbf1 V [libjvm.dylib+0x254242] C2Compiler::compile_method(ciEnv*, ciMethod*, int, DirectiveSet*)+0xe8 V [libjvm.dylib+0x3142ae] CompileBroker::invoke_compiler_on_method(CompileTask*)+0x664 V [libjvm.dylib+0x313a3f] CompileBroker::compiler_thread_loop()+0x283 V [libjvm.dylib+0xabf30b] JavaThread::thread_main_inner()+0x1a1 V [libjvm.dylib+0xabeebd] JavaThread::run()+0x23d V [libjvm.dylib+0xabb861] Thread::call_run()+0x11b V [libjvm.dylib+0x93521c] thread_native_entry(Thread*)+0xe0 C [libsystem_pthread.dylib+0x3661] _pthread_body+0x154 C [libsystem_pthread.dylib+0x350d] _pthread_body+0x0 C [libsystem_pthread.dylib+0x2bf9] thread_start+0xd On 20/02/2020 6:06 pm, Langer, Christoph wrote: > Hi, > > I tested my change for JDK-8237192 in the submit repo. I got this back. > > Can anybody from Oracle please have a look whether the failures could be related to my patch? At first sight and from the information I can see here, I don?t see the relation? > > Thanks > Christoph > > From: do-not-reply at oracle.com > Sent: Mittwoch, 19. Februar 2020 21:16 > To: Langer, Christoph > Subject: [Mach5] mach5-one-clanger-JDK-8237192-20200219-1906-8861422: FAILED, Failed tests: 1 > > Job: mach5-one-clanger-JDK-8237192-20200219-1906-8861422 > BuildId: 2020-02-19-1905201.christoph.langer.source > Failed tests: showing 1 out of 1 > Test > Tier > Platform > Description > serviceability/sa/ClhsdbCDSJstackPrintAll.java > tier1 > macosx-x64-debug > Exception: java.io.IOException: LingeredApp terminated with non-zero exit code ... > Tasks Summary > > * UNABLE_TO_RUN: 1 > * PASSED: 76 > * FAILED: 1 > * NA: 0 > * NOTHING_TO_RUN: 0 > * EXECUTED_WITH_FAILURE: 1 > * HARNESS_ERROR: 0 > * KILLED: 0 > > Build > 1 Failed > > * solaris-sparcv9-open-debug-solaris-sparcv9-build-11 SOURCE_MASTER REASON_INVALID_...d68-25dad266ef13-O20132041 is no longer valid > > Test > 1 Unable to run > > * tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-open-debug-62 Dependency task failed: mach5...s-sparcv9-open-debug-solaris-sparcv9-build-11 > > 1 Executed with failure > > * tier1-debug-open_test_hotspot_jtreg_tier1_serviceability-macosx-x64-debug-59 Results: total: 44, passed: 43, failed: 1, skipped: 0 > From christoph.langer at sap.com Thu Feb 20 08:51:00 2020 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 20 Feb 2020 08:51:00 +0000 Subject: Mach5 test failure when testing JDK-8237192 In-Reply-To: References: Message-ID: Thanks, David for the information. As I don't see a relation from the crash to my change (I didn't touch any hotspot code at least), I guess I'm confident enough to push my patch. If worse comes to worse there's still the option to back it out again... Best regards Christoph > -----Original Message----- > From: David Holmes > Sent: Donnerstag, 20. Februar 2020 09:28 > To: Langer, Christoph ; 'build- > dev at openjdk.java.net' ; 'hotspot- > dev at openjdk.java.net' ; Erik Joelsson > ; Magnus Ihse Bursie > > Subject: Re: Mach5 test failure when testing JDK-8237192 > > Hi Christoph, > > The Solaris failure looks like an infra issue. > > The test failure is a crash - info below. I don't see any open, or > recently fixed, bugs for the same crash. > > David > ----- > > # > # A fatal error has been detected by the Java Runtime Environment: > # > # Internal Error > (/scratch/mesos/slaves/90726e33-be99-4e27-9d68-25dad266ef13- > S3967/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd- > 0196/executors/9c4085d3-922a-46c2-b44a-9835e74ddb36/runs/ecd258eb- > 38e6-4966-b233- > 0cdfa582f713/workspace/open/src/hotspot/share/opto/gcm.cpp:276), > pid=34522, tid=43267 > # assert(false) failed: unscheduable graph > # > # JRE version: Java(TM) SE Runtime Environment (15.0) (fastdebug build > 15-internal+0-2020-02-19-1905201.christoph.langer.source) > # Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug > 15-internal+0-2020-02-19-1905201.christoph.langer.source, mixed mode, > sharing, tiered, compressed oops, g1 gc, bsd-amd64) > # Core dump will be written. Default location: core.34522 > # > # If you would like to submit a bug report, please visit: > # https://bugreport.java.com/bugreport/crash.jsp > # > > --------------- S U M M A R Y ------------ > > Command Line: > -Denv.class.path=/scratch/mesos/slaves/90726e33-be99-4e27-9d68- > 25dad266ef13-S80/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd- > 0196/executors/fb2f58a7-7e16-4340-8783-4207c1d11742/runs/3ca5d907- > 2207-49d1-a673-f3617d83173d/testoutput/test- > support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/1/servi > ceability/sa/ClhsdbCDSJstackPrintAll.d:/scratch/mesos/jib- > master/install/2020-02-19- > 1905201.christoph.langer.source/src.full/open/test/hotspot/jtreg/serviceabil > ity/sa:/scratch/mesos/slaves/90726e33-be99-4e27-9d68-25dad266ef13- > S80/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd- > 0196/executors/fb2f58a7-7e16-4340-8783-4207c1d11742/runs/3ca5d907- > 2207-49d1-a673-f3617d83173d/testoutput/test- > support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/1/test/ > lib:/scratch/mesos/jib-master/install/2020-02-19- > 1905201.christoph.langer.source/src.full/open/test/lib:/scratch/mesos/jib- > master/install/java/re/jtreg/4.2/promoted/all/b16/bundles/jtreg_bin- > 4.2.zip/jtreg/lib/javatest.jar:/scratch/mesos/jib- > master/install/java/re/jtreg/4.2/promoted/all/b16/bundles/jtreg_bin- > 4.2.zip/jtreg/lib/jtreg.jar > -Dapplication.home=/scratch/mesos/jib-master/install/2020-02-19- > 1905201.christoph.langer.source/macosx-x64-debug.jdk/jdk-15/fastdebug > -Xms8m -Djdk.module.main=jdk.hotspot.agent > jdk.hotspot.agent/sun.jvm.hotspot.SALauncher clhsdb --pid=34506 > > Host: scaaa915.us.oracle.com, MacPro6,1 x86_64 3700 MHz, 8 cores, 16G, > Darwin 17.5.0 > Time: Wed Feb 19 19:49:38 2020 GMT elapsed time: 5 seconds (0d 0h 0m 5s) > > --------------- T H R E A D --------------- > > Current thread (0x00007f9158008800): JavaThread "C2 CompilerThread0" > daemon [_thread_in_native, id=43267, > stack(0x0000700007ca8000,0x0000700007da8000)] > > > Current CompileTask: > C2: 5392 726 ! 4 > jdk.internal.reflect.GeneratedConstructorAccessor3::newInstance (53 bytes) > > Stack: [0x0000700007ca8000,0x0000700007da8000], sp=0x0000700007da3b70, > free space=1006k > Native frames: (J=compiled Java code, A=aot compiled Java code, > j=interpreted, Vv=VM code, C=native code) > V [libjvm.dylib+0xb48133] VMError::report_and_die(int, char const*, > char const*, __va_list_tag*, Thread*, unsigned char*, void*, void*, char > const*, int, unsigned long)+0x6e5 > V [libjvm.dylib+0xb4884f] VMError::report_and_die(Thread*, void*, char > const*, int, char const*, char const*, __va_list_tag*)+0x47 > V [libjvm.dylib+0x338454] report_vm_error(char const*, int, char > const*, char const*, ...)+0x145 > V [libjvm.dylib+0x49c0fb] assert_dom(Block*, Block*, Node*, PhaseCFG > const*)+0x15d > V [libjvm.dylib+0x497588] PhaseCFG::schedule_early(VectorSet&, > Node_Stack&)+0x29a > V [libjvm.dylib+0x499c49] PhaseCFG::global_code_motion()+0x1ad > V [libjvm.dylib+0x49a211] PhaseCFG::do_global_code_motion()+0x41 > V [libjvm.dylib+0x304254] Compile::Code_Gen()+0x224 > V [libjvm.dylib+0x301f3b] Compile::Compile(ciEnv*, C2Compiler*, > ciMethod*, int, bool, bool, bool, DirectiveSet*)+0xbf1 > V [libjvm.dylib+0x254242] C2Compiler::compile_method(ciEnv*, > ciMethod*, int, DirectiveSet*)+0xe8 > V [libjvm.dylib+0x3142ae] > CompileBroker::invoke_compiler_on_method(CompileTask*)+0x664 > V [libjvm.dylib+0x313a3f] CompileBroker::compiler_thread_loop()+0x283 > V [libjvm.dylib+0xabf30b] JavaThread::thread_main_inner()+0x1a1 > V [libjvm.dylib+0xabeebd] JavaThread::run()+0x23d > V [libjvm.dylib+0xabb861] Thread::call_run()+0x11b > V [libjvm.dylib+0x93521c] thread_native_entry(Thread*)+0xe0 > C [libsystem_pthread.dylib+0x3661] _pthread_body+0x154 > C [libsystem_pthread.dylib+0x350d] _pthread_body+0x0 > C [libsystem_pthread.dylib+0x2bf9] thread_start+0xd > > On 20/02/2020 6:06 pm, Langer, Christoph wrote: > > Hi, > > > > I tested my change for JDK-8237192 in the submit repo. I got this back. > > > > Can anybody from Oracle please have a look whether the failures could be > related to my patch? At first sight and from the information I can see here, I > don?t see the relation? > > > > Thanks > > Christoph > > > > From: do-not-reply at oracle.com > > Sent: Mittwoch, 19. Februar 2020 21:16 > > To: Langer, Christoph > > Subject: [Mach5] mach5-one-clanger-JDK-8237192-20200219-1906-8861422: > FAILED, Failed tests: 1 > > > > Job: mach5-one-clanger-JDK-8237192-20200219-1906-8861422 > > BuildId: 2020-02-19-1905201.christoph.langer.source > > Failed tests: showing 1 out of 1 > > Test > > Tier > > Platform > > Description > > serviceability/sa/ClhsdbCDSJstackPrintAll.java > > tier1 > > macosx-x64-debug > > Exception: java.io.IOException: LingeredApp terminated with non-zero exit > code ... > > Tasks Summary > > > > * UNABLE_TO_RUN: 1 > > * PASSED: 76 > > * FAILED: 1 > > * NA: 0 > > * NOTHING_TO_RUN: 0 > > * EXECUTED_WITH_FAILURE: 1 > > * HARNESS_ERROR: 0 > > * KILLED: 0 > > > > Build > > 1 Failed > > > > * solaris-sparcv9-open-debug-solaris-sparcv9-build-11 > SOURCE_MASTER REASON_INVALID_...d68-25dad266ef13-O20132041 is no > longer valid > > > > Test > > 1 Unable to run > > > > * tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris- > sparcv9-open-debug-62 Dependency task failed: mach5...s-sparcv9-open- > debug-solaris-sparcv9-build-11 > > > > 1 Executed with failure > > > > * tier1-debug-open_test_hotspot_jtreg_tier1_serviceability-macosx- > x64-debug-59 Results: total: 44, passed: 43, failed: 1, skipped: 0 > > From matthias.baesken at sap.com Thu Feb 20 08:53:41 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 20 Feb 2020 08:53:41 +0000 Subject: RFR [XS]: 8239462: jdk.hotspot.agent misses some ReleaseStringUTFChars calls in case of early returns In-Reply-To: <6ed0d004-3e8c-4b27-f583-06dbacf45173@oracle.com> References: <6ed0d004-3e8c-4b27-f583-06dbacf45173@oracle.com> Message-ID: Hi Alex / Christoph, thanks for the reviews. New webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.1/ - includes LinuxDebuggerLocal.cpp - adds a blank Christoph wanted to have A question (hopefully not a stupid one ?? ): At most places in the coding, GetStringUTFChars success is 1. handled by checking NULL , like this : const char *s = (*env)->GetStringUTFChars(env, p, NULL); if (s == NULL) { // handle failure } 2.At some places , success / failure is not handled at all . 3.Here (e.g. LinuxDebuggerLocal.cpp) success / failure check is done by if (env->ExceptionOccurred()) { ... } Which one is the best / right way to do it (most likely not 2.) ? Best regards, Matthias > > Looks like > src/jdk.hotspot.agent/linux/native/libsaproc/LinuxDebuggerLocal.cpp > has similar issues. It would be nice to fix them as well. > > --alex > > On 02/19/2020 06:21, Baesken, Matthias wrote: > > Hello, please review this small change . > > We miss at a few places ReleaseStringUTFChars calls in the native > > jdk.hotspot.agent coding. > > This happens in case of early returns . > > > > > > Bug/webrev : > > > > https://bugs.openjdk.java.net/browse/JDK-8239462 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.0/ > > > > > > Thanks, Matthias > > From richard.reingruber at sap.com Thu Feb 20 09:03:12 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Thu, 20 Feb 2020 09:03:12 +0000 Subject: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS In-Reply-To: References: Message-ID: Hi Jiangli, thanks for providing feedback. > David's suggestion of using dlsym seems to be more reliable. Regarding the stability of > __pthread_get_minstack in future glibc versions, Florian has added comment to > __pthread_get_minstack about external users to ensure it's not removed until TCB is > separately allocated. Please see: http://51.15.138.76/patch/18989/ and > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. So if I understand you and David correctly, then the test should be skipped if the lookup of __pthread_get_minstack fails. Then the test would be skipped on future glibc versions where __pthread_get_minstack was removed because TLS and TCB are not placed on a threads stack anymore. Actually I would prefer not to skip the test then. I'd like to verify if large data structures can be placed in TLS without stack size adjustment or if there is some kind of regression. That's the point of regression tests after all. In other words: I wouldn't change test coverage of future glibc versions. I just want to exclude glibc versions were the tested feature cannot work. Thanks, Richard. -----Original Message----- From: Jiangli Zhou Sent: Mittwoch, 19. Februar 2020 19:24 To: Reingruber, Richard Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS Hi Richard, David's suggestion of using dlsym seems to be more reliable. Regarding the stability of __pthread_get_minstack in future glibc versions, Florian has added comment to __pthread_get_minstack about external users to ensure it's not removed until TCB is separately allocated. Please see: http://51.15.138.76/patch/18989/ and https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. Best regards, Jiangli On Wed, Feb 19, 2020 at 2:44 AM Reingruber, Richard wrote: > > Hi, > > please review this small test fix > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239449/webrev.0/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8239449 > > On linux systems with older glibc versions (e.g. 2.11) the option -XX:+AdjustStackSizeForTLS has no effect, because the private function __pthread_get_minstack is not provided, and the test fails. With the fix the test is skipped instead. > > Thanks, Richard. From david.holmes at oracle.com Thu Feb 20 09:34:01 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Feb 2020 19:34:01 +1000 Subject: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS In-Reply-To: References: Message-ID: <788370d5-594c-e052-4580-75cc14152f96@oracle.com> On 20/02/2020 7:03 pm, Reingruber, Richard wrote: > Hi Jiangli, > > thanks for providing feedback. > > > David's suggestion of using dlsym seems to be more reliable. Regarding the stability of > > __pthread_get_minstack in future glibc versions, Florian has added comment to > > __pthread_get_minstack about external users to ensure it's not removed until TCB is > > separately allocated. Please see: http://51.15.138.76/patch/18989/ and > > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. > > So if I understand you and David correctly, then the test should be skipped if the lookup of > __pthread_get_minstack fails. > > Then the test would be skipped on future glibc versions where __pthread_get_minstack was removed > because TLS and TCB are not placed on a threads stack anymore. I don't think that is really a significant concern. As Jiangli pointed out any change will be communicated beforehand and we will adjust the VM code and the test as needed. But okay ... I wonder why the VM doesn't use gnu_get_libc_version() ? os_linux.cpp does: size_t n = confstr(_CS_GNU_LIBC_VERSION, NULL, 0); assert(n > 0, "cannot retrieve glibc version"); char *str = (char *)malloc(n, mtInternal); confstr(_CS_GNU_LIBC_VERSION, str, n); I'm okay with the test changes. Thanks, David ----- > Actually I would prefer not to skip the test then. I'd like to verify if large data structures can > be placed in TLS without stack size adjustment or if there is some kind of regression. That's the > point of regression tests after all. > > In other words: I wouldn't change test coverage of future glibc versions. I just want to exclude > glibc versions were the tested feature cannot work. > > Thanks, Richard. > > -----Original Message----- > From: Jiangli Zhou > Sent: Mittwoch, 19. Februar 2020 19:24 > To: Reingruber, Richard > Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS > > Hi Richard, > > David's suggestion of using dlsym seems to be more reliable. Regarding > the stability of __pthread_get_minstack in future glibc versions, > Florian has added comment to __pthread_get_minstack about external > users to ensure it's not removed until TCB is separately allocated. > Please see: http://51.15.138.76/patch/18989/ and > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. > > Best regards, > Jiangli > > > > > On Wed, Feb 19, 2020 at 2:44 AM Reingruber, Richard > wrote: >> >> Hi, >> >> please review this small test fix >> >> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239449/webrev.0/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8239449 >> >> On linux systems with older glibc versions (e.g. 2.11) the option -XX:+AdjustStackSizeForTLS has no effect, because the private function __pthread_get_minstack is not provided, and the test fails. With the fix the test is skipped instead. >> >> Thanks, Richard. From matthias.baesken at sap.com Thu Feb 20 10:25:24 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 20 Feb 2020 10:25:24 +0000 Subject: RFR [XS]: 8239537: cgroup MetricsTester testMemorySubsystem fails sometimes when testing memory.kmem.tcp.usage_in_bytes Message-ID: Hello, please review this small adjustment . The testMemorySubsystem test in MetricsTester.java sometimes fails with messages like this : java.lang.RuntimeException: Test failed for - memory:memory.kmem.tcp.usage_in_bytes, expected [655360], got [786432] at jdk.test.lib.containers.cgroup.MetricsTester.fail(MetricsTester.java:207) at jdk.test.lib.containers.cgroup.MetricsTester.testMemorySubsystem(MetricsTester.java:309) I looked into the code doing the checks , and looks like there is already an error margin allowed in CgroupMetricsTester.java , which is currently 0.1 . This might be changed to a somewhat higher value, e.g. 0.25 to make the test more robust. Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8239537 http://cr.openjdk.java.net/~mbaesken/webrevs/8239537.0/ Thanks, Matthias From aoqi at loongson.cn Thu Feb 20 10:33:10 2020 From: aoqi at loongson.cn (Ao Qi) Date: Thu, 20 Feb 2020 18:33:10 +0800 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: <4ac31ff3-df28-c3c4-5b63-9c2849716d00@oracle.com> References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> <4ac31ff3-df28-c3c4-5b63-9c2849716d00@oracle.com> Message-ID: > Ah! Now I see the problem. In WhiteBox we have: > > if (comp_level > MIN2((CompLevel) TieredStopAtLevel, > CompLevel_highest_tier)) { > tty->print_cr("WB error: invalid compilation level %d", comp_level); > return false; > } > > but TieredStopAtLevel should be ignored when tiered compilation is > disabled, or not present. I actually think it is a bug that the VM > allows you to set TieredStopAtLevel even though it is ignored on those > configurations. The WB code should be changed to something like: > > #ifdef TIERED > CompLevel cl_min = MIN2((CompLevel) TieredStopAtLevel, > CompLevel_highest_tier); > #else > CompLevel cl_min = CompLevel_highest_tier; > #endif > if (comp_level > cl_min) { > tty->print_cr("WB error: invalid compilation level %d", comp_level); > return false; > } > > There are other places in WhiteBox that need similar fixes. Could you give some comments or advice on this one: http://cr.openjdk.java.net/~aoqi/8239423/webrev.01/ ? I think TieredStopAtLevel should be ignored when -XX:-TieredCompilation is used (even a build with tiered configration), so I used TieredCompilation to determine the cl_min, not "#ifdef TIERED". What do you think? Cheers, Ao Qi From david.holmes at oracle.com Thu Feb 20 13:08:01 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 20 Feb 2020 23:08:01 +1000 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> <4ac31ff3-df28-c3c4-5b63-9c2849716d00@oracle.com> Message-ID: <8982aeb0-3e12-f92f-a199-eb4ee1af1af9@oracle.com> On 20/02/2020 8:33 pm, Ao Qi wrote: >> Ah! Now I see the problem. In WhiteBox we have: >> >> if (comp_level > MIN2((CompLevel) TieredStopAtLevel, >> CompLevel_highest_tier)) { >> tty->print_cr("WB error: invalid compilation level %d", comp_level); >> return false; >> } >> >> but TieredStopAtLevel should be ignored when tiered compilation is >> disabled, or not present. I actually think it is a bug that the VM >> allows you to set TieredStopAtLevel even though it is ignored on those >> configurations. The WB code should be changed to something like: >> >> #ifdef TIERED >> CompLevel cl_min = MIN2((CompLevel) TieredStopAtLevel, >> CompLevel_highest_tier); >> #else >> CompLevel cl_min = CompLevel_highest_tier; >> #endif >> if (comp_level > cl_min) { >> tty->print_cr("WB error: invalid compilation level %d", comp_level); >> return false; >> } >> >> There are other places in WhiteBox that need similar fixes. > > Could you give some comments or advice on this one: > http://cr.openjdk.java.net/~aoqi/8239423/webrev.01/ ? > > I think TieredStopAtLevel should be ignored when > -XX:-TieredCompilation is used (even a build with tiered > configration), so I used TieredCompilation to determine the cl_min, > not "#ifdef TIERED". What do you think? Yes you are quite right! That is the correct way to handle this. I assume the test passes in all combinations now? Thanks, David > Cheers, > Ao Qi > From aoqi at loongson.cn Thu Feb 20 13:22:13 2020 From: aoqi at loongson.cn (Ao Qi) Date: Thu, 20 Feb 2020 21:22:13 +0800 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: <8982aeb0-3e12-f92f-a199-eb4ee1af1af9@oracle.com> References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> <4ac31ff3-df28-c3c4-5b63-9c2849716d00@oracle.com> <8982aeb0-3e12-f92f-a199-eb4ee1af1af9@oracle.com> Message-ID: On Thu, Feb 20, 2020 at 9:08 PM David Holmes wrote: > > On 20/02/2020 8:33 pm, Ao Qi wrote: > >> Ah! Now I see the problem. In WhiteBox we have: > >> > >> if (comp_level > MIN2((CompLevel) TieredStopAtLevel, > >> CompLevel_highest_tier)) { > >> tty->print_cr("WB error: invalid compilation level %d", comp_level); > >> return false; > >> } > >> > >> but TieredStopAtLevel should be ignored when tiered compilation is > >> disabled, or not present. I actually think it is a bug that the VM > >> allows you to set TieredStopAtLevel even though it is ignored on those > >> configurations. The WB code should be changed to something like: > >> > >> #ifdef TIERED > >> CompLevel cl_min = MIN2((CompLevel) TieredStopAtLevel, > >> CompLevel_highest_tier); > >> #else > >> CompLevel cl_min = CompLevel_highest_tier; > >> #endif > >> if (comp_level > cl_min) { > >> tty->print_cr("WB error: invalid compilation level %d", comp_level); > >> return false; > >> } > >> > >> There are other places in WhiteBox that need similar fixes. > > > > Could you give some comments or advice on this one: > > http://cr.openjdk.java.net/~aoqi/8239423/webrev.01/ ? > > > > I think TieredStopAtLevel should be ignored when > > -XX:-TieredCompilation is used (even a build with tiered > > configration), so I used TieredCompilation to determine the cl_min, > > not "#ifdef TIERED". What do you think? > > Yes you are quite right! That is the correct way to handle this. > > I assume the test passes in all combinations now? Yes. I have tested all the combinations. Also tested tier1. No regression found so far. Cheers, Ao Qi > > Thanks, > David > > > Cheers, > > Ao Qi > > From mikhailo.seledtsov at oracle.com Thu Feb 20 15:53:45 2020 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Thu, 20 Feb 2020 07:53:45 -0800 Subject: RFR [XS]: 8239537: cgroup MetricsTester testMemorySubsystem fails sometimes when testing memory.kmem.tcp.usage_in_bytes In-Reply-To: References: Message-ID: Looks good to me, Misha On 2/20/20 2:25 AM, Baesken, Matthias wrote: > Hello, please review this small adjustment . > > The testMemorySubsystem test in MetricsTester.java sometimes fails with messages like this : > > java.lang.RuntimeException: Test failed for - memory:memory.kmem.tcp.usage_in_bytes, expected [655360], got [786432] > at jdk.test.lib.containers.cgroup.MetricsTester.fail(MetricsTester.java:207) > at jdk.test.lib.containers.cgroup.MetricsTester.testMemorySubsystem(MetricsTester.java:309) > > I looked into the code doing the checks , and looks like there is already an error margin allowed in CgroupMetricsTester.java , which is currently 0.1 . > This might be changed to a somewhat higher value, e.g. 0.25 to make the test more robust. > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8239537 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8239537.0/ > > Thanks, Matthias From richard.reingruber at sap.com Thu Feb 20 16:22:14 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Thu, 20 Feb 2020 16:22:14 +0000 Subject: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS In-Reply-To: <788370d5-594c-e052-4580-75cc14152f96@oracle.com> References: <788370d5-594c-e052-4580-75cc14152f96@oracle.com> Message-ID: > > I wonder why the VM doesn't use gnu_get_libc_version() ? os_linux.cpp does: > > size_t n = confstr(_CS_GNU_LIBC_VERSION, NULL, 0); > assert(n > 0, "cannot retrieve glibc version"); > char *str = (char *)malloc(n, mtInternal); > confstr(_CS_GNU_LIBC_VERSION, str, n); It used to: https://bugs.openjdk.java.net/browse/JDK-8078513 Personally I would apply - size_t n = confstr(_CS_GNU_LIBC_VERSION, NULL, 0); - assert(n > 0, "cannot retrieve glibc version"); - char *str = (char *)malloc(n, mtInternal); - confstr(_CS_GNU_LIBC_VERSION, str, n); - os::Linux::set_glibc_version(str); - - n = confstr(_CS_GNU_LIBPTHREAD_VERSION, NULL, 0); + os::Linux::set_glibc_version(gnu_get_libc_version()); > I'm okay with the test changes. Thanks! Richard. -----Original Message----- From: David Holmes Sent: Donnerstag, 20. Februar 2020 10:34 To: Reingruber, Richard ; Jiangli Zhou ; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS On 20/02/2020 7:03 pm, Reingruber, Richard wrote: > Hi Jiangli, > > thanks for providing feedback. > > > David's suggestion of using dlsym seems to be more reliable. Regarding the stability of > > __pthread_get_minstack in future glibc versions, Florian has added comment to > > __pthread_get_minstack about external users to ensure it's not removed until TCB is > > separately allocated. Please see: http://51.15.138.76/patch/18989/ and > > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. > > So if I understand you and David correctly, then the test should be skipped if the lookup of > __pthread_get_minstack fails. > > Then the test would be skipped on future glibc versions where __pthread_get_minstack was removed > because TLS and TCB are not placed on a threads stack anymore. I don't think that is really a significant concern. As Jiangli pointed out any change will be communicated beforehand and we will adjust the VM code and the test as needed. But okay ... I wonder why the VM doesn't use gnu_get_libc_version() ? os_linux.cpp does: size_t n = confstr(_CS_GNU_LIBC_VERSION, NULL, 0); assert(n > 0, "cannot retrieve glibc version"); char *str = (char *)malloc(n, mtInternal); confstr(_CS_GNU_LIBC_VERSION, str, n); I'm okay with the test changes. Thanks, David ----- > Actually I would prefer not to skip the test then. I'd like to verify if large data structures can > be placed in TLS without stack size adjustment or if there is some kind of regression. That's the > point of regression tests after all. > > In other words: I wouldn't change test coverage of future glibc versions. I just want to exclude > glibc versions were the tested feature cannot work. > > Thanks, Richard. > > -----Original Message----- > From: Jiangli Zhou > Sent: Mittwoch, 19. Februar 2020 19:24 > To: Reingruber, Richard > Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS > > Hi Richard, > > David's suggestion of using dlsym seems to be more reliable. Regarding > the stability of __pthread_get_minstack in future glibc versions, > Florian has added comment to __pthread_get_minstack about external > users to ensure it's not removed until TCB is separately allocated. > Please see: http://51.15.138.76/patch/18989/ and > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. > > Best regards, > Jiangli > > > > > On Wed, Feb 19, 2020 at 2:44 AM Reingruber, Richard > wrote: >> >> Hi, >> >> please review this small test fix >> >> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239449/webrev.0/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8239449 >> >> On linux systems with older glibc versions (e.g. 2.11) the option -XX:+AdjustStackSizeForTLS has no effect, because the private function __pthread_get_minstack is not provided, and the test fails. With the fix the test is skipped instead. >> >> Thanks, Richard. From jianglizhou at google.com Thu Feb 20 16:32:15 2020 From: jianglizhou at google.com (Jiangli Zhou) Date: Thu, 20 Feb 2020 08:32:15 -0800 Subject: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS In-Reply-To: References: Message-ID: Hi Richard, On Thu, Feb 20, 2020 at 1:03 AM Reingruber, Richard wrote: > > Hi Jiangli, > > thanks for providing feedback. > > > David's suggestion of using dlsym seems to be more reliable. Regarding the stability of > > __pthread_get_minstack in future glibc versions, Florian has added comment to > > __pthread_get_minstack about external users to ensure it's not removed until TCB is > > separately allocated. Please see: http://51.15.138.76/patch/18989/ and > > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. > > So if I understand you and David correctly, then the test should be skipped if the lookup of > __pthread_get_minstack fails. > > Then the test would be skipped on future glibc versions where __pthread_get_minstack was removed > because TLS and TCB are not placed on a threads stack anymore. > > Actually I would prefer not to skip the test then. I'd like to verify if large data structures can > be placed in TLS without stack size adjustment or if there is some kind of regression. That's the > point of regression tests after all. > That's a very valid point. I thought about that as well yesterday. My thinking was that the VM and test would need to be updated to remove the workaround for the TLS issue when no longer needed. Do you know if the proposed check using gnu_get_libc_version() covers all cases? I came across the following archive that mentioned missing symbol in eglibc-2.13 (eglibc however is already deprecated). https://www.mail-archive.com/lfs-dev at linuxfromscratch.org/msg18115.html > In other words: I wouldn't change test coverage of future glibc versions. I just want to exclude > glibc versions were the tested feature cannot work. Best regards, Jiangli > > Thanks, Richard. > > -----Original Message----- > From: Jiangli Zhou > Sent: Mittwoch, 19. Februar 2020 19:24 > To: Reingruber, Richard > Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS > > Hi Richard, > > David's suggestion of using dlsym seems to be more reliable. Regarding > the stability of __pthread_get_minstack in future glibc versions, > Florian has added comment to __pthread_get_minstack about external > users to ensure it's not removed until TCB is separately allocated. > Please see: http://51.15.138.76/patch/18989/ and > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. > > Best regards, > Jiangli > > > > > On Wed, Feb 19, 2020 at 2:44 AM Reingruber, Richard > wrote: > > > > Hi, > > > > please review this small test fix > > > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239449/webrev.0/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8239449 > > > > On linux systems with older glibc versions (e.g. 2.11) the option -XX:+AdjustStackSizeForTLS has no effect, because the private function __pthread_get_minstack is not provided, and the test fails. With the fix the test is skipped instead. > > > > Thanks, Richard. From alexey.menkov at oracle.com Thu Feb 20 20:07:55 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Thu, 20 Feb 2020 12:07:55 -0800 Subject: RFR [XS]: 8239462: jdk.hotspot.agent misses some ReleaseStringUTFChars calls in case of early returns In-Reply-To: References: <6ed0d004-3e8c-4b27-f583-06dbacf45173@oracle.com> Message-ID: Hi Matthias, Looks good in general, but I think it makes sense to fix #2 cases (at least I see them in LinuxDebuggerLocal). If GetStringUTFChars fails, the code will crash. Also I see GetStringUTFChars(str, JNI_FALSE). This look bad as well - 2nd arg is a pointer, so it should be NULL or nullptr. As for #1 and #3 - AFAIU they are both right ways. If GetStringUTFChars fails, it throws OOM and return NULL. And one more thing to consider. LinuxDebuggerLocal_attach0 function looks terrible - 7 ReleaseStringUTFChars calls for 2 GetStringUTFChars. Maybe it make sense to introduce simple wrapper like AutoJavaString in src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp It would make the code simpler and less error prone. --alex On 02/20/2020 00:53, Baesken, Matthias wrote: > Hi Alex / Christoph, thanks for the reviews. > > New webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.1/ > > - includes LinuxDebuggerLocal.cpp > - adds a blank Christoph wanted to have > > > A question (hopefully not a stupid one ?? ): > At most places in the coding, GetStringUTFChars success is > 1. handled by checking NULL , like this : > > const char *s = (*env)->GetStringUTFChars(env, p, NULL); > if (s == NULL) { > // handle failure > } > > 2.At some places , success / failure is not handled at all . > > 3.Here (e.g. LinuxDebuggerLocal.cpp) success / failure check is done by > > if (env->ExceptionOccurred()) { ... } > > Which one is the best / right way to do it (most likely not 2.) ? > > > Best regards, Matthias > > > >> >> Looks like >> src/jdk.hotspot.agent/linux/native/libsaproc/LinuxDebuggerLocal.cpp >> has similar issues. It would be nice to fix them as well. >> >> --alex >> >> On 02/19/2020 06:21, Baesken, Matthias wrote: >>> Hello, please review this small change . >>> We miss at a few places ReleaseStringUTFChars calls in the native >>> jdk.hotspot.agent coding. >>> This happens in case of early returns . >>> >>> >>> Bug/webrev : >>> >>> https://bugs.openjdk.java.net/browse/JDK-8239462 >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.0/ >>> >>> >>> Thanks, Matthias >>> From daniel.daugherty at oracle.com Thu Feb 20 20:30:13 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 20 Feb 2020 15:30:13 -0500 Subject: Mach5 test failure when testing JDK-8237192 In-Reply-To: References: Message-ID: <33bc091b-d9ed-44f3-40e1-729634adbd2f@oracle.com> Greetings, I filed two relevant bugs when I swept the JDK15 CI this AM: ??? JDK-8239566 gtest/GTestWrapper.java fails due to "libstlport.so.1: open failed: No such file or directory" ??? https://bugs.openjdk.java.net/browse/JDK-8239566 ??? JDK-8239565 sa/ClhsdbJhisto.java failed due to "assert(false) failed: unscheduable graph" ??? https://bugs.openjdk.java.net/browse/JDK-8239565 The second bug is similar to a bug I filed earlier this week: ??? JDK-8239367 RunThese30M.java failed due to "assert(false) failed: graph should be schedulable" ??? https://bugs.openjdk.java.net/browse/JDK-8239367 The second bug is also with a different test than is mentioned below: ??? serviceability/sa/ClhsdbCDSJstackPrintAll.java but the assertion and the stack look like they match... Dan On 2/20/20 3:51 AM, Langer, Christoph wrote: > Thanks, David for the information. > > As I don't see a relation from the crash to my change (I didn't touch any hotspot code at least), I guess I'm confident enough to push my patch. If worse comes to worse there's still the option to back it out again... > > Best regards > Christoph > >> -----Original Message----- >> From: David Holmes >> Sent: Donnerstag, 20. Februar 2020 09:28 >> To: Langer, Christoph ; 'build- >> dev at openjdk.java.net' ; 'hotspot- >> dev at openjdk.java.net' ; Erik Joelsson >> ; Magnus Ihse Bursie >> >> Subject: Re: Mach5 test failure when testing JDK-8237192 >> >> Hi Christoph, >> >> The Solaris failure looks like an infra issue. >> >> The test failure is a crash - info below. I don't see any open, or >> recently fixed, bugs for the same crash. >> >> David >> ----- >> >> # >> # A fatal error has been detected by the Java Runtime Environment: >> # >> # Internal Error >> (/scratch/mesos/slaves/90726e33-be99-4e27-9d68-25dad266ef13- >> S3967/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd- >> 0196/executors/9c4085d3-922a-46c2-b44a-9835e74ddb36/runs/ecd258eb- >> 38e6-4966-b233- >> 0cdfa582f713/workspace/open/src/hotspot/share/opto/gcm.cpp:276), >> pid=34522, tid=43267 >> # assert(false) failed: unscheduable graph >> # >> # JRE version: Java(TM) SE Runtime Environment (15.0) (fastdebug build >> 15-internal+0-2020-02-19-1905201.christoph.langer.source) >> # Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug >> 15-internal+0-2020-02-19-1905201.christoph.langer.source, mixed mode, >> sharing, tiered, compressed oops, g1 gc, bsd-amd64) >> # Core dump will be written. Default location: core.34522 >> # >> # If you would like to submit a bug report, please visit: >> # https://bugreport.java.com/bugreport/crash.jsp >> # >> >> --------------- S U M M A R Y ------------ >> >> Command Line: >> -Denv.class.path=/scratch/mesos/slaves/90726e33-be99-4e27-9d68- >> 25dad266ef13-S80/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd- >> 0196/executors/fb2f58a7-7e16-4340-8783-4207c1d11742/runs/3ca5d907- >> 2207-49d1-a673-f3617d83173d/testoutput/test- >> support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/1/servi >> ceability/sa/ClhsdbCDSJstackPrintAll.d:/scratch/mesos/jib- >> master/install/2020-02-19- >> 1905201.christoph.langer.source/src.full/open/test/hotspot/jtreg/serviceabil >> ity/sa:/scratch/mesos/slaves/90726e33-be99-4e27-9d68-25dad266ef13- >> S80/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd- >> 0196/executors/fb2f58a7-7e16-4340-8783-4207c1d11742/runs/3ca5d907- >> 2207-49d1-a673-f3617d83173d/testoutput/test- >> support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/1/test/ >> lib:/scratch/mesos/jib-master/install/2020-02-19- >> 1905201.christoph.langer.source/src.full/open/test/lib:/scratch/mesos/jib- >> master/install/java/re/jtreg/4.2/promoted/all/b16/bundles/jtreg_bin- >> 4.2.zip/jtreg/lib/javatest.jar:/scratch/mesos/jib- >> master/install/java/re/jtreg/4.2/promoted/all/b16/bundles/jtreg_bin- >> 4.2.zip/jtreg/lib/jtreg.jar >> -Dapplication.home=/scratch/mesos/jib-master/install/2020-02-19- >> 1905201.christoph.langer.source/macosx-x64-debug.jdk/jdk-15/fastdebug >> -Xms8m -Djdk.module.main=jdk.hotspot.agent >> jdk.hotspot.agent/sun.jvm.hotspot.SALauncher clhsdb --pid=34506 >> >> Host: scaaa915.us.oracle.com, MacPro6,1 x86_64 3700 MHz, 8 cores, 16G, >> Darwin 17.5.0 >> Time: Wed Feb 19 19:49:38 2020 GMT elapsed time: 5 seconds (0d 0h 0m 5s) >> >> --------------- T H R E A D --------------- >> >> Current thread (0x00007f9158008800): JavaThread "C2 CompilerThread0" >> daemon [_thread_in_native, id=43267, >> stack(0x0000700007ca8000,0x0000700007da8000)] >> >> >> Current CompileTask: >> C2: 5392 726 ! 4 >> jdk.internal.reflect.GeneratedConstructorAccessor3::newInstance (53 bytes) >> >> Stack: [0x0000700007ca8000,0x0000700007da8000], sp=0x0000700007da3b70, >> free space=1006k >> Native frames: (J=compiled Java code, A=aot compiled Java code, >> j=interpreted, Vv=VM code, C=native code) >> V [libjvm.dylib+0xb48133] VMError::report_and_die(int, char const*, >> char const*, __va_list_tag*, Thread*, unsigned char*, void*, void*, char >> const*, int, unsigned long)+0x6e5 >> V [libjvm.dylib+0xb4884f] VMError::report_and_die(Thread*, void*, char >> const*, int, char const*, char const*, __va_list_tag*)+0x47 >> V [libjvm.dylib+0x338454] report_vm_error(char const*, int, char >> const*, char const*, ...)+0x145 >> V [libjvm.dylib+0x49c0fb] assert_dom(Block*, Block*, Node*, PhaseCFG >> const*)+0x15d >> V [libjvm.dylib+0x497588] PhaseCFG::schedule_early(VectorSet&, >> Node_Stack&)+0x29a >> V [libjvm.dylib+0x499c49] PhaseCFG::global_code_motion()+0x1ad >> V [libjvm.dylib+0x49a211] PhaseCFG::do_global_code_motion()+0x41 >> V [libjvm.dylib+0x304254] Compile::Code_Gen()+0x224 >> V [libjvm.dylib+0x301f3b] Compile::Compile(ciEnv*, C2Compiler*, >> ciMethod*, int, bool, bool, bool, DirectiveSet*)+0xbf1 >> V [libjvm.dylib+0x254242] C2Compiler::compile_method(ciEnv*, >> ciMethod*, int, DirectiveSet*)+0xe8 >> V [libjvm.dylib+0x3142ae] >> CompileBroker::invoke_compiler_on_method(CompileTask*)+0x664 >> V [libjvm.dylib+0x313a3f] CompileBroker::compiler_thread_loop()+0x283 >> V [libjvm.dylib+0xabf30b] JavaThread::thread_main_inner()+0x1a1 >> V [libjvm.dylib+0xabeebd] JavaThread::run()+0x23d >> V [libjvm.dylib+0xabb861] Thread::call_run()+0x11b >> V [libjvm.dylib+0x93521c] thread_native_entry(Thread*)+0xe0 >> C [libsystem_pthread.dylib+0x3661] _pthread_body+0x154 >> C [libsystem_pthread.dylib+0x350d] _pthread_body+0x0 >> C [libsystem_pthread.dylib+0x2bf9] thread_start+0xd >> >> On 20/02/2020 6:06 pm, Langer, Christoph wrote: >>> Hi, >>> >>> I tested my change for JDK-8237192 in the submit repo. I got this back. >>> >>> Can anybody from Oracle please have a look whether the failures could be >> related to my patch? At first sight and from the information I can see here, I >> don?t see the relation? >>> Thanks >>> Christoph >>> >>> From: do-not-reply at oracle.com >>> Sent: Mittwoch, 19. Februar 2020 21:16 >>> To: Langer, Christoph >>> Subject: [Mach5] mach5-one-clanger-JDK-8237192-20200219-1906-8861422: >> FAILED, Failed tests: 1 >>> Job: mach5-one-clanger-JDK-8237192-20200219-1906-8861422 >>> BuildId: 2020-02-19-1905201.christoph.langer.source >>> Failed tests: showing 1 out of 1 >>> Test >>> Tier >>> Platform >>> Description >>> serviceability/sa/ClhsdbCDSJstackPrintAll.java >>> tier1 >>> macosx-x64-debug >>> Exception: java.io.IOException: LingeredApp terminated with non-zero exit >> code ... >>> Tasks Summary >>> >>> * UNABLE_TO_RUN: 1 >>> * PASSED: 76 >>> * FAILED: 1 >>> * NA: 0 >>> * NOTHING_TO_RUN: 0 >>> * EXECUTED_WITH_FAILURE: 1 >>> * HARNESS_ERROR: 0 >>> * KILLED: 0 >>> >>> Build >>> 1 Failed >>> >>> * solaris-sparcv9-open-debug-solaris-sparcv9-build-11 >> SOURCE_MASTER REASON_INVALID_...d68-25dad266ef13-O20132041 is no >> longer valid >>> Test >>> 1 Unable to run >>> >>> * tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris- >> sparcv9-open-debug-62 Dependency task failed: mach5...s-sparcv9-open- >> debug-solaris-sparcv9-build-11 >>> 1 Executed with failure >>> >>> * tier1-debug-open_test_hotspot_jtreg_tier1_serviceability-macosx- >> x64-debug-59 Results: total: 44, passed: 43, failed: 1, skipped: 0 From david.holmes at oracle.com Fri Feb 21 02:16:52 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Feb 2020 12:16:52 +1000 Subject: Mach5 test failure when testing JDK-8237192 In-Reply-To: <33bc091b-d9ed-44f3-40e1-729634adbd2f@oracle.com> References: <33bc091b-d9ed-44f3-40e1-729634adbd2f@oracle.com> Message-ID: On 21/02/2020 6:30 am, Daniel D. Daugherty wrote: > Greetings, > > I filed two relevant bugs when I swept the JDK15 CI this AM: > > ??? JDK-8239566 gtest/GTestWrapper.java fails due to "libstlport.so.1: > open failed: No such file or directory" > ??? https://bugs.openjdk.java.net/browse/JDK-8239566 > > ??? JDK-8239565 sa/ClhsdbJhisto.java failed due to "assert(false) > failed: unscheduable graph" > ??? https://bugs.openjdk.java.net/browse/JDK-8239565 > > The second bug is similar to a bug I filed earlier this week: > > ??? JDK-8239367 RunThese30M.java failed due to "assert(false) failed: > graph should be schedulable" > ??? https://bugs.openjdk.java.net/browse/JDK-8239367 I thought I had seen a similar failure, but I searched for unschedulable and so it failed to match. :( David ----- > The second bug is also with a different test than is mentioned below: > > ??? serviceability/sa/ClhsdbCDSJstackPrintAll.java > > but the assertion and the stack look like they match... > > Dan > > > > On 2/20/20 3:51 AM, Langer, Christoph wrote: >> Thanks, David for the information. >> >> As I don't see a relation from the crash to my change (I didn't touch >> any hotspot code at least), I guess I'm confident enough to push my >> patch. If worse comes to worse there's still the option to back it out >> again... >> >> Best regards >> Christoph >> >>> -----Original Message----- >>> From: David Holmes >>> Sent: Donnerstag, 20. Februar 2020 09:28 >>> To: Langer, Christoph ; 'build- >>> dev at openjdk.java.net' ; 'hotspot- >>> dev at openjdk.java.net' ; Erik Joelsson >>> ; Magnus Ihse Bursie >>> >>> Subject: Re: Mach5 test failure when testing JDK-8237192 >>> >>> Hi Christoph, >>> >>> The Solaris failure looks like an infra issue. >>> >>> The test failure is a crash - info below. I don't see any open, or >>> recently fixed, bugs for the same crash. >>> >>> David >>> ----- >>> >>> # >>> # A fatal error has been detected by the Java Runtime Environment: >>> # >>> #? Internal Error >>> (/scratch/mesos/slaves/90726e33-be99-4e27-9d68-25dad266ef13- >>> S3967/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd- >>> 0196/executors/9c4085d3-922a-46c2-b44a-9835e74ddb36/runs/ecd258eb- >>> 38e6-4966-b233- >>> 0cdfa582f713/workspace/open/src/hotspot/share/opto/gcm.cpp:276), >>> pid=34522, tid=43267 >>> #? assert(false) failed: unscheduable graph >>> # >>> # JRE version: Java(TM) SE Runtime Environment (15.0) (fastdebug build >>> 15-internal+0-2020-02-19-1905201.christoph.langer.source) >>> # Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug >>> 15-internal+0-2020-02-19-1905201.christoph.langer.source, mixed mode, >>> sharing, tiered, compressed oops, g1 gc, bsd-amd64) >>> # Core dump will be written. Default location: core.34522 >>> # >>> # If you would like to submit a bug report, please visit: >>> #?? https://bugreport.java.com/bugreport/crash.jsp >>> # >>> >>> ---------------? S U M M A R Y ------------ >>> >>> Command Line: >>> -Denv.class.path=/scratch/mesos/slaves/90726e33-be99-4e27-9d68- >>> 25dad266ef13-S80/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd- >>> 0196/executors/fb2f58a7-7e16-4340-8783-4207c1d11742/runs/3ca5d907- >>> 2207-49d1-a673-f3617d83173d/testoutput/test- >>> support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/1/servi >>> >>> ceability/sa/ClhsdbCDSJstackPrintAll.d:/scratch/mesos/jib- >>> master/install/2020-02-19- >>> 1905201.christoph.langer.source/src.full/open/test/hotspot/jtreg/serviceabil >>> >>> ity/sa:/scratch/mesos/slaves/90726e33-be99-4e27-9d68-25dad266ef13- >>> S80/frameworks/1735e8a2-a1db-478c-8104-60c8b0af87dd- >>> 0196/executors/fb2f58a7-7e16-4340-8783-4207c1d11742/runs/3ca5d907- >>> 2207-49d1-a673-f3617d83173d/testoutput/test- >>> support/jtreg_open_test_hotspot_jtreg_tier1_serviceability/classes/1/test/ >>> >>> lib:/scratch/mesos/jib-master/install/2020-02-19- >>> 1905201.christoph.langer.source/src.full/open/test/lib:/scratch/mesos/jib- >>> >>> master/install/java/re/jtreg/4.2/promoted/all/b16/bundles/jtreg_bin- >>> 4.2.zip/jtreg/lib/javatest.jar:/scratch/mesos/jib- >>> master/install/java/re/jtreg/4.2/promoted/all/b16/bundles/jtreg_bin- >>> 4.2.zip/jtreg/lib/jtreg.jar >>> -Dapplication.home=/scratch/mesos/jib-master/install/2020-02-19- >>> 1905201.christoph.langer.source/macosx-x64-debug.jdk/jdk-15/fastdebug >>> -Xms8m -Djdk.module.main=jdk.hotspot.agent >>> jdk.hotspot.agent/sun.jvm.hotspot.SALauncher clhsdb --pid=34506 >>> >>> Host: scaaa915.us.oracle.com, MacPro6,1 x86_64 3700 MHz, 8 cores, 16G, >>> Darwin 17.5.0 >>> Time: Wed Feb 19 19:49:38 2020 GMT elapsed time: 5 seconds (0d 0h 0m 5s) >>> >>> ---------------? T H R E A D? --------------- >>> >>> Current thread (0x00007f9158008800):? JavaThread "C2 CompilerThread0" >>> daemon [_thread_in_native, id=43267, >>> stack(0x0000700007ca8000,0x0000700007da8000)] >>> >>> >>> Current CompileTask: >>> C2:?? 5392? 726?? !?? 4 >>> jdk.internal.reflect.GeneratedConstructorAccessor3::newInstance (53 >>> bytes) >>> >>> Stack: [0x0000700007ca8000,0x0000700007da8000],? sp=0x0000700007da3b70, >>> free space=1006k >>> Native frames: (J=compiled Java code, A=aot compiled Java code, >>> j=interpreted, Vv=VM code, C=native code) >>> V? [libjvm.dylib+0xb48133]? VMError::report_and_die(int, char const*, >>> char const*, __va_list_tag*, Thread*, unsigned char*, void*, void*, char >>> const*, int, unsigned long)+0x6e5 >>> V? [libjvm.dylib+0xb4884f]? VMError::report_and_die(Thread*, void*, char >>> const*, int, char const*, char const*, __va_list_tag*)+0x47 >>> V? [libjvm.dylib+0x338454]? report_vm_error(char const*, int, char >>> const*, char const*, ...)+0x145 >>> V? [libjvm.dylib+0x49c0fb]? assert_dom(Block*, Block*, Node*, PhaseCFG >>> const*)+0x15d >>> V? [libjvm.dylib+0x497588]? PhaseCFG::schedule_early(VectorSet&, >>> Node_Stack&)+0x29a >>> V? [libjvm.dylib+0x499c49]? PhaseCFG::global_code_motion()+0x1ad >>> V? [libjvm.dylib+0x49a211]? PhaseCFG::do_global_code_motion()+0x41 >>> V? [libjvm.dylib+0x304254]? Compile::Code_Gen()+0x224 >>> V? [libjvm.dylib+0x301f3b]? Compile::Compile(ciEnv*, C2Compiler*, >>> ciMethod*, int, bool, bool, bool, DirectiveSet*)+0xbf1 >>> V? [libjvm.dylib+0x254242]? C2Compiler::compile_method(ciEnv*, >>> ciMethod*, int, DirectiveSet*)+0xe8 >>> V? [libjvm.dylib+0x3142ae] >>> CompileBroker::invoke_compiler_on_method(CompileTask*)+0x664 >>> V? [libjvm.dylib+0x313a3f]? CompileBroker::compiler_thread_loop()+0x283 >>> V? [libjvm.dylib+0xabf30b]? JavaThread::thread_main_inner()+0x1a1 >>> V? [libjvm.dylib+0xabeebd]? JavaThread::run()+0x23d >>> V? [libjvm.dylib+0xabb861]? Thread::call_run()+0x11b >>> V? [libjvm.dylib+0x93521c]? thread_native_entry(Thread*)+0xe0 >>> C? [libsystem_pthread.dylib+0x3661]? _pthread_body+0x154 >>> C? [libsystem_pthread.dylib+0x350d]? _pthread_body+0x0 >>> C? [libsystem_pthread.dylib+0x2bf9]? thread_start+0xd >>> >>> On 20/02/2020 6:06 pm, Langer, Christoph wrote: >>>> Hi, >>>> >>>> I tested my change for JDK-8237192 in the submit repo. I got this back. >>>> >>>> Can anybody from Oracle please have a look whether the failures >>>> could be >>> related to my patch? At first sight and from the information I can >>> see here, I >>> don?t see the relation? >>>> Thanks >>>> Christoph >>>> >>>> From: do-not-reply at oracle.com >>>> Sent: Mittwoch, 19. Februar 2020 21:16 >>>> To: Langer, Christoph >>>> Subject: [Mach5] mach5-one-clanger-JDK-8237192-20200219-1906-8861422: >>> FAILED, Failed tests: 1 >>>> Job: mach5-one-clanger-JDK-8237192-20200219-1906-8861422 >>>> BuildId: 2020-02-19-1905201.christoph.langer.source >>>> Failed tests: showing 1 out of 1 >>>> Test >>>> Tier >>>> Platform >>>> Description >>>> serviceability/sa/ClhsdbCDSJstackPrintAll.java >>>> tier1 >>>> macosx-x64-debug >>>> Exception: java.io.IOException: LingeredApp terminated with non-zero >>>> exit >>> code ... >>>> Tasks Summary >>>> >>>> ??? *?? UNABLE_TO_RUN: 1 >>>> ??? *?? PASSED: 76 >>>> ??? *?? FAILED: 1 >>>> ??? *?? NA: 0 >>>> ??? *?? NOTHING_TO_RUN: 0 >>>> ??? *?? EXECUTED_WITH_FAILURE: 1 >>>> ??? *?? HARNESS_ERROR: 0 >>>> ??? *?? KILLED: 0 >>>> >>>> Build >>>> 1 Failed >>>> >>>> ?????? *?? solaris-sparcv9-open-debug-solaris-sparcv9-build-11 >>> SOURCE_MASTER REASON_INVALID_...d68-25dad266ef13-O20132041 is no >>> longer valid >>>> Test >>>> 1 Unable to run >>>> >>>> ?????? * >>>> tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris- >>> sparcv9-open-debug-62 Dependency task failed: mach5...s-sparcv9-open- >>> debug-solaris-sparcv9-build-11 >>>> 1 Executed with failure >>>> >>>> ?????? * >>>> tier1-debug-open_test_hotspot_jtreg_tier1_serviceability-macosx- >>> x64-debug-59 Results: total: 44, passed: 43, failed: 1, skipped: 0 > From david.holmes at oracle.com Fri Feb 21 02:25:31 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Feb 2020 12:25:31 +1000 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> <4ac31ff3-df28-c3c4-5b63-9c2849716d00@oracle.com> <8982aeb0-3e12-f92f-a199-eb4ee1af1af9@oracle.com> Message-ID: <1a389351-3b80-5dc6-5712-6b4a5e018403@oracle.com> On 20/02/2020 11:22 pm, Ao Qi wrote: > On Thu, Feb 20, 2020 at 9:08 PM David Holmes wrote: >> >> On 20/02/2020 8:33 pm, Ao Qi wrote: >>>> Ah! Now I see the problem. In WhiteBox we have: >>>> >>>> if (comp_level > MIN2((CompLevel) TieredStopAtLevel, >>>> CompLevel_highest_tier)) { >>>> tty->print_cr("WB error: invalid compilation level %d", comp_level); >>>> return false; >>>> } >>>> >>>> but TieredStopAtLevel should be ignored when tiered compilation is >>>> disabled, or not present. I actually think it is a bug that the VM >>>> allows you to set TieredStopAtLevel even though it is ignored on those >>>> configurations. The WB code should be changed to something like: >>>> >>>> #ifdef TIERED >>>> CompLevel cl_min = MIN2((CompLevel) TieredStopAtLevel, >>>> CompLevel_highest_tier); >>>> #else >>>> CompLevel cl_min = CompLevel_highest_tier; >>>> #endif >>>> if (comp_level > cl_min) { >>>> tty->print_cr("WB error: invalid compilation level %d", comp_level); >>>> return false; >>>> } >>>> >>>> There are other places in WhiteBox that need similar fixes. >>> >>> Could you give some comments or advice on this one: >>> http://cr.openjdk.java.net/~aoqi/8239423/webrev.01/ ? >>> >>> I think TieredStopAtLevel should be ignored when >>> -XX:-TieredCompilation is used (even a build with tiered >>> configration), so I used TieredCompilation to determine the cl_min, >>> not "#ifdef TIERED". What do you think? >> >> Yes you are quite right! That is the correct way to handle this. >> >> I assume the test passes in all combinations now? > > Yes. I have tested all the combinations. Also tested tier1. No > regression found so far. We need to find all tests that use the modified WB functions and see if they all pass under normal and non-tiered configurations. Thanks, David > Cheers, > Ao Qi > >> >> Thanks, >> David >> >>> Cheers, >>> Ao Qi >>> > From david.holmes at oracle.com Fri Feb 21 02:43:08 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 21 Feb 2020 12:43:08 +1000 Subject: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS In-Reply-To: References: <788370d5-594c-e052-4580-75cc14152f96@oracle.com> Message-ID: <3460a21e-0fe8-df1b-d56a-00dfb2b3e0d8@oracle.com> On 21/02/2020 2:22 am, Reingruber, Richard wrote: > > > > I wonder why the VM doesn't use gnu_get_libc_version() ? os_linux.cpp does: > > > > size_t n = confstr(_CS_GNU_LIBC_VERSION, NULL, 0); > > assert(n > 0, "cannot retrieve glibc version"); > > char *str = (char *)malloc(n, mtInternal); > > confstr(_CS_GNU_LIBC_VERSION, str, n); > > It used to: https://bugs.openjdk.java.net/browse/JDK-8078513 > > Personally I would apply > > - size_t n = confstr(_CS_GNU_LIBC_VERSION, NULL, 0); > - assert(n > 0, "cannot retrieve glibc version"); > - char *str = (char *)malloc(n, mtInternal); > - confstr(_CS_GNU_LIBC_VERSION, str, n); > - os::Linux::set_glibc_version(str); > - > - n = confstr(_CS_GNU_LIBPTHREAD_VERSION, NULL, 0); > + os::Linux::set_glibc_version(gnu_get_libc_version()); Thomas obviously thinks confstr is superior for some reason. :) Cheers, David > > > I'm okay with the test changes. > > Thanks! > Richard. > > -----Original Message----- > From: David Holmes > Sent: Donnerstag, 20. Februar 2020 10:34 > To: Reingruber, Richard ; Jiangli Zhou ; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS > > On 20/02/2020 7:03 pm, Reingruber, Richard wrote: >> Hi Jiangli, >> >> thanks for providing feedback. >> >> > David's suggestion of using dlsym seems to be more reliable. Regarding the stability of >> > __pthread_get_minstack in future glibc versions, Florian has added comment to >> > __pthread_get_minstack about external users to ensure it's not removed until TCB is >> > separately allocated. Please see: http://51.15.138.76/patch/18989/ and >> > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. >> >> So if I understand you and David correctly, then the test should be skipped if the lookup of >> __pthread_get_minstack fails. >> >> Then the test would be skipped on future glibc versions where __pthread_get_minstack was removed >> because TLS and TCB are not placed on a threads stack anymore. > > I don't think that is really a significant concern. As Jiangli pointed > out any change will be communicated beforehand and we will adjust the VM > code and the test as needed. But okay ... > > I wonder why the VM doesn't use gnu_get_libc_version() ? os_linux.cpp does: > > size_t n = confstr(_CS_GNU_LIBC_VERSION, NULL, 0); > assert(n > 0, "cannot retrieve glibc version"); > char *str = (char *)malloc(n, mtInternal); > confstr(_CS_GNU_LIBC_VERSION, str, n); > > I'm okay with the test changes. > > Thanks, > David > ----- > >> Actually I would prefer not to skip the test then. I'd like to verify if large data structures can >> be placed in TLS without stack size adjustment or if there is some kind of regression. That's the >> point of regression tests after all. >> >> In other words: I wouldn't change test coverage of future glibc versions. I just want to exclude >> glibc versions were the tested feature cannot work. >> >> Thanks, Richard. >> >> -----Original Message----- >> From: Jiangli Zhou >> Sent: Mittwoch, 19. Februar 2020 19:24 >> To: Reingruber, Richard >> Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS >> >> Hi Richard, >> >> David's suggestion of using dlsym seems to be more reliable. Regarding >> the stability of __pthread_get_minstack in future glibc versions, >> Florian has added comment to __pthread_get_minstack about external >> users to ensure it's not removed until TCB is separately allocated. >> Please see: http://51.15.138.76/patch/18989/ and >> https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. >> >> Best regards, >> Jiangli >> >> >> >> >> On Wed, Feb 19, 2020 at 2:44 AM Reingruber, Richard >> wrote: >>> >>> Hi, >>> >>> please review this small test fix >>> >>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239449/webrev.0/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8239449 >>> >>> On linux systems with older glibc versions (e.g. 2.11) the option -XX:+AdjustStackSizeForTLS has no effect, because the private function __pthread_get_minstack is not provided, and the test fails. With the fix the test is skipped instead. >>> >>> Thanks, Richard. From aoqi at loongson.cn Fri Feb 21 02:42:32 2020 From: aoqi at loongson.cn (Ao Qi) Date: Fri, 21 Feb 2020 10:42:32 +0800 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: <1a389351-3b80-5dc6-5712-6b4a5e018403@oracle.com> References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> <4ac31ff3-df28-c3c4-5b63-9c2849716d00@oracle.com> <8982aeb0-3e12-f92f-a199-eb4ee1af1af9@oracle.com> <1a389351-3b80-5dc6-5712-6b4a5e018403@oracle.com> Message-ID: > We need to find all tests that use the modified WB functions and see if > they all pass under normal and non-tiered configurations. I agree. I will try to find these tests. Cheers, Ao Qi > > Thanks, > David > From matthias.baesken at sap.com Fri Feb 21 08:09:26 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 21 Feb 2020 08:09:26 +0000 Subject: RFR [XS]: 8239462: jdk.hotspot.agent misses some ReleaseStringUTFChars calls in case of early returns In-Reply-To: References: <6ed0d004-3e8c-4b27-f583-06dbacf45173@oracle.com> Message-ID: > Also I see GetStringUTFChars(str, JNI_FALSE). This look bad as well - > 2nd arg is a pointer, so it should be NULL or nullptr. Hi looks like there is another one here, do you think these JNI_FALSE params would really cause trouble ? Not even the compiler warns here ... src/java.desktop/unix/native/libawt_xawt/xawt/XlibWrapper.c:824: cname = (char *) (*env)->GetStringUTFChars(env, jstr, JNI_FALSE); Best regards, Matthias > Hi Matthias, > > Looks good in general, but I think it makes sense to fix #2 cases (at > least I see them in LinuxDebuggerLocal). If GetStringUTFChars fails, the > code will crash. > Also I see GetStringUTFChars(str, JNI_FALSE). This look bad as well - > 2nd arg is a pointer, so it should be NULL or nullptr. > > As for #1 and #3 - AFAIU they are both right ways. > If GetStringUTFChars fails, it throws OOM and return NULL. > > And one more thing to consider. > LinuxDebuggerLocal_attach0 function looks terrible - 7 > ReleaseStringUTFChars calls for 2 GetStringUTFChars. > Maybe it make sense to introduce simple wrapper like AutoJavaString in > src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp > It would make the code simpler and less error prone. > > --alex > > On 02/20/2020 00:53, Baesken, Matthias wrote: > > Hi Alex / Christoph, thanks for the reviews. > > > > New webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.1/ > > > > - includes LinuxDebuggerLocal.cpp > > - adds a blank Christoph wanted to have > > > > > > A question (hopefully not a stupid one ?? ): > > At most places in the coding, GetStringUTFChars success is > > 1. handled by checking NULL , like this : > > > > const char *s = (*env)->GetStringUTFChars(env, p, NULL); > > if (s == NULL) { > > // handle failure > > } > > > > 2.At some places , success / failure is not handled at all . > > > > 3.Here (e.g. LinuxDebuggerLocal.cpp) success / failure check is done by > > > > if (env->ExceptionOccurred()) { ... } > > > > Which one is the best / right way to do it (most likely not 2.) ? > > > > > > Best regards, Matthias > > > > > > > >> > >> Looks like > >> src/jdk.hotspot.agent/linux/native/libsaproc/LinuxDebuggerLocal.cpp > >> has similar issues. It would be nice to fix them as well. > >> > >> --alex > >> > >> On 02/19/2020 06:21, Baesken, Matthias wrote: > >>> Hello, please review this small change . > >>> We miss at a few places ReleaseStringUTFChars calls in the native > >>> jdk.hotspot.agent coding. > >>> This happens in case of early returns . > >>> > >>> > >>> Bug/webrev : > >>> > >>> https://bugs.openjdk.java.net/browse/JDK-8239462 > >>> > >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.0/ > >>> > >>> > >>> Thanks, Matthias > >>> From matthias.baesken at sap.com Fri Feb 21 08:32:35 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 21 Feb 2020 08:32:35 +0000 Subject: RFR [XS]: 8239462: jdk.hotspot.agent misses some ReleaseStringUTFChars calls in case of early returns In-Reply-To: References: <6ed0d004-3e8c-4b27-f583-06dbacf45173@oracle.com> Message-ID: Hi Alex , new webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.2/ Best Regards, Matthias > > Hi Matthias, > > Looks good in general, but I think it makes sense to fix #2 cases (at > least I see them in LinuxDebuggerLocal). If GetStringUTFChars fails, the > code will crash. > Also I see GetStringUTFChars(str, JNI_FALSE). This look bad as well - > 2nd arg is a pointer, so it should be NULL or nullptr. > > As for #1 and #3 - AFAIU they are both right ways. > If GetStringUTFChars fails, it throws OOM and return NULL. > > And one more thing to consider. > LinuxDebuggerLocal_attach0 function looks terrible - 7 > ReleaseStringUTFChars calls for 2 GetStringUTFChars. > Maybe it make sense to introduce simple wrapper like AutoJavaString in > src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp > It would make the code simpler and less error prone. > > --alex > From matthias.baesken at sap.com Fri Feb 21 09:19:58 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 21 Feb 2020 09:19:58 +0000 Subject: RFR [XS]: 8239537: cgroup MetricsTester testMemorySubsystem fails sometimes when testing memory.kmem.tcp.usage_in_bytes In-Reply-To: References: Message-ID: Thanks for the review, will push it as XS . Best regards, Matthias > Looks good to me, > > Misha > > On 2/20/20 2:25 AM, Baesken, Matthias wrote: > > Hello, please review this small adjustment . > > > > The testMemorySubsystem test in MetricsTester.java sometimes fails with > messages like this : > > > > java.lang.RuntimeException: Test failed for - > memory:memory.kmem.tcp.usage_in_bytes, expected [655360], got > [786432] > > at > jdk.test.lib.containers.cgroup.MetricsTester.fail(MetricsTester.java:207) > > at > jdk.test.lib.containers.cgroup.MetricsTester.testMemorySubsystem(Metrics > Tester.java:309) > > > > I looked into the code doing the checks , and looks like there is already an > error margin allowed in CgroupMetricsTester.java , which is currently 0.1 . > > This might be changed to a somewhat higher value, e.g. 0.25 to make the > test more robust. > > > > > > Bug/webrev : > > > > https://bugs.openjdk.java.net/browse/JDK-8239537 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8239537.0/ > > > > Thanks, Matthias From stefan.karlsson at oracle.com Fri Feb 21 10:23:44 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 21 Feb 2020 11:23:44 +0100 Subject: [15] RFR 8238633: JVMTI heap walk should consult GC for marking oops In-Reply-To: <3a5d60fd-04d5-da96-3d79-242d43fdec79@redhat.com> References: <8ea20a15-bdb9-27b0-c306-154f539a3674@oracle.com> <3a5d60fd-04d5-da96-3d79-242d43fdec79@redhat.com> Message-ID: <901d7307-cf36-0367-e09c-ff47c76bbc25@oracle.com> Hi Zhengyu, On 2020-02-17 15:51, Zhengyu Gu wrote: > Hi Stefan, > > Thanks for the review and suggestions, updated accordingly: > > http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/ Thanks for moving the code. I think this looks good. If you're up for it, I have a couple of style change suggestions: 1) ObjectMarker uses two verbs to describe the same thing: "mark" and "visit". I propose that we only use "mark" in ObjectMarker and leave the usage of "visited" to the Jvmti code. 2) Some updates to odd whitespaces 3) Using forward declarations in Shenandoah code. I've bundled those changes into webrevs: https://cr.openjdk.java.net/~stefank/8238633/webrev.01.delta https://cr.openjdk.java.net/~stefank/8238633/webrev.01 Regarding performance testing, the HeapWalkTests you used seems to use a very small heap. I think it would be good to redo the measurements on a larger heap. Could you take the HeapWalkTest and add a few GBs of small, linked objects? Thank, StefanK > >> >> --- >> Previously, the calls to 'mark' and 'visited' were inlineable, but >> now every GC has to take a virtual call when marking the objects. My >> guess is that this code is slow anyway, and that it doesn't matter >> too much, but did you measure the effect of that change with, for >> example, G1? >> > I did rough measurement, timing > vmTestbase/nsk/jvmti/unit/heap/HeapWalkTests/TestDescription.java test. > > If you know any tests/benchmarks I should measure, please let me know. > > Thanks, > > -Zhengyu > > >> Thanks, >> StefanK >> >>> Test: >>> ?? hotspot_gc >>> ?? vmTestbase_nsk_jdi >>> ?? vmTestbase_nsk_jvmti >>> >>> Thanks, >>> >>> -Zhengyu >>> >>> >> > From shade at redhat.com Fri Feb 21 10:39:54 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 21 Feb 2020 11:39:54 +0100 Subject: RFR (S) 8239492: [x86] Turn MacroAssembler::verify_oop into macro recording file and line Message-ID: <8bedc6e0-3b2b-a3c1-8a91-1ebce3d6641b@redhat.com> RFE: https://bugs.openjdk.java.net/browse/JDK-8239492 This is done in SPARC and ARM MacroAssemblers, and should be done in x86 too. This should greatly improve the error reports, as it would point to exact place where the verification happened, e.g. in interpreter/stub generation code. Suggested fix: https://cr.openjdk.java.net/~shade/8239492/webrev.02/ Testing: tier1 {x86_32, x86_64} with -XX:+VerifyOops, jdk-submit (clean, except unrelated Solaris gtest failure) -- Thanks, -Aleksey From richard.reingruber at sap.com Fri Feb 21 11:24:59 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 21 Feb 2020 11:24:59 +0000 Subject: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS In-Reply-To: References: Message-ID: Hi Jiangli, > > Actually I would prefer not to skip the test then. I'd like to verify if large data structures can > > be placed in TLS without stack size adjustment or if there is some kind of regression. That's the > > point of regression tests after all. > > > That's a very valid point. I thought about that as well yesterday. My > thinking was that the VM and test would need to be updated to remove > the workaround for the TLS issue when no longer needed. Do you know if > the proposed check using gnu_get_libc_version() covers all cases? I Other C libraries (e.g. MUSL) are not really covered, but I'm not sure if you mean this by covered... > came across the following archive that mentioned missing symbol in > eglibc-2.13 (eglibc however is already deprecated). > https://www.mail-archive.com/lfs-dev at linuxfromscratch.org/msg18115.html I only knew, that 2.11 does not provide __pthread_get_minstack, but then I used https://github.com/bminor/glibc to find out that __pthread_get_minstack was added with 2.14.90 [1]. So 2.15 should be the first stable release to include __pthread_get_minstack. I'll correct this in my fix. Would you be ok with this version of the fix? Thanks and best regards, Richard. [1] version.h at [2] https://github.com/bminor/glibc/blob/2c1094bd700e63a8d7f547b3f5495bedb55c0a08/version.h [2] Repo at [3] https://github.com/bminor/glibc/tree/2c1094bd700e63a8d7f547b3f5495bedb55c0a08 [3] Commit that introduced __pthread_get_minstack https://github.com/bminor/glibc/commit/2c1094bd700e63a8d7f547b3f5495bedb55c0a08 -----Original Message----- From: Jiangli Zhou Sent: Donnerstag, 20. Februar 2020 17:32 To: Reingruber, Richard Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS Hi Richard, On Thu, Feb 20, 2020 at 1:03 AM Reingruber, Richard wrote: > > Hi Jiangli, > > thanks for providing feedback. > > > David's suggestion of using dlsym seems to be more reliable. Regarding the stability of > > __pthread_get_minstack in future glibc versions, Florian has added comment to > > __pthread_get_minstack about external users to ensure it's not removed until TCB is > > separately allocated. Please see: http://51.15.138.76/patch/18989/ and > > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. > > So if I understand you and David correctly, then the test should be skipped if the lookup of > __pthread_get_minstack fails. > > Then the test would be skipped on future glibc versions where __pthread_get_minstack was removed > because TLS and TCB are not placed on a threads stack anymore. > > Actually I would prefer not to skip the test then. I'd like to verify if large data structures can > be placed in TLS without stack size adjustment or if there is some kind of regression. That's the > point of regression tests after all. > That's a very valid point. I thought about that as well yesterday. My thinking was that the VM and test would need to be updated to remove the workaround for the TLS issue when no longer needed. Do you know if the proposed check using gnu_get_libc_version() covers all cases? I came across the following archive that mentioned missing symbol in eglibc-2.13 (eglibc however is already deprecated). https://www.mail-archive.com/lfs-dev at linuxfromscratch.org/msg18115.html > In other words: I wouldn't change test coverage of future glibc versions. I just want to exclude > glibc versions were the tested feature cannot work. Best regards, Jiangli > > Thanks, Richard. > > -----Original Message----- > From: Jiangli Zhou > Sent: Mittwoch, 19. Februar 2020 19:24 > To: Reingruber, Richard > Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS > > Hi Richard, > > David's suggestion of using dlsym seems to be more reliable. Regarding > the stability of __pthread_get_minstack in future glibc versions, > Florian has added comment to __pthread_get_minstack about external > users to ensure it's not removed until TCB is separately allocated. > Please see: http://51.15.138.76/patch/18989/ and > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. > > Best regards, > Jiangli > > > > > On Wed, Feb 19, 2020 at 2:44 AM Reingruber, Richard > wrote: > > > > Hi, > > > > please review this small test fix > > > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239449/webrev.0/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8239449 > > > > On linux systems with older glibc versions (e.g. 2.11) the option -XX:+AdjustStackSizeForTLS has no effect, because the private function __pthread_get_minstack is not provided, and the test fails. With the fix the test is skipped instead. > > > > Thanks, Richard. From richard.reingruber at sap.com Fri Feb 21 11:25:56 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 21 Feb 2020 11:25:56 +0000 Subject: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS In-Reply-To: <3460a21e-0fe8-df1b-d56a-00dfb2b3e0d8@oracle.com> References: <788370d5-594c-e052-4580-75cc14152f96@oracle.com> <3460a21e-0fe8-df1b-d56a-00dfb2b3e0d8@oracle.com> Message-ID: I'll ask him why next time we go get coffee together ;) Cheers, Richard. -----Original Message----- From: David Holmes Sent: Freitag, 21. Februar 2020 03:43 To: Reingruber, Richard ; Jiangli Zhou ; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS On 21/02/2020 2:22 am, Reingruber, Richard wrote: > > > > I wonder why the VM doesn't use gnu_get_libc_version() ? os_linux.cpp does: > > > > size_t n = confstr(_CS_GNU_LIBC_VERSION, NULL, 0); > > assert(n > 0, "cannot retrieve glibc version"); > > char *str = (char *)malloc(n, mtInternal); > > confstr(_CS_GNU_LIBC_VERSION, str, n); > > It used to: https://bugs.openjdk.java.net/browse/JDK-8078513 > > Personally I would apply > > - size_t n = confstr(_CS_GNU_LIBC_VERSION, NULL, 0); > - assert(n > 0, "cannot retrieve glibc version"); > - char *str = (char *)malloc(n, mtInternal); > - confstr(_CS_GNU_LIBC_VERSION, str, n); > - os::Linux::set_glibc_version(str); > - > - n = confstr(_CS_GNU_LIBPTHREAD_VERSION, NULL, 0); > + os::Linux::set_glibc_version(gnu_get_libc_version()); Thomas obviously thinks confstr is superior for some reason. :) Cheers, David > > > I'm okay with the test changes. > > Thanks! > Richard. > > -----Original Message----- > From: David Holmes > Sent: Donnerstag, 20. Februar 2020 10:34 > To: Reingruber, Richard ; Jiangli Zhou ; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS > > On 20/02/2020 7:03 pm, Reingruber, Richard wrote: >> Hi Jiangli, >> >> thanks for providing feedback. >> >> > David's suggestion of using dlsym seems to be more reliable. Regarding the stability of >> > __pthread_get_minstack in future glibc versions, Florian has added comment to >> > __pthread_get_minstack about external users to ensure it's not removed until TCB is >> > separately allocated. Please see: http://51.15.138.76/patch/18989/ and >> > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. >> >> So if I understand you and David correctly, then the test should be skipped if the lookup of >> __pthread_get_minstack fails. >> >> Then the test would be skipped on future glibc versions where __pthread_get_minstack was removed >> because TLS and TCB are not placed on a threads stack anymore. > > I don't think that is really a significant concern. As Jiangli pointed > out any change will be communicated beforehand and we will adjust the VM > code and the test as needed. But okay ... > > I wonder why the VM doesn't use gnu_get_libc_version() ? os_linux.cpp does: > > size_t n = confstr(_CS_GNU_LIBC_VERSION, NULL, 0); > assert(n > 0, "cannot retrieve glibc version"); > char *str = (char *)malloc(n, mtInternal); > confstr(_CS_GNU_LIBC_VERSION, str, n); > > I'm okay with the test changes. > > Thanks, > David > ----- > >> Actually I would prefer not to skip the test then. I'd like to verify if large data structures can >> be placed in TLS without stack size adjustment or if there is some kind of regression. That's the >> point of regression tests after all. >> >> In other words: I wouldn't change test coverage of future glibc versions. I just want to exclude >> glibc versions were the tested feature cannot work. >> >> Thanks, Richard. >> >> -----Original Message----- >> From: Jiangli Zhou >> Sent: Mittwoch, 19. Februar 2020 19:24 >> To: Reingruber, Richard >> Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS >> >> Hi Richard, >> >> David's suggestion of using dlsym seems to be more reliable. Regarding >> the stability of __pthread_get_minstack in future glibc versions, >> Florian has added comment to __pthread_get_minstack about external >> users to ensure it's not removed until TCB is separately allocated. >> Please see: http://51.15.138.76/patch/18989/ and >> https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. >> >> Best regards, >> Jiangli >> >> >> >> >> On Wed, Feb 19, 2020 at 2:44 AM Reingruber, Richard >> wrote: >>> >>> Hi, >>> >>> please review this small test fix >>> >>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239449/webrev.0/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8239449 >>> >>> On linux systems with older glibc versions (e.g. 2.11) the option -XX:+AdjustStackSizeForTLS has no effect, because the private function __pthread_get_minstack is not provided, and the test fails. With the fix the test is skipped instead. >>> >>> Thanks, Richard. From coleen.phillimore at oracle.com Fri Feb 21 12:59:01 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 21 Feb 2020 07:59:01 -0500 Subject: [15] RFR 8238633: JVMTI heap walk should consult GC for marking oops In-Reply-To: <901d7307-cf36-0367-e09c-ff47c76bbc25@oracle.com> References: <8ea20a15-bdb9-27b0-c306-154f539a3674@oracle.com> <3a5d60fd-04d5-da96-3d79-242d43fdec79@redhat.com> <901d7307-cf36-0367-e09c-ff47c76bbc25@oracle.com> Message-ID: Hi, I had a quick look at this, minus the shenandoah code. http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/src/hotspot/share/gc/shared/objectMarker.hpp.html I think this file could have forward declarations of GrowableArray and I didn't see a need for the markWord.hpp include. This change on the whole looks good to me. Coleen On 2/21/20 5:23 AM, Stefan Karlsson wrote: > Hi Zhengyu, > > On 2020-02-17 15:51, Zhengyu Gu wrote: >> Hi Stefan, >> >> Thanks for the review and suggestions, updated accordingly: >> >> http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/ > > Thanks for moving the code. I think this looks good. > > If you're up for it, I have a couple of style change suggestions: > > 1) ObjectMarker uses two verbs to describe the same thing: "mark" and > "visit". I propose that we only use "mark" in ObjectMarker and leave > the usage of "visited" to the Jvmti code. > > 2) Some updates to odd whitespaces > > 3) Using forward declarations in Shenandoah code. > > I've bundled those changes into webrevs: > > https://cr.openjdk.java.net/~stefank/8238633/webrev.01.delta > https://cr.openjdk.java.net/~stefank/8238633/webrev.01 > > Regarding performance testing, the HeapWalkTests you used seems to use > a very small heap. I think it would be good to redo the measurements > on a larger heap. Could you take the HeapWalkTest and add a few GBs of > small, linked objects? > > Thank, > StefanK >> >>> >>> --- >>> Previously, the calls to 'mark' and 'visited' were inlineable, but >>> now every GC has to take a virtual call when marking the objects. My >>> guess is that this code is slow anyway, and that it doesn't matter >>> too much, but did you measure the effect of that change with, for >>> example, G1? >>> >> I did rough measurement, timing >> vmTestbase/nsk/jvmti/unit/heap/HeapWalkTests/TestDescription.java test. >> >> If you know any tests/benchmarks I should measure, please let me know. >> >> Thanks, >> >> -Zhengyu >> >> >>> Thanks, >>> StefanK >>> >>>> Test: >>>> ?? hotspot_gc >>>> ?? vmTestbase_nsk_jdi >>>> ?? vmTestbase_nsk_jvmti >>>> >>>> Thanks, >>>> >>>> -Zhengyu >>>> >>>> >>> >> > From coleen.phillimore at oracle.com Fri Feb 21 13:01:07 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 21 Feb 2020 08:01:07 -0500 Subject: [15] RFR 8238633: JVMTI heap walk should consult GC for marking oops In-Reply-To: References: <8ea20a15-bdb9-27b0-c306-154f539a3674@oracle.com> <3a5d60fd-04d5-da96-3d79-242d43fdec79@redhat.com> <901d7307-cf36-0367-e09c-ff47c76bbc25@oracle.com> Message-ID: Adding serviceability-dev back. Coleen On 2/21/20 7:59 AM, coleen.phillimore at oracle.com wrote: > > Hi, I had a quick look at this, minus the shenandoah code. > > http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/src/hotspot/share/gc/shared/objectMarker.hpp.html > > > I think this file could have forward declarations of GrowableArray and > I didn't see a need for the markWord.hpp include. > > This change on the whole looks good to me. > > Coleen > > On 2/21/20 5:23 AM, Stefan Karlsson wrote: >> Hi Zhengyu, >> >> On 2020-02-17 15:51, Zhengyu Gu wrote: >>> Hi Stefan, >>> >>> Thanks for the review and suggestions, updated accordingly: >>> >>> http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/ >> >> Thanks for moving the code. I think this looks good. >> >> If you're up for it, I have a couple of style change suggestions: >> >> 1) ObjectMarker uses two verbs to describe the same thing: "mark" and >> "visit". I propose that we only use "mark" in ObjectMarker and leave >> the usage of "visited" to the Jvmti code. >> >> 2) Some updates to odd whitespaces >> >> 3) Using forward declarations in Shenandoah code. >> >> I've bundled those changes into webrevs: >> >> https://cr.openjdk.java.net/~stefank/8238633/webrev.01.delta >> https://cr.openjdk.java.net/~stefank/8238633/webrev.01 >> >> Regarding performance testing, the HeapWalkTests you used seems to >> use a very small heap. I think it would be good to redo the >> measurements on a larger heap. Could you take the HeapWalkTest and >> add a few GBs of small, linked objects? >> >> Thank, >> StefanK >>> >>>> >>>> --- >>>> Previously, the calls to 'mark' and 'visited' were inlineable, but >>>> now every GC has to take a virtual call when marking the objects. >>>> My guess is that this code is slow anyway, and that it doesn't >>>> matter too much, but did you measure the effect of that change >>>> with, for example, G1? >>>> >>> I did rough measurement, timing >>> vmTestbase/nsk/jvmti/unit/heap/HeapWalkTests/TestDescription.java test. >>> >>> If you know any tests/benchmarks I should measure, please let me know. >>> >>> Thanks, >>> >>> -Zhengyu >>> >>> >>>> Thanks, >>>> StefanK >>>> >>>>> Test: >>>>> ?? hotspot_gc >>>>> ?? vmTestbase_nsk_jdi >>>>> ?? vmTestbase_nsk_jvmti >>>>> >>>>> Thanks, >>>>> >>>>> -Zhengyu >>>>> >>>>> >>>> >>> >> > From magnus.ihse.bursie at oracle.com Fri Feb 21 16:13:50 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 21 Feb 2020 17:13:50 +0100 Subject: RFR: JDK-8239794 Move -Os from JVM feature 'minimal' to new feature 'opt-size' Message-ID: <39af23bf-d7c4-14c0-233c-e31e1e511369@oracle.com> Prior to JDK-8239450, it was possible to explicitly set the JVM feature 'minimal', even when building variant server. The reason for doing this was to get the code in JvmFeatures.gmk that sets JVM_OPTIMIZATION := SIZE (except for a handful of files). Since this functionality is important in itself (and it has been requested by users in the community), it should be controlled by a separate feature, instead of just as a part of the 'minimal' feature, which signals that the minimal JVM variant is being built. However, when looking into it, I noticed several aspects of the old code that I wanted to address. First of all, the old code had the size optimization silently ignored, if also 'link-time-opt' was enabled. This does not make any sense, since it is completely possible to do both. This code was written in the new Hotspot build system as a way to mimick the old Hotspot build system. I'm guessing that this was initially introduced as a way to enable the size optimization when building minimal, except for arm-32 systems, which were the only system to have link-time-opt enabled. After some head scratching, I implemented the logic in the following way: ?* If opt-size is enabled, the JVM_OPTIMIZATION := SIZE is always performed, regardless of any other features. ?* And contrary, link-time-opt can be enabled/disabled independent on opt-size ?* When building minimal on arm-32, opt-size is not enabled by default, but link-time-opt is (as it was before) ?* When building minimal on any other platform, opt-size is enabled by default, but link-time-opt is not (as it was before) * When building any other variant, opt-size is disabled by default This makes standard builds of minimal with no extra features enabled/disabled behave the same as before, while giving the maximum of flexibility to change this. But I'd like to get input from someone in the Hotspot team that this is a sound approach. Also, this has apparently never been tested by any other compiler than gcc and/or clang, since we explicitly add the compiler flag -fno-optimize-sibling-calls. I wrapped this in a compiler check. Note that this patch is based on JDK-8239789 (Follow-up on JVM feature rewrite). Bug: https://bugs.openjdk.java.net/browse/JDK-8239794 WebRev: http://cr.openjdk.java.net/~ihse/JDK-8239794-create-opt-size-jvm-feature/webrev.01 /Magnus From erik.joelsson at oracle.com Fri Feb 21 16:29:56 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 21 Feb 2020 08:29:56 -0800 Subject: RFR: JDK-8239794 Move -Os from JVM feature 'minimal' to new feature 'opt-size' In-Reply-To: <39af23bf-d7c4-14c0-233c-e31e1e511369@oracle.com> References: <39af23bf-d7c4-14c0-233c-e31e1e511369@oracle.com> Message-ID: <1d49e631-262a-95a6-44b8-047498107b4a@oracle.com> Looks good to me. /Erik On 2020-02-21 08:13, Magnus Ihse Bursie wrote: > Prior to JDK-8239450, it was possible to explicitly set the JVM > feature 'minimal', even when building variant server. The reason for > doing this was to get the code in JvmFeatures.gmk that sets > JVM_OPTIMIZATION := SIZE (except for a handful of files). > > Since this functionality is important in itself (and it has been > requested by users in the community), it should be controlled by a > separate feature, instead of just as a part of the 'minimal' feature, > which signals that the minimal JVM variant is being built. > > However, when looking into it, I noticed several aspects of the old > code that I wanted to address. > > First of all, the old code had the size optimization silently ignored, > if also 'link-time-opt' was enabled. This does not make any sense, > since it is completely possible to do both. This code was written in > the new Hotspot build system as a way to mimick the old Hotspot build > system. I'm guessing that this was initially introduced as a way to > enable the size optimization when building minimal, except for arm-32 > systems, which were the only system to have link-time-opt enabled. > > After some head scratching, I implemented the logic in the following way: > ?* If opt-size is enabled, the JVM_OPTIMIZATION := SIZE is always > performed, regardless of any other features. > ?* And contrary, link-time-opt can be enabled/disabled independent on > opt-size > ?* When building minimal on arm-32, opt-size is not enabled by > default, but link-time-opt is (as it was before) > ?* When building minimal on any other platform, opt-size is enabled by > default, but link-time-opt is not (as it was before) > * When building any other variant, opt-size is disabled by default > > This makes standard builds of minimal with no extra features > enabled/disabled behave the same as before, while giving the maximum > of flexibility to change this. But I'd like to get input from someone > in the Hotspot team that this is a sound approach. > > Also, this has apparently never been tested by any other compiler than > gcc and/or clang, since we explicitly add the compiler flag > -fno-optimize-sibling-calls. I wrapped this in a compiler check. > > Note that this patch is based on JDK-8239789 (Follow-up on JVM feature > rewrite). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8239794 > WebRev: > http://cr.openjdk.java.net/~ihse/JDK-8239794-create-opt-size-jvm-feature/webrev.01 > > /Magnus From magnus.ihse.bursie at oracle.com Fri Feb 21 16:35:46 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 21 Feb 2020 17:35:46 +0100 Subject: RFR: JDK-8239789 Follow-up on JVM feature rewrite In-Reply-To: References: <0f3e2061-5c93-0c7d-fbc7-bdffd268a8cf@oracle.com> Message-ID: On 2020-02-21 17:26, Erik Joelsson wrote: > Baring any component specific opinions on the suggested descriptions > of each feature, the patch looks good to me. Ok, I'm sending a copy of this to hotspot-dev as well for the HS guys to bikeshed about the descriptions. I'll get back in a few days when the dust has settled... :-) /Magnus > > /Erik > > On 2020-02-21 07:05, Magnus Ihse Bursie wrote: >> The JVM feature rewrite was actually somewhat of a step backwards in >> terms of presenting information for the user what different configure >> arguments do. >> >> I've compensated for this by making it far better! :) Now we get a >> list like this: >> >> ? aot???????????????????? enable ahead of time compilation (AOT) >> ? cds???????????????????? enable class data sharing (CDS) >> ? compiler1?????????????? enable hotspot compiler C1 >> ? compiler2?????????????? enable hotspot compiler C2 >> ? dtrace????????????????? enable dtrace support >> ? epsilongc?????????????? include the epsilon (no-op) garbage collector >> ? g1gc??????????????????? include the G1 garbage collector >> ? graal?????????????????? enable Graal (jdk.internal.vm.compiler) >> ? jfr???????????????????? enable Java Flight Recorder (JFR) >> ? jni-check?????????????? enable -Xcheck:jni support >> ? jvmci?????????????????? enable JVM Compiler Interface (JVMCI) >> ? jvmti?????????????????? enable Java Virtual Machine Tool Interface >> (JVM TI) >> ? link-time-opt?????????? enable link time optimization >> ? management????????????? enable java.lang.management API support >> ? minimal???????????????? support building variant 'minimal' >> ? nmt???????????????????? include native memory tracking (NMT) >> ? parallelgc????????????? include the parallel garbage collector >> ? serialgc??????????????? include the serial garbage collector >> ? services??????????????? enable diagnostic services and client >> attaching >> ? shenandoahgc??????????? include the Shenandoah garbage collector >> ? static-build??????????? build static library instead of dynamic >> ? vm-structs????????????? export JVM structures to the >> Serviceablility Agent >> ? zero??????????????????? support building variant 'zero' >> ? zgc???????????????????? include the ZGC garbage collector >> >> Also, I missed using the recommended m4_ prefix on some m4 functions. >> This patch also corrects this. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8239789 >> WebRev: >> http://cr.openjdk.java.net/~ihse/JDK-8239789-jvm-features-followup/webrev.01 >> >> /Magnus From richard.reingruber at sap.com Fri Feb 21 17:32:04 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 21 Feb 2020 17:32:04 +0000 Subject: RFR (S) 8239492: [x86] Turn MacroAssembler::verify_oop into macro recording file and line In-Reply-To: <8bedc6e0-3b2b-a3c1-8a91-1ebce3d6641b@redhat.com> References: <8bedc6e0-3b2b-a3c1-8a91-1ebce3d6641b@redhat.com> Message-ID: Hi Aleksey, > Suggested fix: > https://cr.openjdk.java.net/~shade/8239492/webrev.02/ The RFE is worth the (small) effort and the proposed change looks good to me. I was a little confused as now the verify_oop() calls in TemplateTable get dispatched directely to MacroAssembler, but that's what InterpreterMacroAssembler::verify_oop() used to do there anyway, because all the call sites got the default argument state = atos. I assume you deliberately did the renaming in zVerify.cpp instead of undefining verify_oop as in shenandoahVerifier.cpp. That's good for me too. You should update the years in the copyright headers. Thanks, Richard. // not a Reviewer -----Original Message----- From: hotspot-dev On Behalf Of Aleksey Shipilev Sent: Freitag, 21. Februar 2020 11:40 To: hotspot-dev at openjdk.java.net Subject: RFR (S) 8239492: [x86] Turn MacroAssembler::verify_oop into macro recording file and line RFE: https://bugs.openjdk.java.net/browse/JDK-8239492 This is done in SPARC and ARM MacroAssemblers, and should be done in x86 too. This should greatly improve the error reports, as it would point to exact place where the verification happened, e.g. in interpreter/stub generation code. Suggested fix: https://cr.openjdk.java.net/~shade/8239492/webrev.02/ Testing: tier1 {x86_32, x86_64} with -XX:+VerifyOops, jdk-submit (clean, except unrelated Solaris gtest failure) -- Thanks, -Aleksey From shade at redhat.com Fri Feb 21 17:56:51 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 21 Feb 2020 18:56:51 +0100 Subject: RFR (S) 8239492: [x86] Turn MacroAssembler::verify_oop into macro recording file and line In-Reply-To: References: <8bedc6e0-3b2b-a3c1-8a91-1ebce3d6641b@redhat.com> Message-ID: <01ef7782-46fe-6082-1026-8673b3865b06@redhat.com> On 2/21/20 6:32 PM, Reingruber, Richard wrote: >> Suggested fix: >> https://cr.openjdk.java.net/~shade/8239492/webrev.02/ > > The RFE is worth the (small) effort and the proposed change looks good to me. Thanks. >> I was a little confused as now the verify_oop() calls in TemplateTable get dispatched directely to > MacroAssembler, but that's what InterpreterMacroAssembler::verify_oop() used to do there anyway, > because all the call sites got the default argument state = atos. Yes. Sometimes interp_verify_oop is called with the actual "state", but otherwise it is called with atos. We could, in principle, introduce two separate macro definitions to simulate that default argument, but I'd prefer not to go overly crazy with macros. > I assume you deliberately did the renaming in zVerify.cpp instead of undefining verify_oop as in > shenandoahVerifier.cpp. That's good for me too. Ah, hm. Actually, maybe we should indeed do it Shenandoah-way by undef-ing the macro in the problematic compilation unit. > You should update the years in the copyright headers. Updated. New webrev: https://cr.openjdk.java.net/~shade/8239492/webrev.03/ -- Thanks, -Aleksey From vladimir.x.ivanov at oracle.com Fri Feb 21 18:08:36 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 21 Feb 2020 21:08:36 +0300 Subject: RFR (S) 8239492: [x86] Turn MacroAssembler::verify_oop into macro recording file and line In-Reply-To: <01ef7782-46fe-6082-1026-8673b3865b06@redhat.com> References: <8bedc6e0-3b2b-a3c1-8a91-1ebce3d6641b@redhat.com> <01ef7782-46fe-6082-1026-8673b3865b06@redhat.com> Message-ID: <05c72140-6fb8-4361-105b-ebe1325b6347@oracle.com> > https://cr.openjdk.java.net/~shade/8239492/webrev.03/ Looks good. Best regards, Vladimir Ivanov PS: I don't have a strong opinion about #undefing vs renaming. Probably, slight preference in favor of renaming: I like how it stresses that it's a GC-specific verification. But I'll let ZGC folks to choose what they like more :-) From jianglizhou at google.com Fri Feb 21 18:26:12 2020 From: jianglizhou at google.com (Jiangli Zhou) Date: Fri, 21 Feb 2020 10:26:12 -0800 Subject: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS In-Reply-To: References: Message-ID: Hi Richard, The latest test update looks fine to me. Thank you for digging deeper into this. Best regards, Jiangli On Fri, Feb 21, 2020 at 3:25 AM Reingruber, Richard wrote: > > Hi Jiangli, > > > > Actually I would prefer not to skip the test then. I'd like to verify if large data structures can > > > be placed in TLS without stack size adjustment or if there is some kind of regression. That's the > > > point of regression tests after all. > > > > > > That's a very valid point. I thought about that as well yesterday. My > > thinking was that the VM and test would need to be updated to remove > > the workaround for the TLS issue when no longer needed. Do you know if > > the proposed check using gnu_get_libc_version() covers all cases? I > > Other C libraries (e.g. MUSL) are not really covered, but I'm not sure if you mean this by covered... > > > came across the following archive that mentioned missing symbol in > > eglibc-2.13 (eglibc however is already deprecated). > > > https://www.mail-archive.com/lfs-dev at linuxfromscratch.org/msg18115.html > > I only knew, that 2.11 does not provide __pthread_get_minstack, but then I used > > https://github.com/bminor/glibc > > to find out that __pthread_get_minstack was added with 2.14.90 [1]. So 2.15 should be the first > stable release to include __pthread_get_minstack. I'll correct this in my fix. > > Would you be ok with this version of the fix? > > Thanks and best regards, > Richard. > > [1] version.h at [2] > https://github.com/bminor/glibc/blob/2c1094bd700e63a8d7f547b3f5495bedb55c0a08/version.h > > [2] Repo at [3] > https://github.com/bminor/glibc/tree/2c1094bd700e63a8d7f547b3f5495bedb55c0a08 > > [3] Commit that introduced __pthread_get_minstack > https://github.com/bminor/glibc/commit/2c1094bd700e63a8d7f547b3f5495bedb55c0a08 > > > -----Original Message----- > From: Jiangli Zhou > Sent: Donnerstag, 20. Februar 2020 17:32 > To: Reingruber, Richard > Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS > > Hi Richard, > > On Thu, Feb 20, 2020 at 1:03 AM Reingruber, Richard > wrote: > > > > Hi Jiangli, > > > > thanks for providing feedback. > > > > > David's suggestion of using dlsym seems to be more reliable. Regarding the stability of > > > __pthread_get_minstack in future glibc versions, Florian has added comment to > > > __pthread_get_minstack about external users to ensure it's not removed until TCB is > > > separately allocated. Please see: http://51.15.138.76/patch/18989/ and > > > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. > > > > So if I understand you and David correctly, then the test should be skipped if the lookup of > > __pthread_get_minstack fails. > > > > Then the test would be skipped on future glibc versions where __pthread_get_minstack was removed > > because TLS and TCB are not placed on a threads stack anymore. > > > > Actually I would prefer not to skip the test then. I'd like to verify if large data structures can > > be placed in TLS without stack size adjustment or if there is some kind of regression. That's the > > point of regression tests after all. > > > > That's a very valid point. I thought about that as well yesterday. My > thinking was that the VM and test would need to be updated to remove > the workaround for the TLS issue when no longer needed. Do you know if > the proposed check using gnu_get_libc_version() covers all cases? I > came across the following archive that mentioned missing symbol in > eglibc-2.13 (eglibc however is already deprecated). > > https://www.mail-archive.com/lfs-dev at linuxfromscratch.org/msg18115.html > > > In other words: I wouldn't change test coverage of future glibc versions. I just want to exclude > > glibc versions were the tested feature cannot work. > > Best regards, > Jiangli > > > > Thanks, Richard. > > > > -----Original Message----- > > From: Jiangli Zhou > > Sent: Mittwoch, 19. Februar 2020 19:24 > > To: Reingruber, Richard > > Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > > Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS > > > > Hi Richard, > > > > David's suggestion of using dlsym seems to be more reliable. Regarding > > the stability of __pthread_get_minstack in future glibc versions, > > Florian has added comment to __pthread_get_minstack about external > > users to ensure it's not removed until TCB is separately allocated. > > Please see: http://51.15.138.76/patch/18989/ and > > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. > > > > Best regards, > > Jiangli > > > > > > > > > > On Wed, Feb 19, 2020 at 2:44 AM Reingruber, Richard > > wrote: > > > > > > Hi, > > > > > > please review this small test fix > > > > > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239449/webrev.0/ > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8239449 > > > > > > On linux systems with older glibc versions (e.g. 2.11) the option -XX:+AdjustStackSizeForTLS has no effect, because the private function __pthread_get_minstack is not provided, and the test fails. With the fix the test is skipped instead. > > > > > > Thanks, Richard. From alexey.menkov at oracle.com Fri Feb 21 18:51:15 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Fri, 21 Feb 2020 10:51:15 -0800 Subject: RFR [XS]: 8239462: jdk.hotspot.agent misses some ReleaseStringUTFChars calls in case of early returns In-Reply-To: References: <6ed0d004-3e8c-4b27-f583-06dbacf45173@oracle.com> Message-ID: <425e259a-e905-c80e-8db9-fa39458aaf6b@oracle.com> On 02/21/2020 00:09, Baesken, Matthias wrote: >> Also I see GetStringUTFChars(str, JNI_FALSE). This look bad as well - >> 2nd arg is a pointer, so it should be NULL or nullptr. > > Hi looks like there is another one here, do you think these JNI_FALSE params would really cause trouble ? Not even the compiler warns here ... > > src/java.desktop/unix/native/libawt_xawt/xawt/XlibWrapper.c:824: cname = (char *) (*env)->GetStringUTFChars(env, jstr, JNI_FALSE); This doesn't cause troubles just because both NULL & JNI_FALSE are defines (and are 0). but specify JNI_FALSE as pointer value is confusing. --alex > > Best regards, Matthias > > > >> Hi Matthias, >> >> Looks good in general, but I think it makes sense to fix #2 cases (at >> least I see them in LinuxDebuggerLocal). If GetStringUTFChars fails, the >> code will crash. >> Also I see GetStringUTFChars(str, JNI_FALSE). This look bad as well - >> 2nd arg is a pointer, so it should be NULL or nullptr. >> >> As for #1 and #3 - AFAIU they are both right ways. >> If GetStringUTFChars fails, it throws OOM and return NULL. >> >> And one more thing to consider. >> LinuxDebuggerLocal_attach0 function looks terrible - 7 >> ReleaseStringUTFChars calls for 2 GetStringUTFChars. >> Maybe it make sense to introduce simple wrapper like AutoJavaString in >> src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp >> It would make the code simpler and less error prone. >> >> --alex >> >> On 02/20/2020 00:53, Baesken, Matthias wrote: >>> Hi Alex / Christoph, thanks for the reviews. >>> >>> New webrev : >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.1/ >>> >>> - includes LinuxDebuggerLocal.cpp >>> - adds a blank Christoph wanted to have >>> >>> >>> A question (hopefully not a stupid one ?? ): >>> At most places in the coding, GetStringUTFChars success is >>> 1. handled by checking NULL , like this : >>> >>> const char *s = (*env)->GetStringUTFChars(env, p, NULL); >>> if (s == NULL) { >>> // handle failure >>> } >>> >>> 2.At some places , success / failure is not handled at all . >>> >>> 3.Here (e.g. LinuxDebuggerLocal.cpp) success / failure check is done by >>> >>> if (env->ExceptionOccurred()) { ... } >>> >>> Which one is the best / right way to do it (most likely not 2.) ? >>> >>> >>> Best regards, Matthias >>> >>> >>> >>>> >>>> Looks like >>>> src/jdk.hotspot.agent/linux/native/libsaproc/LinuxDebuggerLocal.cpp >>>> has similar issues. It would be nice to fix them as well. >>>> >>>> --alex >>>> >>>> On 02/19/2020 06:21, Baesken, Matthias wrote: >>>>> Hello, please review this small change . >>>>> We miss at a few places ReleaseStringUTFChars calls in the native >>>>> jdk.hotspot.agent coding. >>>>> This happens in case of early returns . >>>>> >>>>> >>>>> Bug/webrev : >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8239462 >>>>> >>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.0/ >>>>> >>>>> >>>>> Thanks, Matthias >>>>> From kim.barrett at oracle.com Fri Feb 21 20:47:53 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 21 Feb 2020 15:47:53 -0500 Subject: [15] RFR 8238633: JVMTI heap walk should consult GC for marking oops In-Reply-To: References: Message-ID: <1149B0D9-CA32-4491-A2D9-8459EB90D8AB@oracle.com> > On Feb 7, 2020, at 10:53 AM, Zhengyu Gu wrote: > > Hi, > > I would like purpose this change that allows GC to provide ObjectMarker during JVMTI heap walk. > > Currently, JVMTI heap walk uses oop markword's 'marked' pattern to indicate 'visited' oop. > > Unfortunately, it conflicts with Shenandoah, who uses the pattern to indicate 'forwarding'. When JVMTI heap walk occurs in some of Shenandoah's concurrent heap (e.g. concurrent evacuation or concurrent reference updating phases), it can result corrupted heap, as it tries to resolve a real oop header as a forwarding pointer. > > This patch allows GC to provide ObjectMarker for JVMTI to track 'visited' oop, and uses current implementation as default, so that, it has no impact to GCs other than Shenandoah, who provides its own implementation. (Not a review.) I think the jfr leak profiler has the same problem. It too uses the markWord?s ?marked? pattern to indicate an oop it has visited, since JDK-8234173. From zgu at redhat.com Fri Feb 21 21:31:39 2020 From: zgu at redhat.com (Zhengyu Gu) Date: Fri, 21 Feb 2020 16:31:39 -0500 Subject: [15] RFR 8238633: JVMTI heap walk should consult GC for marking oops In-Reply-To: <1149B0D9-CA32-4491-A2D9-8459EB90D8AB@oracle.com> References: <1149B0D9-CA32-4491-A2D9-8459EB90D8AB@oracle.com> Message-ID: Hi Kim, On 2/21/20 3:47 PM, Kim Barrett wrote: >> On Feb 7, 2020, at 10:53 AM, Zhengyu Gu wrote: >> >> Hi, >> >> I would like purpose this change that allows GC to provide ObjectMarker during JVMTI heap walk. >> >> Currently, JVMTI heap walk uses oop markword's 'marked' pattern to indicate 'visited' oop. >> >> Unfortunately, it conflicts with Shenandoah, who uses the pattern to indicate 'forwarding'. When JVMTI heap walk occurs in some of Shenandoah's concurrent heap (e.g. concurrent evacuation or concurrent reference updating phases), it can result corrupted heap, as it tries to resolve a real oop header as a forwarding pointer. >> >> This patch allows GC to provide ObjectMarker for JVMTI to track 'visited' oop, and uses current implementation as default, so that, it has no impact to GCs other than Shenandoah, who provides its own implementation. > > (Not a review.) > > I think the jfr leak profiler has the same problem. It too uses the markWord?s ?marked? pattern > to indicate an oop it has visited, since JDK-8234173. Thanks for pointing it out, I will deal with it next, sigh! -Zhengyu > From alexey.menkov at oracle.com Fri Feb 21 21:47:41 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Fri, 21 Feb 2020 13:47:41 -0800 Subject: RFR [XS]: 8239462: jdk.hotspot.agent misses some ReleaseStringUTFChars calls in case of early returns In-Reply-To: References: <6ed0d004-3e8c-4b27-f583-06dbacf45173@oracle.com> Message-ID: IMO the solution with goto makes it even worse. If you don't want to introduce the wrapper, could you please restore changes in LinuxDebuggerLocal_attach0 from webrev.1 --alex On 02/21/2020 00:32, Baesken, Matthias wrote: > Hi Alex , > > new webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.2/ > > Best Regards, Matthias > > >> >> Hi Matthias, >> >> Looks good in general, but I think it makes sense to fix #2 cases (at >> least I see them in LinuxDebuggerLocal). If GetStringUTFChars fails, the >> code will crash. >> Also I see GetStringUTFChars(str, JNI_FALSE). This look bad as well - >> 2nd arg is a pointer, so it should be NULL or nullptr. >> >> As for #1 and #3 - AFAIU they are both right ways. >> If GetStringUTFChars fails, it throws OOM and return NULL. >> >> And one more thing to consider. >> LinuxDebuggerLocal_attach0 function looks terrible - 7 >> ReleaseStringUTFChars calls for 2 GetStringUTFChars. >> Maybe it make sense to introduce simple wrapper like AutoJavaString in >> src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp >> It would make the code simpler and less error prone. >> >> --alex >> > From david.holmes at oracle.com Mon Feb 24 04:37:39 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 24 Feb 2020 14:37:39 +1000 Subject: RFR: JDK-8239794 Move -Os from JVM feature 'minimal' to new feature 'opt-size' In-Reply-To: <39af23bf-d7c4-14c0-233c-e31e1e511369@oracle.com> References: <39af23bf-d7c4-14c0-233c-e31e1e511369@oracle.com> Message-ID: <8474f0d9-9658-9ce5-4922-74f92ef1d22e@oracle.com> Hi Magnus, This seems reasonable to me. The definition of OPT_SPEED_SRC is likely out of date, but we have no data for modifying that list. People who try opt-size will likely try to refine it. Thanks, David ----- On 22/02/2020 2:13 am, Magnus Ihse Bursie wrote: > Prior to JDK-8239450, it was possible to explicitly set the JVM feature > 'minimal', even when building variant server. The reason for doing this > was to get the code in JvmFeatures.gmk that sets JVM_OPTIMIZATION := > SIZE (except for a handful of files). > > Since this functionality is important in itself (and it has been > requested by users in the community), it should be controlled by a > separate feature, instead of just as a part of the 'minimal' feature, > which signals that the minimal JVM variant is being built. > > However, when looking into it, I noticed several aspects of the old code > that I wanted to address. > > First of all, the old code had the size optimization silently ignored, > if also 'link-time-opt' was enabled. This does not make any sense, since > it is completely possible to do both. This code was written in the new > Hotspot build system as a way to mimick the old Hotspot build system. > I'm guessing that this was initially introduced as a way to enable the > size optimization when building minimal, except for arm-32 systems, > which were the only system to have link-time-opt enabled. > > After some head scratching, I implemented the logic in the following way: > ?* If opt-size is enabled, the JVM_OPTIMIZATION := SIZE is always > performed, regardless of any other features. > ?* And contrary, link-time-opt can be enabled/disabled independent on > opt-size > ?* When building minimal on arm-32, opt-size is not enabled by default, > but link-time-opt is (as it was before) > ?* When building minimal on any other platform, opt-size is enabled by > default, but link-time-opt is not (as it was before) > * When building any other variant, opt-size is disabled by default > > This makes standard builds of minimal with no extra features > enabled/disabled behave the same as before, while giving the maximum of > flexibility to change this. But I'd like to get input from someone in > the Hotspot team that this is a sound approach. > > Also, this has apparently never been tested by any other compiler than > gcc and/or clang, since we explicitly add the compiler flag > -fno-optimize-sibling-calls. I wrapped this in a compiler check. > > Note that this patch is based on JDK-8239789 (Follow-up on JVM feature > rewrite). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8239794 > WebRev: > http://cr.openjdk.java.net/~ihse/JDK-8239794-create-opt-size-jvm-feature/webrev.01 > > > /Magnus From richard.reingruber at sap.com Mon Feb 24 09:17:06 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Mon, 24 Feb 2020 09:17:06 +0000 Subject: RFR (S) 8239492: [x86] Turn MacroAssembler::verify_oop into macro recording file and line In-Reply-To: <01ef7782-46fe-6082-1026-8673b3865b06@redhat.com> References: <8bedc6e0-3b2b-a3c1-8a91-1ebce3d6641b@redhat.com> <01ef7782-46fe-6082-1026-8673b3865b06@redhat.com> Message-ID: > >> I was a little confused as now the verify_oop() calls in TemplateTable get dispatched directely to > > MacroAssembler, but that's what InterpreterMacroAssembler::verify_oop() used to do there anyway, > > because all the call sites got the default argument state = atos. > Yes. Sometimes interp_verify_oop is called with the actual "state", but otherwise it is called with > atos. We could, in principle, introduce two separate macro definitions to simulate that default > argument, but I'd prefer not to go overly crazy with macros. Sure. > > I assume you deliberately did the renaming in zVerify.cpp instead of undefining verify_oop as in > > shenandoahVerifier.cpp. That's good for me too. > Ah, hm. Actually, maybe we should indeed do it Shenandoah-way by undef-ing the macro in the > problematic compilation unit. I'll leave it up to you. Thanks, Richard. -----Original Message----- From: Aleksey Shipilev Sent: Freitag, 21. Februar 2020 18:57 To: Reingruber, Richard ; hotspot-dev at openjdk.java.net Subject: Re: RFR (S) 8239492: [x86] Turn MacroAssembler::verify_oop into macro recording file and line On 2/21/20 6:32 PM, Reingruber, Richard wrote: >> Suggested fix: >> https://cr.openjdk.java.net/~shade/8239492/webrev.02/ > > The RFE is worth the (small) effort and the proposed change looks good to me. Thanks. >> I was a little confused as now the verify_oop() calls in TemplateTable get dispatched directely to > MacroAssembler, but that's what InterpreterMacroAssembler::verify_oop() used to do there anyway, > because all the call sites got the default argument state = atos. Yes. Sometimes interp_verify_oop is called with the actual "state", but otherwise it is called with atos. We could, in principle, introduce two separate macro definitions to simulate that default argument, but I'd prefer not to go overly crazy with macros. > I assume you deliberately did the renaming in zVerify.cpp instead of undefining verify_oop as in > shenandoahVerifier.cpp. That's good for me too. Ah, hm. Actually, maybe we should indeed do it Shenandoah-way by undef-ing the macro in the problematic compilation unit. > You should update the years in the copyright headers. Updated. New webrev: https://cr.openjdk.java.net/~shade/8239492/webrev.03/ -- Thanks, -Aleksey From per.liden at oracle.com Mon Feb 24 10:14:24 2020 From: per.liden at oracle.com (Per Liden) Date: Mon, 24 Feb 2020 11:14:24 +0100 Subject: RFR (S) 8239492: [x86] Turn MacroAssembler::verify_oop into macro recording file and line In-Reply-To: References: <8bedc6e0-3b2b-a3c1-8a91-1ebce3d6641b@redhat.com> <01ef7782-46fe-6082-1026-8673b3865b06@redhat.com> Message-ID: On 2/24/20 10:17 AM, Reingruber, Richard wrote: > >>>> I was a little confused as now the verify_oop() calls in TemplateTable get dispatched directely to >>> MacroAssembler, but that's what InterpreterMacroAssembler::verify_oop() used to do there anyway, >>> because all the call sites got the default argument state = atos. > >> Yes. Sometimes interp_verify_oop is called with the actual "state", but otherwise it is called with >> atos. We could, in principle, introduce two separate macro definitions to simulate that default >> argument, but I'd prefer not to go overly crazy with macros. > > Sure. > >>> I assume you deliberately did the renaming in zVerify.cpp instead of undefining verify_oop as in >>> shenandoahVerifier.cpp. That's good for me too. I would prefer having the two static functions being renamed to z_verify_oops and z_verify_possibly_weak_oop, like you did in webrev.02. thanks, Per > >> Ah, hm. Actually, maybe we should indeed do it Shenandoah-way by undef-ing the macro in the >> problematic compilation unit. > > I'll leave it up to you. > > Thanks, > Richard. > > -----Original Message----- > From: Aleksey Shipilev > Sent: Freitag, 21. Februar 2020 18:57 > To: Reingruber, Richard ; hotspot-dev at openjdk.java.net > Subject: Re: RFR (S) 8239492: [x86] Turn MacroAssembler::verify_oop into macro recording file and line > > On 2/21/20 6:32 PM, Reingruber, Richard wrote: >>> Suggested fix: >>> https://cr.openjdk.java.net/~shade/8239492/webrev.02/ >> >> The RFE is worth the (small) effort and the proposed change looks good to me. > > Thanks. > >>> I was a little confused as now the verify_oop() calls in TemplateTable get dispatched directely to >> MacroAssembler, but that's what InterpreterMacroAssembler::verify_oop() used to do there anyway, >> because all the call sites got the default argument state = atos. > > Yes. Sometimes interp_verify_oop is called with the actual "state", but otherwise it is called with > atos. We could, in principle, introduce two separate macro definitions to simulate that default > argument, but I'd prefer not to go overly crazy with macros. > >> I assume you deliberately did the renaming in zVerify.cpp instead of undefining verify_oop as in >> shenandoahVerifier.cpp. That's good for me too. > > Ah, hm. Actually, maybe we should indeed do it Shenandoah-way by undef-ing the macro in the > problematic compilation unit. > >> You should update the years in the copyright headers. > > Updated. > > New webrev: > https://cr.openjdk.java.net/~shade/8239492/webrev.03/ > From per.liden at oracle.com Mon Feb 24 10:23:15 2020 From: per.liden at oracle.com (Per Liden) Date: Mon, 24 Feb 2020 11:23:15 +0100 Subject: RFR: JDK-8239789 Follow-up on JVM feature rewrite In-Reply-To: References: <0f3e2061-5c93-0c7d-fbc7-bdffd268a8cf@oracle.com> Message-ID: <0f82e345-2c12-05c3-6c37-6a6701a07d67@oracle.com> Hi Magnus, On 2/21/20 5:35 PM, Magnus Ihse Bursie wrote: > On 2020-02-21 17:26, Erik Joelsson wrote: >> Baring any component specific opinions on the suggested descriptions >> of each feature, the patch looks good to me. > Ok, I'm sending a copy of this to hotspot-dev as well for the HS guys to > bikeshed about the descriptions. I'll get back in a few days when the > dust has settled... :-) > [...] >>> ? zgc???????????????????? include the ZGC garbage collector To avoid a tautology, this should be "include the Z garbage collector" :) cheers, Per From shade at redhat.com Mon Feb 24 10:23:17 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 24 Feb 2020 11:23:17 +0100 Subject: RFR (S) 8239492: [x86] Turn MacroAssembler::verify_oop into macro recording file and line In-Reply-To: References: <8bedc6e0-3b2b-a3c1-8a91-1ebce3d6641b@redhat.com> <01ef7782-46fe-6082-1026-8673b3865b06@redhat.com> Message-ID: <96c65ffb-c1fb-2872-94b6-58bca5a07e82@redhat.com> On 2/24/20 11:14 AM, Per Liden wrote: > I would prefer having the two static functions being renamed to > z_verify_oops and z_verify_possibly_weak_oop, like you did in webrev.02. Okay, like this then: https://cr.openjdk.java.net/~shade/8239492/webrev.04/ -- Thanks, -Aleksey From magnus.ihse.bursie at oracle.com Mon Feb 24 10:30:06 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 24 Feb 2020 11:30:06 +0100 Subject: RFR: JDK-8239789 Follow-up on JVM feature rewrite In-Reply-To: <0f82e345-2c12-05c3-6c37-6a6701a07d67@oracle.com> References: <0f3e2061-5c93-0c7d-fbc7-bdffd268a8cf@oracle.com> <0f82e345-2c12-05c3-6c37-6a6701a07d67@oracle.com> Message-ID: <965a49f2-1970-b27f-e956-6705d746279c@oracle.com> On 2020-02-24 11:23, Per Liden wrote: > Hi Magnus, > > On 2/21/20 5:35 PM, Magnus Ihse Bursie wrote: >> On 2020-02-21 17:26, Erik Joelsson wrote: >>> Baring any component specific opinions on the suggested descriptions >>> of each feature, the patch looks good to me. >> Ok, I'm sending a copy of this to hotspot-dev as well for the HS guys >> to bikeshed about the descriptions. I'll get back in a few days when >> the dust has settled... :-) >> > [...] >>>> ? zgc???????????????????? include the ZGC garbage collector > > To avoid a tautology, this should be "include the Z garbage collector" :) Ok! I'll update before pushing. /Magnus > > cheers, > Per From per.liden at oracle.com Mon Feb 24 10:29:28 2020 From: per.liden at oracle.com (Per Liden) Date: Mon, 24 Feb 2020 11:29:28 +0100 Subject: RFR (S) 8239492: [x86] Turn MacroAssembler::verify_oop into macro recording file and line In-Reply-To: <96c65ffb-c1fb-2872-94b6-58bca5a07e82@redhat.com> References: <8bedc6e0-3b2b-a3c1-8a91-1ebce3d6641b@redhat.com> <01ef7782-46fe-6082-1026-8673b3865b06@redhat.com> <96c65ffb-c1fb-2872-94b6-58bca5a07e82@redhat.com> Message-ID: <5d603c9b-2969-7a4b-aaf0-78196a63ad93@oracle.com> On 2/24/20 11:23 AM, Aleksey Shipilev wrote: > On 2/24/20 11:14 AM, Per Liden wrote: >> I would prefer having the two static functions being renamed to >> z_verify_oops and z_verify_possibly_weak_oop, like you did in webrev.02. > Okay, like this then: > https://cr.openjdk.java.net/~shade/8239492/webrev.04/ > Yep, the ZGC part looks good to me. cheers, Per From shade at redhat.com Mon Feb 24 10:35:43 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 24 Feb 2020 11:35:43 +0100 Subject: RFR (S) 8239492: [x86] Turn MacroAssembler::verify_oop into macro recording file and line In-Reply-To: <5d603c9b-2969-7a4b-aaf0-78196a63ad93@oracle.com> References: <8bedc6e0-3b2b-a3c1-8a91-1ebce3d6641b@redhat.com> <01ef7782-46fe-6082-1026-8673b3865b06@redhat.com> <96c65ffb-c1fb-2872-94b6-58bca5a07e82@redhat.com> <5d603c9b-2969-7a4b-aaf0-78196a63ad93@oracle.com> Message-ID: <915f9b4f-f35f-53f2-92ec-6d74be3473a3@redhat.com> On 2/24/20 11:29 AM, Per Liden wrote: > On 2/24/20 11:23 AM, Aleksey Shipilev wrote: >> On 2/24/20 11:14 AM, Per Liden wrote: >>> I would prefer having the two static functions being renamed to >>> z_verify_oops and z_verify_possibly_weak_oop, like you did in webrev.02. >> Okay, like this then: >> https://cr.openjdk.java.net/~shade/8239492/webrev.04/ >> > > Yep, the ZGC part looks good to me. Thanks. Going to push after jdk-submit clears me again. -- -Aleksey From aoqi at loongson.cn Mon Feb 24 11:39:35 2020 From: aoqi at loongson.cn (Ao Qi) Date: Mon, 24 Feb 2020 19:39:35 +0800 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> <4ac31ff3-df28-c3c4-5b63-9c2849716d00@oracle.com> <8982aeb0-3e12-f92f-a199-eb4ee1af1af9@oracle.com> <1a389351-3b80-5dc6-5712-6b4a5e018403@oracle.com> Message-ID: On Fri, Feb 21, 2020 at 10:42 AM Ao Qi wrote: > > > We need to find all tests that use the modified WB functions and see if > > they all pass under normal and non-tiered configurations. I found it's hard to find all the tests that use the modified functions, because not all the tests use the modified functions directly. I try to find them by two ways: - I grep all modified functions' name in test dir and found all the tests contain the names. Some found files are not jtreg tests. They are used by other tests or libs. I also grep these files' name to find tests that use these files. However I missed the situation that other libs use these files. - I added some tty->print at the modified positions and run hotspot (hotspot_all_no_apps), jdk (jdk_stable) and langtools (jdk lib tools) tests. Then I found all jtr files which contain the added print. The test list found by the above two ways are here: http://cr.openjdk.java.net/~aoqi/8239423/all.txt . These tests passed with default and -XX:-TieredCompilation parameter and passed under --with-jvm-features=-compiler1 configuration. I also tested them with c1-only configuration. 2 failures were found, but regression were not found. I also tested tier1-3, both release and fastdebug, with default and -XX:-TieredCompilation parameter. Do I need do some other tests? Cheers, Ao Qi > > I agree. I will try to find these tests. > > Cheers, > Ao Qi > > > > > Thanks, > > David > > From richard.reingruber at sap.com Mon Feb 24 13:38:57 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Mon, 24 Feb 2020 13:38:57 +0000 Subject: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS In-Reply-To: References: Message-ID: Hi Jiangli, Thanks for reviewing and providing feedback. I just pushed the change. Best regards, Richard. -----Original Message----- From: Jiangli Zhou Sent: Freitag, 21. Februar 2020 19:26 To: Reingruber, Richard Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS Hi Richard, The latest test update looks fine to me. Thank you for digging deeper into this. Best regards, Jiangli On Fri, Feb 21, 2020 at 3:25 AM Reingruber, Richard wrote: > > Hi Jiangli, > > > > Actually I would prefer not to skip the test then. I'd like to verify if large data structures can > > > be placed in TLS without stack size adjustment or if there is some kind of regression. That's the > > > point of regression tests after all. > > > > > > That's a very valid point. I thought about that as well yesterday. My > > thinking was that the VM and test would need to be updated to remove > > the workaround for the TLS issue when no longer needed. Do you know if > > the proposed check using gnu_get_libc_version() covers all cases? I > > Other C libraries (e.g. MUSL) are not really covered, but I'm not sure if you mean this by covered... > > > came across the following archive that mentioned missing symbol in > > eglibc-2.13 (eglibc however is already deprecated). > > > https://www.mail-archive.com/lfs-dev at linuxfromscratch.org/msg18115.html > > I only knew, that 2.11 does not provide __pthread_get_minstack, but then I used > > https://github.com/bminor/glibc > > to find out that __pthread_get_minstack was added with 2.14.90 [1]. So 2.15 should be the first > stable release to include __pthread_get_minstack. I'll correct this in my fix. > > Would you be ok with this version of the fix? > > Thanks and best regards, > Richard. > > [1] version.h at [2] > https://github.com/bminor/glibc/blob/2c1094bd700e63a8d7f547b3f5495bedb55c0a08/version.h > > [2] Repo at [3] > https://github.com/bminor/glibc/tree/2c1094bd700e63a8d7f547b3f5495bedb55c0a08 > > [3] Commit that introduced __pthread_get_minstack > https://github.com/bminor/glibc/commit/2c1094bd700e63a8d7f547b3f5495bedb55c0a08 > > > -----Original Message----- > From: Jiangli Zhou > Sent: Donnerstag, 20. Februar 2020 17:32 > To: Reingruber, Richard > Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS > > Hi Richard, > > On Thu, Feb 20, 2020 at 1:03 AM Reingruber, Richard > wrote: > > > > Hi Jiangli, > > > > thanks for providing feedback. > > > > > David's suggestion of using dlsym seems to be more reliable. Regarding the stability of > > > __pthread_get_minstack in future glibc versions, Florian has added comment to > > > __pthread_get_minstack about external users to ensure it's not removed until TCB is > > > separately allocated. Please see: http://51.15.138.76/patch/18989/ and > > > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. > > > > So if I understand you and David correctly, then the test should be skipped if the lookup of > > __pthread_get_minstack fails. > > > > Then the test would be skipped on future glibc versions where __pthread_get_minstack was removed > > because TLS and TCB are not placed on a threads stack anymore. > > > > Actually I would prefer not to skip the test then. I'd like to verify if large data structures can > > be placed in TLS without stack size adjustment or if there is some kind of regression. That's the > > point of regression tests after all. > > > > That's a very valid point. I thought about that as well yesterday. My > thinking was that the VM and test would need to be updated to remove > the workaround for the TLS issue when no longer needed. Do you know if > the proposed check using gnu_get_libc_version() covers all cases? I > came across the following archive that mentioned missing symbol in > eglibc-2.13 (eglibc however is already deprecated). > > https://www.mail-archive.com/lfs-dev at linuxfromscratch.org/msg18115.html > > > In other words: I wouldn't change test coverage of future glibc versions. I just want to exclude > > glibc versions were the tested feature cannot work. > > Best regards, > Jiangli > > > > Thanks, Richard. > > > > -----Original Message----- > > From: Jiangli Zhou > > Sent: Mittwoch, 19. Februar 2020 19:24 > > To: Reingruber, Richard > > Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > > Subject: Re: RFR(S) 8239449: [TESTBUG] test/hotspot/jtreg/runtime/TLS/TestTLS.java: skip test if glibc too old for AdjustStackSizeForTLS > > > > Hi Richard, > > > > David's suggestion of using dlsym seems to be more reliable. Regarding > > the stability of __pthread_get_minstack in future glibc versions, > > Florian has added comment to __pthread_get_minstack about external > > users to ensure it's not removed until TCB is separately allocated. > > Please see: http://51.15.138.76/patch/18989/ and > > https://gcc.gnu.org/bugzilla//show_bug.cgi?id=91628#c2. > > > > Best regards, > > Jiangli > > > > > > > > > > On Wed, Feb 19, 2020 at 2:44 AM Reingruber, Richard > > wrote: > > > > > > Hi, > > > > > > please review this small test fix > > > > > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8239449/webrev.0/ > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8239449 > > > > > > On linux systems with older glibc versions (e.g. 2.11) the option -XX:+AdjustStackSizeForTLS has no effect, because the private function __pthread_get_minstack is not provided, and the test fails. With the fix the test is skipped instead. > > > > > > Thanks, Richard. From igor.ignatyev at oracle.com Mon Feb 24 16:42:30 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 24 Feb 2020 08:42:30 -0800 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> <4ac31ff3-df28-c3c4-5b63-9c2849716d00@oracle.com> <8982aeb0-3e12-f92f-a199-eb4ee1af1af9@oracle.com> <1a389351-3b80-5dc6-5712-6b4a5e018403@oracle.com> Message-ID: Hi > I found it's hard to find all the tests that use the modified > functions, because not all the tests use the modified functions > directly. I'd just change the functions in question to crash JVM, and then all the tests which start to fail are the tests which use these functions directly or indirectly. that's quite similar to your 2nd way. I'll be very surprised if there are any non hotspot tests (modulo jfr) which use this API, so I'd just run hotspot and jdk/jfr tests. > I also tested them with c1-only configuration. 2 failures were > found, but regression were not found. what are these two failures? > Do I need do some other tests? applications/ctw tests definitely use the modified methods, so I'd recommend you to run one tests from there (e.g. applications/ctw/modules/java_base.java) regarding your patch, 'cl_min' name seems to be confusion (at least to me) as it actually works as an upper bound for valid compilation levels. also given you had to change the same code in 3 different places, wouldn't it be better to factor that out? so you'll have smth like: > + if (compLevel < CompLevel_none || compLevel > highestCompLevel()) { > return false; // Intrinsic is not available on a non-existent compilation level. > } I'd like to cycle back on David's idea ("I actually think it is a bug that the VM allows you to set TieredStopAtLevel even though it is ignored on those configurations") Isn't the actual problem in how TieredStopAtLevel flag is handled in cases when TieredCompilation is disabled and/or JVM is built w/o tiered compilation support? I mean if JVM ensured that in such cases TieredStopAtLevel is set to CompLevel_highest_tier, all these WB methods would be correct. Thanks, -- Igor > On Feb 24, 2020, at 3:39 AM, Ao Qi wrote: > > On Fri, Feb 21, 2020 at 10:42 AM Ao Qi wrote: >> >>> We need to find all tests that use the modified WB functions and see if >>> they all pass under normal and non-tiered configurations. > > I found it's hard to find all the tests that use the modified > functions, because not all the tests use the modified functions > directly. I try to find them by two ways: > - I grep all modified functions' name in test dir and found all the > tests contain the names. Some found files are not jtreg tests. They > are used by other tests or libs. I also grep these files' name to find > tests that use these files. However I missed the situation that other > libs use these files. > - I added some tty->print at the modified positions and run hotspot > (hotspot_all_no_apps), jdk (jdk_stable) and langtools (jdk lib tools) > tests. Then I found all jtr files which contain the added print. > The test list found by the above two ways are here: > http://cr.openjdk.java.net/~aoqi/8239423/all.txt . These tests passed > with default and -XX:-TieredCompilation parameter and passed under > --with-jvm-features=-compiler1 configuration. I also tested them with > c1-only configuration. 2 failures were found, but regression were not > found. > > I also tested tier1-3, both release and fastdebug, with default and > -XX:-TieredCompilation parameter. > > Do I need do some other tests? > > Cheers, > Ao Qi > > > > > > > > >> >> I agree. I will try to find these tests. >> >> Cheers, >> Ao Qi >> >>> >>> Thanks, >>> David >>> > From zgu at redhat.com Mon Feb 24 16:49:18 2020 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 24 Feb 2020 11:49:18 -0500 Subject: [15] RFR 8238633: JVMTI heap walk should consult GC for marking oops In-Reply-To: References: <8ea20a15-bdb9-27b0-c306-154f539a3674@oracle.com> <3a5d60fd-04d5-da96-3d79-242d43fdec79@redhat.com> <901d7307-cf36-0367-e09c-ff47c76bbc25@oracle.com> Message-ID: <75082842-c67d-7a60-58b3-c10c67f7646f@redhat.com> Hi all, Updated according to your comments: http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.02/ I modified vmTestbase/nsk/jvmti/unit/heap/HeapWalkTests/TestDescription.java test [1] to walk 300K objects. Without patch: Time: 987431 nsecs Time: 1135390 nsecs Time: 1142519 nsecs Time: 962816 nsecs Time: 1015958 nsecs Avg: 1048822 nsecs With patch: 1105015 nsecs 1142425 nsecs 968057 nsecs 1383838 nsecs 1079885 nsecs Avg: 1135844 nsecs So, it shows about 8% performance hit. Thanks, -Zhengyu [1] http://cr.openjdk.java.net/~zgu/JDK-8238633/test/webrev.00/ On 2/21/20 8:01 AM, coleen.phillimore at oracle.com wrote: > Adding serviceability-dev back. > Coleen > > On 2/21/20 7:59 AM, coleen.phillimore at oracle.com wrote: >> >> Hi, I had a quick look at this, minus the shenandoah code. >> >> http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/src/hotspot/share/gc/shared/objectMarker.hpp.html >> >> >> I think this file could have forward declarations of GrowableArray and >> I didn't see a need for the markWord.hpp include. >> >> This change on the whole looks good to me. >> >> Coleen >> >> On 2/21/20 5:23 AM, Stefan Karlsson wrote: >>> Hi Zhengyu, >>> >>> On 2020-02-17 15:51, Zhengyu Gu wrote: >>>> Hi Stefan, >>>> >>>> Thanks for the review and suggestions, updated accordingly: >>>> >>>> http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/ >>> >>> Thanks for moving the code. I think this looks good. >>> >>> If you're up for it, I have a couple of style change suggestions: >>> >>> 1) ObjectMarker uses two verbs to describe the same thing: "mark" and >>> "visit". I propose that we only use "mark" in ObjectMarker and leave >>> the usage of "visited" to the Jvmti code. >>> >>> 2) Some updates to odd whitespaces >>> >>> 3) Using forward declarations in Shenandoah code. >>> >>> I've bundled those changes into webrevs: >>> >>> https://cr.openjdk.java.net/~stefank/8238633/webrev.01.delta >>> https://cr.openjdk.java.net/~stefank/8238633/webrev.01 >>> >>> Regarding performance testing, the HeapWalkTests you used seems to >>> use a very small heap. I think it would be good to redo the >>> measurements on a larger heap. Could you take the HeapWalkTest and >>> add a few GBs of small, linked objects? >>> >>> Thank, >>> StefanK >>>> >>>>> >>>>> --- >>>>> Previously, the calls to 'mark' and 'visited' were inlineable, but >>>>> now every GC has to take a virtual call when marking the objects. >>>>> My guess is that this code is slow anyway, and that it doesn't >>>>> matter too much, but did you measure the effect of that change >>>>> with, for example, G1? >>>>> >>>> I did rough measurement, timing >>>> vmTestbase/nsk/jvmti/unit/heap/HeapWalkTests/TestDescription.java test. >>>> >>>> If you know any tests/benchmarks I should measure, please let me know. >>>> >>>> Thanks, >>>> >>>> -Zhengyu >>>> >>>> >>>>> Thanks, >>>>> StefanK >>>>> >>>>>> Test: >>>>>> ?? hotspot_gc >>>>>> ?? vmTestbase_nsk_jdi >>>>>> ?? vmTestbase_nsk_jvmti >>>>>> >>>>>> Thanks, >>>>>> >>>>>> -Zhengyu >>>>>> >>>>>> >>>>> >>>> >>> >> > From erik.osterlund at oracle.com Mon Feb 24 17:04:33 2020 From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=) Date: Mon, 24 Feb 2020 18:04:33 +0100 Subject: [15] RFR 8238633: JVMTI heap walk should consult GC for marking oops In-Reply-To: <75082842-c67d-7a60-58b3-c10c67f7646f@redhat.com> References: <75082842-c67d-7a60-58b3-c10c67f7646f@redhat.com> Message-ID: Hi Zhengyu, Can?t your barriers just perform a NULL check on the forwardee instead? forwardee() == NULL never means forwarded, does it? Both JVMTI and JFR just ?mark? the markWord, leaving its forwardee == NULL. That way you can solve the issue in the backend instead, and we don?t need to do anything about JFR either. Or did I miss something? Thanks, /Erik > On 24 Feb 2020, at 17:49, Zhengyu Gu wrote: > > ?Hi all, > > Updated according to your comments: > http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.02/ > > I modified vmTestbase/nsk/jvmti/unit/heap/HeapWalkTests/TestDescription.java test [1] to walk 300K objects. > > Without patch: > Time: 987431 nsecs > Time: 1135390 nsecs > Time: 1142519 nsecs > Time: 962816 nsecs > Time: 1015958 nsecs > > Avg: 1048822 nsecs > > With patch: > 1105015 nsecs > 1142425 nsecs > 968057 nsecs > 1383838 nsecs > 1079885 nsecs > > Avg: 1135844 nsecs > > So, it shows about 8% performance hit. > > Thanks, > > -Zhengyu > > [1] http://cr.openjdk.java.net/~zgu/JDK-8238633/test/webrev.00/ > > > > > >> On 2/21/20 8:01 AM, coleen.phillimore at oracle.com wrote: >> Adding serviceability-dev back. >> Coleen >>> On 2/21/20 7:59 AM, coleen.phillimore at oracle.com wrote: >>> >>> Hi, I had a quick look at this, minus the shenandoah code. >>> >>> http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/src/hotspot/share/gc/shared/objectMarker.hpp.html >>> >>> I think this file could have forward declarations of GrowableArray and I didn't see a need for the markWord.hpp include. >>> >>> This change on the whole looks good to me. >>> >>> Coleen >>> >>> On 2/21/20 5:23 AM, Stefan Karlsson wrote: >>>> Hi Zhengyu, >>>> >>>> On 2020-02-17 15:51, Zhengyu Gu wrote: >>>>> Hi Stefan, >>>>> >>>>> Thanks for the review and suggestions, updated accordingly: >>>>> >>>>> http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/ >>>> >>>> Thanks for moving the code. I think this looks good. >>>> >>>> If you're up for it, I have a couple of style change suggestions: >>>> >>>> 1) ObjectMarker uses two verbs to describe the same thing: "mark" and "visit". I propose that we only use "mark" in ObjectMarker and leave the usage of "visited" to the Jvmti code. >>>> >>>> 2) Some updates to odd whitespaces >>>> >>>> 3) Using forward declarations in Shenandoah code. >>>> >>>> I've bundled those changes into webrevs: >>>> >>>> https://cr.openjdk.java.net/~stefank/8238633/webrev.01.delta >>>> https://cr.openjdk.java.net/~stefank/8238633/webrev.01 >>>> >>>> Regarding performance testing, the HeapWalkTests you used seems to use a very small heap. I think it would be good to redo the measurements on a larger heap. Could you take the HeapWalkTest and add a few GBs of small, linked objects? >>>> >>>> Thank, >>>> StefanK >>>>> >>>>>> >>>>>> --- >>>>>> Previously, the calls to 'mark' and 'visited' were inlineable, but now every GC has to take a virtual call when marking the objects. My guess is that this code is slow anyway, and that it doesn't matter too much, but did you measure the effect of that change with, for example, G1? >>>>>> >>>>> I did rough measurement, timing vmTestbase/nsk/jvmti/unit/heap/HeapWalkTests/TestDescription.java test. >>>>> >>>>> If you know any tests/benchmarks I should measure, please let me know. >>>>> >>>>> Thanks, >>>>> >>>>> -Zhengyu >>>>> >>>>> >>>>>> Thanks, >>>>>> StefanK >>>>>> >>>>>>> Test: >>>>>>> hotspot_gc >>>>>>> vmTestbase_nsk_jdi >>>>>>> vmTestbase_nsk_jvmti >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> -Zhengyu >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> > From zgu at redhat.com Mon Feb 24 20:58:55 2020 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 24 Feb 2020 15:58:55 -0500 Subject: [15] RFR 8238633: JVMTI heap walk should consult GC for marking oops In-Reply-To: References: <75082842-c67d-7a60-58b3-c10c67f7646f@redhat.com> Message-ID: Hi Erik, On 2/24/20 12:04 PM, Erik ?sterlund wrote: > Hi Zhengyu, > > Can?t your barriers just perform a NULL check on the forwardee instead? forwardee() == NULL never means forwarded, does it? Both JVMTI and JFR just ?mark? the markWord, leaving its forwardee == NULL. > > That way you can solve the issue in the backend instead, and we don?t need to do anything about JFR either. Or did I miss something? You are right, this is a much simple solution. But the concern is that, resolve_forward() is the most used barrier, additional null check is undesirable. After offline chat with my colleagues, we realize that it may be ok. As JVMTI/JFR heap walk happens at safepoints, we really don't have to add the null check in regular barrier. Instead, force GC to use different version of resolve_forward with null check. Let me protocol this alternative, will get back you soon. Thank, -Zhengyu > > Thanks, > /Erik > >> On 24 Feb 2020, at 17:49, Zhengyu Gu wrote: >> >> ?Hi all, >> >> Updated according to your comments: >> http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.02/ >> >> I modified vmTestbase/nsk/jvmti/unit/heap/HeapWalkTests/TestDescription.java test [1] to walk 300K objects. >> >> Without patch: >> Time: 987431 nsecs >> Time: 1135390 nsecs >> Time: 1142519 nsecs >> Time: 962816 nsecs >> Time: 1015958 nsecs >> >> Avg: 1048822 nsecs >> >> With patch: >> 1105015 nsecs >> 1142425 nsecs >> 968057 nsecs >> 1383838 nsecs >> 1079885 nsecs >> >> Avg: 1135844 nsecs >> >> So, it shows about 8% performance hit. >> >> Thanks, >> >> -Zhengyu >> >> [1] http://cr.openjdk.java.net/~zgu/JDK-8238633/test/webrev.00/ >> >> >> >> >> >>> On 2/21/20 8:01 AM, coleen.phillimore at oracle.com wrote: >>> Adding serviceability-dev back. >>> Coleen >>>> On 2/21/20 7:59 AM, coleen.phillimore at oracle.com wrote: >>>> >>>> Hi, I had a quick look at this, minus the shenandoah code. >>>> >>>> http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/src/hotspot/share/gc/shared/objectMarker.hpp.html >>>> >>>> I think this file could have forward declarations of GrowableArray and I didn't see a need for the markWord.hpp include. >>>> >>>> This change on the whole looks good to me. >>>> >>>> Coleen >>>> >>>> On 2/21/20 5:23 AM, Stefan Karlsson wrote: >>>>> Hi Zhengyu, >>>>> >>>>> On 2020-02-17 15:51, Zhengyu Gu wrote: >>>>>> Hi Stefan, >>>>>> >>>>>> Thanks for the review and suggestions, updated accordingly: >>>>>> >>>>>> http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/ >>>>> >>>>> Thanks for moving the code. I think this looks good. >>>>> >>>>> If you're up for it, I have a couple of style change suggestions: >>>>> >>>>> 1) ObjectMarker uses two verbs to describe the same thing: "mark" and "visit". I propose that we only use "mark" in ObjectMarker and leave the usage of "visited" to the Jvmti code. >>>>> >>>>> 2) Some updates to odd whitespaces >>>>> >>>>> 3) Using forward declarations in Shenandoah code. >>>>> >>>>> I've bundled those changes into webrevs: >>>>> >>>>> https://cr.openjdk.java.net/~stefank/8238633/webrev.01.delta >>>>> https://cr.openjdk.java.net/~stefank/8238633/webrev.01 >>>>> >>>>> Regarding performance testing, the HeapWalkTests you used seems to use a very small heap. I think it would be good to redo the measurements on a larger heap. Could you take the HeapWalkTest and add a few GBs of small, linked objects? >>>>> >>>>> Thank, >>>>> StefanK >>>>>> >>>>>>> >>>>>>> --- >>>>>>> Previously, the calls to 'mark' and 'visited' were inlineable, but now every GC has to take a virtual call when marking the objects. My guess is that this code is slow anyway, and that it doesn't matter too much, but did you measure the effect of that change with, for example, G1? >>>>>>> >>>>>> I did rough measurement, timing vmTestbase/nsk/jvmti/unit/heap/HeapWalkTests/TestDescription.java test. >>>>>> >>>>>> If you know any tests/benchmarks I should measure, please let me know. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> -Zhengyu >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> StefanK >>>>>>> >>>>>>>> Test: >>>>>>>> hotspot_gc >>>>>>>> vmTestbase_nsk_jdi >>>>>>>> vmTestbase_nsk_jvmti >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> -Zhengyu >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >> > From erik.osterlund at oracle.com Mon Feb 24 21:24:10 2020 From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=) Date: Mon, 24 Feb 2020 22:24:10 +0100 Subject: [15] RFR 8238633: JVMTI heap walk should consult GC for marking oops In-Reply-To: References: Message-ID: <940D46CE-22D6-479B-B8EF-65FFA896C4A0@oracle.com> Hi Zhengyu, > On 24 Feb 2020, at 21:59, Zhengyu Gu wrote: > > ?Hi Erik, > >> On 2/24/20 12:04 PM, Erik ?sterlund wrote: >> Hi Zhengyu, >> Can?t your barriers just perform a NULL check on the forwardee instead? forwardee() == NULL never means forwarded, does it? Both JVMTI and JFR just ?mark? the markWord, leaving its forwardee == NULL. >> That way you can solve the issue in the backend instead, and we don?t need to do anything about JFR either. Or did I miss something? > > You are right, this is a much simple solution. But the concern is that, resolve_forward() is the most used barrier, additional null check is undesirable. > > After offline chat with my colleagues, we realize that it may be ok. As JVMTI/JFR heap walk happens at safepoints, we really don't have to add the null check in regular barrier. Instead, force GC to use different version of resolve_forward with null check. > > Let me protocol this alternative, will get back you soon. The JFR heap walker does use the shared barriers in the safepoint though. So that optimization sounds like it won?t work. Having said that, the null check will be taken only for runtime code, not when going through the JIT. I would be surprised if this very well predicted NULL check used by runtime code would be noticeable, especially since you are probably going to CAS as well in the same path this is taken (the mark word is ?marked?). So perhaps just adding the NULL check in the barrier for the case where the markWord ?is_marked? is the sane thing to do, knowing that the other costs taken in the same path will dominate. Thanks, /Erik > Thank, > > -Zhengyu > >> Thanks, >> /Erik >>>> On 24 Feb 2020, at 17:49, Zhengyu Gu wrote: >>> >>> ?Hi all, >>> >>> Updated according to your comments: >>> http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.02/ >>> >>> I modified vmTestbase/nsk/jvmti/unit/heap/HeapWalkTests/TestDescription.java test [1] to walk 300K objects. >>> >>> Without patch: >>> Time: 987431 nsecs >>> Time: 1135390 nsecs >>> Time: 1142519 nsecs >>> Time: 962816 nsecs >>> Time: 1015958 nsecs >>> >>> Avg: 1048822 nsecs >>> >>> With patch: >>> 1105015 nsecs >>> 1142425 nsecs >>> 968057 nsecs >>> 1383838 nsecs >>> 1079885 nsecs >>> >>> Avg: 1135844 nsecs >>> >>> So, it shows about 8% performance hit. >>> >>> Thanks, >>> >>> -Zhengyu >>> >>> [1] http://cr.openjdk.java.net/~zgu/JDK-8238633/test/webrev.00/ >>> >>> >>> >>> >>> >>>> On 2/21/20 8:01 AM, coleen.phillimore at oracle.com wrote: >>>> Adding serviceability-dev back. >>>> Coleen >>>>> On 2/21/20 7:59 AM, coleen.phillimore at oracle.com wrote: >>>>> >>>>> Hi, I had a quick look at this, minus the shenandoah code. >>>>> >>>>> http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/src/hotspot/share/gc/shared/objectMarker.hpp.html >>>>> >>>>> I think this file could have forward declarations of GrowableArray and I didn't see a need for the markWord.hpp include. >>>>> >>>>> This change on the whole looks good to me. >>>>> >>>>> Coleen >>>>> >>>>> On 2/21/20 5:23 AM, Stefan Karlsson wrote: >>>>>> Hi Zhengyu, >>>>>> >>>>>> On 2020-02-17 15:51, Zhengyu Gu wrote: >>>>>>> Hi Stefan, >>>>>>> >>>>>>> Thanks for the review and suggestions, updated accordingly: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/ >>>>>> >>>>>> Thanks for moving the code. I think this looks good. >>>>>> >>>>>> If you're up for it, I have a couple of style change suggestions: >>>>>> >>>>>> 1) ObjectMarker uses two verbs to describe the same thing: "mark" and "visit". I propose that we only use "mark" in ObjectMarker and leave the usage of "visited" to the Jvmti code. >>>>>> >>>>>> 2) Some updates to odd whitespaces >>>>>> >>>>>> 3) Using forward declarations in Shenandoah code. >>>>>> >>>>>> I've bundled those changes into webrevs: >>>>>> >>>>>> https://cr.openjdk.java.net/~stefank/8238633/webrev.01.delta >>>>>> https://cr.openjdk.java.net/~stefank/8238633/webrev.01 >>>>>> >>>>>> Regarding performance testing, the HeapWalkTests you used seems to use a very small heap. I think it would be good to redo the measurements on a larger heap. Could you take the HeapWalkTest and add a few GBs of small, linked objects? >>>>>> >>>>>> Thank, >>>>>> StefanK >>>>>>> >>>>>>>> >>>>>>>> --- >>>>>>>> Previously, the calls to 'mark' and 'visited' were inlineable, but now every GC has to take a virtual call when marking the objects. My guess is that this code is slow anyway, and that it doesn't matter too much, but did you measure the effect of that change with, for example, G1? >>>>>>>> >>>>>>> I did rough measurement, timing vmTestbase/nsk/jvmti/unit/heap/HeapWalkTests/TestDescription.java test. >>>>>>> >>>>>>> If you know any tests/benchmarks I should measure, please let me know. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> -Zhengyu >>>>>>> >>>>>>> >>>>>>>> Thanks, >>>>>>>> StefanK >>>>>>>> >>>>>>>>> Test: >>>>>>>>> hotspot_gc >>>>>>>>> vmTestbase_nsk_jdi >>>>>>>>> vmTestbase_nsk_jvmti >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> -Zhengyu >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>> > From zgu at redhat.com Mon Feb 24 22:38:14 2020 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 24 Feb 2020 17:38:14 -0500 Subject: [15] RFR 8238633: JVMTI heap walk should consult GC for marking oops In-Reply-To: <940D46CE-22D6-479B-B8EF-65FFA896C4A0@oracle.com> References: <940D46CE-22D6-479B-B8EF-65FFA896C4A0@oracle.com> Message-ID: <3d8553da-f7a8-fe84-d923-b10c5b47775e@redhat.com> > > The JFR heap walker does use the shared barriers in the safepoint though. So that optimization sounds like it won?t work. Okay. > > Having said that, the null check will be taken only for runtime code, not when going through the JIT. I would be surprised if this very well predicted NULL check used by runtime code would be noticeable, especially since you are probably going to CAS as well in the same path this is taken (the mark word is ?marked?). > > So perhaps just adding the NULL check in the barrier for the case where the markWord ?is_marked? is the sane thing to do, knowing that the other costs taken in the same path will dominate. I have this patch, exactly what you suggested. I will let Aleksey run his numbers. diff -r ef1e608a5ecc src/hotspot/share/gc/shenandoah/shenandoahForwarding.inline.hpp --- a/src/hotspot/share/gc/shenandoah/shenandoahForwarding.inline.hpp Mon Feb 24 15:03:28 2020 +0100 +++ b/src/hotspot/share/gc/shenandoah/shenandoahForwarding.inline.hpp Mon Feb 24 12:56:08 2020 -0500 @@ -37,10 +37,12 @@ inline HeapWord* ShenandoahForwarding::get_forwardee_raw_unchecked(oop obj) { markWord mark = obj->mark_raw(); if (mark.is_marked()) { - return (HeapWord*) mark.clear_lock_bits().to_pointer(); - } else { - return cast_from_oop(obj); + HeapWord* fwdptr = (HeapWord*)mark.clear_lock_bits().to_pointer(); + if (fwdptr != NULL) { + return fwdptr; + } } + return cast_from_oop(obj); } Thanks, -Zhengyu > > Thanks, > /Erik > >> Thank, >> >> -Zhengyu >> >>> Thanks, >>> /Erik >>>>> On 24 Feb 2020, at 17:49, Zhengyu Gu wrote: >>>> >>>> ?Hi all, >>>> >>>> Updated according to your comments: >>>> http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.02/ >>>> >>>> I modified vmTestbase/nsk/jvmti/unit/heap/HeapWalkTests/TestDescription.java test [1] to walk 300K objects. >>>> >>>> Without patch: >>>> Time: 987431 nsecs >>>> Time: 1135390 nsecs >>>> Time: 1142519 nsecs >>>> Time: 962816 nsecs >>>> Time: 1015958 nsecs >>>> >>>> Avg: 1048822 nsecs >>>> >>>> With patch: >>>> 1105015 nsecs >>>> 1142425 nsecs >>>> 968057 nsecs >>>> 1383838 nsecs >>>> 1079885 nsecs >>>> >>>> Avg: 1135844 nsecs >>>> >>>> So, it shows about 8% performance hit. >>>> >>>> Thanks, >>>> >>>> -Zhengyu >>>> >>>> [1] http://cr.openjdk.java.net/~zgu/JDK-8238633/test/webrev.00/ >>>> >>>> >>>> >>>> >>>> >>>>> On 2/21/20 8:01 AM, coleen.phillimore at oracle.com wrote: >>>>> Adding serviceability-dev back. >>>>> Coleen >>>>>> On 2/21/20 7:59 AM, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> Hi, I had a quick look at this, minus the shenandoah code. >>>>>> >>>>>> http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/src/hotspot/share/gc/shared/objectMarker.hpp.html >>>>>> >>>>>> I think this file could have forward declarations of GrowableArray and I didn't see a need for the markWord.hpp include. >>>>>> >>>>>> This change on the whole looks good to me. >>>>>> >>>>>> Coleen >>>>>> >>>>>> On 2/21/20 5:23 AM, Stefan Karlsson wrote: >>>>>>> Hi Zhengyu, >>>>>>> >>>>>>> On 2020-02-17 15:51, Zhengyu Gu wrote: >>>>>>>> Hi Stefan, >>>>>>>> >>>>>>>> Thanks for the review and suggestions, updated accordingly: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~zgu/JDK-8238633/webrev.01/ >>>>>>> >>>>>>> Thanks for moving the code. I think this looks good. >>>>>>> >>>>>>> If you're up for it, I have a couple of style change suggestions: >>>>>>> >>>>>>> 1) ObjectMarker uses two verbs to describe the same thing: "mark" and "visit". I propose that we only use "mark" in ObjectMarker and leave the usage of "visited" to the Jvmti code. >>>>>>> >>>>>>> 2) Some updates to odd whitespaces >>>>>>> >>>>>>> 3) Using forward declarations in Shenandoah code. >>>>>>> >>>>>>> I've bundled those changes into webrevs: >>>>>>> >>>>>>> https://cr.openjdk.java.net/~stefank/8238633/webrev.01.delta >>>>>>> https://cr.openjdk.java.net/~stefank/8238633/webrev.01 >>>>>>> >>>>>>> Regarding performance testing, the HeapWalkTests you used seems to use a very small heap. I think it would be good to redo the measurements on a larger heap. Could you take the HeapWalkTest and add a few GBs of small, linked objects? >>>>>>> >>>>>>> Thank, >>>>>>> StefanK >>>>>>>> >>>>>>>>> >>>>>>>>> --- >>>>>>>>> Previously, the calls to 'mark' and 'visited' were inlineable, but now every GC has to take a virtual call when marking the objects. My guess is that this code is slow anyway, and that it doesn't matter too much, but did you measure the effect of that change with, for example, G1? >>>>>>>>> >>>>>>>> I did rough measurement, timing vmTestbase/nsk/jvmti/unit/heap/HeapWalkTests/TestDescription.java test. >>>>>>>> >>>>>>>> If you know any tests/benchmarks I should measure, please let me know. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> -Zhengyu >>>>>>>> >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> StefanK >>>>>>>>> >>>>>>>>>> Test: >>>>>>>>>> hotspot_gc >>>>>>>>>> vmTestbase_nsk_jdi >>>>>>>>>> vmTestbase_nsk_jvmti >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> -Zhengyu >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >> > From aoqi at loongson.cn Tue Feb 25 08:34:58 2020 From: aoqi at loongson.cn (Ao Qi) Date: Tue, 25 Feb 2020 16:34:58 +0800 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> <4ac31ff3-df28-c3c4-5b63-9c2849716d00@oracle.com> <8982aeb0-3e12-f92f-a199-eb4ee1af1af9@oracle.com> <1a389351-3b80-5dc6-5712-6b4a5e018403@oracle.com> Message-ID: On Tue, Feb 25, 2020 at 12:42 AM Igor Ignatyev wrote: > > Hi > > I found it's hard to find all the tests that use the modified > functions, because not all the tests use the modified functions > directly. > > > I'd just change the functions in question to crash JVM, and then all the tests which start to fail are the tests which use these functions directly or indirectly. that's quite similar to your 2nd way. > I'll be very surprised if there are any non hotspot tests (modulo jfr) which use this API, so I'd just run hotspot and jdk/jfr tests. Thanks, Igor. I am doing. I selected tests with TEST="test/hotspot test/jdk/jdk/jfr". It would take some time. > > I also tested them with c1-only configuration. 2 failures were > > found, but regression were not found. > > what are these two failures? Their names and part of stderr are in below. test/hotspot/jtreg/compiler/jvmci/compilerToVM/IsMatureVsReprofileTest.java: ---------------------- STDERR: java.lang.RuntimeException: Multiple times invoked method should have MDO: expected 0 to not equal 0 at jdk.test.lib.Asserts.fail(Asserts.java:594) at jdk.test.lib.Asserts.assertNotEquals(Asserts.java:394) at jdk.test.lib.Asserts.assertNE(Asserts.java:367) at compiler.jvmci.compilerToVM.IsMatureVsReprofileTest.test(IsMatureVsReprofileTest.java:82) at compiler.jvmci.compilerToVM.IsMatureVsReprofileTest.main(IsMatureVsReprofileTest.java:65) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:564) at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127) at java.base/java.lang.Thread.run(Thread.java:832) JavaTest Message: Test threw exception: java.lang.RuntimeException: Multiple times invoked method should have MDO: expected 0 to not equal 0 JavaTest Message: shutting down test ---------------------- test/hotspot/jtreg/compiler/jvmci/compilerToVM/ReprofileTest.java: ---------------------- STDERR: java.lang.RuntimeException: public int compiler.jvmci.compilerToVM.DummyClass.dummyInstanceFunction() : profiling info wasn't changed after 1500 invocations: expected DefaultProfilingInfo<> to not equal DefaultProfilingInfo<> at jdk.test.lib.Asserts.fail(Asserts.java:594) at jdk.test.lib.Asserts.assertNotEquals(Asserts.java:394) at jdk.test.lib.Asserts.assertNE(Asserts.java:367) at compiler.jvmci.compilerToVM.ReprofileTest.runSanityTest(ReprofileTest.java:101) at java.base/java.util.ArrayList.forEach(ArrayList.java:1510) at compiler.jvmci.compilerToVM.ReprofileTest.main(ReprofileTest.java:64) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:564) at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127) at java.base/java.lang.Thread.run(Thread.java:832) JavaTest Message: Test threw exception: java.lang.RuntimeException: public int compiler.jvmci.compilerToVM.DummyClass.dummyInstanceFunction() : profiling info wasn't changed after 1500 invocations: expected DefaultProfilingInfo<> to not equal DefaultProfilingInfo<> JavaTest Message: shutting down test ---------------------- > > Do I need do some other tests? > > applications/ctw tests definitely use the modified methods, so I'd recommend you to run one tests from there (e.g. applications/ctw/modules/java_base.java) I will also do these tests. > > regarding your patch, 'cl_min' name seems to be confusion (at least to me) as it actually works as an upper bound for valid compilation levels. also given you had to change the same code in 3 different places, wouldn't it be better to factor that out? so you'll have smth like: > > + if (compLevel < CompLevel_none || compLevel > highestCompLevel()) { > return false; // Intrinsic is not available on a non-existent compilation level. > } like this: http://cr.openjdk.java.net/~aoqi/8239423/webrev.02/ ? Thanks, Ao Qi > > > I'd like to cycle back on David's idea ("I actually think it is a bug that the VM allows you to set TieredStopAtLevel even though it is ignored on those configurations") Isn't the actual problem in how TieredStopAtLevel flag is handled in cases when TieredCompilation is disabled and/or JVM is built w/o tiered compilation support? I mean if JVM ensured that in such cases TieredStopAtLevel is set to CompLevel_highest_tier, all these WB methods would be correct. > > > Thanks, > -- Igor > > On Feb 24, 2020, at 3:39 AM, Ao Qi wrote: > > On Fri, Feb 21, 2020 at 10:42 AM Ao Qi wrote: > > > We need to find all tests that use the modified WB functions and see if > they all pass under normal and non-tiered configurations. > > > I found it's hard to find all the tests that use the modified > functions, because not all the tests use the modified functions > directly. I try to find them by two ways: > - I grep all modified functions' name in test dir and found all the > tests contain the names. Some found files are not jtreg tests. They > are used by other tests or libs. I also grep these files' name to find > tests that use these files. However I missed the situation that other > libs use these files. > - I added some tty->print at the modified positions and run hotspot > (hotspot_all_no_apps), jdk (jdk_stable) and langtools (jdk lib tools) > tests. Then I found all jtr files which contain the added print. > The test list found by the above two ways are here: > http://cr.openjdk.java.net/~aoqi/8239423/all.txt . These tests passed > with default and -XX:-TieredCompilation parameter and passed under > --with-jvm-features=-compiler1 configuration. I also tested them with > c1-only configuration. 2 failures were found, but regression were not > found. > > I also tested tier1-3, both release and fastdebug, with default and > -XX:-TieredCompilation parameter. > > Do I need do some other tests? > > Cheers, > Ao Qi > > > > > > > > > > I agree. I will try to find these tests. > > Cheers, > Ao Qi > > > Thanks, > David > > > From matthias.baesken at sap.com Tue Feb 25 16:20:35 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 25 Feb 2020 16:20:35 +0000 Subject: RFR [XS]: 8239462: jdk.hotspot.agent misses some ReleaseStringUTFChars calls in case of early returns In-Reply-To: References: <6ed0d004-3e8c-4b27-f583-06dbacf45173@oracle.com> Message-ID: New webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.3/ Best regards, Matthias > IMO the solution with goto makes it even worse. > If you don't want to introduce the wrapper, could you please restore > changes in LinuxDebuggerLocal_attach0 from webrev.1 > > --alex > > On 02/21/2020 00:32, Baesken, Matthias wrote: > > Hi Alex , > > > > new webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.2/ > > > > Best Regards, Matthias > > > > > >> > >> Hi Matthias, > >> > >> Looks good in general, but I think it makes sense to fix #2 cases (at > >> least I see them in LinuxDebuggerLocal). If GetStringUTFChars fails, the > >> code will crash. > >> Also I see GetStringUTFChars(str, JNI_FALSE). This look bad as well - > >> 2nd arg is a pointer, so it should be NULL or nullptr. > >> > >> As for #1 and #3 - AFAIU they are both right ways. > >> If GetStringUTFChars fails, it throws OOM and return NULL. > >> > >> And one more thing to consider. > >> LinuxDebuggerLocal_attach0 function looks terrible - 7 > >> ReleaseStringUTFChars calls for 2 GetStringUTFChars. > >> Maybe it make sense to introduce simple wrapper like AutoJavaString in > >> src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp > >> It would make the code simpler and less error prone. > >> > >> --alex > >> > > From sgehwolf at redhat.com Tue Feb 25 19:22:26 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 25 Feb 2020 20:22:26 +0100 Subject: RFR(S): 8239785: Cgroups: Incorrect detection logic on old systems in hotspot Message-ID: <362678e8bc8231c205c6b3327d08dd5179154e23.camel@redhat.com> Hi, Could I please get reviews of this cgroup-related patch? After JDK- 8230305 old systems with no mounted cgroup controllers would get detected as cgroups v2 (wrong). Then, when initializing the v2 subsystem, it would fail. The trace about cgroupv2 mount point not found is misleading in this case. While the outcome is similar pre/post patch (NULL cgroup subsystem), I'd like to be explicit about this case. The suggested fix is to look at /proc/self/mountinfo in order to correctly detect whether this is cgroup v2 or cgroup v1 with no mounted controllers. In the latter case we just stop initialization as we'd fail later in cgroupv1 code anyway. This aligns hotspot code with core- libs after JDK-8239559. Bug: https://bugs.openjdk.java.net/browse/JDK-8239785 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/ Testing: jdk-submit, hotspot docker tests on cgroup v1 and cgroup v2. All pass. Thoughts? Thanks, Severin From alexey.menkov at oracle.com Tue Feb 25 19:30:28 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Tue, 25 Feb 2020 11:30:28 -0800 Subject: RFR [XS]: 8239462: jdk.hotspot.agent misses some ReleaseStringUTFChars calls in case of early returns In-Reply-To: References: <6ed0d004-3e8c-4b27-f583-06dbacf45173@oracle.com> Message-ID: Hi Matthias, LGTM --alex On 02/25/2020 08:20, Baesken, Matthias wrote: > > New webrev : > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.3/ > > > Best regards, Matthias > > >> IMO the solution with goto makes it even worse. >> If you don't want to introduce the wrapper, could you please restore >> changes in LinuxDebuggerLocal_attach0 from webrev.1 >> >> --alex >> >> On 02/21/2020 00:32, Baesken, Matthias wrote: >>> Hi Alex , >>> >>> new webrev : >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.2/ >>> >>> Best Regards, Matthias >>> >>> >>>> >>>> Hi Matthias, >>>> >>>> Looks good in general, but I think it makes sense to fix #2 cases (at >>>> least I see them in LinuxDebuggerLocal). If GetStringUTFChars fails, the >>>> code will crash. >>>> Also I see GetStringUTFChars(str, JNI_FALSE). This look bad as well - >>>> 2nd arg is a pointer, so it should be NULL or nullptr. >>>> >>>> As for #1 and #3 - AFAIU they are both right ways. >>>> If GetStringUTFChars fails, it throws OOM and return NULL. >>>> >>>> And one more thing to consider. >>>> LinuxDebuggerLocal_attach0 function looks terrible - 7 >>>> ReleaseStringUTFChars calls for 2 GetStringUTFChars. >>>> Maybe it make sense to introduce simple wrapper like AutoJavaString in >>>> src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp >>>> It would make the code simpler and less error prone. >>>> >>>> --alex >>>> >>> From bob.vandette at oracle.com Tue Feb 25 20:08:26 2020 From: bob.vandette at oracle.com (Bob Vandette) Date: Tue, 25 Feb 2020 15:08:26 -0500 Subject: RFR(S): 8239785: Cgroups: Incorrect detection logic on old systems in hotspot In-Reply-To: <362678e8bc8231c205c6b3327d08dd5179154e23.camel@redhat.com> References: <362678e8bc8231c205c6b3327d08dd5179154e23.camel@redhat.com> Message-ID: <2A934185-9B81-4371-B985-F0C3C39B3FD3@oracle.com> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.sdiff.html If both cgroupv1 and v2 controllers are mounted, you might have some problems. You might want to only look for cgroup2 mounts if is_cgroupV2 is true otherwise only look for cgroup mounts. 194 cg_infos[i]._mount_path = os::strdup(tmp_mount_point); This will stomp the pointer: 213 cg_infos[memory_idx]._mount_path = os::strdup(tmpmount); You should free all strdup?d pointers in cg_infos in places where you fail and return NULL. 81 cg_infos[memory_idx]._name = os::strdup(name); 213 cg_infos[memory_idx]._mount_path = os::strdup(tmpmount); 214 cg_infos[memory_idx]._root_mount_path = os::strdup(tmproot); Bob. > On Feb 25, 2020, at 2:22 PM, Severin Gehwolf wrote: > > Hi, > > Could I please get reviews of this cgroup-related patch? After JDK- > 8230305 old systems with no mounted cgroup controllers would get > detected as cgroups v2 (wrong). Then, when initializing the v2 > subsystem, it would fail. The trace about cgroupv2 mount point not > found is misleading in this case. While the outcome is similar pre/post > patch (NULL cgroup subsystem), I'd like to be explicit about this case. > > The suggested fix is to look at /proc/self/mountinfo in order to > correctly detect whether this is cgroup v2 or cgroup v1 with no mounted > controllers. In the latter case we just stop initialization as we'd > fail later in cgroupv1 code anyway. This aligns hotspot code with core- > libs after JDK-8239559. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8239785 > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/ > > Testing: jdk-submit, hotspot docker tests on cgroup v1 and cgroup v2. All pass. > > Thoughts? > > Thanks, > Severin > From david.holmes at oracle.com Wed Feb 26 01:05:23 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 26 Feb 2020 11:05:23 +1000 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> <4ac31ff3-df28-c3c4-5b63-9c2849716d00@oracle.com> <8982aeb0-3e12-f92f-a199-eb4ee1af1af9@oracle.com> <1a389351-3b80-5dc6-5712-6b4a5e018403@oracle.com> Message-ID: Hi Igor, On 25/02/2020 2:42 am, Igor Ignatyev wrote: > Hi > >> I found it's hard to find all the tests that use the modified >> functions, because not all the tests use the modified functions >> directly. > > I'd just change the functions in question to crash JVM, and then all the > tests which start to fail are the tests which use these functions > directly or indirectly. that's quite similar to your 2nd way. > I'll be very surprised if there are any non hotspot tests (modulo jfr) > which use this API, so I'd just run hotspot and jdk/jfr tests. > >> I also tested them with c1-only configuration. 2 failures were >> found, but regression were not?found. > what are these two failures? > >> Do I need do some other tests? > applications/ctw tests definitely use the modified methods, so I'd > recommend you to run one tests from there > (e.g.?applications/ctw/modules/java_base.java) > > regarding your patch, 'cl_min' name seems to be confusion (at least to > me) as it actually works as an upper bound for valid compilation levels. Yes that is my fault. I use cl_min because it captured the result of the MIN2 macro. But it is actually the lesser of two possible upper bounds > also given you had to change the same code in 3 different places, > wouldn't it be better to factor that out? so you'll have smth like: >> + if (compLevel < CompLevel_none || compLevel > highestCompLevel()) { >> return false; // Intrinsic is not available on a non-existent compilation level. >> } Yes a file static function would be useful to capture this. > I'd like to cycle back on David's idea ("I actually think it is a bug > that the VM allows you to set TieredStopAtLevel even though it is > ignored on those configurations") Isn't the actual problem in how > TieredStopAtLevel flag is handled in cases when TieredCompilation is > disabled and/or JVM is built w/o tiered compilation support? I mean if > JVM ensured that in such cases TieredStopAtLevel is set to > CompLevel_highest_tier, all these WB methods would be correct. TieredStopAtLevel is an input flag. I don't think it makes sense to treat it as an output flag, and not only ignore what the user sets but replace it with another value when there is no tiered. Ultimately this is a design decision of the compiler team, but as it stands any code that uses TieredStopAtLevel has to ignore it when there is no tiered. Cheers, David ----- > Thanks, > -- Igor > >> On Feb 24, 2020, at 3:39 AM, Ao Qi > > wrote: >> >> On Fri, Feb 21, 2020 at 10:42 AM Ao Qi > > wrote: >>> >>>> We need to find all tests that use the modified WB functions and see if >>>> they all pass under normal and non-tiered configurations. >> >> I found it's hard to find all the tests that use the modified >> functions, because not all the tests use the modified functions >> directly. I try to find them by two ways: >> ?- I grep all modified functions' name in test dir and found all the >> tests contain the names. Some found files are not jtreg tests. They >> are used by other tests or libs. I also grep these files' name to find >> tests that use these files. However I missed the situation that other >> libs use these files. >> ?- I added some tty->print at the modified positions and run hotspot >> (hotspot_all_no_apps), jdk (jdk_stable) and langtools (jdk lib tools) >> tests. Then I found all jtr files which contain the added print. >> The test list found by the above two ways are here: >> http://cr.openjdk.java.net/~aoqi/8239423/all.txt . These tests passed >> with default and -XX:-TieredCompilation parameter and passed under >> --with-jvm-features=-compiler1 configuration. I also tested them with >> c1-only configuration. 2 failures were found, but regression were not >> found. >> >> I also tested tier1-3, both release and fastdebug, with default and >> -XX:-TieredCompilation parameter. >> >> Do I need do some other tests? >> >> Cheers, >> Ao Qi >> >> >> >> >> >> >> >> >>> >>> I agree. I will try to find these tests. >>> >>> Cheers, >>> Ao Qi >>> >>>> >>>> Thanks, >>>> David >>>> >> > From david.holmes at oracle.com Wed Feb 26 01:51:58 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 26 Feb 2020 11:51:58 +1000 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> <4ac31ff3-df28-c3c4-5b63-9c2849716d00@oracle.com> <8982aeb0-3e12-f92f-a199-eb4ee1af1af9@oracle.com> <1a389351-3b80-5dc6-5712-6b4a5e018403@oracle.com> Message-ID: <5af5cfe4-e235-f0c0-cb57-a9e1254be642@oracle.com> Hi, On 25/02/2020 6:34 pm, Ao Qi wrote: > On Tue, Feb 25, 2020 at 12:42 AM Igor Ignatyev wrote: >> >> Hi >> >> I found it's hard to find all the tests that use the modified >> functions, because not all the tests use the modified functions >> directly. >> >> >> I'd just change the functions in question to crash JVM, and then all the tests which start to fail are the tests which use these functions directly or indirectly. that's quite similar to your 2nd way. >> I'll be very surprised if there are any non hotspot tests (modulo jfr) which use this API, so I'd just run hotspot and jdk/jfr tests. > > Thanks, Igor. I am doing. I selected tests with TEST="test/hotspot > test/jdk/jdk/jfr". It would take some time. > >> >> I also tested them with c1-only configuration. 2 failures were >> >> found, but regression were not found. >> >> what are these two failures? > > Their names and part of stderr are in below. > > test/hotspot/jtreg/compiler/jvmci/compilerToVM/IsMatureVsReprofileTest.java: > ---------------------- > STDERR: > java.lang.RuntimeException: Multiple times invoked method should have > MDO: expected 0 to not equal 0 > at jdk.test.lib.Asserts.fail(Asserts.java:594) > at jdk.test.lib.Asserts.assertNotEquals(Asserts.java:394) > at jdk.test.lib.Asserts.assertNE(Asserts.java:367) > at compiler.jvmci.compilerToVM.IsMatureVsReprofileTest.test(IsMatureVsReprofileTest.java:82) > at compiler.jvmci.compilerToVM.IsMatureVsReprofileTest.main(IsMatureVsReprofileTest.java:65) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:564) > at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127) > at java.base/java.lang.Thread.run(Thread.java:832) > > JavaTest Message: Test threw exception: java.lang.RuntimeException: > Multiple times invoked method should have MDO: expected 0 to not equal > 0 > JavaTest Message: shutting down test > ---------------------- > > test/hotspot/jtreg/compiler/jvmci/compilerToVM/ReprofileTest.java: > ---------------------- > STDERR: > java.lang.RuntimeException: public int > compiler.jvmci.compilerToVM.DummyClass.dummyInstanceFunction() : > profiling info wasn't changed after 1500 invocations: expected > DefaultProfilingInfo<> to not equal DefaultProfilingInfo<> > at jdk.test.lib.Asserts.fail(Asserts.java:594) > at jdk.test.lib.Asserts.assertNotEquals(Asserts.java:394) > at jdk.test.lib.Asserts.assertNE(Asserts.java:367) > at compiler.jvmci.compilerToVM.ReprofileTest.runSanityTest(ReprofileTest.java:101) > at java.base/java.util.ArrayList.forEach(ArrayList.java:1510) > at compiler.jvmci.compilerToVM.ReprofileTest.main(ReprofileTest.java:64) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:564) > at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127) > at java.base/java.lang.Thread.run(Thread.java:832) > > JavaTest Message: Test threw exception: java.lang.RuntimeException: > public int compiler.jvmci.compilerToVM.DummyClass.dummyInstanceFunction() > : profiling info wasn't changed after 1500 invocations: expected > DefaultProfilingInfo<> to not equal DefaultProfilingInfo<> > JavaTest Message: shutting down test > ---------------------- Those tests don't look like they handle a C1-only configuration to me. I'd just file a bug and let the JVMCI folk deal with that. >> >> Do I need do some other tests? >> >> applications/ctw tests definitely use the modified methods, so I'd recommend you to run one tests from there (e.g. applications/ctw/modules/java_base.java) > > I will also do these tests. > >> >> regarding your patch, 'cl_min' name seems to be confusion (at least to me) as it actually works as an upper bound for valid compilation levels. also given you had to change the same code in 3 different places, wouldn't it be better to factor that out? so you'll have smth like: >> >> + if (compLevel < CompLevel_none || compLevel > highestCompLevel()) { >> return false; // Intrinsic is not available on a non-existent compilation level. >> } > > like this: http://cr.openjdk.java.net/~aoqi/8239423/webrev.02/ ? That looks good to me! Thanks, David > Thanks, > Ao Qi > >> >> >> I'd like to cycle back on David's idea ("I actually think it is a bug that the VM allows you to set TieredStopAtLevel even though it is ignored on those configurations") Isn't the actual problem in how TieredStopAtLevel flag is handled in cases when TieredCompilation is disabled and/or JVM is built w/o tiered compilation support? I mean if JVM ensured that in such cases TieredStopAtLevel is set to CompLevel_highest_tier, all these WB methods would be correct. >> >> >> Thanks, >> -- Igor >> >> On Feb 24, 2020, at 3:39 AM, Ao Qi wrote: >> >> On Fri, Feb 21, 2020 at 10:42 AM Ao Qi wrote: >> >> >> We need to find all tests that use the modified WB functions and see if >> they all pass under normal and non-tiered configurations. >> >> >> I found it's hard to find all the tests that use the modified >> functions, because not all the tests use the modified functions >> directly. I try to find them by two ways: >> - I grep all modified functions' name in test dir and found all the >> tests contain the names. Some found files are not jtreg tests. They >> are used by other tests or libs. I also grep these files' name to find >> tests that use these files. However I missed the situation that other >> libs use these files. >> - I added some tty->print at the modified positions and run hotspot >> (hotspot_all_no_apps), jdk (jdk_stable) and langtools (jdk lib tools) >> tests. Then I found all jtr files which contain the added print. >> The test list found by the above two ways are here: >> http://cr.openjdk.java.net/~aoqi/8239423/all.txt . These tests passed >> with default and -XX:-TieredCompilation parameter and passed under >> --with-jvm-features=-compiler1 configuration. I also tested them with >> c1-only configuration. 2 failures were found, but regression were not >> found. >> >> I also tested tier1-3, both release and fastdebug, with default and >> -XX:-TieredCompilation parameter. >> >> Do I need do some other tests? >> >> Cheers, >> Ao Qi >> >> >> >> >> >> >> >> >> >> I agree. I will try to find these tests. >> >> Cheers, >> Ao Qi >> >> >> Thanks, >> David >> >> >> > From igor.ignatyev at oracle.com Wed Feb 26 03:40:31 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 25 Feb 2020 19:40:31 -0800 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: <5af5cfe4-e235-f0c0-cb57-a9e1254be642@oracle.com> References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> <4ac31ff3-df28-c3c4-5b63-9c2849716d00@oracle.com> <8982aeb0-3e12-f92f-a199-eb4ee1af1af9@oracle.com> <1a389351-3b80-5dc6-5712-6b4a5e018403@oracle.com> <5af5cfe4-e235-f0c0-cb57-a9e1254be642@oracle.com> Message-ID: Hi, > Those tests don't look like they handle a C1-only configuration to me. I'd just file a bug and let the JVMCI folk deal with that. I concur, actually I'm surprised that these tests weren't filtered out in c1-only, they require JVM under test to have jvmci feature which IIRC you weren't able to include if you don't have c2. anyhow, these tests expect c2 or graal compiler to be available, so please go ahead and file a bug. >> like this: http://cr.openjdk.java.net/~aoqi/8239423/webrev.02/ ? > That looks good to me! +1 -- Igor > On Feb 25, 2020, at 5:51 PM, David Holmes wrote: > > Hi, > > On 25/02/2020 6:34 pm, Ao Qi wrote: >> On Tue, Feb 25, 2020 at 12:42 AM Igor Ignatyev wrote: >>> >>> Hi >>> >>> I found it's hard to find all the tests that use the modified >>> functions, because not all the tests use the modified functions >>> directly. >>> >>> >>> I'd just change the functions in question to crash JVM, and then all the tests which start to fail are the tests which use these functions directly or indirectly. that's quite similar to your 2nd way. >>> I'll be very surprised if there are any non hotspot tests (modulo jfr) which use this API, so I'd just run hotspot and jdk/jfr tests. >> Thanks, Igor. I am doing. I selected tests with TEST="test/hotspot >> test/jdk/jdk/jfr". It would take some time. >>> >>> I also tested them with c1-only configuration. 2 failures were >>> >>> found, but regression were not found. >>> >>> what are these two failures? >> Their names and part of stderr are in below. >> test/hotspot/jtreg/compiler/jvmci/compilerToVM/IsMatureVsReprofileTest.java: >> ---------------------- >> STDERR: >> java.lang.RuntimeException: Multiple times invoked method should have >> MDO: expected 0 to not equal 0 >> at jdk.test.lib.Asserts.fail(Asserts.java:594) >> at jdk.test.lib.Asserts.assertNotEquals(Asserts.java:394) >> at jdk.test.lib.Asserts.assertNE(Asserts.java:367) >> at compiler.jvmci.compilerToVM.IsMatureVsReprofileTest.test(IsMatureVsReprofileTest.java:82) >> at compiler.jvmci.compilerToVM.IsMatureVsReprofileTest.main(IsMatureVsReprofileTest.java:65) >> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native >> Method) >> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.base/java.lang.reflect.Method.invoke(Method.java:564) >> at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127) >> at java.base/java.lang.Thread.run(Thread.java:832) >> JavaTest Message: Test threw exception: java.lang.RuntimeException: >> Multiple times invoked method should have MDO: expected 0 to not equal >> 0 >> JavaTest Message: shutting down test >> ---------------------- >> test/hotspot/jtreg/compiler/jvmci/compilerToVM/ReprofileTest.java: >> ---------------------- >> STDERR: >> java.lang.RuntimeException: public int >> compiler.jvmci.compilerToVM.DummyClass.dummyInstanceFunction() : >> profiling info wasn't changed after 1500 invocations: expected >> DefaultProfilingInfo<> to not equal DefaultProfilingInfo<> >> at jdk.test.lib.Asserts.fail(Asserts.java:594) >> at jdk.test.lib.Asserts.assertNotEquals(Asserts.java:394) >> at jdk.test.lib.Asserts.assertNE(Asserts.java:367) >> at compiler.jvmci.compilerToVM.ReprofileTest.runSanityTest(ReprofileTest.java:101) >> at java.base/java.util.ArrayList.forEach(ArrayList.java:1510) >> at compiler.jvmci.compilerToVM.ReprofileTest.main(ReprofileTest.java:64) >> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native >> Method) >> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.base/java.lang.reflect.Method.invoke(Method.java:564) >> at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127) >> at java.base/java.lang.Thread.run(Thread.java:832) >> JavaTest Message: Test threw exception: java.lang.RuntimeException: >> public int compiler.jvmci.compilerToVM.DummyClass.dummyInstanceFunction() >> : profiling info wasn't changed after 1500 invocations: expected >> DefaultProfilingInfo<> to not equal DefaultProfilingInfo<> >> JavaTest Message: shutting down test >> ---------------------- > > Those tests don't look like they handle a C1-only configuration to me. I'd just file a bug and let the JVMCI folk deal with that. > >>> >>> Do I need do some other tests? >>> >>> applications/ctw tests definitely use the modified methods, so I'd recommend you to run one tests from there (e.g. applications/ctw/modules/java_base.java) >> I will also do these tests. >>> >>> regarding your patch, 'cl_min' name seems to be confusion (at least to me) as it actually works as an upper bound for valid compilation levels. also given you had to change the same code in 3 different places, wouldn't it be better to factor that out? so you'll have smth like: >>> >>> + if (compLevel < CompLevel_none || compLevel > highestCompLevel()) { >>> return false; // Intrinsic is not available on a non-existent compilation level. >>> } >> like this: http://cr.openjdk.java.net/~aoqi/8239423/webrev.02/ ? > > That looks good to me! > > Thanks, > David > >> Thanks, >> Ao Qi >>> >>> >>> I'd like to cycle back on David's idea ("I actually think it is a bug that the VM allows you to set TieredStopAtLevel even though it is ignored on those configurations") Isn't the actual problem in how TieredStopAtLevel flag is handled in cases when TieredCompilation is disabled and/or JVM is built w/o tiered compilation support? I mean if JVM ensured that in such cases TieredStopAtLevel is set to CompLevel_highest_tier, all these WB methods would be correct. >>> >>> >>> Thanks, >>> -- Igor >>> >>> On Feb 24, 2020, at 3:39 AM, Ao Qi wrote: >>> >>> On Fri, Feb 21, 2020 at 10:42 AM Ao Qi wrote: >>> >>> >>> We need to find all tests that use the modified WB functions and see if >>> they all pass under normal and non-tiered configurations. >>> >>> >>> I found it's hard to find all the tests that use the modified >>> functions, because not all the tests use the modified functions >>> directly. I try to find them by two ways: >>> - I grep all modified functions' name in test dir and found all the >>> tests contain the names. Some found files are not jtreg tests. They >>> are used by other tests or libs. I also grep these files' name to find >>> tests that use these files. However I missed the situation that other >>> libs use these files. >>> - I added some tty->print at the modified positions and run hotspot >>> (hotspot_all_no_apps), jdk (jdk_stable) and langtools (jdk lib tools) >>> tests. Then I found all jtr files which contain the added print. >>> The test list found by the above two ways are here: >>> http://cr.openjdk.java.net/~aoqi/8239423/all.txt . These tests passed >>> with default and -XX:-TieredCompilation parameter and passed under >>> --with-jvm-features=-compiler1 configuration. I also tested them with >>> c1-only configuration. 2 failures were found, but regression were not >>> found. >>> >>> I also tested tier1-3, both release and fastdebug, with default and >>> -XX:-TieredCompilation parameter. >>> >>> Do I need do some other tests? >>> >>> Cheers, >>> Ao Qi >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> I agree. I will try to find these tests. >>> >>> Cheers, >>> Ao Qi >>> >>> >>> Thanks, >>> David From aoqi at loongson.cn Wed Feb 26 04:01:23 2020 From: aoqi at loongson.cn (Ao Qi) Date: Wed, 26 Feb 2020 12:01:23 +0800 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: <5af5cfe4-e235-f0c0-cb57-a9e1254be642@oracle.com> References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> <4ac31ff3-df28-c3c4-5b63-9c2849716d00@oracle.com> <8982aeb0-3e12-f92f-a199-eb4ee1af1af9@oracle.com> <1a389351-3b80-5dc6-5712-6b4a5e018403@oracle.com> <5af5cfe4-e235-f0c0-cb57-a9e1254be642@oracle.com> Message-ID: On Wed, Feb 26, 2020 at 9:51 AM David Holmes wrote: > > Hi, > > On 25/02/2020 6:34 pm, Ao Qi wrote: > > On Tue, Feb 25, 2020 at 12:42 AM Igor Ignatyev wrote: > >> > >> Hi > >> > >> I found it's hard to find all the tests that use the modified > >> functions, because not all the tests use the modified functions > >> directly. > >> > >> > >> I'd just change the functions in question to crash JVM, and then all the tests which start to fail are the tests which use these functions directly or indirectly. that's quite similar to your 2nd way. > >> I'll be very surprised if there are any non hotspot tests (modulo jfr) which use this API, so I'd just run hotspot and jdk/jfr tests. The tests list found by this way is http://cr.openjdk.java.net/~aoqi/8239423/all_found_by_crash.txt . All the tests passed with default and -XX:-TieredCompilation parameter, also passed under -compiler1 and -compiler2 configuration. > > > > Thanks, Igor. I am doing. I selected tests with TEST="test/hotspot > > test/jdk/jdk/jfr". It would take some time. > > > >> > >> I also tested them with c1-only configuration. 2 failures were > >> > >> found, but regression were not found. > >> > >> what are these two failures? > > > > Their names and part of stderr are in below. > > > > test/hotspot/jtreg/compiler/jvmci/compilerToVM/IsMatureVsReprofileTest.java: > > ---------------------- > > STDERR: > > java.lang.RuntimeException: Multiple times invoked method should have > > MDO: expected 0 to not equal 0 > > at jdk.test.lib.Asserts.fail(Asserts.java:594) > > at jdk.test.lib.Asserts.assertNotEquals(Asserts.java:394) > > at jdk.test.lib.Asserts.assertNE(Asserts.java:367) > > at compiler.jvmci.compilerToVM.IsMatureVsReprofileTest.test(IsMatureVsReprofileTest.java:82) > > at compiler.jvmci.compilerToVM.IsMatureVsReprofileTest.main(IsMatureVsReprofileTest.java:65) > > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > > Method) > > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.base/java.lang.reflect.Method.invoke(Method.java:564) > > at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127) > > at java.base/java.lang.Thread.run(Thread.java:832) > > > > JavaTest Message: Test threw exception: java.lang.RuntimeException: > > Multiple times invoked method should have MDO: expected 0 to not equal > > 0 > > JavaTest Message: shutting down test > > ---------------------- > > > > test/hotspot/jtreg/compiler/jvmci/compilerToVM/ReprofileTest.java: > > ---------------------- > > STDERR: > > java.lang.RuntimeException: public int > > compiler.jvmci.compilerToVM.DummyClass.dummyInstanceFunction() : > > profiling info wasn't changed after 1500 invocations: expected > > DefaultProfilingInfo<> to not equal DefaultProfilingInfo<> > > at jdk.test.lib.Asserts.fail(Asserts.java:594) > > at jdk.test.lib.Asserts.assertNotEquals(Asserts.java:394) > > at jdk.test.lib.Asserts.assertNE(Asserts.java:367) > > at compiler.jvmci.compilerToVM.ReprofileTest.runSanityTest(ReprofileTest.java:101) > > at java.base/java.util.ArrayList.forEach(ArrayList.java:1510) > > at compiler.jvmci.compilerToVM.ReprofileTest.main(ReprofileTest.java:64) > > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > > Method) > > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.base/java.lang.reflect.Method.invoke(Method.java:564) > > at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127) > > at java.base/java.lang.Thread.run(Thread.java:832) > > > > JavaTest Message: Test threw exception: java.lang.RuntimeException: > > public int compiler.jvmci.compilerToVM.DummyClass.dummyInstanceFunction() > > : profiling info wasn't changed after 1500 invocations: expected > > DefaultProfilingInfo<> to not equal DefaultProfilingInfo<> > > JavaTest Message: shutting down test > > ---------------------- > > Those tests don't look like they handle a C1-only configuration to me. > I'd just file a bug and let the JVMCI folk deal with that. > > >> > >> Do I need do some other tests? > >> > >> applications/ctw tests definitely use the modified methods, so I'd recommend you to run one tests from there (e.g. applications/ctw/modules/java_base.java) These tests are in the list. > > > > I will also do these tests. > > > >> > >> regarding your patch, 'cl_min' name seems to be confusion (at least to me) as it actually works as an upper bound for valid compilation levels. also given you had to change the same code in 3 different places, wouldn't it be better to factor that out? so you'll have smth like: > >> > >> + if (compLevel < CompLevel_none || compLevel > highestCompLevel()) { > >> return false; // Intrinsic is not available on a non-existent compilation level. > >> } > > > > like this: http://cr.openjdk.java.net/~aoqi/8239423/webrev.02/ ? > > That looks good to me! Thanks, David! Cheers, Ao Qi > > Thanks, > David > > > Thanks, > > Ao Qi > > > >> > >> > >> I'd like to cycle back on David's idea ("I actually think it is a bug that the VM allows you to set TieredStopAtLevel even though it is ignored on those configurations") Isn't the actual problem in how TieredStopAtLevel flag is handled in cases when TieredCompilation is disabled and/or JVM is built w/o tiered compilation support? I mean if JVM ensured that in such cases TieredStopAtLevel is set to CompLevel_highest_tier, all these WB methods would be correct. > >> > >> > >> Thanks, > >> -- Igor > >> > >> On Feb 24, 2020, at 3:39 AM, Ao Qi wrote: > >> > >> On Fri, Feb 21, 2020 at 10:42 AM Ao Qi wrote: > >> > >> > >> We need to find all tests that use the modified WB functions and see if > >> they all pass under normal and non-tiered configurations. > >> > >> > >> I found it's hard to find all the tests that use the modified > >> functions, because not all the tests use the modified functions > >> directly. I try to find them by two ways: > >> - I grep all modified functions' name in test dir and found all the > >> tests contain the names. Some found files are not jtreg tests. They > >> are used by other tests or libs. I also grep these files' name to find > >> tests that use these files. However I missed the situation that other > >> libs use these files. > >> - I added some tty->print at the modified positions and run hotspot > >> (hotspot_all_no_apps), jdk (jdk_stable) and langtools (jdk lib tools) > >> tests. Then I found all jtr files which contain the added print. > >> The test list found by the above two ways are here: > >> http://cr.openjdk.java.net/~aoqi/8239423/all.txt . These tests passed > >> with default and -XX:-TieredCompilation parameter and passed under > >> --with-jvm-features=-compiler1 configuration. I also tested them with > >> c1-only configuration. 2 failures were found, but regression were not > >> found. > >> > >> I also tested tier1-3, both release and fastdebug, with default and > >> -XX:-TieredCompilation parameter. > >> > >> Do I need do some other tests? > >> > >> Cheers, > >> Ao Qi > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> I agree. I will try to find these tests. > >> > >> Cheers, > >> Ao Qi > >> > >> > >> Thanks, > >> David > >> > >> > >> > > From david.holmes at oracle.com Wed Feb 26 04:39:17 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 26 Feb 2020 14:39:17 +1000 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: References: <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> <4ac31ff3-df28-c3c4-5b63-9c2849716d00@oracle.com> <8982aeb0-3e12-f92f-a199-eb4ee1af1af9@oracle.com> <1a389351-3b80-5dc6-5712-6b4a5e018403@oracle.com> <5af5cfe4-e235-f0c0-cb57-a9e1254be642@oracle.com> Message-ID: <9dcf0c9d-9719-0670-f84b-8acf2eeace5c@oracle.com> I can sponsor this. Cheers, David On 26/02/2020 2:17 pm, Ao Qi wrote: > On Wed, Feb 26, 2020 at 11:40 AM Igor Ignatyev wrote: >> >> Hi, >> >> Those tests don't look like they handle a C1-only configuration to me. I'd just file a bug and let the JVMCI folk deal with that. >> >> I concur, actually I'm surprised that these tests weren't filtered out in c1-only, they require JVM under test to have jvmci feature which IIRC you weren't able to include if you don't have c2. anyhow, these tests expect c2 or graal compiler to be available, so please go ahead and file a bug. >> >> like this: http://cr.openjdk.java.net/~aoqi/8239423/webrev.02/ ? >> >> That looks good to me! >> >> +1 > > Thank you, Igor. > > Would someone please sponsor this patch? > > Thanks, > Ao Qi > From aoqi at loongson.cn Wed Feb 26 04:17:27 2020 From: aoqi at loongson.cn (Ao Qi) Date: Wed, 26 Feb 2020 12:17:27 +0800 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: References: <063c45f6-b8f1-9862-c207-40be94027439@oracle.com> <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> <4ac31ff3-df28-c3c4-5b63-9c2849716d00@oracle.com> <8982aeb0-3e12-f92f-a199-eb4ee1af1af9@oracle.com> <1a389351-3b80-5dc6-5712-6b4a5e018403@oracle.com> <5af5cfe4-e235-f0c0-cb57-a9e1254be642@oracle.com> Message-ID: On Wed, Feb 26, 2020 at 11:40 AM Igor Ignatyev wrote: > > Hi, > > Those tests don't look like they handle a C1-only configuration to me. I'd just file a bug and let the JVMCI folk deal with that. > > I concur, actually I'm surprised that these tests weren't filtered out in c1-only, they require JVM under test to have jvmci feature which IIRC you weren't able to include if you don't have c2. anyhow, these tests expect c2 or graal compiler to be available, so please go ahead and file a bug. > > like this: http://cr.openjdk.java.net/~aoqi/8239423/webrev.02/ ? > > That looks good to me! > > +1 Thank you, Igor. Would someone please sponsor this patch? Thanks, Ao Qi From aoqi at loongson.cn Wed Feb 26 05:23:01 2020 From: aoqi at loongson.cn (Ao Qi) Date: Wed, 26 Feb 2020 13:23:01 +0800 Subject: RFR: 8239423: [TESTBUG] jdk/jfr/jvm/TestJFRIntrinsic.java failed with -XX:-TieredCompilation In-Reply-To: <9dcf0c9d-9719-0670-f84b-8acf2eeace5c@oracle.com> References: <630a7817-a3f0-77d4-a054-3bdc4c5ce7cb@oracle.com> <0f2b9cc2-6e81-674e-93bb-f0fd74453471@oracle.com> <4ac31ff3-df28-c3c4-5b63-9c2849716d00@oracle.com> <8982aeb0-3e12-f92f-a199-eb4ee1af1af9@oracle.com> <1a389351-3b80-5dc6-5712-6b4a5e018403@oracle.com> <5af5cfe4-e235-f0c0-cb57-a9e1254be642@oracle.com> <9dcf0c9d-9719-0670-f84b-8acf2eeace5c@oracle.com> Message-ID: Thank you, David! Cheer, Ao Qi On Wed, Feb 26, 2020 at 12:39 PM David Holmes wrote: > > I can sponsor this. > > Cheers, > David > > On 26/02/2020 2:17 pm, Ao Qi wrote: > > On Wed, Feb 26, 2020 at 11:40 AM Igor Ignatyev wrote: > >> > >> Hi, > >> > >> Those tests don't look like they handle a C1-only configuration to me. I'd just file a bug and let the JVMCI folk deal with that. > >> > >> I concur, actually I'm surprised that these tests weren't filtered out in c1-only, they require JVM under test to have jvmci feature which IIRC you weren't able to include if you don't have c2. anyhow, these tests expect c2 or graal compiler to be available, so please go ahead and file a bug. > >> > >> like this: http://cr.openjdk.java.net/~aoqi/8239423/webrev.02/ ? > >> > >> That looks good to me! > >> > >> +1 > > > > Thank you, Igor. > > > > Would someone please sponsor this patch? > > > > Thanks, > > Ao Qi > > From serguei.spitsyn at oracle.com Wed Feb 26 07:38:49 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 25 Feb 2020 23:38:49 -0800 Subject: RFR [XS]: 8239462: jdk.hotspot.agent misses some ReleaseStringUTFChars calls in case of early returns In-Reply-To: References: <6ed0d004-3e8c-4b27-f583-06dbacf45173@oracle.com> Message-ID: <5ae2b1d1-f01b-1112-8400-cd9ea6d6cd14@oracle.com> Hi Matthias, LGTM++ Thanks, Serguei On 2/25/20 11:30, Alex Menkov wrote: > Hi Matthias, > > LGTM > > --alex > > On 02/25/2020 08:20, Baesken, Matthias wrote: >> >> New webrev : >> >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.3/ >> >> >> Best regards, Matthias >> >> >>> IMO the solution with goto makes it even worse. >>> If you don't want to introduce the wrapper, could you please restore >>> changes in LinuxDebuggerLocal_attach0 from webrev.1 >>> >>> --alex >>> >>> On 02/21/2020 00:32, Baesken, Matthias wrote: >>>> Hi Alex , >>>> >>>> new webrev : >>>> >>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8239462.2/ >>>> >>>> Best Regards, Matthias >>>> >>>> >>>>> >>>>> Hi Matthias, >>>>> >>>>> Looks good in general, but I think it makes sense to fix #2 cases (at >>>>> least I see them in LinuxDebuggerLocal). If GetStringUTFChars >>>>> fails, the >>>>> code will crash. >>>>> Also I see GetStringUTFChars(str, JNI_FALSE). This look bad as well - >>>>> 2nd arg is a pointer, so it should be NULL or nullptr. >>>>> >>>>> As for #1 and #3 - AFAIU they are both right ways. >>>>> If GetStringUTFChars fails, it throws OOM and return NULL. >>>>> >>>>> And one more thing to consider. >>>>> LinuxDebuggerLocal_attach0 function looks terrible - 7 >>>>> ReleaseStringUTFChars calls for 2 GetStringUTFChars. >>>>> Maybe it make sense to introduce simple wrapper like >>>>> AutoJavaString in >>>>> src/jdk.hotspot.agent/windows/native/libsaproc/sawindbg.cpp >>>>> It would make the code simpler and less error prone. >>>>> >>>>> --alex >>>>> >>>> From zgu at redhat.com Wed Feb 26 22:16:40 2020 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 26 Feb 2020 17:16:40 -0500 Subject: [15] RFR 8238633: JVMTI heap walk should consult GC for marking oops In-Reply-To: <3d8553da-f7a8-fe84-d923-b10c5b47775e@redhat.com> References: <940D46CE-22D6-479B-B8EF-65FFA896C4A0@oracle.com> <3d8553da-f7a8-fe84-d923-b10c5b47775e@redhat.com> Message-ID: <701adbd0-d66d-1002-d2e3-77ea02847613@redhat.com> Hi, >> So perhaps just adding the NULL check in the barrier for the case >> where the markWord ?is_marked? is the sane thing to do, knowing that >> the other costs taken in the same path will dominate. > > I have this patch, exactly what you suggested. I will let Aleksey run > his numbers. Aleksey is happy with the solution, as we are able to split compiler barrier and runtime barrier. Therefore, I would like to withdraw this CR. Thanks all of you! -Zhengyu From huangjia at loongson.cn Fri Feb 28 13:08:14 2020 From: huangjia at loongson.cn (Jia Huang) Date: Fri, 28 Feb 2020 21:08:14 +0800 Subject: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp failed when the number of tests is greater than or equal to 2 Message-ID: <64960cc9-5021-d265-ab30-9cf9a91cc6e3@loongson.cn> Hi all, JBS: https://bugs.openjdk.java.net/browse/JDK-8240222 Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ gtest/jfr/test_networkUtilization.cpp failed when the number of tests is greater than or equal to 2. As shown in the following code, the test failed because "traceid id" is a static local variable. When the test runs a second time, "id" continues to grow, resulting in the return value of "i" being "_interfaces.end()". ---------------------------------------------------------------------------- test/hotspot/gtest/jfr/test_networkUtilization.cpp static const MockNetworkInterface& get_interface(traceid id) { ? std::list::const_iterator i = _interfaces.begin(); ? for (; i != _interfaces.end(); ++i) { ??? if (i->id == id) { ????? break; ??? } ? } ? return *i; } ---------------------------------------------------------------------------- Testing: ??? - make run-test TEST=gtest:JfrTestNetworkUtilization GTEST=REPEAT=2 CONF=server-release ??? - make run-test TEST=gtest:JfrTestNetworkUtilization GTEST=REPEAT=2 CONF=server-fastdebug Thanks a lot Best regards, Jia From david.holmes at oracle.com Fri Feb 28 13:16:26 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 28 Feb 2020 23:16:26 +1000 Subject: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp failed when the number of tests is greater than or equal to 2 In-Reply-To: <64960cc9-5021-d265-ab30-9cf9a91cc6e3@loongson.cn> References: <64960cc9-5021-d265-ab30-9cf9a91cc6e3@loongson.cn> Message-ID: <8f143c78-6bba-5f11-21ad-3096115287e1@oracle.com> Hi Jia, On 28/02/2020 11:08 pm, Jia Huang wrote: > Hi all, > > JBS: https://bugs.openjdk.java.net/browse/JDK-8240222 > Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ > > gtest/jfr/test_networkUtilization.cpp failed when the number of tests is > greater than or equal to 2. > As shown in the following code, the test failed because "traceid id" is > a static local variable. > When the test runs a second time, "id" continues to grow, resulting in > the return value of "i" being "_interfaces.end()". Interesting! I'm not at all familiar with the details of gtest and how it actually runs things. But I would expect that we have many tests that rely on starting from a cleanly initialized VM state, or test state, and so can't simply be repeated (it would be like editing an arbitrary test and putting the body of main into a for-loop!). So I'm not sure what the right way to fix this is: - don't use REPEAT? - declare the test as not repeatable somehow? - change the test to make it repeatable? Cheers, David > ---------------------------------------------------------------------------- > > test/hotspot/gtest/jfr/test_networkUtilization.cpp > > static const MockNetworkInterface& get_interface(traceid id) { > ? std::list::const_iterator i = _interfaces.begin(); > ? for (; i != _interfaces.end(); ++i) { > ??? if (i->id == id) { > ????? break; > ??? } > ? } > ? return *i; > } > ---------------------------------------------------------------------------- > > > > Testing: > ??? - make run-test TEST=gtest:JfrTestNetworkUtilization GTEST=REPEAT=2 > CONF=server-release > ??? - make run-test TEST=gtest:JfrTestNetworkUtilization GTEST=REPEAT=2 > CONF=server-fastdebug > > Thanks a lot > > Best regards, > Jia > From sgehwolf at redhat.com Fri Feb 28 16:42:13 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 28 Feb 2020 17:42:13 +0100 Subject: RFR(S): 8239785: Cgroups: Incorrect detection logic on old systems in hotspot In-Reply-To: <2A934185-9B81-4371-B985-F0C3C39B3FD3@oracle.com> References: <362678e8bc8231c205c6b3327d08dd5179154e23.camel@redhat.com> <2A934185-9B81-4371-B985-F0C3C39B3FD3@oracle.com> Message-ID: <215e52601b2da8600cbfd47ef73634720a7736d2.camel@redhat.com> Hi Bob, Thanks for the review! On Tue, 2020-02-25 at 15:08 -0500, Bob Vandette wrote: > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.sdiff.html > > If both cgroupv1 and v2 controllers are mounted, you might have some problems. You might want to only look for > cgroup2 mounts if is_cgroupV2 is true otherwise only look for cgroup mounts. OK. > 194 cg_infos[i]._mount_path = os::strdup(tmp_mount_point); > This will stomp the pointer: > 213 cg_infos[memory_idx]._mount_path = os::strdup(tmpmount); You are right. If we are on a hybrid hierarchy and for some reason the cgroup v2 mount shows up *after* cgroup mounts in /proc/self/mountinfo, we'd get the mount path wrong. So ordering matters! The reason this worked for me on hybrid is that cgroup2 mounts come *before* cgroup mounts on the system I've tested. > You should free all strdup?d pointers in cg_infos in places where you fail and return NULL. > > 81 cg_infos[memory_idx]._name = os::strdup(name); > 213 cg_infos[memory_idx]._mount_path = os::strdup(tmpmount); > 214 cg_infos[memory_idx]._root_mount_path = os::strdup(tmproot); Done now. More thoughts? Updated webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/03/webrev/ Thanks, Severin > > On Feb 25, 2020, at 2:22 PM, Severin Gehwolf wrote: > > > > Hi, > > > > Could I please get reviews of this cgroup-related patch? After JDK- > > 8230305 old systems with no mounted cgroup controllers would get > > detected as cgroups v2 (wrong). Then, when initializing the v2 > > subsystem, it would fail. The trace about cgroupv2 mount point not > > found is misleading in this case. While the outcome is similar pre/post > > patch (NULL cgroup subsystem), I'd like to be explicit about this case. > > > > The suggested fix is to look at /proc/self/mountinfo in order to > > correctly detect whether this is cgroup v2 or cgroup v1 with no mounted > > controllers. In the latter case we just stop initialization as we'd > > fail later in cgroupv1 code anyway. This aligns hotspot code with core- > > libs after JDK-8239559. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8239785 > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/ > > > > Testing: jdk-submit, hotspot docker tests on cgroup v1 and cgroup v2. All pass. > > > > Thoughts? > > > > Thanks, > > Severin > > From bob.vandette at oracle.com Fri Feb 28 20:08:23 2020 From: bob.vandette at oracle.com (Bob Vandette) Date: Fri, 28 Feb 2020 15:08:23 -0500 Subject: RFR(S): 8239785: Cgroups: Incorrect detection logic on old systems in hotspot In-Reply-To: <215e52601b2da8600cbfd47ef73634720a7736d2.camel@redhat.com> References: <362678e8bc8231c205c6b3327d08dd5179154e23.camel@redhat.com> <2A934185-9B81-4371-B985-F0C3C39B3FD3@oracle.com> <215e52601b2da8600cbfd47ef73634720a7736d2.camel@redhat.com> Message-ID: It still looks like the _mount_path can be corrupted if both cgroupv1 and cgroupv2 are mounted. You?ll only initialize the _mount_path if cgroupv2 is detected in the cgroups file but if cgroupv1 is also mounted, you?ll still clobber the _mount_path. Maybe you should just do the cgroupv2 processing if /proc/cgroups has cgroupv2 enabled and if that fails continue looking for cgroup v1 mounts. Do you really need to do this? The BSS section should be zeroed. 55 for (int i = 0; i < CG_INFO_LENGTH; i++) { 56 cg_infos[i]._name = NULL; 57 cg_infos[i]._cgroup_path = NULL; 58 cg_infos[i]._root_mount_path = NULL; 59 cg_infos[i]._mount_path = NULL; 60 } You don?t need to check for NULL in cleanup, os::free can handle NULL pointers. 339 void CgroupSubsystemFactory::cleanup(CgroupInfo* cg_infos) { 340 assert(cg_infos != NULL, "Invariant"); 341 for (int i = 0; i < CG_INFO_LENGTH; i++) { 342 if (cg_infos[i]._name != NULL) { 343 os::free(cg_infos[i]._name); 344 } 345 if (cg_infos[i]._cgroup_path != NULL) { 346 os::free(cg_infos[i]._cgroup_path); 347 } 348 if (cg_infos[i]._root_mount_path != NULL) { 349 os::free(cg_infos[i]._root_mount_path); 350 } 351 if (cg_infos[i]._mount_path != NULL) { 352 os::free(cg_infos[i]._mount_path); 353 } 354 } 355 } Bob. > On Feb 28, 2020, at 11:42 AM, Severin Gehwolf wrote: > > Hi Bob, > > Thanks for the review! > > On Tue, 2020-02-25 at 15:08 -0500, Bob Vandette wrote: >> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.sdiff.html >> >> If both cgroupv1 and v2 controllers are mounted, you might have some problems. You might want to only look for >> cgroup2 mounts if is_cgroupV2 is true otherwise only look for cgroup mounts. > > OK. > >> 194 cg_infos[i]._mount_path = os::strdup(tmp_mount_point); >> This will stomp the pointer: >> 213 cg_infos[memory_idx]._mount_path = os::strdup(tmpmount); > > You are right. If we are on a hybrid hierarchy and for some reason the > cgroup v2 mount shows up *after* cgroup mounts in /proc/self/mountinfo, > we'd get the mount path wrong. So ordering matters! The reason this > worked for me on hybrid is that cgroup2 mounts come *before* cgroup > mounts on the system I've tested. > >> You should free all strdup?d pointers in cg_infos in places where you fail and return NULL. >> >> 81 cg_infos[memory_idx]._name = os::strdup(name); >> 213 cg_infos[memory_idx]._mount_path = os::strdup(tmpmount); >> 214 cg_infos[memory_idx]._root_mount_path = os::strdup(tmproot); > > Done now. More thoughts? > > Updated webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/03/webrev/ > > Thanks, > Severin > >>> On Feb 25, 2020, at 2:22 PM, Severin Gehwolf wrote: >>> >>> Hi, >>> >>> Could I please get reviews of this cgroup-related patch? After JDK- >>> 8230305 old systems with no mounted cgroup controllers would get >>> detected as cgroups v2 (wrong). Then, when initializing the v2 >>> subsystem, it would fail. The trace about cgroupv2 mount point not >>> found is misleading in this case. While the outcome is similar pre/post >>> patch (NULL cgroup subsystem), I'd like to be explicit about this case. >>> >>> The suggested fix is to look at /proc/self/mountinfo in order to >>> correctly detect whether this is cgroup v2 or cgroup v1 with no mounted >>> controllers. In the latter case we just stop initialization as we'd >>> fail later in cgroupv1 code anyway. This aligns hotspot code with core- >>> libs after JDK-8239559. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8239785 >>> webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/ >>> >>> Testing: jdk-submit, hotspot docker tests on cgroup v1 and cgroup v2. All pass. >>> >>> Thoughts? >>> >>> Thanks, >>> Severin >>> > From kim.barrett at oracle.com Sat Feb 29 01:44:13 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 28 Feb 2020 20:44:13 -0500 Subject: RFR[T]: 8240246: Avoid cast_to_oop from char* Message-ID: <6AEAFE1F-8318-415C-AC4B-070D891FCE13@oracle.com> Please review this trivial change of a cast_to_oop argument from char* to address. This removes the only call of that function with a char* argument. CR: https://bugs.openjdk.java.net/browse/JDK-8240246 Webrev: https://cr.openjdk.java.net/~kbarrett/8240246/open.00/ Testing: mach5 tier1 From huangjia at loongson.cn Sat Feb 29 03:53:03 2020 From: huangjia at loongson.cn (Jia Huang) Date: Sat, 29 Feb 2020 11:53:03 +0800 Subject: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp failed when the number of tests is greater than or equal to 2 In-Reply-To: <8f143c78-6bba-5f11-21ad-3096115287e1@oracle.com> References: <64960cc9-5021-d265-ab30-9cf9a91cc6e3@loongson.cn> <8f143c78-6bba-5f11-21ad-3096115287e1@oracle.com> Message-ID: <4f6c2613-de5a-cc28-f208-1a763270a5d0@loongson.cn> Hi David, ? 2020/2/28 21:16, David Holmes ??: > Hi Jia, > > On 28/02/2020 11:08 pm, Jia Huang wrote: >> Hi all, >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8240222 >> Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ >> >> gtest/jfr/test_networkUtilization.cpp failed when the number of tests >> is greater than or equal to 2. >> As shown in the following code, the test failed because "traceid id" >> is a static local variable. >> When the test runs a second time, "id" continues to grow, resulting >> in the return value of "i" being "_interfaces.end()". > > Interesting! I'm not at all familiar with the details of gtest and how > it actually runs things. But I would expect that we have many tests > that rely on starting from a cleanly initialized VM state, or test > state, and so can't simply be repeated (it would be like editing an > arbitrary test and putting the body of main into a for-loop!). I'm sorry that my last description may be a little incomplete. JfrTestNetworkUtilization is a group of tests,it includes four individual tests: ? --JfrTestNetworkUtilization.RequestFunctionBasic_test_vm ? --JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm ? --JfrTestNetworkUtilization.InterfaceRemoved_test_vm ? --JfrTestNetworkUtilization.InterfaceReset_test_vm If you only run a individual test once, such as the following Testing, it still fails. The reason for the failure, as I said last time, is also because of static local variables. In the "RequestFunctionMultiple_test_vm" test, the sequence number of "eth0" is 2. However, the initial value of the static local variable "id" is 0. Testting: ?? --make run-test TEST=gtest:JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm GTEST=REPEAT=1 CONF=release ?? --make run-test TEST=gtest:JfrTestNetworkUtilization.InterfaceRemoved_test_vm GTEST=REPEAT=1 CONF=release ?? --make run-test TEST=gtest:JfrTestNetworkUtilization.InterfaceReset_test_vm GTEST=REPEAT=1 CONF=release > > So I'm not sure what the right way to fix this is: > - don't use REPEAT? > - declare the test as not repeatable somehow? > - change the test to make it repeatable? In the doc/testing.md or doc/testing.html, Gtest keywords inclue "REPEAT", so I think this means that all GTEST tests should be able to use the "REPEAT parameter. Best regards, Jia > > Cheers, > David > >> ---------------------------------------------------------------------------- >> >> test/hotspot/gtest/jfr/test_networkUtilization.cpp >> >> static const MockNetworkInterface& get_interface(traceid id) { >> ?? std::list::const_iterator i = >> _interfaces.begin(); >> ?? for (; i != _interfaces.end(); ++i) { >> ???? if (i->id == id) { >> ?????? break; >> ???? } >> ?? } >> ?? return *i; >> } >> ---------------------------------------------------------------------------- >> >> >> >> Testing: >> ???? - make run-test TEST=gtest:JfrTestNetworkUtilization >> GTEST=REPEAT=2 CONF=server-release >> ???? - make run-test TEST=gtest:JfrTestNetworkUtilization >> GTEST=REPEAT=2 CONF=server-fastdebug >> >> Thanks a lot >> >> Best regards, >> Jia >> From david.holmes at oracle.com Sat Feb 29 05:11:21 2020 From: david.holmes at oracle.com (David Holmes) Date: Sat, 29 Feb 2020 15:11:21 +1000 Subject: RFR[T]: 8240246: Avoid cast_to_oop from char* In-Reply-To: <6AEAFE1F-8318-415C-AC4B-070D891FCE13@oracle.com> References: <6AEAFE1F-8318-415C-AC4B-070D891FCE13@oracle.com> Message-ID: <3c8258e8-81ba-87a5-f3c7-18bc45fd3f71@oracle.com> Hi Kim, On 29/02/2020 11:44 am, Kim Barrett wrote: > Please review this trivial change of a cast_to_oop argument from char* > to address. This removes the only call of that function with a char* > argument. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8240246 > > Webrev: > https://cr.openjdk.java.net/~kbarrett/8240246/open.00/ Okay - trivial change. But any reason not to simply use uint8_t directly? Thanks, David > Testing: > mach5 tier1 > > From kim.barrett at oracle.com Sat Feb 29 08:24:06 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 29 Feb 2020 03:24:06 -0500 Subject: RFR[T]: 8240246: Avoid cast_to_oop from char* In-Reply-To: <3c8258e8-81ba-87a5-f3c7-18bc45fd3f71@oracle.com> References: <6AEAFE1F-8318-415C-AC4B-070D891FCE13@oracle.com> <3c8258e8-81ba-87a5-f3c7-18bc45fd3f71@oracle.com> Message-ID: > On Feb 29, 2020, at 12:11 AM, David Holmes wrote: > > Hi Kim, > > On 29/02/2020 11:44 am, Kim Barrett wrote: >> Please review this trivial change of a cast_to_oop argument from char* >> to address. This removes the only call of that function with a char* >> argument. >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8240246 >> Webrev: >> https://cr.openjdk.java.net/~kbarrett/8240246/open.00/ > > Okay - trivial change. But any reason not to simply use uint8_t directly? The desired type here is ?address?. These are in globalDefinitions.hpp: typedef unsigned char u_char; typedef u_char* address; I could have used RemovePointer
::type :) > > Thanks, > David > >> Testing: >> mach5 tier1 From kim.barrett at oracle.com Sat Feb 29 08:35:48 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 29 Feb 2020 03:35:48 -0500 Subject: RFR[T]: 8240246: Avoid cast_to_oop from char* In-Reply-To: References: <6AEAFE1F-8318-415C-AC4B-070D891FCE13@oracle.com> <3c8258e8-81ba-87a5-f3c7-18bc45fd3f71@oracle.com> Message-ID: > On Feb 29, 2020, at 3:24 AM, Kim Barrett wrote: > > I could have used RemovePointer
::type :) Of course, that doesn?t work. I should know better than to write anything code-like this late; sleepy? From kim.barrett at oracle.com Sat Feb 29 08:37:47 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 29 Feb 2020 03:37:47 -0500 Subject: RFR[T]: 8240246: Avoid cast_to_oop from char* In-Reply-To: References: <6AEAFE1F-8318-415C-AC4B-070D891FCE13@oracle.com> <3c8258e8-81ba-87a5-f3c7-18bc45fd3f71@oracle.com> Message-ID: > On Feb 29, 2020, at 3:35 AM, Kim Barrett wrote: > >> On Feb 29, 2020, at 3:24 AM, Kim Barrett wrote: >> >> I could have used RemovePointer
::type :) > > Of course, that doesn?t work. I should know better than to write anything code-like this late; sleepy? Or maybe it does. Yeah, I?m definitely too tired for this. Just ignore all my replies.