From kim.barrett at oracle.com Sun Dec 1 23:10:53 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 1 Dec 2019 18:10:53 -0500 Subject: RFR: 8234779: Provide idiom for declaring classes noncopyable In-Reply-To: References: <233a779d-ae3b-4856-9c44-dd81bfceab6e@oracle.com> <129579A4-01C4-4F62-9582-06BE19CB13C0@oracle.com> Message-ID: > On Nov 29, 2019, at 6:04 PM, John Rose wrote: > > On Nov 27, 2019, at 1:49 PM, Kim Barrett wrote: >> >> The proposed macro significantly reduces that wordiness. Far more >> importantly, it makes the intent entirely self-evident; there's no >> need for any explanatory comments. > > Or to put it another way, the explanatory comments can be centralized > in the header file which defines the macro. And the macro can be given > a name which explains the intent. The name and comments can reflect > HotSpot-specific ?house rules? (local design rules and conventions). > > On Nov 29, 2019, at 1:45 PM, Kim Barrett wrote: >> ? I also like putting repetitive code behind names to make it >> easier to chunk and understand. > > +1 A well-chosen macro name can be easier to read than a chunk of boilerplate. > This is especially applicable to us since C++ boilerplate evolves over time, and > as a highly portable system we don?t have the ability to track one particular > dialect of C++. But even if we did, we'd still have complex ?house rules? to > enforce and document, and macros play a role there. > > I don?t think that learning the ?house macros? for HotSpot is an excessive > burden for people learning to work on HotSpot. Kim?s proposal seems to > be yet another one of these macros. > > Thanks, Kim and Per, for marshaling the arguments pro and con. > > ? John Thanks John. I've also decided to take your earlier advice (from off-list) to use an external terminating semicolon, rather than having it in the macro expansion (webrev updated in place). Besides potentially helping editors indent properly (which is not a problem for at least some editors), some uses looked very odd in context without a trailing semicolon. (The location of the terminating semicolon is rendered moot by C++11, but I can't prove all compilers currently used to build JDK support empty declarations in classes.) While I prefer macros whose expansions are "complete" (where that makes sense for the macro), that's not a strong preference. From david.holmes at oracle.com Mon Dec 2 00:27:55 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 2 Dec 2019 10:27:55 +1000 Subject: RFR: 8234796: Refactor Handshake::execute to take a HandshakeOperation In-Reply-To: <1745284a-545f-7bf5-533a-945d917cfe08@oracle.com> References: <1ff4ff37-c65e-43e7-a845-bc6ce06750f0@oracle.com> <9dfd8c82-a7b2-fe4d-4631-31d0d5b29ff0@oracle.com> <5e7f1ea4-147b-a7cc-2aed-7993adcb6cfb@oracle.com> <1745284a-545f-7bf5-533a-945d917cfe08@oracle.com> Message-ID: <2932f39a-3ce3-b4bf-0e4e-e0103b6f7c05@oracle.com> v3 and v4 updates still appear fine. Thanks, David On 29/11/2019 9:52 pm, Robbin Ehn wrote: > Hi, Shenandoah just added a handshake, here is the additional fix. > > http://cr.openjdk.java.net/~rehn/8234796/v4/full/ > http://cr.openjdk.java.net/~rehn/8234796/v4/inc/ > > Thanks, Robbin > > On 11/28/19 3:29 PM, Robbin Ehn wrote: >> Thanks David! >> >> Since I had no compile issues, fixing includes for the ThreadClosure >> move slipped my mind. >> >> Inc: >> http://cr.openjdk.java.net/~rehn/8234796/v3/inc/webrev/ >> Full: >> http://cr.openjdk.java.net/~rehn/8234796/v3/full/webrev/ >> >> Built fastdebug and release for x64 win/lin/osx, aarch64, ppc, >> solaris-sprac. >> And without precompiled header locally. >> arm32 and x86 have prior build issues and do not compile. >> >> Thanks, Robbin >> >> On 2019-11-28 07:21, David Holmes wrote: >>> Hi Robbin, >>> >>> On 28/11/2019 1:25 am, Robbin Ehn wrote: >>>> Hi all, please review. >>>> >>>> Here is the result after Per's suggestion: >>>> http://cr.openjdk.java.net/~rehn/8234796/v2/full/webrev/index.html >>>> (incremental made no sense) >>>> >>>> Due to circular dependency between thread.hpp and handshake.hpp, I >>>> moved the ThreadClosure to iterator.hpp, as was suggested offline. >>> >>> That all looks good to me! Thanks for splitting these up. >>> >>> Thanks, >>> David >>> ----- >>> >>>> Passes t1-3 >>>> >>>> Thanks, Robbin >>>> >>>> On 11/26/19 2:07 PM, Robbin Ehn wrote: >>>>> Hi all, please review. >>>>> >>>>> Issue: >>>>> https://bugs.openjdk.java.net/browse/JDK-8234796 >>>>> Code: >>>>> http://cr.openjdk.java.net/~rehn/8234796/full/webrev/ >>>>> >>>>> The handshake code needs more information about the handshake >>>>> operation. >>>>> We change type from ThreadClosure to HandshakeOperation in >>>>> Handshake::execute. >>>>> This enables us to add more details to the HandshakeOperation as >>>>> needed going forward. >>>>> >>>>> Tested t1 and t1-3 together with the logging improvements in 8234742. >>>>> >>>>> It was requested that "HandshakeOperation()" would take the name >>>>> instead having "virtual const char* name();". Which is in this patch. >>>>> >>>>> Thanks, Robbin From david.holmes at oracle.com Mon Dec 2 01:01:27 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 2 Dec 2019 11:01:27 +1000 Subject: 8234397: add OS uptime information to os::print_os_info output In-Reply-To: References: <30E06543-E081-429B-8293-8CA81D1F6870@sap.com> Message-ID: <038f060e-b253-e33f-bc03-a56c0735a90f@oracle.com> Hi Matthias, On 29/11/2019 11:58 pm, Baesken, Matthias wrote: > Hi, here is a new webrev, this time with nicer output (see os.cpp) . > > http://cr.openjdk.java.net/~mbaesken/webrevs/8234397.3/ src/hotspot/os/bsd/os_bsd.cpp This link may be more general than the Apple "iOS" one. https://man.openbsd.org/sysctl.2 + time_t bootsec = boottime.tv_sec, currsec = time(NULL); Please write the two declarations on separate lines. --- src/hotspot/os/posix/os_posix.cpp + // unfortunately it does not work on macOS Still no mention of Linux --- src/hotspot/share/runtime/os.cpp + if (minutes < 10) { + st->print_cr("%s %llu days %llu:0%llu hours", startStr, days, hours, minutes); + } else { + st->print_cr("%s %llu days %llu:%llu hours", startStr, days, hours, minutes); + } You should just use a zero-padded width of 2 to get 01..09 for minutes. Is %llu supported by all compilers (32-bit as well)? --- src/hotspot/os/windows/os_windows.cpp + unsigned long long ticks = GetTickCount64(); + os::print_dhm(st, "OS uptime:", ticks); GetTickCount64 returns milliseconds since boot not seconds. --- Thanks, David > Best regards, Matthias > > >> -----Original Message----- >> From: Langer, Christoph >> Sent: Donnerstag, 28. November 2019 11:55 >> To: Baesken, Matthias ; Schmidt, Lutz >> ; David Holmes ; >> 'hotspot-dev at openjdk.java.net' >> Subject: RE: 8234397: add OS uptime information to os::print_os_info output >> >> Hi Matthias, >> >> I'd like to see the uptime information in hs_err files. >> >> I, however, would rather like to see a more readable output like "OS uptime >> 10 days 3:10". I understand that's some more formatting effort but on the >> other hand you'd not need floating point calculations. >> >> As for os_windows.cpp: Why don't you spend a >> os::win32::print_uptime_info(st); method to align with the other >> implementations? >> >> Best regards >> Christoph > From per.liden at oracle.com Mon Dec 2 09:04:57 2019 From: per.liden at oracle.com (Per Liden) Date: Mon, 2 Dec 2019 10:04:57 +0100 Subject: RFR: 8234779: Provide idiom for declaring classes noncopyable In-Reply-To: References: <233a779d-ae3b-4856-9c44-dd81bfceab6e@oracle.com> <129579A4-01C4-4F62-9582-06BE19CB13C0@oracle.com> Message-ID: Hi Kim & John, On 12/2/19 12:10 AM, Kim Barrett wrote: >> On Nov 29, 2019, at 6:04 PM, John Rose wrote: >> >> On Nov 27, 2019, at 1:49 PM, Kim Barrett wrote: >>> >>> The proposed macro significantly reduces that wordiness. Far more >>> importantly, it makes the intent entirely self-evident; there's no >>> need for any explanatory comments. >> >> Or to put it another way, the explanatory comments can be centralized >> in the header file which defines the macro. And the macro can be given >> a name which explains the intent. The name and comments can reflect >> HotSpot-specific ?house rules? (local design rules and conventions). >> >> On Nov 29, 2019, at 1:45 PM, Kim Barrett wrote: >>> ? I also like putting repetitive code behind names to make it >>> easier to chunk and understand. >> >> +1 A well-chosen macro name can be easier to read than a chunk of boilerplate. >> This is especially applicable to us since C++ boilerplate evolves over time, and >> as a highly portable system we don?t have the ability to track one particular >> dialect of C++. But even if we did, we'd still have complex ?house rules? to >> enforce and document, and macros play a role there. >> >> I don?t think that learning the ?house macros? for HotSpot is an excessive >> burden for people learning to work on HotSpot. Kim?s proposal seems to >> be yet another one of these macros. >> >> Thanks, Kim and Per, for marshaling the arguments pro and con. It seems I'm not going to convince anyone, so let's just agree to disagree. I withdraw my objection. >> >> ? John > > Thanks John. > > I've also decided to take your earlier advice (from off-list) to use > an external terminating semicolon, rather than having it in the macro > expansion (webrev updated in place). Besides potentially helping > editors indent properly (which is not a problem for at least some > editors), some uses looked very odd in context without a trailing +1. An external semicolon makes it look less foreign. I'd also like to suggest that this type of macro belongs in globalDefinitions.hpp rather than macros.hpp, since it feels more akin to ALWAYSINLINE/ATTRIBUTE_ALIGNED/etc than INCLUDE_JVMTI/X86_ONLY/etc. cheers, Per > semicolon. (The location of the terminating semicolon is rendered moot > by C++11, but I can't prove all compilers currently used to build JDK > support empty declarations in classes.) While I prefer macros whose > expansions are "complete" (where that makes sense for the macro), > that's not a strong preference. > From Joshua.Zhu at arm.com Mon Dec 2 09:10:20 2019 From: Joshua.Zhu at arm.com (Joshua Zhu (Arm Technology China)) Date: Mon, 2 Dec 2019 09:10:20 +0000 Subject: 8233948: AArch64: Incorrect mapping between OptoReg and VMReg for high 64 bits of Vector Register In-Reply-To: References: Message-ID: Hi Andrew Dinn, > I think this is a good start but there is more work to do to clean up method > RegisterSaver::save_live_registers defined in file sharedRuntime_aarch64.cpp. > It would be good to do that clean up as part of this patch so it is all consistent. Thanks for your review! > 128 class FloatRegisterImpl: public AbstractRegisterImpl { > 129 public: > 130 enum { > 131 number_of_registers = 32, > 132 max_slots_per_register = 4, > save_slots_per_register = 2, > extra_save_slots_per_register = 2 > > The 2 new tags are needed because sharedRuntime_aarch64.cpp normally > only saves 2 slots per register but it occasionally needs to save all 4. Make sense. Bottom 64 bits of SVE vector register still overlaps with floating-point register, Therefore I will make extra_save_slots_per_register = max_slots_per_register - save_slots_per_register; so that we just need to improve the value of max_slots_per_register for SVE in future. > 607 const int FPUStateSizeInWords = 32 * 2; > > So, that can now be redefined as > > 607 const int FPUStateSizeInWords = > FloatRegisterImpl::number_of_registers * > FloatRegisterImpl::save_slots_per_register; > > We then need to redefine the code at lines 108 - 110 to use the enum values: > > 108 rfp_off = r0_off + > (RegisterImpl::number_of_registers - 2) * > RegisterImpl::max_slots_per_register, > 109 return_off = rfp_off + > RegisterImpl::max_slots_per_register, // slot for return address > 110 reg_save_size = return_off + > RegisterImpl::max_slots_per_register}; Thanks for your reminder! Make sense. > Finally, we can method edit save_live_registers at the point where it allows > space for the extra vector register content. That needs to be updated to use > the relevant constants: > > 116 if (save_vectors) { > 117 // Save upper half of vector registers > 118 int vect_words = FloatRegisterImpl::number_of_registers * > FloatRegisterImpl::extra_save_slots_per_register; > 119 additional_frame_words += vect_words; Here the computation of vect_words should be: int vect_words = FloatRegisterImpl::number_of_registers * FloatRegisterImpl::extra_save_slots_per_register / VMRegImpl::slots_per_word; > Could you prepare a new webrev with these extra changes in and check it is > ok? Yes, of course. I updated webrev according to your review comments. http://cr.openjdk.java.net/~jzhu/8233948/webrev.01/ > Also, could you report what testing you did before and after your change > (other than checking the dump output). You will probably need to repeat it to > ensure these extra changes are ok. Oh, Sorry for that. I did not mention my testing in my previous mail (My initial RFR mail is too long). I ran full jtreg tests without new failure introduced. I will re-run it against the new webrev and will let you know the results. Thanks again for your review. Best Regards, Joshua From robbin.ehn at oracle.com Mon Dec 2 09:18:43 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 2 Dec 2019 10:18:43 +0100 Subject: RFR: 8234796: Refactor Handshake::execute to take a HandshakeOperation In-Reply-To: <2932f39a-3ce3-b4bf-0e4e-e0103b6f7c05@oracle.com> References: <1ff4ff37-c65e-43e7-a845-bc6ce06750f0@oracle.com> <9dfd8c82-a7b2-fe4d-4631-31d0d5b29ff0@oracle.com> <5e7f1ea4-147b-a7cc-2aed-7993adcb6cfb@oracle.com> <1745284a-545f-7bf5-533a-945d917cfe08@oracle.com> <2932f39a-3ce3-b4bf-0e4e-e0103b6f7c05@oracle.com> Message-ID: <85d9da18-c654-2e90-2be9-c9a3132afb9d@oracle.com> Thanks David! /Robbin On 2019-12-02 01:27, David Holmes wrote: > v3 and v4 updates still appear fine. > > Thanks, > David > > On 29/11/2019 9:52 pm, Robbin Ehn wrote: >> Hi, Shenandoah just added a handshake, here is the additional fix. >> >> http://cr.openjdk.java.net/~rehn/8234796/v4/full/ >> http://cr.openjdk.java.net/~rehn/8234796/v4/inc/ >> >> Thanks, Robbin >> >> On 11/28/19 3:29 PM, Robbin Ehn wrote: >>> Thanks David! >>> >>> Since I had no compile issues, fixing includes for the ThreadClosure move >>> slipped my mind. >>> >>> Inc: >>> http://cr.openjdk.java.net/~rehn/8234796/v3/inc/webrev/ >>> Full: >>> http://cr.openjdk.java.net/~rehn/8234796/v3/full/webrev/ >>> >>> Built fastdebug and release for x64 win/lin/osx, aarch64, ppc, solaris-sprac. >>> And without precompiled header locally. >>> arm32 and x86 have prior build issues and do not compile. >>> >>> Thanks, Robbin >>> >>> On 2019-11-28 07:21, David Holmes wrote: >>>> Hi Robbin, >>>> >>>> On 28/11/2019 1:25 am, Robbin Ehn wrote: >>>>> Hi all, please review. >>>>> >>>>> Here is the result after Per's suggestion: >>>>> http://cr.openjdk.java.net/~rehn/8234796/v2/full/webrev/index.html >>>>> (incremental made no sense) >>>>> >>>>> Due to circular dependency between thread.hpp and handshake.hpp, I moved >>>>> the ThreadClosure to iterator.hpp, as was suggested offline. >>>> >>>> That all looks good to me! Thanks for splitting these up. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> Passes t1-3 >>>>> >>>>> Thanks, Robbin >>>>> >>>>> On 11/26/19 2:07 PM, Robbin Ehn wrote: >>>>>> Hi all, please review. >>>>>> >>>>>> Issue: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8234796 >>>>>> Code: >>>>>> http://cr.openjdk.java.net/~rehn/8234796/full/webrev/ >>>>>> >>>>>> The handshake code needs more information about the handshake operation. >>>>>> We change type from ThreadClosure to HandshakeOperation in >>>>>> Handshake::execute. >>>>>> This enables us to add more details to the HandshakeOperation as needed >>>>>> going forward. >>>>>> >>>>>> Tested t1 and t1-3 together with the logging improvements in 8234742. >>>>>> >>>>>> It was requested that "HandshakeOperation()" would take the name instead >>>>>> having "virtual const char* name();". Which is in this patch. >>>>>> >>>>>> Thanks, Robbin From erik.osterlund at oracle.com Mon Dec 2 10:09:18 2019 From: erik.osterlund at oracle.com (erik.osterlund at oracle.com) Date: Mon, 2 Dec 2019 11:09:18 +0100 Subject: RFR: ZGC: Add support for JFR leak profiler Message-ID: <27862ac1-e86b-406a-7819-85a13fbb24a3@oracle.com> Hi, The JFR leak profiler is currently not supported on ZGC, because it has not been accessorized appropriately. This patch adds the required Access API sprinkling necessary, to enable this for ZGC. I did the following: 1) Renamed UnifiedOop to UnifiedOopRef, because it describes the reference to an oop, not the oop itself. 2) Changed UnifiedOopRef to be a POD, instead of an AllStatic that passes around its encoded data as ?? const oop* (even though it might be a tagged pointer to a narrow oop), improving type safty. 3) Added another tag bit to UnifiedOopRef, allowing it to keep track of whether the described location is ?? IN_NATIVE or IN_HEAP. The dereference function performs the appropriate NativeAccess/HeapAccess with the ?? AS_NO_KEEPALIVE decotator, which is fine because the leak profiler does not create any edges to oops ?? found during traversal. 4) Removed code blob oop iteration, because it can't be supported appropriately by the current UnifiedOopRef ?? mechanism (because oops are misaligned, and UnifiedOopRef requires tag bits - I believe this has never ?? worked properly) 6) Accessorized loads on strong roots to NativeAccess::oop_load. JFR leak profiler never ?? creates new references to oops that are visited, so it does not need to keep things alive 7) Explicitly told the oop iteration closures to not visit referents. The default referent strategy used ?? before is DO_DISCOVERY, which doesn't seem right. 8) Fixed a pre-existing issue where the array index reported in the leak chain was calculated incorrectly ?? as the byte offset from the array base. It should be scaled by the element length of the array. Ran some JFR leak profiler tests, and some ad-hoc leak profiling in various applications. Also checked that there were no observable perf regressions in the iterations, and there were not. Bug: https://bugs.openjdk.java.net/browse/JDK-8235174 Webrev: http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/ Thanks, /Erik From stefan.karlsson at oracle.com Mon Dec 2 10:10:59 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 2 Dec 2019 11:10:59 +0100 Subject: RFR: 8213415: BitMap::word_index_round_up overflow problems In-Reply-To: References: <24C2BDC9-AC35-4804-95C8-1B59747C1494@oracle.com> <5bd19480-d570-b73d-70fe-10e93ef2ecb8@oracle.com> <3F4F27AC-4812-4E55-99DF-25F0A7BD991D@oracle.com> <2FB27F12-B779-419E-A501-715248CF2309@oracle.com> <4CD1F762-EAF0-497B-A307-2E9CABFCDD14@oracle.com> Message-ID: <9b9a8a11-1e8f-8dae-c57d-40e6073511de@oracle.com> On 2019-11-29 21:37, Kim Barrett wrote: >> On Nov 29, 2019, at 3:03 AM, Thomas St?fe wrote: >> >> Side note, I was interested in using smaller types because long term I would like to have a BitMap class in cases where I today use little hand written bitmaps. As it is now, BitMap has a pointer and a size, which makes it a 16byte structure on 64 bit, which is rather fat. The indirection is also often unwanted. I would like to have a BitMap class which contains directly the data as member(s), e.g. one where it just has a 16bit word or, maybe, an array of multiple words. That would make this structure a lot smaller and better suited to be included in space sensitive structures. > There was some discussion here in Oracle about this sort of thing a > while ago. Looking back over that discussion, I don't think we got > very far with a statically sized bitmap, just some handwaving. I think > the interest is there, but ideas (and time to persue them!) are > needed. FWIW, see the usage of BitMapView in zLiveMap.hpp and zLiveMap.inline.hpp. The bitmap backing is a single word inside ZLiveMap: ? BitMap::bm_word_t _segment_live_bits; and the size is constant: ? static const size_t nsegments = 64; The views are created with: inline BitMapView ZLiveMap::segment_live_bits() { ? return BitMapView(&_segment_live_bits, nsegments); } and then used with: inline bool ZLiveMap::is_segment_live(BitMap::idx_t segment) const { ? return segment_live_bits().par_at(segment); } All this gets inlined as expected. It doesn't support sizes less than a word, but it does support arrays of multiple words. StefanK From stefan.karlsson at oracle.com Mon Dec 2 14:45:16 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 2 Dec 2019 15:45:16 +0100 Subject: RFR: 8234426: Sweeper should not CompiledIC::set_to_clean with ICStubs for is_unloading() nmethods In-Reply-To: References: Message-ID: <950a64c6-92a4-0f99-9f4d-47725283a6ba@oracle.com> Hi Erik, After discussing this with Erik, I think this looks good. Just to recapture some of my questions and thoughts: The nmethod can be in three interesting states when the sweeper executes this code: 1) is_alive && !is_unloading 2) is_alive && is_unloading 3) !is_alive && is_unloading The transition from (2) to (3) happens when the GC is unloading the nmethod concurrently while the sweeper is processing the nmethod. It's only interesting to bring up state (3), because this is the reason why we can't assert that the nmethod is_alive. The interesting transition happens between (1) and (2), and this transition is well-synchronized between the GC and sweeper by means of the synchronization protocol in nmethod::is_unloading. In (1) all oops are good and the nmethod is guaranteed to not be unloaded. In (2) at least one of the oops are bad and the nmethod is on its way to become unloaded The sweeper communicates that it's processing an nmethod by exposing it to the root set of the threads. We are OK with exposing (1) nmethods, but we must make sure we never expose (2) nmethods. The only time the sweeper-processed nmethod is exposed to the root set of the threads is when we safepoint. The only path that can safepoint in set_to_clean(bool in_use) is when we're using ICStubs. However, we don't need to use ICStubs when an nmethod is not "in use". is_unloading nmethods (2) are not "in use", otherwise they would have been found by root scanning and/or entry barriers and would have stayed !is_unloading. Therefore the code was changed to set_to_clean(!is_unloading()). Thanks, StefanK On 2019-11-22 11:49, erik.osterlund at oracle.com wrote: > Hi, > > When the sweeper processes an nmethod, it will clean inline caches if > it is_alive(). > Today, the cleaning will utilize transitional states (using ICStubs) > if the nmethod is_alive(), > which is always true for the sweeper. If it runs out of ICStubs, it > might have to safepoint > to refill them. When it does, the currently processed nmethod might be > is_unloading(). > That is not a problem for the GC per se (safepoint operation fusing > with mark end), but it > is a problem for heap walkers that get confused that an nmethod > reachable from a thread is unloading > and hence has dead oops in it. This sweeper nmethod is the *only* > nmethod that violates an > invariant that nmethods reachable from threads (Thread::nmethods_do) > are not unloading. > > By simply changing the condition to not use ICStubs when the nmethod > is_unloading(), we > get this neat invariant, and code gets less confused about this. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8234426 > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8234426/webrev.00/ > > Thanks, > /Erik From coleen.phillimore at oracle.com Mon Dec 2 16:40:39 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 2 Dec 2019 11:40:39 -0500 Subject: RFR: 8234779: Provide idiom for declaring classes noncopyable In-Reply-To: References: <233a779d-ae3b-4856-9c44-dd81bfceab6e@oracle.com> <129579A4-01C4-4F62-9582-06BE19CB13C0@oracle.com> Message-ID: <739746c7-3910-0aba-5985-41d3e13b1ed9@oracle.com> Hi, I saw this thread while on vacation and thought I would hate it, but I really don't.? I for one, would greatly appreciate not having to get all the syntax quite right when declaring a class not copyable (or ask Kim!). I did expect the macro to be in globalDefinitions.hpp as well though. Thanks, Coleen On 12/2/19 4:04 AM, Per Liden wrote: > Hi Kim & John, > > On 12/2/19 12:10 AM, Kim Barrett wrote: >>> On Nov 29, 2019, at 6:04 PM, John Rose wrote: >>> >>> On Nov 27, 2019, at 1:49 PM, Kim Barrett >>> wrote: >>>> >>>> The proposed macro significantly reduces that wordiness. Far more >>>> importantly, it makes the intent entirely self-evident; there's no >>>> need for any explanatory comments. >>> >>> Or to put it another way, the explanatory comments can be centralized >>> in the header file which defines the macro.? And the macro can be given >>> a name which explains the intent.? The name and comments can reflect >>> HotSpot-specific ?house rules? (local design rules and conventions). >>> >>> On Nov 29, 2019, at 1:45 PM, Kim Barrett >>> wrote: >>>> ? I also like putting repetitive code behind names to make it >>>> easier to chunk and understand. >>> >>> +1? A well-chosen macro name can be easier to read than a chunk of >>> boilerplate. >>> This is especially applicable to us since C++ boilerplate evolves >>> over time, and >>> as a highly portable system we don?t have the ability to track one >>> particular >>> dialect of C++.? But even if we did, we'd still have complex ?house >>> rules? to >>> enforce and document, and macros play a role there. >>> >>> I don?t think that learning the ?house macros? for HotSpot is an >>> excessive >>> burden for people learning to work on HotSpot.? Kim?s proposal seems to >>> be yet another one of these macros. >>> >>> Thanks, Kim and Per, for marshaling the arguments pro and con. > > It seems I'm not going to convince anyone, so let's just agree to > disagree. I withdraw my objection. > >>> >>> ? John >> >> Thanks John. >> >> I've also decided to take your earlier advice (from off-list) to use >> an external terminating semicolon, rather than having it in the macro >> expansion (webrev updated in place). Besides potentially helping >> editors indent properly (which is not a problem for at least some >> editors), some uses looked very odd in context without a trailing > > +1. An external semicolon makes it look less foreign. > > I'd also like to suggest that this type of macro belongs in > globalDefinitions.hpp rather than macros.hpp, since it feels more akin > to ALWAYSINLINE/ATTRIBUTE_ALIGNED/etc than INCLUDE_JVMTI/X86_ONLY/etc. > > cheers, > Per > >> semicolon. (The location of the terminating semicolon is rendered moot >> by C++11, but I can't prove all compilers currently used to build JDK >> support empty declarations in classes.) While I prefer macros whose >> expansions are "complete" (where that makes sense for the macro), >> that's not a strong preference. >> From erik.osterlund at oracle.com Mon Dec 2 17:01:54 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 2 Dec 2019 18:01:54 +0100 Subject: RFR: 8234426: Sweeper should not CompiledIC::set_to_clean with ICStubs for is_unloading() nmethods In-Reply-To: <950a64c6-92a4-0f99-9f4d-47725283a6ba@oracle.com> References: <950a64c6-92a4-0f99-9f4d-47725283a6ba@oracle.com> Message-ID: <8eecd145-dbb3-687c-e1c8-273db8c1111a@oracle.com> Hi Stefan, Thank you for the review. Your analysis is exactly right! Thanks, /Erik On 2019-12-02 15:45, Stefan Karlsson wrote: > Hi Erik, > > After discussing this with Erik, I think this looks good. Just to > recapture some of my questions and thoughts: > > The nmethod can be in three interesting states when the sweeper > executes this code: > 1) is_alive && !is_unloading > 2) is_alive && is_unloading > 3) !is_alive && is_unloading > > The transition from (2) to (3) happens when the GC is unloading the > nmethod concurrently while the sweeper is processing the nmethod. It's > only interesting to bring up state (3), because this is the reason why > we can't assert that the nmethod is_alive. > > The interesting transition happens between (1) and (2), and this > transition is well-synchronized between the GC and sweeper by means of > the synchronization protocol in nmethod::is_unloading. > > In (1) all oops are good and the nmethod is guaranteed to not be > unloaded. > In (2) at least one of the oops are bad and the nmethod is on its way > to become unloaded > > The sweeper communicates that it's processing an nmethod by exposing > it to the root set of the threads. We are OK with exposing (1) > nmethods, but we must make sure we never expose (2) nmethods. > > The only time the sweeper-processed nmethod is exposed to the root set > of the threads is when we safepoint. The only path that can safepoint > in set_to_clean(bool in_use) is when we're using ICStubs. However, we > don't need to use ICStubs when an nmethod is not "in use". > is_unloading nmethods (2) are not "in use", otherwise they would have > been found by root scanning and/or entry barriers and would have > stayed !is_unloading. Therefore the code was changed to > set_to_clean(!is_unloading()). > > Thanks, > StefanK > > On 2019-11-22 11:49, erik.osterlund at oracle.com wrote: >> Hi, >> >> When the sweeper processes an nmethod, it will clean inline caches if >> it is_alive(). >> Today, the cleaning will utilize transitional states (using ICStubs) >> if the nmethod is_alive(), >> which is always true for the sweeper. If it runs out of ICStubs, it >> might have to safepoint >> to refill them. When it does, the currently processed nmethod might >> be is_unloading(). >> That is not a problem for the GC per se (safepoint operation fusing >> with mark end), but it >> is a problem for heap walkers that get confused that an nmethod >> reachable from a thread is unloading >> and hence has dead oops in it. This sweeper nmethod is the *only* >> nmethod that violates an >> invariant that nmethods reachable from threads (Thread::nmethods_do) >> are not unloading. >> >> By simply changing the condition to not use ICStubs when the nmethod >> is_unloading(), we >> get this neat invariant, and code gets less confused about this. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8234426 >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8234426/webrev.00/ >> >> Thanks, >> /Erik > From gerard.ziemski at oracle.com Mon Dec 2 17:21:51 2019 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Mon, 2 Dec 2019 11:21:51 -0600 Subject: RFR: 8234741: enhance os::get_core_path on macOS In-Reply-To: References: <495770f4-122c-436c-6606-9f9dd79d1e90@oracle.com> Message-ID: <86f37254-d12c-214b-17fa-0c1d924e4ac0@oracle.com> Looks good. Thank you! On 11/28/19 5:09 AM, Baesken, Matthias wrote: > Thanks for the review. > >> Please also add os:: to the other call to current_process_id() in line 3787 > Okay. > > Gerard, may I add you as reviewer ? > > > Best regards, Matthias > > > >> -----Original Message----- >> From: Langer, Christoph >> Sent: Donnerstag, 28. November 2019 12:03 >> To: Baesken, Matthias ; gerard ziemski >> >> Cc: hotspot-dev developers >> Subject: RE: RFR: 8234741: enhance os::get_core_path on macOS >> >> Hi Matthias, >> >> this looks good to me. >> >> Please also add os:: to the other call to current_process_id() in line 3787 (the >> else branch). No need for another webrev, though. >> >> /Christoph >> >>> -----Original Message----- >>> From: hotspot-dev On Behalf >> Of >>> Baesken, Matthias >>> Sent: Mittwoch, 27. November 2019 11:09 >>> To: gerard ziemski >>> Cc: hotspot-dev developers >>> Subject: RE: RFR: 8234741: enhance os::get_core_path on macOS >>> >>> Hi Gerard, thanks for your input . >>> >>> New webrev : >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8234741.1/ >>> >>> >>> Best regards, Matthias >>> >>> >>>> hi Matthias, >>>> >>>> I had to look up "kern.corefile" option, of which I was not previously >>>> aware - it looks like a nice enhancement. >>>> >>>> I'd like to suggest a few small cleanups though: >>>> >>>> #1 add "os::" prefix to "current_process_id()" call >>>> #2 restrict the scope of "os::current_process_id()" to only the branch >>>> of "if" that needs it >>>> #3 expand on the "tail" comment a bit to explain why it might be needed >>>> >>>> Perhaps something like this: >>>> >>>> ? char coreinfo[MAX_PATH]; >>>> ?? size_t sz = sizeof(coreinfo); >>>> ?? int ret = sysctlbyname("kern.corefile", coreinfo, &sz, NULL, 0); >>>> ?? if (ret == 0) { >>>> ???? char *pid_pos = strstr(coreinfo, "%P"); >>>> ???? const char* tail = (pid_pos != NULL) ? (pid_pos + 2) : ""; // skip >>>> over the "%P" to preserve any optional custom user pattern (i.e. %N, >> %U) >>>> ???? if (pid_pos != NULL) { >>>> ?????? *pid_pos = '\0'; >>>> ?????? n = jio_snprintf(buffer, bufferSize, "%s%d%s", coreinfo, >>>> os::current_process_id(), tail); >>>> ???? } else { >>>> ?????? n = jio_snprintf(buffer, bufferSize, "%s", coreinfo); >>>> ???? } >>>> ?? } >>>> >>>> BTW. I'm glad you agree to remove the unrelated AWT change from this >> fix >>>> and let the client team handle it. >>>> >>>> From coleen.phillimore at oracle.com Mon Dec 2 17:28:18 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 2 Dec 2019 12:28:18 -0500 Subject: RFR: 8234796: Refactor Handshake::execute to take a HandshakeOperation In-Reply-To: <1745284a-545f-7bf5-533a-945d917cfe08@oracle.com> References: <1ff4ff37-c65e-43e7-a845-bc6ce06750f0@oracle.com> <9dfd8c82-a7b2-fe4d-4631-31d0d5b29ff0@oracle.com> <5e7f1ea4-147b-a7cc-2aed-7993adcb6cfb@oracle.com> <1745284a-545f-7bf5-533a-945d917cfe08@oracle.com> Message-ID: <51722147-0a22-d1bb-6eec-2d2716c6b543@oracle.com> v4 looks good to me.? A lot better than v0. http://cr.openjdk.java.net/~rehn/8234796/v4/full/webrev/src/hotspot/share/runtime/deoptimization.cpp.udiff.html Thank you for changing these names from TC. Thanks also for moving ThreadClosure to iterator.hpp because that's where I would have looked for it first. Coleen On 11/29/19 6:52 AM, Robbin Ehn wrote: > Hi, Shenandoah just added a handshake, here is the additional fix. > > http://cr.openjdk.java.net/~rehn/8234796/v4/full/ > http://cr.openjdk.java.net/~rehn/8234796/v4/inc/ > > Thanks, Robbin > > On 11/28/19 3:29 PM, Robbin Ehn wrote: >> Thanks David! >> >> Since I had no compile issues, fixing includes for the ThreadClosure >> move slipped my mind. >> >> Inc: >> http://cr.openjdk.java.net/~rehn/8234796/v3/inc/webrev/ >> Full: >> http://cr.openjdk.java.net/~rehn/8234796/v3/full/webrev/ >> >> Built fastdebug and release for x64 win/lin/osx, aarch64, ppc, >> solaris-sprac. >> And without precompiled header locally. >> arm32 and x86 have prior build issues and do not compile. >> >> Thanks, Robbin >> >> On 2019-11-28 07:21, David Holmes wrote: >>> Hi Robbin, >>> >>> On 28/11/2019 1:25 am, Robbin Ehn wrote: >>>> Hi all, please review. >>>> >>>> Here is the result after Per's suggestion: >>>> http://cr.openjdk.java.net/~rehn/8234796/v2/full/webrev/index.html >>>> (incremental made no sense) >>>> >>>> Due to circular dependency between thread.hpp and handshake.hpp, I >>>> moved the ThreadClosure to iterator.hpp, as was suggested offline. >>> >>> That all looks good to me! Thanks for splitting these up. >>> >>> Thanks, >>> David >>> ----- >>> >>>> Passes t1-3 >>>> >>>> Thanks, Robbin >>>> >>>> On 11/26/19 2:07 PM, Robbin Ehn wrote: >>>>> Hi all, please review. >>>>> >>>>> Issue: >>>>> https://bugs.openjdk.java.net/browse/JDK-8234796 >>>>> Code: >>>>> http://cr.openjdk.java.net/~rehn/8234796/full/webrev/ >>>>> >>>>> The handshake code needs more information about the handshake >>>>> operation. >>>>> We change type from ThreadClosure to HandshakeOperation in >>>>> Handshake::execute. >>>>> This enables us to add more details to the HandshakeOperation as >>>>> needed going forward. >>>>> >>>>> Tested t1 and t1-3 together with the logging improvements in 8234742. >>>>> >>>>> It was requested that "HandshakeOperation()" would take the name >>>>> instead having "virtual const char* name();". Which is in this patch. >>>>> >>>>> Thanks, Robbin From kim.barrett at oracle.com Mon Dec 2 19:49:02 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 2 Dec 2019 14:49:02 -0500 Subject: RFR: 8234779: Provide idiom for declaring classes noncopyable In-Reply-To: References: <233a779d-ae3b-4856-9c44-dd81bfceab6e@oracle.com> <129579A4-01C4-4F62-9582-06BE19CB13C0@oracle.com> Message-ID: <4ECD9AC3-A67D-4587-B9DF-A00F713B8D79@oracle.com> > On Dec 2, 2019, at 4:04 AM, Per Liden wrote: > I'd also like to suggest that this type of macro belongs in globalDefinitions.hpp rather than macros.hpp, since it feels more akin to ALWAYSINLINE/ATTRIBUTE_ALIGNED/etc than INCLUDE_JVMTI/X86_ONLY/etc. Yeah, now that you point it out, macros.hpp is definitely the wrong place for it. I personally don?t care for the dumping ground that is globalDefinitions.hpp and would prefer smaller and more focused headers, but that?s an entirely different discussion. I?ll move this new macro there and clean up the patch accordingly (there are #include changes that will need adjusting). From fujie at loongson.cn Tue Dec 3 01:43:58 2019 From: fujie at loongson.cn (Jie Fu) Date: Tue, 3 Dec 2019 09:43:58 +0800 Subject: RFR: 8235218: Minimal VM is broken after JDK-8173361 Message-ID: Hi all, May I get reviews for the small fix? JBS:??? https://bugs.openjdk.java.net/browse/JDK-8235218 Webrev: http://cr.openjdk.java.net/~jiefu/8235218/webrev.00/ Thanks a lot. Best regards, Jie From kim.barrett at oracle.com Tue Dec 3 03:12:13 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 2 Dec 2019 22:12:13 -0500 Subject: RFR: 8213415: BitMap::word_index_round_up overflow problems In-Reply-To: <9b9a8a11-1e8f-8dae-c57d-40e6073511de@oracle.com> References: <24C2BDC9-AC35-4804-95C8-1B59747C1494@oracle.com> <5bd19480-d570-b73d-70fe-10e93ef2ecb8@oracle.com> <3F4F27AC-4812-4E55-99DF-25F0A7BD991D@oracle.com> <2FB27F12-B779-419E-A501-715248CF2309@oracle.com> <4CD1F762-EAF0-497B-A307-2E9CABFCDD14@oracle.com> <9b9a8a11-1e8f-8dae-c57d-40e6073511de@oracle.com> Message-ID: <23446461-F38E-4350-8001-EED9982CD2DF@oracle.com> > On Dec 2, 2019, at 5:10 AM, Stefan Karlsson wrote: > > On 2019-11-29 21:37, Kim Barrett wrote: >>> On Nov 29, 2019, at 3:03 AM, Thomas St?fe wrote: >>> >>> Side note, I was interested in using smaller types because long term I would like to have a BitMap class in cases where I today use little hand written bitmaps. As it is now, BitMap has a pointer and a size, which makes it a 16byte structure on 64 bit, which is rather fat. The indirection is also often unwanted. I would like to have a BitMap class which contains directly the data as member(s), e.g. one where it just has a 16bit word or, maybe, an array of multiple words. That would make this structure a lot smaller and better suited to be included in space sensitive structures. >> There was some discussion here in Oracle about this sort of thing a >> while ago. Looking back over that discussion, I don't think we got >> very far with a statically sized bitmap, just some handwaving. I think >> the interest is there, but ideas (and time to persue them!) are >> needed. > > FWIW, see the usage of BitMapView in zLiveMap.hpp and zLiveMap.inline.hpp. Thanks - I?d forgotten about that. From vladimir.kozlov at oracle.com Tue Dec 3 03:33:41 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 2 Dec 2019 19:33:41 -0800 Subject: Question on "JEP: JVMCI based JIT Compiler pre-compiled as shared library" In-Reply-To: <5b917002-fdbf-6f25-cf7a-79890fab97b2@redhat.com> References: <4c28f33e-a72e-b3cd-87de-f6266d9caae2@oracle.com> <5b917002-fdbf-6f25-cf7a-79890fab97b2@redhat.com> Message-ID: <0d2c1b57-1ba1-50be-2c47-d97b80191f30@oracle.com> Hi Andrew, On 11/28/19 7:40 AM, Andrew Dinn wrote: > Hi Vladimir, > > On 22/11/2019 18:23, Vladimir Kozlov wrote: > >> As you remember during this JVMLS we talked about our plan to transition >> to Graal from C2 in a future. And using AOT'ed (SVM'ed) Graal (libgraal) >> is important part of this transition, 8220623 is part of that. Most work >> is done by GraalVM group in Oracle. They just released 19.3 version of >> GraalVM that is based on JDK 11 and using libgraal by default which use >> JVMCI changes from 8220623. > > I am not sure that last statement is 100% correct :-) My point was that 8220623 JVMCI changes are used by GraalVM (regardless what is used as base). > > It depends what you mean by 'based' on jdk11. The latest GraalVM relies > on a specific downstream jdk11 tree maintained by Oracle. GraalVM users > have to download a build derived from that tree in order to be able to > use the Graal JIT and Substrate native image generator to build native > images and shared libraries including, I believe, libgraal. I understand > that the changes made to that downstream repo are public i.e. Oracle's > GraalVM team have published sources for this tree which include the > relevant backports of upstream patches. Some changes pushed into GraalVM open repo are different from what was pushed into private Oracle JDK 11u. The sum of changes should be the same but I can't guarantee it because we (in HotSpot group) only fully tested Oracle JDK 11u. > > Red Hat as maintainers of jdk11u would very much like to correct that > situation (as, I believe, would the GraalVM team). We have a list of all > the JIRAs whose patches have been backported. Is there any reason you > are aware of for the jdk11u maintainers not to backport the relevant > patches from the jdk dev tree to jdk11u? It is up to you. But it may help that a code in Open and Oracle JDK 11u is in sync. The main reason we backported these changes is to help GraalVM release to avoid building their own JDK 11u. > >> On OpenJDK side we plan to release libgraal EA based on Metropolis >> repository, as I said during JVMLS. This is tracked by RFE [1]. RFE's >> description is outdated since Metropolis repo is based on JDK 14 now and >> I need to update it. There is also Graal's PR, Bob V. is working on, to >> adjust upstream Graal/SVM code to API changes done in JDK 14. > > That's very good news. I'm very much looking forward to seeing > Metropolis acquire both a JIT and a VM implemented in Java. Thanks, Vladimir > > regards, > > > Andrew Dinn > ----------- > From john.r.rose at oracle.com Tue Dec 3 05:04:50 2019 From: john.r.rose at oracle.com (John Rose) Date: Mon, 2 Dec 2019 21:04:50 -0800 Subject: Bounds Check Elimination with Fast-Range In-Reply-To: <2BAEB1D3-46B7-4BA9-81A3-4F5E7B47B82A@gmail.com> References: <19544187-6670-4A65-AF47-297F38E8D555@gmail.com> <87r22549fg.fsf@oldenburg2.str.redhat.com> <1AB809E9-27B3-423E-AB1A-448D6B4689F6@oracle.com> <877e3x0wji.fsf@oldenburg2.str.redhat.com> <2BAEB1D3-46B7-4BA9-81A3-4F5E7B47B82A@gmail.com> Message-ID: <1590A038-CA8D-4174-8FC3-F3BAC1A93E57@oracle.com> On Nov 18, 2019, at 8:43 PM, August Nagro wrote: > > And here?s a tangent to thing to think about: is growing HashMap?s backing array by powers of 2 actually a good thing, when the HashMap gets large? What if you instead wanted to grow by powers of 1.5, or even grow probabilistically, based on the collision rate, allocation pressure, or other data? With fast-range you can do this if you want. And without the performance hit of %! It's an interesting tangent; let me follow you along it for a bit. I think there may be interesting compromises between a size menu having only powers of two and a size-to-order policy, or a menu of powers of bases not related to 2, such as 1.5. In particular, if you can manage a ratio near to 1.414 you can cut your fragmentation costs roughly in half by putting all powers of 2 on the menu, plus those values times 1.414. (Or likewise a pair of multipliers near to 1.260 and 1.587.) If there were a clever ALU operation (or table lookup) cheaper than a full multiply or remainder that would get the effect of a fast range reduction just for the items on the menu, then it would be a good trade-off to stick to the menu, where the discounts apply, rather than pay for full-custom sizes. Both 1.4 (7/5) and 1.5 (3/2) look attractively simple, requiring some wrangling of approximate moduli of 3 or 5 or 7. Or for 2^(1/3), 1.25 (5/4) and 1.66? (5/3)) are also simple ratios. Or maybe there is a non-obvious number (not a simple ratio) which has a surprisingly simple modulus approximation?though I?m not holding my breath on that one. Since I?m hoping to stay under the cost of a multiply, I admit it?s a tight ceiling, but I think there might be something there between the two extremes (2^n only and arbitrary sizes). ? John From david.holmes at oracle.com Tue Dec 3 05:15:12 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 3 Dec 2019 15:15:12 +1000 Subject: RFR: 8235218: Minimal VM is broken after JDK-8173361 In-Reply-To: References: Message-ID: Hi Jie, On 3/12/2019 11:43 am, Jie Fu wrote: > Hi all, > > May I get reviews for the small fix? > > JBS:??? https://bugs.openjdk.java.net/browse/JDK-8235218 > Webrev: http://cr.openjdk.java.net/~jiefu/8235218/webrev.00/ This fix looks fine - and trivial IMO. I'm not convinced we're handling the absence of JVMTI at the right level in all this code, but that's a different discussion. Thanks, David > Thanks a lot. > Best regards, > Jie > From fujie at loongson.cn Tue Dec 3 05:46:39 2019 From: fujie at loongson.cn (Jie Fu) Date: Tue, 3 Dec 2019 13:46:39 +0800 Subject: RFR: 8235218: Minimal VM is broken after JDK-8173361 In-Reply-To: References: Message-ID: Thanks David for your review and comments. Best regards, Jie On 2019/12/3 ??1:15, David Holmes wrote: > Hi Jie, > > On 3/12/2019 11:43 am, Jie Fu wrote: >> Hi all, >> >> May I get reviews for the small fix? >> >> JBS:??? https://bugs.openjdk.java.net/browse/JDK-8235218 >> Webrev: http://cr.openjdk.java.net/~jiefu/8235218/webrev.00/ > > This fix looks fine - and trivial IMO. > > I'm not convinced we're handling the absence of JVMTI at the right > level in all this code, but that's a different discussion. > > Thanks, > David > >> Thanks a lot. >> Best regards, >> Jie >> From john.r.rose at oracle.com Tue Dec 3 06:06:39 2019 From: john.r.rose at oracle.com (John Rose) Date: Mon, 2 Dec 2019 22:06:39 -0800 Subject: Bounds Check Elimination with Fast-Range In-Reply-To: <877e3x0wji.fsf@oldenburg2.str.redhat.com> References: <19544187-6670-4A65-AF47-297F38E8D555@gmail.com> <87r22549fg.fsf@oldenburg2.str.redhat.com> <1AB809E9-27B3-423E-AB1A-448D6B4689F6@oracle.com> <877e3x0wji.fsf@oldenburg2.str.redhat.com> Message-ID: <28CF6CF4-3E20-489C-9659-C50E4222EC22@oracle.com> On Nov 18, 2019, at 12:17 PM, Florian Weimer wrote: > >> int bucket = fr(h * M); // M = 0x2357BD or something >> >> or maybe something fast and sloppy like: >> >> int bucket = fr(h + (h << 8)); Surely this one works, since fr is the final operation. The shift/add is just a mixing step to precondition the input. Just for the record, I?d like to keep brainstorming a bit more, though surely this sort of thing is of limited interest. So, just a little more from me on this. If we had BITR I?d want to try something like fr(h - bitr(h)). But what I keep wishing for a good one- or two-cycle instruction that will mix all input bits into all the output bits, so that any change in one input bit is likely to cause cascading changes in many output bits, perhaps even 50% on average. A pair of AES steps is a good example of this. I think AES would be superior to multiply (for mixing) when used on 128 bit payloads or larger, so it looks appealing (to me) for vectorizable hashing applications. Though it is overkill on scalars, I think it points in promising directions for scalars also. >> >> or even: >> >> int bucket = fr(h) ^ (h & (N-1)); > Does this really work? I don't think so. Oops, you are right. Something like it might work, though. The idea, on paper, is that h & (N-1) is less than N, for any N >=1. And if N-1 has a high enough pop-count the information content is close to 50% of h (though maybe 50% of the bits are masked out). The xor of two quasi-independent values both less than N is, well, less than 2^(ceil lg N), not N, which is a bug. Oops. There are ways to quickly combine two values less than N and reduce the result to less than N: You do a conditional subtract of N if the sum is >= N. So the tactical area I?m trying to explore here is to take two reduced hashes developed in parallel, which depend on different bits of the input, and combine them into a single stronger hash (not by ^). Maybe (I can?t resist hacking at it some more): int h1 = H1(h), h2 = H2(h); int bucket = CCA(h1 - h2, N); // where H1 := fr, H2(h) := (h & (N-1)) // where CCA(x, N) := x + ((x >> 31) & N) // Conditional Compensating Add In this framework, a better H2 for favoring the low bits of h might be H2(h) := ((1< I think this kind of perturbation is quite expensive. Arm's BITR should > be helpful here. Yes, BITR is a helpful building block. If I understand you correctly, it needs to be combined with other operations, such as multiply, shift, xor, etc., and can overcome biases towards high bits or towards low bits that come from the simple arithmetic definitions of the other mixing operations. The hack with CCA(h1 - h2, N) seems competitive with a BITR-based mixing step, since H2 can be very simple. A scalar variant of two AES steps (with xor of a second register or constant parameter at both stages) would be a better building block for strongly mixing bits. Or some other shallow linear network with a layer of non-linear S-boxes. > But even though this operation is commonly needed and > easily implemented in hardware, it's rarely found in CPUs. Yep; the whole cottage industry of building clever mixing functions out of hand calculator functions could be sidelined if CPUs gave us good cheap mixing primitives out of the box. The crypto literature is full of them, and many are designed to be easy to implement in silicon. ? John P.S. I mention AES because I?m familiar with that bit of crypto tech, and also because I actually tried it out once on the smhasher quality benchmark. No surprise in hindsight; it passes the quality tests with just two rounds. Given that it is as cheap as multiplication, and handles twice as many bits at a time, but requires two steps for full mixing, it would seem to be competitive with multiplication as a mixing step. It has no built-in biases towards high or low bits, so that?s an advantage over multiplication. Why two rounds? The one-round version has flaws, as a hash function, which are obvious on inspection of the simple structure of an AES round. Not every output bit is data-dependent on every input bit of one round, but two rounds swirls them all together. Are back-to-back AES rounds expensive? Maybe, although that?s how the instructions are designed to be used, about 10 of them back to back to do real crypto. From thomas.stuefe at gmail.com Tue Dec 3 06:57:11 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 3 Dec 2019 07:57:11 +0100 Subject: RFR: 8213415: BitMap::word_index_round_up overflow problems In-Reply-To: References: <24C2BDC9-AC35-4804-95C8-1B59747C1494@oracle.com> <5bd19480-d570-b73d-70fe-10e93ef2ecb8@oracle.com> <3F4F27AC-4812-4E55-99DF-25F0A7BD991D@oracle.com> <2FB27F12-B779-419E-A501-715248CF2309@oracle.com> <4CD1F762-EAF0-497B-A307-2E9CABFCDD14@oracle.com> Message-ID: Hi Kim, After realizing my thinking error, I'm fine with the patch as it is. Thanks, Thomas On Fri, Nov 29, 2019, 18:57 Kim Barrett wrote: > > On Nov 29, 2019, at 3:03 AM, Thomas St?fe > wrote: > > > > Hi Kim, > > > > On Thu, Nov 28, 2019 at 11:01 PM Kim Barrett > wrote: > > > On Nov 28, 2019, at 7:42 AM, Thomas St?fe > wrote: > > > So, if I understand this correctly, we need this since we use the same > type for bit and word indices and since we have a n:1 relationship between > those two the max. bit index is necessary smaller than _MAX? > > > > The point is to avoid overflow of the type used for bit indices when > > aligning a value up to a multiple of the word size. This doesn't > > really have anything to do with using the same types for bit indices > > and word indices, though using different types might affect the > > details of some of the calculations, and the range for the word type > > would need to be suitably chosen to accomodate the bit range. > > > > > > I still in the dark. In your current version max_size_in_words() and > max_size_in_bits() there is an overflow, since both bit- and word indexes > use the same type. With 64bit I come to: FFFFFFFF.FFFFFFC0 for max word > index, 3FFFFFF.FFFFFFFF for max bit index. For 64bit types this does not > matter much, but if we ever were to use smaller types, e.g. uint16_t, it > would matter. Also, I find it surprising that max bit index is smaller than > max word index. > > You have the values backward. > > The max bit index is certainly not the smaller, since it is a multiple of > max word size. > > > > From kim.barrett at oracle.com Tue Dec 3 07:22:45 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 3 Dec 2019 02:22:45 -0500 Subject: RFR: 8213415: BitMap::word_index_round_up overflow problems In-Reply-To: References: <24C2BDC9-AC35-4804-95C8-1B59747C1494@oracle.com> <5bd19480-d570-b73d-70fe-10e93ef2ecb8@oracle.com> <3F4F27AC-4812-4E55-99DF-25F0A7BD991D@oracle.com> <2FB27F12-B779-419E-A501-715248CF2309@oracle.com> <4CD1F762-EAF0-497B-A307-2E9CABFCDD14@oracle.com> Message-ID: <19E959E2-DE3A-4C0A-BA2F-962690279AA3@oracle.com> > On Dec 3, 2019, at 1:57 AM, Thomas St?fe wrote: > > Hi Kim, > > After realizing my thinking error, I'm fine with the patch as it is. Thanks. > > Thanks, Thomas > > On Fri, Nov 29, 2019, 18:57 Kim Barrett wrote: > > On Nov 29, 2019, at 3:03 AM, Thomas St?fe wrote: > > > > Hi Kim, > > > > On Thu, Nov 28, 2019 at 11:01 PM Kim Barrett wrote: > > > On Nov 28, 2019, at 7:42 AM, Thomas St?fe wrote: > > > So, if I understand this correctly, we need this since we use the same type for bit and word indices and since we have a n:1 relationship between those two the max. bit index is necessary smaller than _MAX? > > > > The point is to avoid overflow of the type used for bit indices when > > aligning a value up to a multiple of the word size. This doesn't > > really have anything to do with using the same types for bit indices > > and word indices, though using different types might affect the > > details of some of the calculations, and the range for the word type > > would need to be suitably chosen to accomodate the bit range. > > > > > > I still in the dark. In your current version max_size_in_words() and max_size_in_bits() there is an overflow, since both bit- and word indexes use the same type. With 64bit I come to: FFFFFFFF.FFFFFFC0 for max word index, 3FFFFFF.FFFFFFFF for max bit index. For 64bit types this does not matter much, but if we ever were to use smaller types, e.g. uint16_t, it would matter. Also, I find it surprising that max bit index is smaller than max word index. > > You have the values backward. > > The max bit index is certainly not the smaller, since it is a multiple of max word size. From kim.barrett at oracle.com Tue Dec 3 07:27:12 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 3 Dec 2019 02:27:12 -0500 Subject: RFR: 8234779: Provide idiom for declaring classes noncopyable In-Reply-To: <4ECD9AC3-A67D-4587-B9DF-A00F713B8D79@oracle.com> References: <233a779d-ae3b-4856-9c44-dd81bfceab6e@oracle.com> <129579A4-01C4-4F62-9582-06BE19CB13C0@oracle.com> <4ECD9AC3-A67D-4587-B9DF-A00F713B8D79@oracle.com> Message-ID: > On Dec 2, 2019, at 2:49 PM, Kim Barrett wrote: > >> On Dec 2, 2019, at 4:04 AM, Per Liden wrote: >> I'd also like to suggest that this type of macro belongs in globalDefinitions.hpp rather than macros.hpp, since it feels more akin to ALWAYSINLINE/ATTRIBUTE_ALIGNED/etc than INCLUDE_JVMTI/X86_ONLY/etc. > > Yeah, now that you point it out, macros.hpp is definitely the wrong place for it. > I personally don?t care for the dumping ground that is globalDefinitions.hpp and > would prefer smaller and more focused headers, but that?s an entirely different > discussion. I?ll move this new macro there and clean up the patch accordingly > (there are #include changes that will need adjusting). Moved macro to globalDefinitions.hpp and updated #includes. New webrevs: full: https://cr.openjdk.java.net/~kbarrett/8234779/open.01/ incr: https://cr.openjdk.java.net/~kbarrett/8234779/open.01.inc/ Testing: mach5 tier1 (linux-x64, windows-x64, solaris-x64, macosx-x64) linux fastdebug builds for the following: 1. platforms: aarch64, arm32, ppc64le, s390x. 2. x86_64 with shenandoah included. 3. x86_64 minimal configuration. From per.liden at oracle.com Tue Dec 3 07:53:11 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 3 Dec 2019 08:53:11 +0100 Subject: RFR: 8234779: Provide idiom for declaring classes noncopyable In-Reply-To: References: <233a779d-ae3b-4856-9c44-dd81bfceab6e@oracle.com> <129579A4-01C4-4F62-9582-06BE19CB13C0@oracle.com> <4ECD9AC3-A67D-4587-B9DF-A00F713B8D79@oracle.com> Message-ID: On 12/3/19 8:27 AM, Kim Barrett wrote: >> On Dec 2, 2019, at 2:49 PM, Kim Barrett wrote: >> >>> On Dec 2, 2019, at 4:04 AM, Per Liden wrote: >>> I'd also like to suggest that this type of macro belongs in globalDefinitions.hpp rather than macros.hpp, since it feels more akin to ALWAYSINLINE/ATTRIBUTE_ALIGNED/etc than INCLUDE_JVMTI/X86_ONLY/etc. >> >> Yeah, now that you point it out, macros.hpp is definitely the wrong place for it. >> I personally don?t care for the dumping ground that is globalDefinitions.hpp and >> would prefer smaller and more focused headers, but that?s an entirely different >> discussion. I?ll move this new macro there and clean up the patch accordingly >> (there are #include changes that will need adjusting). > > Moved macro to globalDefinitions.hpp and updated #includes. > > New webrevs: > full: https://cr.openjdk.java.net/~kbarrett/8234779/open.01/ > incr: https://cr.openjdk.java.net/~kbarrett/8234779/open.01.inc/ Thanks for moving the macro. Looks good. /Per > > Testing: > mach5 tier1 (linux-x64, windows-x64, solaris-x64, macosx-x64) > > linux fastdebug builds for the following: > 1. platforms: aarch64, arm32, ppc64le, s390x. > 2. x86_64 with shenandoah included. > 3. x86_64 minimal configuration. > From david.holmes at oracle.com Tue Dec 3 08:04:44 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 3 Dec 2019 18:04:44 +1000 Subject: RFR: 8234779: Provide idiom for declaring classes noncopyable In-Reply-To: References: <233a779d-ae3b-4856-9c44-dd81bfceab6e@oracle.com> <129579A4-01C4-4F62-9582-06BE19CB13C0@oracle.com> <4ECD9AC3-A67D-4587-B9DF-A00F713B8D79@oracle.com> Message-ID: <5326a54a-4160-1c26-48b9-c8af451ffd6a@oracle.com> Hi Kim, There is a bit of an include file inconsistency, but not your fault. In places you have added: #include "utilities/globalDefinitions.hpp" to files that should already (but don't) include macros.hpp. And globalDefinitions.hpp itself includes macros.hpp which "fixes" that. But then in other places we have: +#include "utilities/globalDefinitions.hpp" #include "utilities/macros.hpp" Either all sites should include both files or else only globalDefinitions.hpp. But perhaps a later cleanup. Thanks, David On 3/12/2019 5:27 pm, Kim Barrett wrote: >> On Dec 2, 2019, at 2:49 PM, Kim Barrett wrote: >> >>> On Dec 2, 2019, at 4:04 AM, Per Liden wrote: >>> I'd also like to suggest that this type of macro belongs in globalDefinitions.hpp rather than macros.hpp, since it feels more akin to ALWAYSINLINE/ATTRIBUTE_ALIGNED/etc than INCLUDE_JVMTI/X86_ONLY/etc. >> >> Yeah, now that you point it out, macros.hpp is definitely the wrong place for it. >> I personally don?t care for the dumping ground that is globalDefinitions.hpp and >> would prefer smaller and more focused headers, but that?s an entirely different >> discussion. I?ll move this new macro there and clean up the patch accordingly >> (there are #include changes that will need adjusting). > > Moved macro to globalDefinitions.hpp and updated #includes. > > New webrevs: > full: https://cr.openjdk.java.net/~kbarrett/8234779/open.01/ > incr: https://cr.openjdk.java.net/~kbarrett/8234779/open.01.inc/ > > Testing: > mach5 tier1 (linux-x64, windows-x64, solaris-x64, macosx-x64) > > linux fastdebug builds for the following: > 1. platforms: aarch64, arm32, ppc64le, s390x. > 2. x86_64 with shenandoah included. > 3. x86_64 minimal configuration. > From tobias.hartmann at oracle.com Tue Dec 3 08:08:51 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 3 Dec 2019 09:08:51 +0100 Subject: RFR: 8234426: Sweeper should not CompiledIC::set_to_clean with ICStubs for is_unloading() nmethods In-Reply-To: <8eecd145-dbb3-687c-e1c8-273db8c1111a@oracle.com> References: <950a64c6-92a4-0f99-9f4d-47725283a6ba@oracle.com> <8eecd145-dbb3-687c-e1c8-273db8c1111a@oracle.com> Message-ID: Hi Erik, this looks good to me too. Thanks to Stefan for the additional details. Should we assert(from->is_alive() || from->is_unloading()) for sanity? Best regards, Tobias On 02.12.19 18:01, Erik ?sterlund wrote: > Hi Stefan, > > Thank you for the review. Your analysis is exactly right! > > Thanks, > /Erik > > On 2019-12-02 15:45, Stefan Karlsson wrote: >> Hi Erik, >> >> After discussing this with Erik, I think this looks good. Just to recapture some of my questions >> and thoughts: >> >> The nmethod can be in three interesting states when the sweeper executes this code: >> 1) is_alive && !is_unloading >> 2) is_alive && is_unloading >> 3) !is_alive && is_unloading >> >> The transition from (2) to (3) happens when the GC is unloading the nmethod concurrently while the >> sweeper is processing the nmethod. It's only interesting to bring up state (3), because this is >> the reason why we can't assert that the nmethod is_alive. >> >> The interesting transition happens between (1) and (2), and this transition is well-synchronized >> between the GC and sweeper by means of the synchronization protocol in nmethod::is_unloading. >> >> In (1) all oops are good and the nmethod is guaranteed to not be unloaded. >> In (2) at least one of the oops are bad and the nmethod is on its way to become unloaded >> >> The sweeper communicates that it's processing an nmethod by exposing it to the root set of the >> threads. We are OK with exposing (1) nmethods, but we must make sure we never expose (2) nmethods. >> >> The only time the sweeper-processed nmethod is exposed to the root set of the threads is when we >> safepoint. The only path that can safepoint in set_to_clean(bool in_use) is when we're using >> ICStubs. However, we don't need to use ICStubs when an nmethod is not "in use". is_unloading >> nmethods (2) are not "in use", otherwise they would have been found by root scanning and/or entry >> barriers and would have stayed !is_unloading. Therefore the code was changed to >> set_to_clean(!is_unloading()). >> >> Thanks, >> StefanK >> >> On 2019-11-22 11:49, erik.osterlund at oracle.com wrote: >>> Hi, >>> >>> When the sweeper processes an nmethod, it will clean inline caches if it is_alive(). >>> Today, the cleaning will utilize transitional states (using ICStubs) if the nmethod is_alive(), >>> which is always true for the sweeper. If it runs out of ICStubs, it might have to safepoint >>> to refill them. When it does, the currently processed nmethod might be is_unloading(). >>> That is not a problem for the GC per se (safepoint operation fusing with mark end), but it >>> is a problem for heap walkers that get confused that an nmethod reachable from a thread is unloading >>> and hence has dead oops in it. This sweeper nmethod is the *only* nmethod that violates an >>> invariant that nmethods reachable from threads (Thread::nmethods_do) are not unloading. >>> >>> By simply changing the condition to not use ICStubs when the nmethod is_unloading(), we >>> get this neat invariant, and code gets less confused about this. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8234426 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8234426/webrev.00/ >>> >>> Thanks, >>> /Erik >> > From thomas.stuefe at gmail.com Tue Dec 3 08:28:07 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 3 Dec 2019 09:28:07 +0100 Subject: RFR: 8213415: BitMap::word_index_round_up overflow problems In-Reply-To: <9b9a8a11-1e8f-8dae-c57d-40e6073511de@oracle.com> References: <24C2BDC9-AC35-4804-95C8-1B59747C1494@oracle.com> <5bd19480-d570-b73d-70fe-10e93ef2ecb8@oracle.com> <3F4F27AC-4812-4E55-99DF-25F0A7BD991D@oracle.com> <2FB27F12-B779-419E-A501-715248CF2309@oracle.com> <4CD1F762-EAF0-497B-A307-2E9CABFCDD14@oracle.com> <9b9a8a11-1e8f-8dae-c57d-40e6073511de@oracle.com> Message-ID: Hi Stefan, On Mon, Dec 2, 2019 at 11:11 AM Stefan Karlsson wrote: > On 2019-11-29 21:37, Kim Barrett wrote: > >> On Nov 29, 2019, at 3:03 AM, Thomas St?fe > wrote: > >> > >> Side note, I was interested in using smaller types because long term I > would like to have a BitMap class in cases where I today use little hand > written bitmaps. As it is now, BitMap has a pointer and a size, which makes > it a 16byte structure on 64 bit, which is rather fat. The indirection is > also often unwanted. I would like to have a BitMap class which contains > directly the data as member(s), e.g. one where it just has a 16bit word or, > maybe, an array of multiple words. That would make this structure a lot > smaller and better suited to be included in space sensitive structures. > > There was some discussion here in Oracle about this sort of thing a > > while ago. Looking back over that discussion, I don't think we got > > very far with a statically sized bitmap, just some handwaving. I think > > the interest is there, but ideas (and time to persue them!) are > > needed. > > FWIW, see the usage of BitMapView in zLiveMap.hpp and zLiveMap.inline.hpp. > > The bitmap backing is a single word inside ZLiveMap: > BitMap::bm_word_t _segment_live_bits; > > and the size is constant: > static const size_t nsegments = 64; > > The views are created with: > inline BitMapView ZLiveMap::segment_live_bits() { > return BitMapView(&_segment_live_bits, nsegments); > } > > and then used with: > inline bool ZLiveMap::is_segment_live(BitMap::idx_t segment) const { > return segment_live_bits().par_at(segment); > } > > All this gets inlined as expected. > > It doesn't support sizes less than a word, but it does support arrays of > multiple words. > > StefanK > I was aware that we could put the Bitmap over existing memory, but it did not occur to me to do this just temporarily for one operation... So, all that business above (Bitmap construction, copying etc) gets optimized away to a simple load and an or? Thanks, Thomas From robbin.ehn at oracle.com Tue Dec 3 08:59:38 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 3 Dec 2019 09:59:38 +0100 Subject: RFR: 8234796: Refactor Handshake::execute to take a HandshakeOperation In-Reply-To: <51722147-0a22-d1bb-6eec-2d2716c6b543@oracle.com> References: <1ff4ff37-c65e-43e7-a845-bc6ce06750f0@oracle.com> <9dfd8c82-a7b2-fe4d-4631-31d0d5b29ff0@oracle.com> <5e7f1ea4-147b-a7cc-2aed-7993adcb6cfb@oracle.com> <1745284a-545f-7bf5-533a-945d917cfe08@oracle.com> <51722147-0a22-d1bb-6eec-2d2716c6b543@oracle.com> Message-ID: <4b252d52-3b4b-19b7-d912-c278604dad31@oracle.com> Thanks Coleen! /Robbin On 12/2/19 6:28 PM, coleen.phillimore at oracle.com wrote: > v4 looks good to me.? A lot better than v0. > > http://cr.openjdk.java.net/~rehn/8234796/v4/full/webrev/src/hotspot/share/runtime/deoptimization.cpp.udiff.html > > > Thank you for changing these names from TC. > > Thanks also for moving ThreadClosure to iterator.hpp because that's where I > would have looked for it first. > > Coleen > > On 11/29/19 6:52 AM, Robbin Ehn wrote: >> Hi, Shenandoah just added a handshake, here is the additional fix. >> >> http://cr.openjdk.java.net/~rehn/8234796/v4/full/ >> http://cr.openjdk.java.net/~rehn/8234796/v4/inc/ >> >> Thanks, Robbin >> >> On 11/28/19 3:29 PM, Robbin Ehn wrote: >>> Thanks David! >>> >>> Since I had no compile issues, fixing includes for the ThreadClosure move >>> slipped my mind. >>> >>> Inc: >>> http://cr.openjdk.java.net/~rehn/8234796/v3/inc/webrev/ >>> Full: >>> http://cr.openjdk.java.net/~rehn/8234796/v3/full/webrev/ >>> >>> Built fastdebug and release for x64 win/lin/osx, aarch64, ppc, solaris-sprac. >>> And without precompiled header locally. >>> arm32 and x86 have prior build issues and do not compile. >>> >>> Thanks, Robbin >>> >>> On 2019-11-28 07:21, David Holmes wrote: >>>> Hi Robbin, >>>> >>>> On 28/11/2019 1:25 am, Robbin Ehn wrote: >>>>> Hi all, please review. >>>>> >>>>> Here is the result after Per's suggestion: >>>>> http://cr.openjdk.java.net/~rehn/8234796/v2/full/webrev/index.html >>>>> (incremental made no sense) >>>>> >>>>> Due to circular dependency between thread.hpp and handshake.hpp, I moved >>>>> the ThreadClosure to iterator.hpp, as was suggested offline. >>>> >>>> That all looks good to me! Thanks for splitting these up. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> Passes t1-3 >>>>> >>>>> Thanks, Robbin >>>>> >>>>> On 11/26/19 2:07 PM, Robbin Ehn wrote: >>>>>> Hi all, please review. >>>>>> >>>>>> Issue: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8234796 >>>>>> Code: >>>>>> http://cr.openjdk.java.net/~rehn/8234796/full/webrev/ >>>>>> >>>>>> The handshake code needs more information about the handshake operation. >>>>>> We change type from ThreadClosure to HandshakeOperation in >>>>>> Handshake::execute. >>>>>> This enables us to add more details to the HandshakeOperation as needed >>>>>> going forward. >>>>>> >>>>>> Tested t1 and t1-3 together with the logging improvements in 8234742. >>>>>> >>>>>> It was requested that "HandshakeOperation()" would take the name instead >>>>>> having "virtual const char* name();". Which is in this patch. >>>>>> >>>>>> Thanks, Robbin > From thomas.schatzl at oracle.com Tue Dec 3 09:01:51 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 3 Dec 2019 10:01:51 +0100 Subject: RFR: 8213415: BitMap::word_index_round_up overflow problems In-Reply-To: <19E959E2-DE3A-4C0A-BA2F-962690279AA3@oracle.com> References: <24C2BDC9-AC35-4804-95C8-1B59747C1494@oracle.com> <5bd19480-d570-b73d-70fe-10e93ef2ecb8@oracle.com> <3F4F27AC-4812-4E55-99DF-25F0A7BD991D@oracle.com> <2FB27F12-B779-419E-A501-715248CF2309@oracle.com> <4CD1F762-EAF0-497B-A307-2E9CABFCDD14@oracle.com> <19E959E2-DE3A-4C0A-BA2F-962690279AA3@oracle.com> Message-ID: Hi, On 03.12.19 08:22, Kim Barrett wrote: >> On Dec 3, 2019, at 1:57 AM, Thomas St?fe wrote: >> >> Hi Kim, >> >> After realizing my thinking error, I'm fine with the patch as it is. > > Thanks. > >> the latest patch (with the comment changes) is still fine with me. Thomas From stefan.karlsson at oracle.com Tue Dec 3 09:26:48 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 3 Dec 2019 10:26:48 +0100 Subject: RFR: 8213415: BitMap::word_index_round_up overflow problems In-Reply-To: References: <24C2BDC9-AC35-4804-95C8-1B59747C1494@oracle.com> <5bd19480-d570-b73d-70fe-10e93ef2ecb8@oracle.com> <3F4F27AC-4812-4E55-99DF-25F0A7BD991D@oracle.com> <2FB27F12-B779-419E-A501-715248CF2309@oracle.com> <4CD1F762-EAF0-497B-A307-2E9CABFCDD14@oracle.com> <9b9a8a11-1e8f-8dae-c57d-40e6073511de@oracle.com> Message-ID: <60210bb7-3cba-0383-00ea-ff5c4b875699@oracle.com> Hi Thomas, On 2019-12-03 09:28, Thomas St?fe wrote: > Hi Stefan, > > On Mon, Dec 2, 2019 at 11:11 AM Stefan Karlsson > > wrote: > > On 2019-11-29 21:37, Kim Barrett wrote: > >> On Nov 29, 2019, at 3:03 AM, Thomas St?fe > > wrote: > >> > >> Side note, I was interested in using smaller types because long > term I would like to have a BitMap class in cases where I today use > little hand written bitmaps. As it is now, BitMap has a pointer and > a size, which makes it a 16byte structure on 64 bit, which is rather > fat. The indirection is also often unwanted. I would like to have a > BitMap class which contains directly the data as member(s), e.g. one > where it just has a 16bit word or, maybe, an array of multiple > words. That would make this structure a lot smaller and better > suited to be included in space sensitive structures. > > There was some discussion here in Oracle about this sort of thing a > > while ago. Looking back over that discussion, I don't think we got > > very far with a statically sized bitmap, just some handwaving. I > think > > the interest is there, but ideas (and time to persue them!) are > > needed. > > FWIW, see the usage of BitMapView in zLiveMap.hpp and > zLiveMap.inline.hpp. > > The bitmap backing is a single word inside ZLiveMap: > ?? BitMap::bm_word_t _segment_live_bits; > > and the size is constant: > ?? static const size_t nsegments = 64; > > The views are created with: > inline BitMapView ZLiveMap::segment_live_bits() { > ?? return BitMapView(&_segment_live_bits, nsegments); > } > > and then used with: > inline bool ZLiveMap::is_segment_live(BitMap::idx_t segment) const { > ?? return segment_live_bits().par_at(segment); > } > > All this gets inlined as expected. > > It doesn't support sizes less than a word, but it does support > arrays of > multiple words. > > StefanK > > > I was aware that we could put the Bitmap over existing memory, but it > did not occur to me to do this just temporarily for one operation... So, > all that business above (Bitmap construction, copying etc) gets > optimized away to a simple load and an or? There are some extra masking and shifting as well. Some of that could probably be optimized away with a special-built bitmap. If we take this function as an example: void ZLiveMap::reset_segment(BitMap::idx_t segment) { bool contention = false; if (!claim_segment(segment)) { with claim_segment: inline bool ZLiveMap::claim_segment(BitMap::idx_t segment) { return segment_claim_bits().par_set_bit(segment, memory_order_acq_rel); } and par_set_bit: inline bool BitMap::par_set_bit(idx_t bit, atomic_memory_order memory_order) { verify_index(bit); volatile bm_word_t* const addr = word_addr(bit); const bm_word_t mask = bit_mask(bit); and we correlate that with the generated code: 0000000000e00910 <_ZN8ZLiveMap13reset_segmentEm>: e00910: 55 push %rbp e00911: 48 89 f1 mov %rsi,%rcx // segment e00914: 83 e1 3f and $0x3f,%ecx // bit_in_word e00917: 48 89 e5 mov %rsp,%rbp e0091a: 41 57 push %r15 e0091c: 41 56 push %r14 e0091e: 49 89 f6 mov %rsi,%r14 e00921: 41 55 push %r13 e00923: 49 c1 ee 06 shr $0x6,%r14 // word_index e00927: 49 89 fd mov %rdi,%r13 e0092a: 41 54 push %r12 e0092c: 49 c1 e6 03 shl $0x3,%r14 e00930: 49 89 f4 mov %rsi,%r12 e00933: 53 push %rbx e00934: 4a 8d 7c 37 18 lea 0x18(%rdi,%r14,1),%rdi // word_addr e00939: bb 01 00 00 00 mov $0x1,%ebx e0093e: 48 d3 e3 shl %cl,%rbx // bit_mask e00941: 48 83 ec 08 sub $0x8,%rsp e00945: 48 8b 0f mov (%rdi),%rcx // load e00948: eb 16 jmp e00960 <_ZN8ZLiveMap13reset_segmentEm+0x50> e0094a: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1) e00950: 48 89 c8 mov %rcx,%rax e00953: f0 48 0f b1 17 lock cmpxchg %rdx,(%rdi) e00958: 48 39 c1 cmp %rax,%rcx e0095b: 74 43 je e009a0 <_ZN8ZLiveMap13reset_segmentEm+0x90> There's no traces of the temporary bitmap view, as far as I can see. StefanK > > Thanks, Thomas > > From vladimir.x.ivanov at oracle.com Tue Dec 3 09:32:48 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 3 Dec 2019 12:32:48 +0300 Subject: Bounds Check Elimination with Fast-Range In-Reply-To: <28CF6CF4-3E20-489C-9659-C50E4222EC22@oracle.com> References: <19544187-6670-4A65-AF47-297F38E8D555@gmail.com> <87r22549fg.fsf@oldenburg2.str.redhat.com> <1AB809E9-27B3-423E-AB1A-448D6B4689F6@oracle.com> <877e3x0wji.fsf@oldenburg2.str.redhat.com> <28CF6CF4-3E20-489C-9659-C50E4222EC22@oracle.com> Message-ID: <5ec39d06-8f9c-f731-bd8f-7eeef85147f2@oracle.com> > but two rounds swirls them all together. Are back-to-back AES rounds > expensive? Maybe, although that?s how the instructions are designed to > be used, about 10 of them back to back to do real crypto. Throughput-oriented implementation should work fine for crypto purposes, but AESENC does look very good on recent Intel micro-architectures from latency perspective as well (data from [1] [2]): it improved from 8/1 on Sandy Bridge and 7/1 on Haswell to 4/1 on Skylake and it's listed (on uops.info [2]) as 3/1 on Ice Lake which is on par with IMUL (while processing twice as much bits). And vector variant (VAESENC) has the same latency as scalar (8->7->4->3 [3]) which looks very appealing for throughput-oriented use cases. Best regards, Vladimir Ivanov [1] https://www.agner.org/optimize/instruction_tables.pdf [2] https://uops.info/html-lat/ICL/AESENC_XMM_XMM-Measurements.html [3] https://uops.info/html-instr/VAESENC_XMM_XMM_XMM.html From erik.osterlund at oracle.com Tue Dec 3 09:59:23 2019 From: erik.osterlund at oracle.com (erik.osterlund at oracle.com) Date: Tue, 3 Dec 2019 10:59:23 +0100 Subject: RFR: 8234426: Sweeper should not CompiledIC::set_to_clean with ICStubs for is_unloading() nmethods In-Reply-To: References: <950a64c6-92a4-0f99-9f4d-47725283a6ba@oracle.com> <8eecd145-dbb3-687c-e1c8-273db8c1111a@oracle.com> Message-ID: Hi Tobias, Thank you for the review. We could indeed add an assert like that. Another possibly better assert is assert(!from->is_zombie()). The thing is that is_unloading is strictly monotonic - it goes from false to true (iff GC kills an oop). Therefore, nmethods that are unloaded are always is_unloading(), so your assert will only ever catch zombies (the only flavour of !is_alive() that can be !is_unloading()). However, the sweeper flips unloaded nmethods to zombies today, and these mysterious zombies are also is_unloading() (because they were once unloaded by the GC before they became zombies). Your assert will still allow such zombies, but they should in fact never have their inline caches cleaned. So in one way, asserting that the CompiledMethod is !is_zombie() is strictly more precise than your assert. But it might also be less obvious to the reader. Unless I put a big comment in there describing what I explained above, which of course I could do. Here is an attempt at that: http://cr.openjdk.java.net/~eosterlund/8234426/webrev.01/ Incremental: http://cr.openjdk.java.net/~eosterlund/8234426/webrev.00_01/ What do you think? Thanks, /Erik On 12/3/19 9:08 AM, Tobias Hartmann wrote: > Hi Erik, > > this looks good to me too. Thanks to Stefan for the additional details. > > Should we assert(from->is_alive() || from->is_unloading()) for sanity? > > Best regards, > Tobias > > On 02.12.19 18:01, Erik ?sterlund wrote: >> Hi Stefan, >> >> Thank you for the review. Your analysis is exactly right! >> >> Thanks, >> /Erik >> >> On 2019-12-02 15:45, Stefan Karlsson wrote: >>> Hi Erik, >>> >>> After discussing this with Erik, I think this looks good. Just to recapture some of my questions >>> and thoughts: >>> >>> The nmethod can be in three interesting states when the sweeper executes this code: >>> 1) is_alive && !is_unloading >>> 2) is_alive && is_unloading >>> 3) !is_alive && is_unloading >>> >>> The transition from (2) to (3) happens when the GC is unloading the nmethod concurrently while the >>> sweeper is processing the nmethod. It's only interesting to bring up state (3), because this is >>> the reason why we can't assert that the nmethod is_alive. >>> >>> The interesting transition happens between (1) and (2), and this transition is well-synchronized >>> between the GC and sweeper by means of the synchronization protocol in nmethod::is_unloading. >>> >>> In (1) all oops are good and the nmethod is guaranteed to not be unloaded. >>> In (2) at least one of the oops are bad and the nmethod is on its way to become unloaded >>> >>> The sweeper communicates that it's processing an nmethod by exposing it to the root set of the >>> threads. We are OK with exposing (1) nmethods, but we must make sure we never expose (2) nmethods. >>> >>> The only time the sweeper-processed nmethod is exposed to the root set of the threads is when we >>> safepoint. The only path that can safepoint in set_to_clean(bool in_use) is when we're using >>> ICStubs. However, we don't need to use ICStubs when an nmethod is not "in use". is_unloading >>> nmethods (2) are not "in use", otherwise they would have been found by root scanning and/or entry >>> barriers and would have stayed !is_unloading. Therefore the code was changed to >>> set_to_clean(!is_unloading()). >>> >>> Thanks, >>> StefanK >>> >>> On 2019-11-22 11:49, erik.osterlund at oracle.com wrote: >>>> Hi, >>>> >>>> When the sweeper processes an nmethod, it will clean inline caches if it is_alive(). >>>> Today, the cleaning will utilize transitional states (using ICStubs) if the nmethod is_alive(), >>>> which is always true for the sweeper. If it runs out of ICStubs, it might have to safepoint >>>> to refill them. When it does, the currently processed nmethod might be is_unloading(). >>>> That is not a problem for the GC per se (safepoint operation fusing with mark end), but it >>>> is a problem for heap walkers that get confused that an nmethod reachable from a thread is unloading >>>> and hence has dead oops in it. This sweeper nmethod is the *only* nmethod that violates an >>>> invariant that nmethods reachable from threads (Thread::nmethods_do) are not unloading. >>>> >>>> By simply changing the condition to not use ICStubs when the nmethod is_unloading(), we >>>> get this neat invariant, and code gets less confused about this. >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8234426 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8234426/webrev.00/ >>>> >>>> Thanks, >>>> /Erik From matthias.baesken at sap.com Tue Dec 3 10:48:23 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 3 Dec 2019 10:48:23 +0000 Subject: RFR [XS]: 8235243: handle VS2017 15.9 and VS2019 in abstract_vm_version Message-ID: Hello, please review this small enhancement for Visual Studio compiler version handling. The C++ compiler handling coding in abstract_vm_version.cpp currently misses handling nicely VS2017 15.9 ("update 9") and VS2019. This leads to bad output like "unknown MS VC++:1916" in the hs_err file. A list of _MSC_VER can be found for example here : https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8235243 http://cr.openjdk.java.net/~mbaesken/webrevs/8235243.0/ Thanks, Matthias From Joshua.Zhu at arm.com Tue Dec 3 12:26:15 2019 From: Joshua.Zhu at arm.com (Joshua Zhu (Arm Technology China)) Date: Tue, 3 Dec 2019 12:26:15 +0000 Subject: 8233948: AArch64: Incorrect mapping between OptoReg and VMReg for high 64 bits of Vector Register In-Reply-To: References: Message-ID: Hi Andrew Dinn, > > Could you prepare a new webrev with these extra changes in and check > > it is ok? > > Yes, of course. I updated webrev according to your review comments. > http://cr.openjdk.java.net/~jzhu/8233948/webrev.01/ > > Also, could you report what testing you did before and after your > > change (other than checking the dump output). You will probably need > > to repeat it to ensure these extra changes are ok. > > Oh, Sorry for that. I did not mention my testing in my previous mail (My > initial RFR mail is too long). > I ran full jtreg tests without new failure introduced. > I will re-run it against the new webrev and will let you know the results. > Thanks again for your review. Testing: Passed hotspot_all_no_apps, jdk_core and langtools:tier1 without new failure introduced. Please let me know if any comments. Thanks! Best Regards, Joshua From tobias.hartmann at oracle.com Tue Dec 3 12:27:57 2019 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 3 Dec 2019 13:27:57 +0100 Subject: RFR: 8234426: Sweeper should not CompiledIC::set_to_clean with ICStubs for is_unloading() nmethods In-Reply-To: References: <950a64c6-92a4-0f99-9f4d-47725283a6ba@oracle.com> <8eecd145-dbb3-687c-e1c8-273db8c1111a@oracle.com> Message-ID: Hi Erik, right, your assert is better and the comment explaining the details is very helpful. Looks good to me! Best regards, Tobias On 03.12.19 10:59, erik.osterlund at oracle.com wrote: > Hi Tobias, > > Thank you for the review. > We could indeed add an assert like that. Another possibly better assert is assert(!from->is_zombie()). > The thing is that is_unloading is strictly monotonic - it goes from false to true (iff GC kills an > oop). > Therefore, nmethods that are unloaded are always is_unloading(), so your assert will only ever catch > zombies > (the only flavour of !is_alive() that can be !is_unloading()). However, the sweeper flips unloaded > nmethods to zombies today, and these mysterious zombies are also is_unloading() (because they were once > unloaded by the GC before they became zombies). Your assert will still allow such zombies, but they > should in fact never have their inline caches cleaned. > > So in one way, asserting that the CompiledMethod is !is_zombie() is strictly more precise than > your assert. But it might also be less obvious to the reader. Unless I put a big comment in there > describing what I explained above, which of course I could do. Here is an attempt at that: > > http://cr.openjdk.java.net/~eosterlund/8234426/webrev.01/ > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8234426/webrev.00_01/ > > What do you think? > > Thanks, > /Erik > > On 12/3/19 9:08 AM, Tobias Hartmann wrote: >> Hi Erik, >> >> this looks good to me too. Thanks to Stefan for the additional details. >> >> Should we assert(from->is_alive() || from->is_unloading()) for sanity? >> >> Best regards, >> Tobias >> >> On 02.12.19 18:01, Erik ?sterlund wrote: >>> Hi Stefan, >>> >>> Thank you for the review. Your analysis is exactly right! >>> >>> Thanks, >>> /Erik >>> >>> On 2019-12-02 15:45, Stefan Karlsson wrote: >>>> Hi Erik, >>>> >>>> After discussing this with Erik, I think this looks good. Just to recapture some of my questions >>>> and thoughts: >>>> >>>> The nmethod can be in three interesting states when the sweeper executes this code: >>>> 1) is_alive && !is_unloading >>>> 2) is_alive && is_unloading >>>> 3) !is_alive && is_unloading >>>> >>>> The transition from (2) to (3) happens when the GC is unloading the nmethod concurrently while the >>>> sweeper is processing the nmethod. It's only interesting to bring up state (3), because this is >>>> the reason why we can't assert that the nmethod is_alive. >>>> >>>> The interesting transition happens between (1) and (2), and this transition is well-synchronized >>>> between the GC and sweeper by means of the synchronization protocol in nmethod::is_unloading. >>>> >>>> In (1) all oops are good and the nmethod is guaranteed to not be unloaded. >>>> In (2) at least one of the oops are bad and the nmethod is on its way to become unloaded >>>> >>>> The sweeper communicates that it's processing an nmethod by exposing it to the root set of the >>>> threads. We are OK with exposing (1) nmethods, but we must make sure we never expose (2) nmethods. >>>> >>>> The only time the sweeper-processed nmethod is exposed to the root set of the threads is when we >>>> safepoint. The only path that can safepoint in set_to_clean(bool in_use) is when we're using >>>> ICStubs. However, we don't need to use ICStubs when an nmethod is not "in use". is_unloading >>>> nmethods (2) are not "in use", otherwise they would have been found by root scanning and/or entry >>>> barriers and would have stayed !is_unloading. Therefore the code was changed to >>>> set_to_clean(!is_unloading()). >>>> >>>> Thanks, >>>> StefanK >>>> >>>> On 2019-11-22 11:49, erik.osterlund at oracle.com wrote: >>>>> Hi, >>>>> >>>>> When the sweeper processes an nmethod, it will clean inline caches if it is_alive(). >>>>> Today, the cleaning will utilize transitional states (using ICStubs) if the nmethod is_alive(), >>>>> which is always true for the sweeper. If it runs out of ICStubs, it might have to safepoint >>>>> to refill them. When it does, the currently processed nmethod might be is_unloading(). >>>>> That is not a problem for the GC per se (safepoint operation fusing with mark end), but it >>>>> is a problem for heap walkers that get confused that an nmethod reachable from a thread is >>>>> unloading >>>>> and hence has dead oops in it. This sweeper nmethod is the *only* nmethod that violates an >>>>> invariant that nmethods reachable from threads (Thread::nmethods_do) are not unloading. >>>>> >>>>> By simply changing the condition to not use ICStubs when the nmethod is_unloading(), we >>>>> get this neat invariant, and code gets less confused about this. >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8234426 >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8234426/webrev.00/ >>>>> >>>>> Thanks, >>>>> /Erik > From erik.osterlund at oracle.com Tue Dec 3 12:35:41 2019 From: erik.osterlund at oracle.com (erik.osterlund at oracle.com) Date: Tue, 3 Dec 2019 13:35:41 +0100 Subject: RFR: 8234426: Sweeper should not CompiledIC::set_to_clean with ICStubs for is_unloading() nmethods In-Reply-To: References: <950a64c6-92a4-0f99-9f4d-47725283a6ba@oracle.com> <8eecd145-dbb3-687c-e1c8-273db8c1111a@oracle.com> Message-ID: Hi Tobias, Thanks for the review! /Erik On 12/3/19 1:27 PM, Tobias Hartmann wrote: > Hi Erik, > > right, your assert is better and the comment explaining the details is very helpful. > > Looks good to me! > > Best regards, > Tobias > > On 03.12.19 10:59, erik.osterlund at oracle.com wrote: >> Hi Tobias, >> >> Thank you for the review. >> We could indeed add an assert like that. Another possibly better assert is assert(!from->is_zombie()). >> The thing is that is_unloading is strictly monotonic - it goes from false to true (iff GC kills an >> oop). >> Therefore, nmethods that are unloaded are always is_unloading(), so your assert will only ever catch >> zombies >> (the only flavour of !is_alive() that can be !is_unloading()). However, the sweeper flips unloaded >> nmethods to zombies today, and these mysterious zombies are also is_unloading() (because they were once >> unloaded by the GC before they became zombies). Your assert will still allow such zombies, but they >> should in fact never have their inline caches cleaned. >> >> So in one way, asserting that the CompiledMethod is !is_zombie() is strictly more precise than >> your assert. But it might also be less obvious to the reader. Unless I put a big comment in there >> describing what I explained above, which of course I could do. Here is an attempt at that: >> >> http://cr.openjdk.java.net/~eosterlund/8234426/webrev.01/ >> >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8234426/webrev.00_01/ >> >> What do you think? >> >> Thanks, >> /Erik >> >> On 12/3/19 9:08 AM, Tobias Hartmann wrote: >>> Hi Erik, >>> >>> this looks good to me too. Thanks to Stefan for the additional details. >>> >>> Should we assert(from->is_alive() || from->is_unloading()) for sanity? >>> >>> Best regards, >>> Tobias >>> >>> On 02.12.19 18:01, Erik ?sterlund wrote: >>>> Hi Stefan, >>>> >>>> Thank you for the review. Your analysis is exactly right! >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2019-12-02 15:45, Stefan Karlsson wrote: >>>>> Hi Erik, >>>>> >>>>> After discussing this with Erik, I think this looks good. Just to recapture some of my questions >>>>> and thoughts: >>>>> >>>>> The nmethod can be in three interesting states when the sweeper executes this code: >>>>> 1) is_alive && !is_unloading >>>>> 2) is_alive && is_unloading >>>>> 3) !is_alive && is_unloading >>>>> >>>>> The transition from (2) to (3) happens when the GC is unloading the nmethod concurrently while the >>>>> sweeper is processing the nmethod. It's only interesting to bring up state (3), because this is >>>>> the reason why we can't assert that the nmethod is_alive. >>>>> >>>>> The interesting transition happens between (1) and (2), and this transition is well-synchronized >>>>> between the GC and sweeper by means of the synchronization protocol in nmethod::is_unloading. >>>>> >>>>> In (1) all oops are good and the nmethod is guaranteed to not be unloaded. >>>>> In (2) at least one of the oops are bad and the nmethod is on its way to become unloaded >>>>> >>>>> The sweeper communicates that it's processing an nmethod by exposing it to the root set of the >>>>> threads. We are OK with exposing (1) nmethods, but we must make sure we never expose (2) nmethods. >>>>> >>>>> The only time the sweeper-processed nmethod is exposed to the root set of the threads is when we >>>>> safepoint. The only path that can safepoint in set_to_clean(bool in_use) is when we're using >>>>> ICStubs. However, we don't need to use ICStubs when an nmethod is not "in use". is_unloading >>>>> nmethods (2) are not "in use", otherwise they would have been found by root scanning and/or entry >>>>> barriers and would have stayed !is_unloading. Therefore the code was changed to >>>>> set_to_clean(!is_unloading()). >>>>> >>>>> Thanks, >>>>> StefanK >>>>> >>>>> On 2019-11-22 11:49, erik.osterlund at oracle.com wrote: >>>>>> Hi, >>>>>> >>>>>> When the sweeper processes an nmethod, it will clean inline caches if it is_alive(). >>>>>> Today, the cleaning will utilize transitional states (using ICStubs) if the nmethod is_alive(), >>>>>> which is always true for the sweeper. If it runs out of ICStubs, it might have to safepoint >>>>>> to refill them. When it does, the currently processed nmethod might be is_unloading(). >>>>>> That is not a problem for the GC per se (safepoint operation fusing with mark end), but it >>>>>> is a problem for heap walkers that get confused that an nmethod reachable from a thread is >>>>>> unloading >>>>>> and hence has dead oops in it. This sweeper nmethod is the *only* nmethod that violates an >>>>>> invariant that nmethods reachable from threads (Thread::nmethods_do) are not unloading. >>>>>> >>>>>> By simply changing the condition to not use ICStubs when the nmethod is_unloading(), we >>>>>> get this neat invariant, and code gets less confused about this. >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8234426 >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8234426/webrev.00/ >>>>>> >>>>>> Thanks, >>>>>> /Erik From coleen.phillimore at oracle.com Tue Dec 3 13:13:26 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 3 Dec 2019 08:13:26 -0500 Subject: RFR: 8235218: Minimal VM is broken after JDK-8173361 In-Reply-To: References: Message-ID: <7c8a51f1-5cd0-bfbb-86f0-0684c2e87eea@oracle.com> Thanks for fixing and David? for pushing this. Coleen On 12/2/19 8:43 PM, Jie Fu wrote: > Hi all, > > May I get reviews for the small fix? > > JBS:??? https://bugs.openjdk.java.net/browse/JDK-8235218 > Webrev: http://cr.openjdk.java.net/~jiefu/8235218/webrev.00/ > > Thanks a lot. > Best regards, > Jie > From per.liden at oracle.com Tue Dec 3 13:15:24 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 3 Dec 2019 14:15:24 +0100 Subject: RFR: 8234662: Sweeper should keep current nmethod alive before yielding for ICStub refills In-Reply-To: References: Message-ID: <9f44d85e-d737-87aa-70a4-020ae713d4cb@oracle.com> On 11/22/19 4:03 PM, erik.osterlund at oracle.com wrote: > Hi, > > Today, when GCs scan the stack, all nmethods roots found from threads > are nmethods that have gone through an nmethod entry barrier (in the > case of concurrent class unloading), except the super special sweeper > nmethod that is currently being processed when it runs out of ICStubs > and need to safepoint to refill ICStubs, which with the fantastic > safepoint coalescing optimization that we all know and love, can get > fused into a GC safepoint, which will find the sweeper nmethod and be > confused why it has not gone through an entry barrier before it yielded > to the safepoint like everyone else. > > This causes some headache, and I would like to harmonize this better by > simply calling the nmethod entry barrier on that nmethod in the sweeper, > before yielding to the safepoint. Then there is no need to treat it as a > special case. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8234662 > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8234662/webrev.00/ Looks good to me. /Per From stefan.karlsson at oracle.com Tue Dec 3 13:15:36 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 3 Dec 2019 14:15:36 +0100 Subject: RFR: 8234662: Sweeper should keep current nmethod alive before yielding for ICStub refills In-Reply-To: References: Message-ID: Looks good. StefanK On 2019-11-22 16:03, erik.osterlund at oracle.com wrote: > Hi, > > Today, when GCs scan the stack, all nmethods roots found from threads > are nmethods that have gone through an nmethod entry barrier (in the > case of concurrent class unloading), except the super special sweeper > nmethod that is currently being processed when it runs out of ICStubs > and need to safepoint to refill ICStubs, which with the fantastic > safepoint coalescing optimization that we all know and love, can get > fused into a GC safepoint, which will find the sweeper nmethod and be > confused why it has not gone through an entry barrier before it yielded > to the safepoint like everyone else. > > This causes some headache, and I would like to harmonize this better by > simply calling the nmethod entry barrier on that nmethod in the sweeper, > before yielding to the safepoint. Then there is no need to treat it as a > special case. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8234662 > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8234662/webrev.00/ > > Thanks. > /Erik From erik.osterlund at oracle.com Tue Dec 3 13:16:11 2019 From: erik.osterlund at oracle.com (erik.osterlund at oracle.com) Date: Tue, 3 Dec 2019 14:16:11 +0100 Subject: RFR: 8234662: Sweeper should keep current nmethod alive before yielding for ICStub refills In-Reply-To: <9f44d85e-d737-87aa-70a4-020ae713d4cb@oracle.com> References: <9f44d85e-d737-87aa-70a4-020ae713d4cb@oracle.com> Message-ID: <79b1a1e5-1aa0-12da-6f6b-f135ec0d170f@oracle.com> Hi Per, Thanks for the review. /Erik On 12/3/19 2:15 PM, Per Liden wrote: > > On 11/22/19 4:03 PM, erik.osterlund at oracle.com wrote: >> Hi, >> >> Today, when GCs scan the stack, all nmethods roots found from threads >> are nmethods that have gone through an nmethod entry barrier (in the >> case of concurrent class unloading), except the super special sweeper >> nmethod that is currently being processed when it runs out of ICStubs >> and need to safepoint to refill ICStubs, which with the fantastic >> safepoint coalescing optimization that we all know and love, can get >> fused into a GC safepoint, which will find the sweeper nmethod and be >> confused why it has not gone through an entry barrier before it >> yielded to the safepoint like everyone else. >> >> This causes some headache, and I would like to harmonize this better >> by simply calling the nmethod entry barrier on that nmethod in the >> sweeper, before yielding to the safepoint. Then there is no need to >> treat it as a special case. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8234662 >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8234662/webrev.00/ > > Looks good to me. > > /Per From erik.osterlund at oracle.com Tue Dec 3 13:19:23 2019 From: erik.osterlund at oracle.com (erik.osterlund at oracle.com) Date: Tue, 3 Dec 2019 14:19:23 +0100 Subject: RFR: 8234662: Sweeper should keep current nmethod alive before yielding for ICStub refills In-Reply-To: References: Message-ID: <6f276fe3-49e8-6788-4685-19cf92ac2e14@oracle.com> Hi Stefan, Thanks for the review. /Erik On 12/3/19 2:15 PM, Stefan Karlsson wrote: > Looks good. > > StefanK > > On 2019-11-22 16:03, erik.osterlund at oracle.com wrote: >> Hi, >> >> Today, when GCs scan the stack, all nmethods roots found from threads >> are nmethods that have gone through an nmethod entry barrier (in the >> case of concurrent class unloading), except the super special sweeper >> nmethod that is currently being processed when it runs out of ICStubs >> and need to safepoint to refill ICStubs, which with the fantastic >> safepoint coalescing optimization that we all know and love, can get >> fused into a GC safepoint, which will find the sweeper nmethod and be >> confused why it has not gone through an entry barrier before it >> yielded to the safepoint like everyone else. >> >> This causes some headache, and I would like to harmonize this better >> by simply calling the nmethod entry barrier on that nmethod in the >> sweeper, before yielding to the safepoint. Then there is no need to >> treat it as a special case. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8234662 >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8234662/webrev.00/ >> >> Thanks. >> /Erik From david.holmes at oracle.com Tue Dec 3 13:32:03 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 3 Dec 2019 23:32:03 +1000 Subject: RFR: 8235218: Minimal VM is broken after JDK-8173361 In-Reply-To: <7c8a51f1-5cd0-bfbb-86f0-0684c2e87eea@oracle.com> References: <7c8a51f1-5cd0-bfbb-86f0-0684c2e87eea@oracle.com> Message-ID: On 3/12/2019 11:13 pm, coleen.phillimore at oracle.com wrote: > Thanks for fixing and David? for pushing this. Jie is a committer :) I just had to review. David > Coleen > > On 12/2/19 8:43 PM, Jie Fu wrote: >> Hi all, >> >> May I get reviews for the small fix? >> >> JBS:??? https://bugs.openjdk.java.net/browse/JDK-8235218 >> Webrev: http://cr.openjdk.java.net/~jiefu/8235218/webrev.00/ >> >> Thanks a lot. >> Best regards, >> Jie >> > From martin.doerr at sap.com Tue Dec 3 13:44:40 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 3 Dec 2019 13:44:40 +0000 Subject: RFR [XS]: 8235243: handle VS2017 15.9 and VS2019 in abstract_vm_version In-Reply-To: References: Message-ID: Hi Matthias, looks good. Thanks for updating. Best regards, Martin > -----Original Message----- > From: hotspot-dev On Behalf Of > Baesken, Matthias > Sent: Dienstag, 3. Dezember 2019 11:48 > To: 'hotspot-dev at openjdk.java.net' > Subject: RFR [XS]: 8235243: handle VS2017 15.9 and VS2019 in > abstract_vm_version > > Hello, please review this small enhancement for Visual Studio compiler > version handling. > > The C++ compiler handling coding in abstract_vm_version.cpp currently > misses handling nicely VS2017 15.9 ("update 9") and VS2019. > This leads to bad output like "unknown MS VC++:1916" in the hs_err file. > > A list of _MSC_VER can be found for example here : > > https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B > > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8235243 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235243.0/ > > > Thanks, Matthias From matthias.baesken at sap.com Tue Dec 3 15:08:22 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 3 Dec 2019 15:08:22 +0000 Subject: RFR [XS]: 8235243: handle VS2017 15.9 and VS2019 in abstract_vm_version In-Reply-To: References: Message-ID: Thanks for the review ! Will push it as XS tomorrow, in case no objections come up . Best regards, Matthias > > Hi Matthias, > > looks good. Thanks for updating. > > Best regards, > Martin > > > > -----Original Message----- > > From: hotspot-dev On Behalf > Of > > Baesken, Matthias > > Sent: Dienstag, 3. Dezember 2019 11:48 > > To: 'hotspot-dev at openjdk.java.net' > > Subject: RFR [XS]: 8235243: handle VS2017 15.9 and VS2019 in > > abstract_vm_version > > > > Hello, please review this small enhancement for Visual Studio compiler > > version handling. > > > > The C++ compiler handling coding in abstract_vm_version.cpp currently > > misses handling nicely VS2017 15.9 ("update 9") and VS2019. > > This leads to bad output like "unknown MS VC++:1916" in the hs_err file. > > > > A list of _MSC_VER can be found for example here : > > > > https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B > > > > > > > > Bug/webrev : > > > > https://bugs.openjdk.java.net/browse/JDK-8235243 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235243.0/ > > > > > > Thanks, Matthias From martin.doerr at sap.com Tue Dec 3 15:29:44 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 3 Dec 2019 15:29:44 +0000 Subject: RFR [XS]: 8235243: handle VS2017 15.9 and VS2019 in abstract_vm_version In-Reply-To: References: Message-ID: Hi Matthias, I'm ok with applying the trivial rule. Best regards, Martin > -----Original Message----- > From: Baesken, Matthias > Sent: Dienstag, 3. Dezember 2019 16:08 > To: Doerr, Martin ; 'hotspot- > dev at openjdk.java.net' > Subject: RE: RFR [XS]: 8235243: handle VS2017 15.9 and VS2019 in > abstract_vm_version > > Thanks for the review ! Will push it as XS tomorrow, in case no objections > come up . > > Best regards, Matthias > > > > > > Hi Matthias, > > > > looks good. Thanks for updating. > > > > Best regards, > > Martin > > > > > > > -----Original Message----- > > > From: hotspot-dev On Behalf > > Of > > > Baesken, Matthias > > > Sent: Dienstag, 3. Dezember 2019 11:48 > > > To: 'hotspot-dev at openjdk.java.net' > > > Subject: RFR [XS]: 8235243: handle VS2017 15.9 and VS2019 in > > > abstract_vm_version > > > > > > Hello, please review this small enhancement for Visual Studio compiler > > > version handling. > > > > > > The C++ compiler handling coding in abstract_vm_version.cpp currently > > > misses handling nicely VS2017 15.9 ("update 9") and VS2019. > > > This leads to bad output like "unknown MS VC++:1916" in the hs_err file. > > > > > > A list of _MSC_VER can be found for example here : > > > > > > https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B > > > > > > > > > > > > Bug/webrev : > > > > > > https://bugs.openjdk.java.net/browse/JDK-8235243 > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235243.0/ > > > > > > > > > Thanks, Matthias From matthias.baesken at sap.com Tue Dec 3 15:38:04 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 3 Dec 2019 15:38:04 +0000 Subject: 8234397: add OS uptime information to os::print_os_info output In-Reply-To: <038f060e-b253-e33f-bc03-a56c0735a90f@oracle.com> References: <30E06543-E081-429B-8293-8CA81D1F6870@sap.com> <038f060e-b253-e33f-bc03-a56c0735a90f@oracle.com> Message-ID: Hi David, here is a new webrev based on your comments : http://cr.openjdk.java.net/~mbaesken/webrevs/8234397.4/ > > Is %llu supported by all compilers (32-bit as well)? > Not sure , I checked and it works with VS2017/32bit. But I don't know about other compilers. So I better use long / %ld. Btw I noticed that os::print_date_and_time ... // print elapsed time in a human-readable format: Contains very similar coding to what I do in os::print_dhm --> should this maybe be unified ? Best regards, Matthias > Hi Matthias, > > On 29/11/2019 11:58 pm, Baesken, Matthias wrote: > > Hi, here is a new webrev, this time with nicer output (see os.cpp) . > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8234397.3/ > > src/hotspot/os/bsd/os_bsd.cpp > > This link may be more general than the Apple "iOS" one. > > https://man.openbsd.org/sysctl.2 > > + time_t bootsec = boottime.tv_sec, currsec = time(NULL); > > Please write the two declarations on separate lines. > > --- > > src/hotspot/os/posix/os_posix.cpp > > + // unfortunately it does not work on macOS > > Still no mention of Linux > > --- > > src/hotspot/share/runtime/os.cpp > > + if (minutes < 10) { > + st->print_cr("%s %llu days %llu:0%llu hours", startStr, days, > hours, minutes); > + } else { > + st->print_cr("%s %llu days %llu:%llu hours", startStr, days, > hours, minutes); > + } > > You should just use a zero-padded width of 2 to get 01..09 for minutes. > > Is %llu supported by all compilers (32-bit as well)? > > --- > > src/hotspot/os/windows/os_windows.cpp > > + unsigned long long ticks = GetTickCount64(); > + os::print_dhm(st, "OS uptime:", ticks); > > GetTickCount64 returns milliseconds since boot not seconds. > > --- > > Thanks, > David > From patricio.chilano.mateo at oracle.com Tue Dec 3 17:52:00 2019 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Tue, 3 Dec 2019 14:52:00 -0300 Subject: RFR: 8234742: Improve handshake logging In-Reply-To: References: <31e700c7-d2fe-2977-e1e2-13e3cc10927d@oracle.com> Message-ID: <672781fe-d004-2503-5b06-5ec9022ed737@oracle.com> Hi Robbin, v3 looks good to me! Patricio On 11/27/19 10:51 AM, Robbin Ehn wrote: > v3 which is also rebased on latest 8234796: > Inc: > http://cr.openjdk.java.net/~rehn/8234742/v3/inc/webrev/index.html > Full: > http://cr.openjdk.java.net/~rehn/8234742/v3/full/webrev/index.html > > t1-3 > > Thanks, Robbin > >> Thanks, >> David >> ----- >> >>> Thanks, Robbin >>> >>> On 11/25/19 5:33 PM, Robbin Ehn wrote: >>>> Hi all, please review. >>>> >>>> There is little useful information in the handshaking logs. >>>> This changes the handshakes logs similar to safepoint logs, so the >>>> basic need of >>>> what handshake operation and how long it took easily can be tracked. >>>> Also the per thread log is a bit enhanced. >>>> >>>> The refactoring using HandshakeOperation instead of a ThreadClosure >>>> is not >>>> merely for this change. Other changes in the pipeline also require >>>> a more >>>> complex HandshakeOperation. >>>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8234742 >>>> Changeset: >>>> http://cr.openjdk.java.net/~rehn/8234742/full/webrev/ >>>> >>>> Passes t1-3. >>>> >>>> Thanks, Robbin >>>> >>>> Examples: >>>> -Xlog:handshake,safepoint >>>> >>>> [7.148s][info][safepoint] Safepoint "ZMarkStart", Time since last: >>>> 381873579 ns, Reaching safepoint: 451132 ns, At safepoint: 491202 >>>> ns, Total: 942334 ns >>>> [7.151s][info][handshake] Handshake "ZMarkFlushAndFreeStacks", >>>> Targeted threads: 25, Executed by targeted threads: 8, Total >>>> completion time: 46884 ns >>>> [7.151s][info][handshake] Handshake "ZMarkFlushAndFreeStacks", >>>> Targeted threads: 25, Executed by targeted threads: 10, Total >>>> completion time: 94547 ns >>>> [7.152s][info][handshake] Handshake "ZMarkFlushAndFreeStacks", >>>> Targeted threads: 25, Executed by targeted threads: 10, Total >>>> completion time: 33545 ns >>>> [7.154s][info][safepoint] Safepoint "ZMarkEnd", Time since last: >>>> 4697901 ns, Reaching safepoint: 218800 ns, At safepoint: 1462059 >>>> ns, Total: 1680859 ns >>>> [7.156s][info][handshake] Handshake "ZRendezvous", Targeted >>>> threads: 25, Executed by targeted threads: 10, Total completion >>>> time: 37291 ns >>>> [7.157s][info][safepoint] Safepoint "ZVerify", Time since last: >>>> 2201206 ns, Reaching safepoint: 295463 ns, At safepoint: 928077 ns, >>>> Total: 1223540 ns >>>> [7.161s][info][safepoint] Safepoint "ZRelocateStart", Time since >>>> last: 3161645 ns, Reaching safepoint: 206278 ns, At safepoint: >>>> 357284 ns, Total: 563562 ns >>>> [8.162s][info][safepoint] Safepoint "Cleanup", Time since last: >>>> 1000123769 ns, Reaching safepoint: 526489 ns, At safepoint: 23345 >>>> ns, Total: 549834 ns >>>> [8.182s][info][handshake] Handshake "RevokeOneBias", Targeted >>>> threads: 1, Executed by targeted threads: 0, Total completion time: >>>> 41322 ns >>>> >>>> -Xlog:handshake*=trace >>>> >>>> [1.259s][trace][handshake ] Threads signaled, begin processing >>>> blocked threads by VMThtread >>>> [1.259s][trace][handshake ] Processing handshake by VMThtread >>>> [1.259s][debug][handshake,task ] Operation: ZRendezvous for thread >>>> 0x00007f2594022800, is_vm_thread: true, completed in 487 ns >>>> [1.259s][debug][handshake,task ] Operation: ZRendezvous for thread >>>> 0x00007f259459e000, is_vm_thread: false, completed in 1233 ns >>>> [1.259s][trace][handshake ] Processing handshake by VMThtread >>>> [1.259s][debug][handshake,task ] Operation: ZRendezvous for thread >>>> 0x00007f25945a0000, is_vm_thread: false, completed in 669 ns >>>> [1.259s][debug][handshake,task ] Operation: ZRendezvous for thread >>>> 0x00007f259428a800, is_vm_thread: true, completed in 462 ns >>>> [1.259s][debug][handshake,task ] Operation: ZRendezvous for thread >>>> 0x00007f25945b3800, is_vm_thread: false, completed in 574 ns >>>> ... >>>> [1.259s][trace][handshake ] Processing handshake by VMThtread >>>> [1.259s][debug][handshake,task ] Operation: ZRendezvous for thread >>>> 0x00007f25945b6000, is_vm_thread: true, completed in 100 ns >>>> [1.259s][trace][handshake ] Processing handshake by VMThtread >>>> [1.259s][debug][handshake,task ] Operation: ZRendezvous for thread >>>> 0x00007f25945b7800, is_vm_thread: true, completed in 103 ns >>>> [1.260s][info ][handshake ] Handshake "ZRendezvous", Targeted >>>> threads: 28, Executed by targeted threads: 4, Total completion >>>> time: 629534 ns >>>> [1.260s][debug][handshake,task ] Operation: ZRendezvous for thread >>>> 0x00007f25945a3800, is_vm_thread: false, completed in 608 ns From coleen.phillimore at oracle.com Tue Dec 3 18:21:15 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 3 Dec 2019 13:21:15 -0500 Subject: RFR (XS) 8235273: nmethodLocker not needed for COMPILED_METHOD_UNLOAD events Message-ID: <3630a545-2b2c-e17c-bbe5-98ae8508ae38@oracle.com> Summary: remove unnecessary nmethodLocker See bug for more details.? Tested with tier2-8. open webrev at http://cr.openjdk.java.net/~coleenp/2019/8235273.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8235273 (Note, this has a trivial merge with the change for JDK-8212160). Thanks, Coleen From kim.barrett at oracle.com Tue Dec 3 19:49:16 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 3 Dec 2019 14:49:16 -0500 Subject: RFR: 8234779: Provide idiom for declaring classes noncopyable In-Reply-To: <5326a54a-4160-1c26-48b9-c8af451ffd6a@oracle.com> References: <233a779d-ae3b-4856-9c44-dd81bfceab6e@oracle.com> <129579A4-01C4-4F62-9582-06BE19CB13C0@oracle.com> <4ECD9AC3-A67D-4587-B9DF-A00F713B8D79@oracle.com> <5326a54a-4160-1c26-48b9-c8af451ffd6a@oracle.com> Message-ID: <915A772C-7444-44F8-A4F4-0C00EB8E5E73@oracle.com> > On Dec 3, 2019, at 3:04 AM, David Holmes wrote: > > Hi Kim, > > There is a bit of an include file inconsistency, but not your fault. In places you have added: > > #include "utilities/globalDefinitions.hpp" > > to files that should already (but don't) include macros.hpp. And globalDefinitions.hpp itself includes macros.hpp which "fixes" that. But then in other places we have: > > +#include "utilities/globalDefinitions.hpp" > #include "utilities/macros.hpp" > > Either all sites should include both files or else only globalDefinitions.hpp. But perhaps a later cleanup. Not all places that #include globalDefinitions.hpp need to explicitly #include macros.hpp. To avoid implicit #include dependencies macros.hpp ought to be directly #included if the file directly uses definitions from there. I'm sure there are files that use things from globalDefinitions.hpp but not from macros.hpp. I'm equally sure there are files that ought to be directly including one or the other, or both, but aren't. Addressing and maintaining that would require some significant work, probably involving adding some tooling similar to https://github.com/include-what-you-use/include-what-you-use to our build process. From kim.barrett at oracle.com Tue Dec 3 19:50:10 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 3 Dec 2019 14:50:10 -0500 Subject: RFR: 8213415: BitMap::word_index_round_up overflow problems In-Reply-To: References: <24C2BDC9-AC35-4804-95C8-1B59747C1494@oracle.com> <5bd19480-d570-b73d-70fe-10e93ef2ecb8@oracle.com> <3F4F27AC-4812-4E55-99DF-25F0A7BD991D@oracle.com> <2FB27F12-B779-419E-A501-715248CF2309@oracle.com> <4CD1F762-EAF0-497B-A307-2E9CABFCDD14@oracle.com> <19E959E2-DE3A-4C0A-BA2F-962690279AA3@oracle.com> Message-ID: > On Dec 3, 2019, at 4:01 AM, Thomas Schatzl wrote: > > Hi, > > On 03.12.19 08:22, Kim Barrett wrote: >>> On Dec 3, 2019, at 1:57 AM, Thomas St?fe wrote: >>> >>> Hi Kim, >>> >>> After realizing my thinking error, I'm fine with the patch as it is. >> Thanks. >>> > > the latest patch (with the comment changes) is still fine with me. > > Thomas Thanks. From coleen.phillimore at oracle.com Tue Dec 3 20:35:18 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 3 Dec 2019 15:35:18 -0500 Subject: RFR: 8234779: Provide idiom for declaring classes noncopyable In-Reply-To: References: <233a779d-ae3b-4856-9c44-dd81bfceab6e@oracle.com> <129579A4-01C4-4F62-9582-06BE19CB13C0@oracle.com> <4ECD9AC3-A67D-4587-B9DF-A00F713B8D79@oracle.com> Message-ID: <924d7cc0-b13b-d683-72f7-e54131c0f34b@oracle.com> This change looks good to me.? thanks. Coleen On 12/3/19 2:27 AM, Kim Barrett wrote: >> On Dec 2, 2019, at 2:49 PM, Kim Barrett wrote: >> >>> On Dec 2, 2019, at 4:04 AM, Per Liden wrote: >>> I'd also like to suggest that this type of macro belongs in globalDefinitions.hpp rather than macros.hpp, since it feels more akin to ALWAYSINLINE/ATTRIBUTE_ALIGNED/etc than INCLUDE_JVMTI/X86_ONLY/etc. >> Yeah, now that you point it out, macros.hpp is definitely the wrong place for it. >> I personally don?t care for the dumping ground that is globalDefinitions.hpp and >> would prefer smaller and more focused headers, but that?s an entirely different >> discussion. I?ll move this new macro there and clean up the patch accordingly >> (there are #include changes that will need adjusting). > Moved macro to globalDefinitions.hpp and updated #includes. > > New webrevs: > full: https://cr.openjdk.java.net/~kbarrett/8234779/open.01/ > incr: https://cr.openjdk.java.net/~kbarrett/8234779/open.01.inc/ > > Testing: > mach5 tier1 (linux-x64, windows-x64, solaris-x64, macosx-x64) > > linux fastdebug builds for the following: > 1. platforms: aarch64, arm32, ppc64le, s390x. > 2. x86_64 with shenandoah included. > 3. x86_64 minimal configuration. > From kim.barrett at oracle.com Tue Dec 3 20:51:39 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 3 Dec 2019 15:51:39 -0500 Subject: RFR: 8234779: Provide idiom for declaring classes noncopyable In-Reply-To: <924d7cc0-b13b-d683-72f7-e54131c0f34b@oracle.com> References: <233a779d-ae3b-4856-9c44-dd81bfceab6e@oracle.com> <129579A4-01C4-4F62-9582-06BE19CB13C0@oracle.com> <4ECD9AC3-A67D-4587-B9DF-A00F713B8D79@oracle.com> <924d7cc0-b13b-d683-72f7-e54131c0f34b@oracle.com> Message-ID: <1447E1BA-FB85-4E10-B24F-9E1FFECE1C36@oracle.com> > On Dec 3, 2019, at 3:35 PM, coleen.phillimore at oracle.com wrote: > > This change looks good to me. thanks. Thanks. From david.holmes at oracle.com Wed Dec 4 04:31:16 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 4 Dec 2019 14:31:16 +1000 Subject: 8234397: add OS uptime information to os::print_os_info output In-Reply-To: References: <30E06543-E081-429B-8293-8CA81D1F6870@sap.com> <038f060e-b253-e33f-bc03-a56c0735a90f@oracle.com> Message-ID: Hi Matthias, On 4/12/2019 1:38 am, Baesken, Matthias wrote: > Hi David, here is a new webrev based on your comments : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8234397.4/ Thanks for the updates. >> >> Is %llu supported by all compilers (32-bit as well)? >> > > Not sure , I checked and it works with VS2017/32bit. But I don't know about other compilers. So I better use long / %ld. I think int would be fine - 2^31 seconds is a long time. > > Btw I noticed that > > os::print_date_and_time > ... > // print elapsed time in a human-readable format: > > Contains very similar coding to what I do in os::print_dhm --> should this maybe be unified ? I don't really see a way to unify. You might hoist the constants out to be static in the file for sharing: const int secs_per_day = 86400; const int secs_per_hour = 3600; const int secs_per_min = 60; and add others as needed. Thanks, David > > Best regards, Matthias > > > >> Hi Matthias, >> >> On 29/11/2019 11:58 pm, Baesken, Matthias wrote: >>> Hi, here is a new webrev, this time with nicer output (see os.cpp) . >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8234397.3/ >> >> src/hotspot/os/bsd/os_bsd.cpp >> >> This link may be more general than the Apple "iOS" one. >> >> https://man.openbsd.org/sysctl.2 >> >> + time_t bootsec = boottime.tv_sec, currsec = time(NULL); >> >> Please write the two declarations on separate lines. >> >> --- >> >> src/hotspot/os/posix/os_posix.cpp >> >> + // unfortunately it does not work on macOS >> >> Still no mention of Linux >> >> --- >> >> src/hotspot/share/runtime/os.cpp >> >> + if (minutes < 10) { >> + st->print_cr("%s %llu days %llu:0%llu hours", startStr, days, >> hours, minutes); >> + } else { >> + st->print_cr("%s %llu days %llu:%llu hours", startStr, days, >> hours, minutes); >> + } >> >> You should just use a zero-padded width of 2 to get 01..09 for minutes. >> >> Is %llu supported by all compilers (32-bit as well)? >> >> --- >> >> src/hotspot/os/windows/os_windows.cpp >> >> + unsigned long long ticks = GetTickCount64(); >> + os::print_dhm(st, "OS uptime:", ticks); >> >> GetTickCount64 returns milliseconds since boot not seconds. >> >> --- >> >> Thanks, >> David >> > From david.holmes at oracle.com Wed Dec 4 04:49:56 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 4 Dec 2019 14:49:56 +1000 Subject: RFR (XS) 8235273: nmethodLocker not needed for COMPILED_METHOD_UNLOAD events In-Reply-To: <3630a545-2b2c-e17c-bbe5-98ae8508ae38@oracle.com> References: <3630a545-2b2c-e17c-bbe5-98ae8508ae38@oracle.com> Message-ID: <8530df0a-c67c-3159-4b96-8bde4e925434@oracle.com> Hi Coleen, That all seems fine to me. Thanks, David On 4/12/2019 4:21 am, coleen.phillimore at oracle.com wrote: > Summary: remove unnecessary nmethodLocker > > See bug for more details.? Tested with tier2-8. > > open webrev at http://cr.openjdk.java.net/~coleenp/2019/8235273.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8235273 > > (Note, this has a trivial merge with the change for JDK-8212160). > > Thanks, > Coleen From david.holmes at oracle.com Wed Dec 4 04:57:41 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 4 Dec 2019 14:57:41 +1000 Subject: RFR [XS]: 8235243: handle VS2017 15.9 and VS2019 in abstract_vm_version In-Reply-To: References: Message-ID: Looks fine to me too. (As usual I lament the fact there is no _MSVC_VER_STR available ) Thanks, David On 3/12/2019 8:48 pm, Baesken, Matthias wrote: > Hello, please review this small enhancement for Visual Studio compiler version handling. > > The C++ compiler handling coding in abstract_vm_version.cpp currently misses handling nicely VS2017 15.9 ("update 9") and VS2019. > This leads to bad output like "unknown MS VC++:1916" in the hs_err file. > > A list of _MSC_VER can be found for example here : > > https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B > > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8235243 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235243.0/ > > > Thanks, Matthias > From david.holmes at oracle.com Wed Dec 4 05:13:49 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 4 Dec 2019 15:13:49 +1000 Subject: RFR: 8234779: Provide idiom for declaring classes noncopyable In-Reply-To: <915A772C-7444-44F8-A4F4-0C00EB8E5E73@oracle.com> References: <233a779d-ae3b-4856-9c44-dd81bfceab6e@oracle.com> <129579A4-01C4-4F62-9582-06BE19CB13C0@oracle.com> <4ECD9AC3-A67D-4587-B9DF-A00F713B8D79@oracle.com> <5326a54a-4160-1c26-48b9-c8af451ffd6a@oracle.com> <915A772C-7444-44F8-A4F4-0C00EB8E5E73@oracle.com> Message-ID: <2acbd2d6-715a-bdcb-b22b-a8b5cc8b4f23@oracle.com> On 4/12/2019 5:49 am, Kim Barrett wrote: >> On Dec 3, 2019, at 3:04 AM, David Holmes wrote: >> >> Hi Kim, >> >> There is a bit of an include file inconsistency, but not your fault. In places you have added: >> >> #include "utilities/globalDefinitions.hpp" >> >> to files that should already (but don't) include macros.hpp. And globalDefinitions.hpp itself includes macros.hpp which "fixes" that. But then in other places we have: >> >> +#include "utilities/globalDefinitions.hpp" >> #include "utilities/macros.hpp" >> >> Either all sites should include both files or else only globalDefinitions.hpp. But perhaps a later cleanup. > > Not all places that #include globalDefinitions.hpp need to > explicitly #include macros.hpp. By "all" I meant all files that use both headers. > To avoid implicit #include > dependencies macros.hpp ought to be directly #included if the file > directly uses definitions from there. I'm sure there are files that > use things from globalDefinitions.hpp but not from macros.hpp. I'm > equally sure there are files that ought to be directly including one > or the other, or both, but aren't. Addressing and maintaining that > would require some significant work, probably involving adding some > tooling similar to > https://github.com/include-what-you-use/include-what-you-use > to our build process. A tool for managing includes? What a novel idea ;-) Cheers, David ----- From christoph.langer at sap.com Wed Dec 4 08:06:05 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 4 Dec 2019 08:06:05 +0000 Subject: native debug symbols support on Windows Message-ID: Hi, I'm currently looking into native debug symbols support for Windows. The OpenJDK build system supports these two configure flags --with-native-debug-symbols= (among a few other options which I don't want to discuss here). So, the name implies that for "internal", debug symbols should be contained in the binaries. And "external" should create separate files that contain the debug symbols. However, to my knowledge, Windows would always make use of external symbol files, named *.pdb. And there is no way to have the debug symbols included in the binaries. Is that correct or am I wrong in this assumption? If it's true, I guess --with-native-debug-symbols=internal would not make sense on Windows and should rather be rejected by configure. Otherwise, if we were to support --with-native-debug-symbols=internal, the build is broken for target "test-image" when it comes to building/copying the gtest image. I'd like to fix either the one way or the other. What do people think? Thanks for your help Christoph From robbin.ehn at oracle.com Wed Dec 4 08:10:14 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 4 Dec 2019 09:10:14 +0100 Subject: RFR: 8234742: Improve handshake logging In-Reply-To: <672781fe-d004-2503-5b06-5ec9022ed737@oracle.com> References: <31e700c7-d2fe-2977-e1e2-13e3cc10927d@oracle.com> <672781fe-d004-2503-5b06-5ec9022ed737@oracle.com> Message-ID: <304951b2-2f2c-b620-c6ee-74ec6c36c673@oracle.com> Thanks Patricio! /Robbin On 2019-12-03 18:52, Patricio Chilano wrote: > Hi Robbin, > > v3 looks good to me! > > Patricio > On 11/27/19 10:51 AM, Robbin Ehn wrote: >> v3 which is also rebased on latest 8234796: >> Inc: >> http://cr.openjdk.java.net/~rehn/8234742/v3/inc/webrev/index.html >> Full: >> http://cr.openjdk.java.net/~rehn/8234742/v3/full/webrev/index.html >> >> t1-3 >> >> Thanks, Robbin >> >>> Thanks, >>> David >>> ----- >>> >>>> Thanks, Robbin >>>> >>>> On 11/25/19 5:33 PM, Robbin Ehn wrote: >>>>> Hi all, please review. >>>>> >>>>> There is little useful information in the handshaking logs. >>>>> This changes the handshakes logs similar to safepoint logs, so the >>>>> basic need of >>>>> what handshake operation and how long it took easily can be tracked. >>>>> Also the per thread log is a bit enhanced. >>>>> >>>>> The refactoring using HandshakeOperation instead of a >>>>> ThreadClosure is not >>>>> merely for this change. Other changes in the pipeline also require >>>>> a more >>>>> complex HandshakeOperation. >>>>> >>>>> Issue: >>>>> https://bugs.openjdk.java.net/browse/JDK-8234742 >>>>> Changeset: >>>>> http://cr.openjdk.java.net/~rehn/8234742/full/webrev/ >>>>> >>>>> Passes t1-3. >>>>> >>>>> Thanks, Robbin >>>>> >>>>> Examples: >>>>> -Xlog:handshake,safepoint >>>>> >>>>> [7.148s][info][safepoint] Safepoint "ZMarkStart", Time since last: >>>>> 381873579 ns, Reaching safepoint: 451132 ns, At safepoint: 491202 >>>>> ns, Total: 942334 ns >>>>> [7.151s][info][handshake] Handshake "ZMarkFlushAndFreeStacks", >>>>> Targeted threads: 25, Executed by targeted threads: 8, Total >>>>> completion time: 46884 ns >>>>> [7.151s][info][handshake] Handshake "ZMarkFlushAndFreeStacks", >>>>> Targeted threads: 25, Executed by targeted threads: 10, Total >>>>> completion time: 94547 ns >>>>> [7.152s][info][handshake] Handshake "ZMarkFlushAndFreeStacks", >>>>> Targeted threads: 25, Executed by targeted threads: 10, Total >>>>> completion time: 33545 ns >>>>> [7.154s][info][safepoint] Safepoint "ZMarkEnd", Time since last: >>>>> 4697901 ns, Reaching safepoint: 218800 ns, At safepoint: 1462059 >>>>> ns, Total: 1680859 ns >>>>> [7.156s][info][handshake] Handshake "ZRendezvous", Targeted >>>>> threads: 25, Executed by targeted threads: 10, Total completion >>>>> time: 37291 ns >>>>> [7.157s][info][safepoint] Safepoint "ZVerify", Time since last: >>>>> 2201206 ns, Reaching safepoint: 295463 ns, At safepoint: 928077 >>>>> ns, Total: 1223540 ns >>>>> [7.161s][info][safepoint] Safepoint "ZRelocateStart", Time since >>>>> last: 3161645 ns, Reaching safepoint: 206278 ns, At safepoint: >>>>> 357284 ns, Total: 563562 ns >>>>> [8.162s][info][safepoint] Safepoint "Cleanup", Time since last: >>>>> 1000123769 ns, Reaching safepoint: 526489 ns, At safepoint: 23345 >>>>> ns, Total: 549834 ns >>>>> [8.182s][info][handshake] Handshake "RevokeOneBias", Targeted >>>>> threads: 1, Executed by targeted threads: 0, Total completion >>>>> time: 41322 ns >>>>> >>>>> -Xlog:handshake*=trace >>>>> >>>>> [1.259s][trace][handshake ] Threads signaled, begin processing >>>>> blocked threads by VMThtread >>>>> [1.259s][trace][handshake ] Processing handshake by VMThtread >>>>> [1.259s][debug][handshake,task ] Operation: ZRendezvous for thread >>>>> 0x00007f2594022800, is_vm_thread: true, completed in 487 ns >>>>> [1.259s][debug][handshake,task ] Operation: ZRendezvous for thread >>>>> 0x00007f259459e000, is_vm_thread: false, completed in 1233 ns >>>>> [1.259s][trace][handshake ] Processing handshake by VMThtread >>>>> [1.259s][debug][handshake,task ] Operation: ZRendezvous for thread >>>>> 0x00007f25945a0000, is_vm_thread: false, completed in 669 ns >>>>> [1.259s][debug][handshake,task ] Operation: ZRendezvous for thread >>>>> 0x00007f259428a800, is_vm_thread: true, completed in 462 ns >>>>> [1.259s][debug][handshake,task ] Operation: ZRendezvous for thread >>>>> 0x00007f25945b3800, is_vm_thread: false, completed in 574 ns >>>>> ... >>>>> [1.259s][trace][handshake ] Processing handshake by VMThtread >>>>> [1.259s][debug][handshake,task ] Operation: ZRendezvous for thread >>>>> 0x00007f25945b6000, is_vm_thread: true, completed in 100 ns >>>>> [1.259s][trace][handshake ] Processing handshake by VMThtread >>>>> [1.259s][debug][handshake,task ] Operation: ZRendezvous for thread >>>>> 0x00007f25945b7800, is_vm_thread: true, completed in 103 ns >>>>> [1.260s][info ][handshake ] Handshake "ZRendezvous", Targeted >>>>> threads: 28, Executed by targeted threads: 4, Total completion >>>>> time: 629534 ns >>>>> [1.260s][debug][handshake,task ] Operation: ZRendezvous for thread >>>>> 0x00007f25945a3800, is_vm_thread: false, completed in 608 ns > From matthias.baesken at sap.com Wed Dec 4 08:11:43 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 4 Dec 2019 08:11:43 +0000 Subject: 8234397: add OS uptime information to os::print_os_info output In-Reply-To: References: <30E06543-E081-429B-8293-8CA81D1F6870@sap.com> <038f060e-b253-e33f-bc03-a56c0735a90f@oracle.com> Message-ID: Hi David, I think I stay away from the unification . May I add you as a reviewer ? Best regards, Matthias > > Hi Matthias, > > On 4/12/2019 1:38 am, Baesken, Matthias wrote: > > Hi David, here is a new webrev based on your comments : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8234397.4/ > > Thanks for the updates. > > >> > >> Is %llu supported by all compilers (32-bit as well)? > >> > > > > Not sure , I checked and it works with VS2017/32bit. But I don't know > about other compilers. So I better use long / %ld. > > I think int would be fine - 2^31 seconds is a long time. > > > > > Btw I noticed that > > > > os::print_date_and_time > > ... > > // print elapsed time in a human-readable format: > > > > Contains very similar coding to what I do in os::print_dhm --> should this > maybe be unified ? > > I don't really see a way to unify. You might hoist the constants out to > be static in the file for sharing: > > const int secs_per_day = 86400; > const int secs_per_hour = 3600; > const int secs_per_min = 60; > > and add others as needed. > From matthias.baesken at sap.com Wed Dec 4 08:49:47 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 4 Dec 2019 08:49:47 +0000 Subject: 8235325: build failure on Linux after 8235243 - was : RE: RFR [XS]: 8235243: handle VS2017 15.9 and VS2019 in abstract_vm_version Message-ID: Thanks fort he reviews . Unfortunately I noticed a bit too late that a " was missing. Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8235325 http://cr.openjdk.java.net/~mbaesken/webrevs/8235325.0/ Best regards, Matthias > Looks fine to me too. > > (As usual I lament the fact there is no _MSVC_VER_STR available ) > > Thanks, > David > > On 3/12/2019 8:48 pm, Baesken, Matthias wrote: > > Hello, please review this small enhancement for Visual Studio compiler > version handling. > > > > The C++ compiler handling coding in abstract_vm_version.cpp currently > misses handling nicely VS2017 15.9 ("update 9") and VS2019. > > This leads to bad output like "unknown MS VC++:1916" in the hs_err file. > > > > A list of _MSC_VER can be found for example here : > > > > https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B > > > > > > > > Bug/webrev : > > > > https://bugs.openjdk.java.net/browse/JDK-8235243 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235243.0/ > > > > > > Thanks, Matthias > > From christoph.langer at sap.com Wed Dec 4 08:51:29 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 4 Dec 2019 08:51:29 +0000 Subject: 8235325: build failure on Linux after 8235243 - was : RE: RFR [XS]: 8235243: handle VS2017 15.9 and VS2019 in abstract_vm_version In-Reply-To: References: Message-ID: Hi Matthias, Looks good and trivial. Please push ASAP to resolve the build failures. ?? Thanks Christoph > -----Original Message----- > From: Baesken, Matthias > Sent: Mittwoch, 4. Dezember 2019 09:50 > To: David Holmes ; 'hotspot- > dev at openjdk.java.net' > Cc: Langer, Christoph > Subject: 8235325: build failure on Linux after 8235243 - was : RE: RFR [XS]: > 8235243: handle VS2017 15.9 and VS2019 in abstract_vm_version > > Thanks fort he reviews . Unfortunately I noticed a bit too late that a " was > missing. > Bug/webrev : > > > https://bugs.openjdk.java.net/browse/JDK-8235325 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235325.0/ > > > Best regards, Matthias > > > > Looks fine to me too. > > > > (As usual I lament the fact there is no _MSVC_VER_STR available ) > > > > Thanks, > > David > > > > On 3/12/2019 8:48 pm, Baesken, Matthias wrote: > > > Hello, please review this small enhancement for Visual Studio compiler > > version handling. > > > > > > The C++ compiler handling coding in abstract_vm_version.cpp currently > > misses handling nicely VS2017 15.9 ("update 9") and VS2019. > > > This leads to bad output like "unknown MS VC++:1916" in the hs_err file. > > > > > > A list of _MSC_VER can be found for example here : > > > > > > https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B > > > > > > > > > > > > Bug/webrev : > > > > > > https://bugs.openjdk.java.net/browse/JDK-8235243 > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235243.0/ > > > > > > > > > Thanks, Matthias > > > From serguei.spitsyn at oracle.com Wed Dec 4 10:14:26 2019 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 4 Dec 2019 02:14:26 -0800 Subject: RFR (XS) 8235273: nmethodLocker not needed for COMPILED_METHOD_UNLOAD events In-Reply-To: <8530df0a-c67c-3159-4b96-8bde4e925434@oracle.com> References: <3630a545-2b2c-e17c-bbe5-98ae8508ae38@oracle.com> <8530df0a-c67c-3159-4b96-8bde4e925434@oracle.com> Message-ID: <7cf9f5a3-f8c5-1981-b7b9-b83b5cdf7fe9@oracle.com> Hi Coleen, +1 Thanks, Serguei On 12/3/19 20:49, David Holmes wrote: > Hi Coleen, > > That all seems fine to me. > > Thanks, > David > > On 4/12/2019 4:21 am, coleen.phillimore at oracle.com wrote: >> Summary: remove unnecessary nmethodLocker >> >> See bug for more details.? Tested with tier2-8. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2019/8235273.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8235273 >> >> (Note, this has a trivial merge with the change for JDK-8212160). >> >> Thanks, >> Coleen From david.holmes at oracle.com Wed Dec 4 11:40:36 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 4 Dec 2019 21:40:36 +1000 Subject: 8234397: add OS uptime information to os::print_os_info output In-Reply-To: References: <30E06543-E081-429B-8293-8CA81D1F6870@sap.com> <038f060e-b253-e33f-bc03-a56c0735a90f@oracle.com> Message-ID: <20a034cf-09d2-5065-0806-1ffc0ede2e23@oracle.com> On 4/12/2019 6:11 pm, Baesken, Matthias wrote: > Hi David, I think I stay away from the unification . > May I add you as a reviewer ? Of course. David > Best regards, Matthias > >> >> Hi Matthias, >> >> On 4/12/2019 1:38 am, Baesken, Matthias wrote: >>> Hi David, here is a new webrev based on your comments : >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8234397.4/ >> >> Thanks for the updates. >> >>>> >>>> Is %llu supported by all compilers (32-bit as well)? >>>> >>> >>> Not sure , I checked and it works with VS2017/32bit. But I don't know >> about other compilers. So I better use long / %ld. >> >> I think int would be fine - 2^31 seconds is a long time. >> >>> >>> Btw I noticed that >>> >>> os::print_date_and_time >>> ... >>> // print elapsed time in a human-readable format: >>> >>> Contains very similar coding to what I do in os::print_dhm --> should this >> maybe be unified ? >> >> I don't really see a way to unify. You might hoist the constants out to >> be static in the file for sharing: >> >> const int secs_per_day = 86400; >> const int secs_per_hour = 3600; >> const int secs_per_min = 60; >> >> and add others as needed. >> > From coleen.phillimore at oracle.com Wed Dec 4 12:24:36 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 4 Dec 2019 07:24:36 -0500 Subject: RFR (XS) 8235273: nmethodLocker not needed for COMPILED_METHOD_UNLOAD events In-Reply-To: <7cf9f5a3-f8c5-1981-b7b9-b83b5cdf7fe9@oracle.com> References: <3630a545-2b2c-e17c-bbe5-98ae8508ae38@oracle.com> <8530df0a-c67c-3159-4b96-8bde4e925434@oracle.com> <7cf9f5a3-f8c5-1981-b7b9-b83b5cdf7fe9@oracle.com> Message-ID: <332991c8-acdb-9f7d-dc58-7f5e95b28770@oracle.com> Thanks David and Serguei! Coleen On 12/4/19 5:14 AM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > +1 > > Thanks, > Serguei > > > On 12/3/19 20:49, David Holmes wrote: >> Hi Coleen, >> >> That all seems fine to me. >> >> Thanks, >> David >> >> On 4/12/2019 4:21 am, coleen.phillimore at oracle.com wrote: >>> Summary: remove unnecessary nmethodLocker >>> >>> See bug for more details.? Tested with tier2-8. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2019/8235273.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8235273 >>> >>> (Note, this has a trivial merge with the change for JDK-8212160). >>> >>> Thanks, >>> Coleen > From stefan.karlsson at oracle.com Wed Dec 4 13:55:06 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 4 Dec 2019 14:55:06 +0100 Subject: RFR: ZGC: Add support for JFR leak profiler In-Reply-To: <27862ac1-e86b-406a-7819-85a13fbb24a3@oracle.com> References: <27862ac1-e86b-406a-7819-85a13fbb24a3@oracle.com> Message-ID: Hi Erik, This looks good to me! I think it would be good for the code to limit the number of UnifedOopRef::addr calls, since those are unsafe ways into the the addr and _could_ be abused to go around the UnifiedOopRef encapsulation. Maybe something for the future. Just a small nit: https://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/src/hotspot/share/jfr/leakprofiler/checkpoint/eventEmitter.cpp.frames.html Now that you've limited the use of object_addr, maybe you could also limit its scope by moving 113 const oop* object_addr = sample->object_addr(); to around: 123 edge = edge_store->put(UnifiedOopRef::encode_in_native(object_addr)); Thanks, StefanK On 2019-12-02 11:09, erik.osterlund at oracle.com wrote: > Hi, > > The JFR leak profiler is currently not supported on ZGC, because it > has not been accessorized appropriately. > This patch adds the required Access API sprinkling necessary, to > enable this for ZGC. I did the following: > > 1) Renamed UnifiedOop to UnifiedOopRef, because it describes the > reference to an oop, not the oop itself. > 2) Changed UnifiedOopRef to be a POD, instead of an AllStatic that > passes around its encoded data as > ?? const oop* (even though it might be a tagged pointer to a narrow > oop), improving type safty. > 3) Added another tag bit to UnifiedOopRef, allowing it to keep track > of whether the described location is > ?? IN_NATIVE or IN_HEAP. The dereference function performs the > appropriate NativeAccess/HeapAccess with the > ?? AS_NO_KEEPALIVE decotator, which is fine because the leak profiler > does not create any edges to oops > ?? found during traversal. > 4) Removed code blob oop iteration, because it can't be supported > appropriately by the current UnifiedOopRef > ?? mechanism (because oops are misaligned, and UnifiedOopRef requires > tag bits - I believe this has never > ?? worked properly) > 6) Accessorized loads on strong roots to > NativeAccess::oop_load. JFR leak profiler never > ?? creates new references to oops that are visited, so it does not > need to keep things alive > 7) Explicitly told the oop iteration closures to not visit referents. > The default referent strategy used > ?? before is DO_DISCOVERY, which doesn't seem right. > 8) Fixed a pre-existing issue where the array index reported in the > leak chain was calculated incorrectly > ?? as the byte offset from the array base. It should be scaled by the > element length of the array. > > Ran some JFR leak profiler tests, and some ad-hoc leak profiling in > various applications. Also checked that > there were no observable perf regressions in the iterations, and there > were not. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8235174 > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/ > > Thanks, > /Erik From erik.joelsson at oracle.com Wed Dec 4 14:11:00 2019 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 4 Dec 2019 06:11:00 -0800 Subject: native debug symbols support on Windows In-Reply-To: References: Message-ID: Correct, with the Microsoft toolchain there is no support for internal. I don't know what happens to the build if you try to configure it that way. Feel free to come up with a reasonable behavior. /Erik On 2019-12-04 00:06, Langer, Christoph wrote: > Hi, > > I'm currently looking into native debug symbols support for Windows. > > The OpenJDK build system supports these two configure flags --with-native-debug-symbols= (among a few other options which I don't want to discuss here). > > So, the name implies that for "internal", debug symbols should be contained in the binaries. And "external" should create separate files that contain the debug symbols. However, to my knowledge, Windows would always make use of external symbol files, named *.pdb. And there is no way to have the debug symbols included in the binaries. Is that correct or am I wrong in this assumption? > > If it's true, I guess --with-native-debug-symbols=internal would not make sense on Windows and should rather be rejected by configure. Otherwise, if we were to support --with-native-debug-symbols=internal, the build is broken for target "test-image" when it comes to building/copying the gtest image. > > I'd like to fix either the one way or the other. What do people think? > > Thanks for your help > Christoph > From bob.vandette at oracle.com Wed Dec 4 14:26:18 2019 From: bob.vandette at oracle.com (Bob Vandette) Date: Wed, 4 Dec 2019 09:26:18 -0500 Subject: native debug symbols support on Windows In-Reply-To: References: Message-ID: <9F2FEA9C-202F-4413-B281-36C469B25A66@oracle.com> There seems to be an option that will include debug information in generated .obj files. Assuming this option is supported in the versions of Visual Studio we use, it could be used to implement ?internal? native debug symbols. /Z7 The /Z7 option produces object files that also contain full symbolic debugging information for use with the debugger. These object files and the built executable can be substantially larger than files that have no debugging information. The symbolic debugging information includes the names and types of variables, as well as functions and line numbers. No PDB file is produced. Bob. > On Dec 4, 2019, at 9:11 AM, Erik Joelsson wrote: > > Correct, with the Microsoft toolchain there is no support for internal. I don't know what happens to the build if you try to configure it that way. Feel free to come up with a reasonable behavior. > > /Erik > > On 2019-12-04 00:06, Langer, Christoph wrote: >> Hi, >> >> I'm currently looking into native debug symbols support for Windows. >> >> The OpenJDK build system supports these two configure flags --with-native-debug-symbols= (among a few other options which I don't want to discuss here). >> >> So, the name implies that for "internal", debug symbols should be contained in the binaries. And "external" should create separate files that contain the debug symbols. However, to my knowledge, Windows would always make use of external symbol files, named *.pdb. And there is no way to have the debug symbols included in the binaries. Is that correct or am I wrong in this assumption? >> >> If it's true, I guess --with-native-debug-symbols=internal would not make sense on Windows and should rather be rejected by configure. Otherwise, if we were to support --with-native-debug-symbols=internal, the build is broken for target "test-image" when it comes to building/copying the gtest image. >> >> I'd like to fix either the one way or the other. What do people think? >> >> Thanks for your help >> Christoph >> From erik.joelsson at oracle.com Wed Dec 4 14:46:10 2019 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 4 Dec 2019 06:46:10 -0800 Subject: native debug symbols support on Windows In-Reply-To: <9F2FEA9C-202F-4413-B281-36C469B25A66@oracle.com> References: <9F2FEA9C-202F-4413-B281-36C469B25A66@oracle.com> Message-ID: <45a8b20e-1e91-f2da-a431-8f775a216d03@oracle.com> Hello, On 2019-12-04 06:26, Bob Vandette wrote: > There seems to be an option that will include debug information in generated .obj files. Assuming this option is supported in the > versions of Visual Studio we use, it could be used to implement ?internal? native debug symbols. > > /Z7 We already use this when compiling, but we still link to external pdb files. I was not aware of being able to link with the symbol information internal. While this seems like it could be implemented, my question is, does anyone need it? The internal symbols on Linux was something the Linux distros wanted as they like to move it out in a uniform manner later. I can't really see a need for this on Windows, but I certainly wouldn't object if someone else do and wants to implement the support in the OpenJDK build. /Erik > The /Z7 option produces object files that also contain full symbolic debugging information for use with the debugger. These object files and the built executable can be substantially larger than files that have no debugging information. The symbolic debugging information includes the names and types of variables, as well as functions and line numbers. No PDB file is produced. > > Bob. > > >> On Dec 4, 2019, at 9:11 AM, Erik Joelsson wrote: >> >> Correct, with the Microsoft toolchain there is no support for internal. I don't know what happens to the build if you try to configure it that way. Feel free to come up with a reasonable behavior. >> >> /Erik >> >> On 2019-12-04 00:06, Langer, Christoph wrote: >>> Hi, >>> >>> I'm currently looking into native debug symbols support for Windows. >>> >>> The OpenJDK build system supports these two configure flags --with-native-debug-symbols= (among a few other options which I don't want to discuss here). >>> >>> So, the name implies that for "internal", debug symbols should be contained in the binaries. And "external" should create separate files that contain the debug symbols. However, to my knowledge, Windows would always make use of external symbol files, named *.pdb. And there is no way to have the debug symbols included in the binaries. Is that correct or am I wrong in this assumption? >>> >>> If it's true, I guess --with-native-debug-symbols=internal would not make sense on Windows and should rather be rejected by configure. Otherwise, if we were to support --with-native-debug-symbols=internal, the build is broken for target "test-image" when it comes to building/copying the gtest image. >>> >>> I'd like to fix either the one way or the other. What do people think? >>> >>> Thanks for your help >>> Christoph >>> From christoph.langer at sap.com Wed Dec 4 14:50:20 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 4 Dec 2019 14:50:20 +0000 Subject: 8234397: add OS uptime information to os::print_os_info output In-Reply-To: References: <30E06543-E081-429B-8293-8CA81D1F6870@sap.com> <038f060e-b253-e33f-bc03-a56c0735a90f@oracle.com> Message-ID: Hi Matthias, your latest version looks fine to me, too. Thanks for doing this. /Christoph > -----Original Message----- > From: Baesken, Matthias > Sent: Mittwoch, 4. Dezember 2019 09:12 > To: David Holmes ; Langer, Christoph > ; Schmidt, Lutz ; > 'hotspot-dev at openjdk.java.net' > Subject: RE: 8234397: add OS uptime information to os::print_os_info output > > Hi David, I think I stay away from the unification . > May I add you as a reviewer ? > > Best regards, Matthias > > > > > Hi Matthias, > > > > On 4/12/2019 1:38 am, Baesken, Matthias wrote: > > > Hi David, here is a new webrev based on your comments : > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8234397.4/ > > > > Thanks for the updates. > > > > >> > > >> Is %llu supported by all compilers (32-bit as well)? > > >> > > > > > > Not sure , I checked and it works with VS2017/32bit. But I don't know > > about other compilers. So I better use long / %ld. > > > > I think int would be fine - 2^31 seconds is a long time. > > > > > > > > Btw I noticed that > > > > > > os::print_date_and_time > > > ... > > > // print elapsed time in a human-readable format: > > > > > > Contains very similar coding to what I do in os::print_dhm --> should > this > > maybe be unified ? > > > > I don't really see a way to unify. You might hoist the constants out to > > be static in the file for sharing: > > > > const int secs_per_day = 86400; > > const int secs_per_hour = 3600; > > const int secs_per_min = 60; > > > > and add others as needed. > > From christoph.langer at sap.com Wed Dec 4 15:09:00 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 4 Dec 2019 15:09:00 +0000 Subject: native debug symbols support on Windows In-Reply-To: <45a8b20e-1e91-f2da-a431-8f775a216d03@oracle.com> References: <9F2FEA9C-202F-4413-B281-36C469B25A66@oracle.com> <45a8b20e-1e91-f2da-a431-8f775a216d03@oracle.com> Message-ID: Hi, thanks for your comments. The reason why I want to have (at least basic) internal debugging information is to have helpful callstacks in hs_err files on crashes. So, Bob, I don't think the executables can contain debug information, just the compiled .obj files. When it comes to linking, the linker either generates a pdb file or nothing. That's at least how I read the MSDN documentation (of linker options /debug [0] and /pdb [1]). Maybe an idea to implement "internal" debug symbols for Windows would be to copy the pdb files to the release bundle as well. But actually, it's a strange interpretation of "internal" in that case... Thanks Christoph [0] https://docs.microsoft.com/en-us/cpp/build/reference/debug-generate-debug-info?view=vs-2019 [1] https://docs.microsoft.com/en-us/cpp/build/reference/pdb-use-program-database?view=vs-2019 > -----Original Message----- > From: Erik Joelsson > Sent: Mittwoch, 4. Dezember 2019 15:46 > To: Bob Vandette > Cc: Langer, Christoph ; build- > dev at openjdk.java.net; core-libs-dev at openjdk.java.net; hotspot-dev > developers > Subject: Re: native debug symbols support on Windows > > Hello, > > On 2019-12-04 06:26, Bob Vandette wrote: > > There seems to be an option that will include debug information in > generated .obj files. Assuming this option is supported in the > > versions of Visual Studio we use, it could be used to implement ?internal? > native debug symbols. > > > > /Z7 > > We already use this when compiling, but we still link to external pdb > files. I was not aware of being able to link with the symbol information > internal. > > While this seems like it could be implemented, my question is, does > anyone need it? The internal symbols on Linux was something the Linux > distros wanted as they like to move it out in a uniform manner later. I > can't really see a need for this on Windows, but I certainly wouldn't > object if someone else do and wants to implement the support in the > OpenJDK build. > > /Erik > > > The /Z7 option produces object files that also contain full symbolic > debugging information for use with the debugger. These object files and the > built executable can be substantially larger than files that have no debugging > information. The symbolic debugging information includes the names and > types of variables, as well as functions and line numbers. No PDB file is > produced. > > > > Bob. > > > > > >> On Dec 4, 2019, at 9:11 AM, Erik Joelsson > wrote: > >> > >> Correct, with the Microsoft toolchain there is no support for internal. I > don't know what happens to the build if you try to configure it that way. Feel > free to come up with a reasonable behavior. > >> > >> /Erik > >> > >> On 2019-12-04 00:06, Langer, Christoph wrote: > >>> Hi, > >>> > >>> I'm currently looking into native debug symbols support for Windows. > >>> > >>> The OpenJDK build system supports these two configure flags --with- > native-debug-symbols= (among a few other options > which I don't want to discuss here). > >>> > >>> So, the name implies that for "internal", debug symbols should be > contained in the binaries. And "external" should create separate files that > contain the debug symbols. However, to my knowledge, Windows would > always make use of external symbol files, named *.pdb. And there is no way > to have the debug symbols included in the binaries. Is that correct or am I > wrong in this assumption? > >>> > >>> If it's true, I guess --with-native-debug-symbols=internal would not > make sense on Windows and should rather be rejected by configure. > Otherwise, if we were to support --with-native-debug-symbols=internal, the > build is broken for target "test-image" when it comes to building/copying the > gtest image. > >>> > >>> I'd like to fix either the one way or the other. What do people think? > >>> > >>> Thanks for your help > >>> Christoph > >>> From bob.vandette at oracle.com Wed Dec 4 15:21:35 2019 From: bob.vandette at oracle.com (Bob Vandette) Date: Wed, 4 Dec 2019 10:21:35 -0500 Subject: native debug symbols support on Windows In-Reply-To: References: <9F2FEA9C-202F-4413-B281-36C469B25A66@oracle.com> <45a8b20e-1e91-f2da-a431-8f775a216d03@oracle.com> Message-ID: Oh well, this sentence in [0] says it all. "It is not possible to create an .exe or .dll that contains debug information. Debug information is always placed in a .obj or .pdb file.? I guess you need to copy pdb files if you runtime debug information. Bob. > On Dec 4, 2019, at 10:09 AM, Langer, Christoph wrote: > > Hi, > > thanks for your comments. > > The reason why I want to have (at least basic) internal debugging information is to have helpful callstacks in hs_err files on crashes. > > So, Bob, I don't think the executables can contain debug information, just the compiled .obj files. When it comes to linking, the linker either generates a pdb file or nothing. That's at least how I read the MSDN documentation (of linker options /debug [0] and /pdb [1]). > > Maybe an idea to implement "internal" debug symbols for Windows would be to copy the pdb files to the release bundle as well. But actually, it's a strange interpretation of "internal" in that case... > > Thanks > Christoph > > [0] https://docs.microsoft.com/en-us/cpp/build/reference/debug-generate-debug-info?view=vs-2019 > [1] https://docs.microsoft.com/en-us/cpp/build/reference/pdb-use-program-database?view=vs-2019 > >> -----Original Message----- >> From: Erik Joelsson >> Sent: Mittwoch, 4. Dezember 2019 15:46 >> To: Bob Vandette >> Cc: Langer, Christoph ; build- >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; hotspot-dev >> developers >> Subject: Re: native debug symbols support on Windows >> >> Hello, >> >> On 2019-12-04 06:26, Bob Vandette wrote: >>> There seems to be an option that will include debug information in >> generated .obj files. Assuming this option is supported in the >>> versions of Visual Studio we use, it could be used to implement ?internal? >> native debug symbols. >>> >>> /Z7 >> >> We already use this when compiling, but we still link to external pdb >> files. I was not aware of being able to link with the symbol information >> internal. >> >> While this seems like it could be implemented, my question is, does >> anyone need it? The internal symbols on Linux was something the Linux >> distros wanted as they like to move it out in a uniform manner later. I >> can't really see a need for this on Windows, but I certainly wouldn't >> object if someone else do and wants to implement the support in the >> OpenJDK build. >> >> /Erik >> >>> The /Z7 option produces object files that also contain full symbolic >> debugging information for use with the debugger. These object files and the >> built executable can be substantially larger than files that have no debugging >> information. The symbolic debugging information includes the names and >> types of variables, as well as functions and line numbers. No PDB file is >> produced. >>> >>> Bob. >>> >>> >>>> On Dec 4, 2019, at 9:11 AM, Erik Joelsson >> wrote: >>>> >>>> Correct, with the Microsoft toolchain there is no support for internal. I >> don't know what happens to the build if you try to configure it that way. Feel >> free to come up with a reasonable behavior. >>>> >>>> /Erik >>>> >>>> On 2019-12-04 00:06, Langer, Christoph wrote: >>>>> Hi, >>>>> >>>>> I'm currently looking into native debug symbols support for Windows. >>>>> >>>>> The OpenJDK build system supports these two configure flags --with- >> native-debug-symbols= (among a few other options >> which I don't want to discuss here). >>>>> >>>>> So, the name implies that for "internal", debug symbols should be >> contained in the binaries. And "external" should create separate files that >> contain the debug symbols. However, to my knowledge, Windows would >> always make use of external symbol files, named *.pdb. And there is no way >> to have the debug symbols included in the binaries. Is that correct or am I >> wrong in this assumption? >>>>> >>>>> If it's true, I guess --with-native-debug-symbols=internal would not >> make sense on Windows and should rather be rejected by configure. >> Otherwise, if we were to support --with-native-debug-symbols=internal, the >> build is broken for target "test-image" when it comes to building/copying the >> gtest image. >>>>> >>>>> I'd like to fix either the one way or the other. What do people think? >>>>> >>>>> Thanks for your help >>>>> Christoph >>>>> From erik.osterlund at oracle.com Wed Dec 4 16:16:40 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 4 Dec 2019 17:16:40 +0100 Subject: RFR: ZGC: Add support for JFR leak profiler In-Reply-To: References: <27862ac1-e86b-406a-7819-85a13fbb24a3@oracle.com> Message-ID: <6e08f198-6555-8d24-0be2-b9b6b76874ce@oracle.com> Hi Stefan, Thanks for the review. On 2019-12-04 14:55, Stefan Karlsson wrote: > Hi Erik, > > This looks good to me! Thanks! > I think it would be good for the code to limit the number of > UnifedOopRef::addr calls, since those are unsafe ways into the the > addr and _could_ be abused to go around the UnifiedOopRef > encapsulation. Maybe something for the future. Yeah. I wrote that function because I noticed that the address was used with literally every single address looking type I could imagine, in different places (uintptr_t, address, char*, oop*, void*, you name it). So I just wrote the addr function to be able to perform this change as mechanically as possible. > Just a small nit: > > https://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/src/hotspot/share/jfr/leakprofiler/checkpoint/eventEmitter.cpp.frames.html > > > Now that you've limited the use of object_addr, maybe you could also > limit its scope by moving > > 113?? const oop* object_addr = sample->object_addr(); > > to around: > > 123 edge = edge_store->put(UnifiedOopRef::encode_in_native(object_addr)); Absolutely. Here is a new webrev with your proposed change, and with some changes from Markus (big thanks to Markus for looking at this in great detail): http://cr.openjdk.java.net/~eosterlund/8235174/webrev.01/ Incremental: http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00_01/ His changes mostly massage some types and order some includes. For example Markus prefers passing around UnifiedOopRef as non-const (now that it is a POD, it's like passing around an integer). The most important change though, was fixing a bug in my patch by making sure that ObjectSampler::weak_oops_do actually clears the oops that are dead. I thought it did, like the other weak_oops_do functions. Turns out that it has not been needed until now. The oops were not cleared, yet the ObjectSample is reused as some kind of cached object (presumably to call malloc less). The consequence was that when the ObjectSample gets subsequently reused with a dead oop in it, set_object() now using NativeAccess::oop_store with my patch crashes things, because the G1 barrier backend SATB enqueues what was there in memory before (which is dead garbage memory). Now one might argue that it's really silly of G1 barriers to SATB enqueue on non-strong stores (because the very snapshot that SATB tracks is the strong graph, which is the graph being marked concurrently, nothing else). But I would like to change that heuristic weirdness in a separate change. In this patch I am happy with the object simply being cleared in weak_oops_do, like it is in all other weak_oops_do functions. That fixes the issue. Before there was a separate _dead boolean to track that the ObjectSample is dead. Now that the oop is cleared by the GC when it dies, the boolean is no longer needed, because the is_dead() function can simply look at the oop. Thanks, /Erik > Thanks, > StefanK > > On 2019-12-02 11:09, erik.osterlund at oracle.com wrote: >> Hi, >> >> The JFR leak profiler is currently not supported on ZGC, because it >> has not been accessorized appropriately. >> This patch adds the required Access API sprinkling necessary, to >> enable this for ZGC. I did the following: >> >> 1) Renamed UnifiedOop to UnifiedOopRef, because it describes the >> reference to an oop, not the oop itself. >> 2) Changed UnifiedOopRef to be a POD, instead of an AllStatic that >> passes around its encoded data as >> ?? const oop* (even though it might be a tagged pointer to a narrow >> oop), improving type safty. >> 3) Added another tag bit to UnifiedOopRef, allowing it to keep track >> of whether the described location is >> ?? IN_NATIVE or IN_HEAP. The dereference function performs the >> appropriate NativeAccess/HeapAccess with the >> ?? AS_NO_KEEPALIVE decotator, which is fine because the leak profiler >> does not create any edges to oops >> ?? found during traversal. >> 4) Removed code blob oop iteration, because it can't be supported >> appropriately by the current UnifiedOopRef >> ?? mechanism (because oops are misaligned, and UnifiedOopRef requires >> tag bits - I believe this has never >> ?? worked properly) >> 6) Accessorized loads on strong roots to >> NativeAccess::oop_load. JFR leak profiler never >> ?? creates new references to oops that are visited, so it does not >> need to keep things alive >> 7) Explicitly told the oop iteration closures to not visit referents. >> The default referent strategy used >> ?? before is DO_DISCOVERY, which doesn't seem right. >> 8) Fixed a pre-existing issue where the array index reported in the >> leak chain was calculated incorrectly >> ?? as the byte offset from the array base. It should be scaled by the >> element length of the array. >> >> Ran some JFR leak profiler tests, and some ad-hoc leak profiling in >> various applications. Also checked that >> there were no observable perf regressions in the iterations, and >> there were not. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8235174 >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/ >> >> Thanks, >> /Erik > From markus.gronlund at oracle.com Wed Dec 4 17:15:42 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 4 Dec 2019 09:15:42 -0800 (PST) Subject: RFR: ZGC: Add support for JFR leak profiler In-Reply-To: <6e08f198-6555-8d24-0be2-b9b6b76874ce@oracle.com> References: <27862ac1-e86b-406a-7819-85a13fbb24a3@oracle.com> <6e08f198-6555-8d24-0be2-b9b6b76874ce@oracle.com> Message-ID: <1d080813-cd13-44b5-aa38-8c3ed6ae271e@default> Hi Erik, Looks good. Thank you very much for getting the correct accessors in place and providing an improved type for representing references. Markus -----Original Message----- From: Erik ?sterlund Sent: den 4 december 2019 17:17 To: Stefan Karlsson ; hotspot-dev developers Cc: Markus Gronlund ; Erik Gahlin Subject: Re: RFR: ZGC: Add support for JFR leak profiler Hi Stefan, Thanks for the review. On 2019-12-04 14:55, Stefan Karlsson wrote: > Hi Erik, > > This looks good to me! Thanks! > I think it would be good for the code to limit the number of > UnifedOopRef::addr calls, since those are unsafe ways into the the > addr and _could_ be abused to go around the UnifiedOopRef > encapsulation. Maybe something for the future. Yeah. I wrote that function because I noticed that the address was used with literally every single address looking type I could imagine, in different places (uintptr_t, address, char*, oop*, void*, you name it). So I just wrote the addr function to be able to perform this change as mechanically as possible. > Just a small nit: > > https://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/src/hotspot/ > share/jfr/leakprofiler/checkpoint/eventEmitter.cpp.frames.html > > > Now that you've limited the use of object_addr, maybe you could also > limit its scope by moving > > 113?? const oop* object_addr = sample->object_addr(); > > to around: > > 123 edge = > edge_store->put(UnifiedOopRef::encode_in_native(object_addr)); Absolutely. Here is a new webrev with your proposed change, and with some changes from Markus (big thanks to Markus for looking at this in great detail): http://cr.openjdk.java.net/~eosterlund/8235174/webrev.01/ Incremental: http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00_01/ His changes mostly massage some types and order some includes. For example Markus prefers passing around UnifiedOopRef as non-const (now that it is a POD, it's like passing around an integer). The most important change though, was fixing a bug in my patch by making sure that ObjectSampler::weak_oops_do actually clears the oops that are dead. I thought it did, like the other weak_oops_do functions. Turns out that it has not been needed until now. The oops were not cleared, yet the ObjectSample is reused as some kind of cached object (presumably to call malloc less). The consequence was that when the ObjectSample gets subsequently reused with a dead oop in it, set_object() now using NativeAccess::oop_store with my patch crashes things, because the G1 barrier backend SATB enqueues what was there in memory before (which is dead garbage memory). Now one might argue that it's really silly of G1 barriers to SATB enqueue on non-strong stores (because the very snapshot that SATB tracks is the strong graph, which is the graph being marked concurrently, nothing else). But I would like to change that heuristic weirdness in a separate change. In this patch I am happy with the object simply being cleared in weak_oops_do, like it is in all other weak_oops_do functions. That fixes the issue. Before there was a separate _dead boolean to track that the ObjectSample is dead. Now that the oop is cleared by the GC when it dies, the boolean is no longer needed, because the is_dead() function can simply look at the oop. Thanks, /Erik > Thanks, > StefanK > > On 2019-12-02 11:09, erik.osterlund at oracle.com wrote: >> Hi, >> >> The JFR leak profiler is currently not supported on ZGC, because it >> has not been accessorized appropriately. >> This patch adds the required Access API sprinkling necessary, to >> enable this for ZGC. I did the following: >> >> 1) Renamed UnifiedOop to UnifiedOopRef, because it describes the >> reference to an oop, not the oop itself. >> 2) Changed UnifiedOopRef to be a POD, instead of an AllStatic that >> passes around its encoded data as >> ?? const oop* (even though it might be a tagged pointer to a narrow >> oop), improving type safty. >> 3) Added another tag bit to UnifiedOopRef, allowing it to keep track >> of whether the described location is >> ?? IN_NATIVE or IN_HEAP. The dereference function performs the >> appropriate NativeAccess/HeapAccess with the >> ?? AS_NO_KEEPALIVE decotator, which is fine because the leak profiler >> does not create any edges to oops >> ?? found during traversal. >> 4) Removed code blob oop iteration, because it can't be supported >> appropriately by the current UnifiedOopRef >> ?? mechanism (because oops are misaligned, and UnifiedOopRef requires >> tag bits - I believe this has never >> ?? worked properly) >> 6) Accessorized loads on strong roots to >> NativeAccess::oop_load. JFR leak profiler never >> ?? creates new references to oops that are visited, so it does not >> need to keep things alive >> 7) Explicitly told the oop iteration closures to not visit referents. >> The default referent strategy used >> ?? before is DO_DISCOVERY, which doesn't seem right. >> 8) Fixed a pre-existing issue where the array index reported in the >> leak chain was calculated incorrectly >> ?? as the byte offset from the array base. It should be scaled by the >> element length of the array. >> >> Ran some JFR leak profiler tests, and some ad-hoc leak profiling in >> various applications. Also checked that there were no observable perf >> regressions in the iterations, and there were not. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8235174 >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/ >> >> Thanks, >> /Erik > From erik.gahlin at oracle.com Wed Dec 4 17:25:25 2019 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Wed, 4 Dec 2019 18:25:25 +0100 Subject: RFR: ZGC: Add support for JFR leak profiler In-Reply-To: <6e08f198-6555-8d24-0be2-b9b6b76874ce@oracle.com> References: <27862ac1-e86b-406a-7819-85a13fbb24a3@oracle.com> <6e08f198-6555-8d24-0be2-b9b6b76874ce@oracle.com> Message-ID: <9fae214f-557b-d90d-a595-c9d0c26e32a3@oracle.com> Looks good. Erik On 2019-12-04 17:16, Erik ?sterlund wrote: > Hi Stefan, > > Thanks for the review. > > On 2019-12-04 14:55, Stefan Karlsson wrote: >> Hi Erik, >> >> This looks good to me! > > Thanks! > >> I think it would be good for the code to limit the number of >> UnifedOopRef::addr calls, since those are unsafe ways into the the >> addr and _could_ be abused to go around the UnifiedOopRef >> encapsulation. Maybe something for the future. > > Yeah. I wrote that function because I noticed that the address was > used with literally every single > address looking type I could imagine, in different places (uintptr_t, > address, char*, oop*, void*, > you name it). So I just wrote the addr function to be able to > perform this change as mechanically > as possible. > >> Just a small nit: >> >> https://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/src/hotspot/share/jfr/leakprofiler/checkpoint/eventEmitter.cpp.frames.html >> >> >> Now that you've limited the use of object_addr, maybe you could also >> limit its scope by moving >> >> 113?? const oop* object_addr = sample->object_addr(); >> >> to around: >> >> 123 edge = >> edge_store->put(UnifiedOopRef::encode_in_native(object_addr)); > > Absolutely. > > Here is a new webrev with your proposed change, and with some changes > from Markus (big thanks to Markus for looking at this in great detail): > http://cr.openjdk.java.net/~eosterlund/8235174/webrev.01/ > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00_01/ > > His changes mostly massage some types and order some includes. For > example Markus prefers passing around UnifiedOopRef > as non-const (now that it is a POD, it's like passing around an integer). > > The most important change though, was fixing a bug in my patch by > making sure that ObjectSampler::weak_oops_do > actually clears the oops that are dead. I thought it did, like the > other weak_oops_do functions. Turns out > that it has not been needed until now. The oops were not cleared, yet > the ObjectSample is reused as some > kind of cached object (presumably to call malloc less). The > consequence was that when the ObjectSample gets subsequently reused > with a dead oop in it, set_object() now using > NativeAccess::oop_store > with my patch crashes things, because the G1 barrier backend SATB > enqueues what was there in memory before > (which is dead garbage memory). > > Now one might argue that it's really silly of G1 barriers to SATB > enqueue on non-strong stores (because the very > snapshot that SATB tracks is the strong graph, which is the graph > being marked concurrently, nothing else). But I > would like to change that heuristic weirdness in a separate change. In > this patch I am happy with the object simply > being cleared in weak_oops_do, like it is in all other weak_oops_do > functions. That fixes the issue. > > Before there was a separate _dead boolean to track that the > ObjectSample is dead. Now that the oop is cleared > by the GC when it dies, the boolean is no longer needed, because the > is_dead() function can simply look at the oop. > > Thanks, > /Erik > >> Thanks, >> StefanK >> >> On 2019-12-02 11:09, erik.osterlund at oracle.com wrote: >>> Hi, >>> >>> The JFR leak profiler is currently not supported on ZGC, because it >>> has not been accessorized appropriately. >>> This patch adds the required Access API sprinkling necessary, to >>> enable this for ZGC. I did the following: >>> >>> 1) Renamed UnifiedOop to UnifiedOopRef, because it describes the >>> reference to an oop, not the oop itself. >>> 2) Changed UnifiedOopRef to be a POD, instead of an AllStatic that >>> passes around its encoded data as >>> ?? const oop* (even though it might be a tagged pointer to a narrow >>> oop), improving type safty. >>> 3) Added another tag bit to UnifiedOopRef, allowing it to keep track >>> of whether the described location is >>> ?? IN_NATIVE or IN_HEAP. The dereference function performs the >>> appropriate NativeAccess/HeapAccess with the >>> ?? AS_NO_KEEPALIVE decotator, which is fine because the leak >>> profiler does not create any edges to oops >>> ?? found during traversal. >>> 4) Removed code blob oop iteration, because it can't be supported >>> appropriately by the current UnifiedOopRef >>> ?? mechanism (because oops are misaligned, and UnifiedOopRef >>> requires tag bits - I believe this has never >>> ?? worked properly) >>> 6) Accessorized loads on strong roots to >>> NativeAccess::oop_load. JFR leak profiler never >>> ?? creates new references to oops that are visited, so it does not >>> need to keep things alive >>> 7) Explicitly told the oop iteration closures to not visit >>> referents. The default referent strategy used >>> ?? before is DO_DISCOVERY, which doesn't seem right. >>> 8) Fixed a pre-existing issue where the array index reported in the >>> leak chain was calculated incorrectly >>> ?? as the byte offset from the array base. It should be scaled by >>> the element length of the array. >>> >>> Ran some JFR leak profiler tests, and some ad-hoc leak profiling in >>> various applications. Also checked that >>> there were no observable perf regressions in the iterations, and >>> there were not. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8235174 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/ >>> >>> Thanks, >>> /Erik >> > From erik.osterlund at oracle.com Wed Dec 4 19:04:53 2019 From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=) Date: Wed, 4 Dec 2019 20:04:53 +0100 Subject: RFR: ZGC: Add support for JFR leak profiler In-Reply-To: <1d080813-cd13-44b5-aa38-8c3ed6ae271e@default> References: <1d080813-cd13-44b5-aa38-8c3ed6ae271e@default> Message-ID: Hi Markus, Thanks for the review and for catching the bug! /Erik > On 4 Dec 2019, at 18:15, Markus Gronlund wrote: > > ?Hi Erik, > > Looks good. > > Thank you very much for getting the correct accessors in place and providing an improved type for representing references. > > Markus > > -----Original Message----- > From: Erik ?sterlund > Sent: den 4 december 2019 17:17 > To: Stefan Karlsson ; hotspot-dev developers > Cc: Markus Gronlund ; Erik Gahlin > Subject: Re: RFR: ZGC: Add support for JFR leak profiler > > Hi Stefan, > > Thanks for the review. > >> On 2019-12-04 14:55, Stefan Karlsson wrote: >> Hi Erik, >> >> This looks good to me! > > Thanks! > >> I think it would be good for the code to limit the number of >> UnifedOopRef::addr calls, since those are unsafe ways into the the >> addr and _could_ be abused to go around the UnifiedOopRef >> encapsulation. Maybe something for the future. > > Yeah. I wrote that function because I noticed that the address was used with literally every single address looking type I could imagine, in different places (uintptr_t, address, char*, oop*, void*, you name it). So I just wrote the addr function to be able to perform this change as mechanically as possible. > >> Just a small nit: >> >> https://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/src/hotspot/ >> share/jfr/leakprofiler/checkpoint/eventEmitter.cpp.frames.html >> >> >> Now that you've limited the use of object_addr, maybe you could also >> limit its scope by moving >> >> 113 const oop* object_addr = sample->object_addr(); >> >> to around: >> >> 123 edge = >> edge_store->put(UnifiedOopRef::encode_in_native(object_addr)); > > Absolutely. > > Here is a new webrev with your proposed change, and with some changes from Markus (big thanks to Markus for looking at this in great detail): > http://cr.openjdk.java.net/~eosterlund/8235174/webrev.01/ > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00_01/ > > His changes mostly massage some types and order some includes. For example Markus prefers passing around UnifiedOopRef as non-const (now that it is a POD, it's like passing around an integer). > > The most important change though, was fixing a bug in my patch by making sure that ObjectSampler::weak_oops_do actually clears the oops that are dead. I thought it did, like the other weak_oops_do functions. Turns out that it has not been needed until now. The oops were not cleared, yet the ObjectSample is reused as some kind of cached object (presumably to call malloc less). The consequence was that when the ObjectSample gets subsequently reused with a dead oop in it, set_object() now using NativeAccess::oop_store > with my patch crashes things, because the G1 barrier backend SATB enqueues what was there in memory before (which is dead garbage memory). > > Now one might argue that it's really silly of G1 barriers to SATB enqueue on non-strong stores (because the very snapshot that SATB tracks is the strong graph, which is the graph being marked concurrently, nothing else). But I would like to change that heuristic weirdness in a separate change. In this patch I am happy with the object simply being cleared in weak_oops_do, like it is in all other weak_oops_do functions. That fixes the issue. > > Before there was a separate _dead boolean to track that the ObjectSample is dead. Now that the oop is cleared by the GC when it dies, the boolean is no longer needed, because the > is_dead() function can simply look at the oop. > > Thanks, > /Erik > >> Thanks, >> StefanK >> >>> On 2019-12-02 11:09, erik.osterlund at oracle.com wrote: >>> Hi, >>> >>> The JFR leak profiler is currently not supported on ZGC, because it >>> has not been accessorized appropriately. >>> This patch adds the required Access API sprinkling necessary, to >>> enable this for ZGC. I did the following: >>> >>> 1) Renamed UnifiedOop to UnifiedOopRef, because it describes the >>> reference to an oop, not the oop itself. >>> 2) Changed UnifiedOopRef to be a POD, instead of an AllStatic that >>> passes around its encoded data as >>> const oop* (even though it might be a tagged pointer to a narrow >>> oop), improving type safty. >>> 3) Added another tag bit to UnifiedOopRef, allowing it to keep track >>> of whether the described location is >>> IN_NATIVE or IN_HEAP. The dereference function performs the >>> appropriate NativeAccess/HeapAccess with the >>> AS_NO_KEEPALIVE decotator, which is fine because the leak profiler >>> does not create any edges to oops >>> found during traversal. >>> 4) Removed code blob oop iteration, because it can't be supported >>> appropriately by the current UnifiedOopRef >>> mechanism (because oops are misaligned, and UnifiedOopRef requires >>> tag bits - I believe this has never >>> worked properly) >>> 6) Accessorized loads on strong roots to >>> NativeAccess::oop_load. JFR leak profiler never >>> creates new references to oops that are visited, so it does not >>> need to keep things alive >>> 7) Explicitly told the oop iteration closures to not visit referents. >>> The default referent strategy used >>> before is DO_DISCOVERY, which doesn't seem right. >>> 8) Fixed a pre-existing issue where the array index reported in the >>> leak chain was calculated incorrectly >>> as the byte offset from the array base. It should be scaled by the >>> element length of the array. >>> >>> Ran some JFR leak profiler tests, and some ad-hoc leak profiling in >>> various applications. Also checked that there were no observable perf >>> regressions in the iterations, and there were not. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8235174 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/ >>> >>> Thanks, >>> /Erik >> > From erik.osterlund at oracle.com Wed Dec 4 19:05:44 2019 From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=) Date: Wed, 4 Dec 2019 20:05:44 +0100 Subject: RFR: ZGC: Add support for JFR leak profiler In-Reply-To: <9fae214f-557b-d90d-a595-c9d0c26e32a3@oracle.com> References: <9fae214f-557b-d90d-a595-c9d0c26e32a3@oracle.com> Message-ID: <6AAA240A-5983-4E6E-8BA3-71C6776E13EE@oracle.com> Hi Erik, Thanks for the review! /Erik > On 4 Dec 2019, at 18:25, Erik Gahlin wrote: > > ?Looks good. > > Erik > >> On 2019-12-04 17:16, Erik ?sterlund wrote: >> Hi Stefan, >> >> Thanks for the review. >> >>> On 2019-12-04 14:55, Stefan Karlsson wrote: >>> Hi Erik, >>> >>> This looks good to me! >> >> Thanks! >> >>> I think it would be good for the code to limit the number of UnifedOopRef::addr calls, since those are unsafe ways into the the addr and _could_ be abused to go around the UnifiedOopRef encapsulation. Maybe something for the future. >> >> Yeah. I wrote that function because I noticed that the address was used with literally every single >> address looking type I could imagine, in different places (uintptr_t, address, char*, oop*, void*, >> you name it). So I just wrote the addr function to be able to perform this change as mechanically >> as possible. >> >>> Just a small nit: >>> >>> https://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/src/hotspot/share/jfr/leakprofiler/checkpoint/eventEmitter.cpp.frames.html >>> >>> Now that you've limited the use of object_addr, maybe you could also limit its scope by moving >>> >>> 113 const oop* object_addr = sample->object_addr(); >>> >>> to around: >>> >>> 123 edge = edge_store->put(UnifiedOopRef::encode_in_native(object_addr)); >> >> Absolutely. >> >> Here is a new webrev with your proposed change, and with some changes from Markus (big thanks to Markus for looking at this in great detail): >> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.01/ >> >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00_01/ >> >> His changes mostly massage some types and order some includes. For example Markus prefers passing around UnifiedOopRef >> as non-const (now that it is a POD, it's like passing around an integer). >> >> The most important change though, was fixing a bug in my patch by making sure that ObjectSampler::weak_oops_do >> actually clears the oops that are dead. I thought it did, like the other weak_oops_do functions. Turns out >> that it has not been needed until now. The oops were not cleared, yet the ObjectSample is reused as some >> kind of cached object (presumably to call malloc less). The consequence was that when the ObjectSample gets subsequently reused with a dead oop in it, set_object() now using NativeAccess::oop_store >> with my patch crashes things, because the G1 barrier backend SATB enqueues what was there in memory before >> (which is dead garbage memory). >> >> Now one might argue that it's really silly of G1 barriers to SATB enqueue on non-strong stores (because the very >> snapshot that SATB tracks is the strong graph, which is the graph being marked concurrently, nothing else). But I >> would like to change that heuristic weirdness in a separate change. In this patch I am happy with the object simply >> being cleared in weak_oops_do, like it is in all other weak_oops_do functions. That fixes the issue. >> >> Before there was a separate _dead boolean to track that the ObjectSample is dead. Now that the oop is cleared >> by the GC when it dies, the boolean is no longer needed, because the is_dead() function can simply look at the oop. >> >> Thanks, >> /Erik >> >>> Thanks, >>> StefanK >>> >>> On 2019-12-02 11:09, erik.osterlund at oracle.com wrote: >>>> Hi, >>>> >>>> The JFR leak profiler is currently not supported on ZGC, because it has not been accessorized appropriately. >>>> This patch adds the required Access API sprinkling necessary, to enable this for ZGC. I did the following: >>>> >>>> 1) Renamed UnifiedOop to UnifiedOopRef, because it describes the reference to an oop, not the oop itself. >>>> 2) Changed UnifiedOopRef to be a POD, instead of an AllStatic that passes around its encoded data as >>>> const oop* (even though it might be a tagged pointer to a narrow oop), improving type safty. >>>> 3) Added another tag bit to UnifiedOopRef, allowing it to keep track of whether the described location is >>>> IN_NATIVE or IN_HEAP. The dereference function performs the appropriate NativeAccess/HeapAccess with the >>>> AS_NO_KEEPALIVE decotator, which is fine because the leak profiler does not create any edges to oops >>>> found during traversal. >>>> 4) Removed code blob oop iteration, because it can't be supported appropriately by the current UnifiedOopRef >>>> mechanism (because oops are misaligned, and UnifiedOopRef requires tag bits - I believe this has never >>>> worked properly) >>>> 6) Accessorized loads on strong roots to NativeAccess::oop_load. JFR leak profiler never >>>> creates new references to oops that are visited, so it does not need to keep things alive >>>> 7) Explicitly told the oop iteration closures to not visit referents. The default referent strategy used >>>> before is DO_DISCOVERY, which doesn't seem right. >>>> 8) Fixed a pre-existing issue where the array index reported in the leak chain was calculated incorrectly >>>> as the byte offset from the array base. It should be scaled by the element length of the array. >>>> >>>> Ran some JFR leak profiler tests, and some ad-hoc leak profiling in various applications. Also checked that >>>> there were no observable perf regressions in the iterations, and there were not. >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8235174 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/ >>>> >>>> Thanks, >>>> /Erik >>> >> From igor.ignatyev at oracle.com Wed Dec 4 19:52:27 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 4 Dec 2019 11:52:27 -0800 Subject: RFR(T) : 8235353 : clean up hotspot problem lists Message-ID: http://cr.openjdk.java.net/~iignatyev//8235353/webrev.00 > 9 lines changed: 0 ins; 0 del; 9 mod; Hi all, could you please review this small and trivial cleanup which returns serviceablility/sa tests back to execution on linux-ppc64. the tests were problem listed due to 8211767[1], which is closed as a dup of resolved 8228649[2]. [1] https://bugs.openjdk.java.net/browse/JDK-8211767 [2] https://bugs.openjdk.java.net/browse/JDK-8228649 Thanks, -- Igor From vladimir.kozlov at oracle.com Wed Dec 4 20:01:21 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 4 Dec 2019 12:01:21 -0800 Subject: RFR(T) : 8235353 : clean up hotspot problem lists In-Reply-To: References: Message-ID: I am fine with changes but we need to ask PPC64 supporter to verify that tests passed now. Thanks, Vladimir K On 12/4/19 11:52 AM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8235353/webrev.00 >> 9 lines changed: 0 ins; 0 del; 9 mod; > > Hi all, > > could you please review this small and trivial cleanup which returns serviceablility/sa tests back to execution on linux-ppc64. the tests were problem listed due to 8211767[1], which is closed as a dup of resolved 8228649[2]. > > [1] https://bugs.openjdk.java.net/browse/JDK-8211767 > [2] https://bugs.openjdk.java.net/browse/JDK-8228649 > > Thanks, > -- Igor > From erik.joelsson at oracle.com Wed Dec 4 22:14:21 2019 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 4 Dec 2019 14:14:21 -0800 Subject: native debug symbols support on Windows In-Reply-To: References: <9F2FEA9C-202F-4413-B281-36C469B25A66@oracle.com> <45a8b20e-1e91-f2da-a431-8f775a216d03@oracle.com> Message-ID: <87a76d15-eabc-913b-bbd2-1ebf594d5f8c@oracle.com> On 2019-12-04 07:09, Langer, Christoph wrote: > Hi, > > thanks for your comments. > > The reason why I want to have (at least basic) internal debugging information is to have helpful callstacks in hs_err files on crashes. We go to rather great length to provide this in our testing. The bundles target creates a symbols bundle that can be overlayed on top of the main bundle. The RunTests.gmk makefile can be supplied with the variable SYMBOLS_IMAGE_DIR (which should point to where the symbols bundle is unzipped if it's separate to the main JDK image) and sets up _NT_SYMBOL_PATH when running tests. /Erik > > So, Bob, I don't think the executables can contain debug information, just the compiled .obj files. When it comes to linking, the linker either generates a pdb file or nothing. That's at least how I read the MSDN documentation (of linker options /debug [0] and /pdb [1]). > > Maybe an idea to implement "internal" debug symbols for Windows would be to copy the pdb files to the release bundle as well. But actually, it's a strange interpretation of "internal" in that case... > > Thanks > Christoph > > [0] https://docs.microsoft.com/en-us/cpp/build/reference/debug-generate-debug-info?view=vs-2019 > [1] https://docs.microsoft.com/en-us/cpp/build/reference/pdb-use-program-database?view=vs-2019 > >> -----Original Message----- >> From: Erik Joelsson >> Sent: Mittwoch, 4. Dezember 2019 15:46 >> To: Bob Vandette >> Cc: Langer, Christoph ; build- >> dev at openjdk.java.net; core-libs-dev at openjdk.java.net; hotspot-dev >> developers >> Subject: Re: native debug symbols support on Windows >> >> Hello, >> >> On 2019-12-04 06:26, Bob Vandette wrote: >>> There seems to be an option that will include debug information in >> generated .obj files. Assuming this option is supported in the >>> versions of Visual Studio we use, it could be used to implement ?internal? >> native debug symbols. >>> /Z7 >> We already use this when compiling, but we still link to external pdb >> files. I was not aware of being able to link with the symbol information >> internal. >> >> While this seems like it could be implemented, my question is, does >> anyone need it? The internal symbols on Linux was something the Linux >> distros wanted as they like to move it out in a uniform manner later. I >> can't really see a need for this on Windows, but I certainly wouldn't >> object if someone else do and wants to implement the support in the >> OpenJDK build. >> >> /Erik >> >>> The /Z7 option produces object files that also contain full symbolic >> debugging information for use with the debugger. These object files and the >> built executable can be substantially larger than files that have no debugging >> information. The symbolic debugging information includes the names and >> types of variables, as well as functions and line numbers. No PDB file is >> produced. >>> Bob. >>> >>> >>>> On Dec 4, 2019, at 9:11 AM, Erik Joelsson >> wrote: >>>> Correct, with the Microsoft toolchain there is no support for internal. I >> don't know what happens to the build if you try to configure it that way. Feel >> free to come up with a reasonable behavior. >>>> /Erik >>>> >>>> On 2019-12-04 00:06, Langer, Christoph wrote: >>>>> Hi, >>>>> >>>>> I'm currently looking into native debug symbols support for Windows. >>>>> >>>>> The OpenJDK build system supports these two configure flags --with- >> native-debug-symbols= (among a few other options >> which I don't want to discuss here). >>>>> So, the name implies that for "internal", debug symbols should be >> contained in the binaries. And "external" should create separate files that >> contain the debug symbols. However, to my knowledge, Windows would >> always make use of external symbol files, named *.pdb. And there is no way >> to have the debug symbols included in the binaries. Is that correct or am I >> wrong in this assumption? >>>>> If it's true, I guess --with-native-debug-symbols=internal would not >> make sense on Windows and should rather be rejected by configure. >> Otherwise, if we were to support --with-native-debug-symbols=internal, the >> build is broken for target "test-image" when it comes to building/copying the >> gtest image. >>>>> I'd like to fix either the one way or the other. What do people think? >>>>> >>>>> Thanks for your help >>>>> Christoph >>>>> From igor.ignatyev at oracle.com Thu Dec 5 02:08:23 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 4 Dec 2019 18:08:23 -0800 Subject: RFR(T) : 8235353 : clean up hotspot problem lists In-Reply-To: References: Message-ID: <9405A3B3-8045-4863-9BD4-FD293E2C62F9@oracle.com> Martin, Goetz. could you please check that these 9 tests still pass on PPC? -- Igor > On Dec 4, 2019, at 12:01 PM, Vladimir Kozlov wrote: > > I am fine with changes but we need to ask PPC64 supporter to verify that tests passed now. > > Thanks, > Vladimir K > > On 12/4/19 11:52 AM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8235353/webrev.00 >>> 9 lines changed: 0 ins; 0 del; 9 mod; >> Hi all, >> could you please review this small and trivial cleanup which returns serviceablility/sa tests back to execution on linux-ppc64. the tests were problem listed due to 8211767[1], which is closed as a dup of resolved 8228649[2]. >> [1] https://bugs.openjdk.java.net/browse/JDK-8211767 >> [2] https://bugs.openjdk.java.net/browse/JDK-8228649 >> Thanks, >> -- Igor From christoph.langer at sap.com Thu Dec 5 11:06:21 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 5 Dec 2019 11:06:21 +0000 Subject: RFR (S): 8235403: Further cleanup to test serviceability/sa/ClhsdbCDSCore.java Message-ID: Hi, please review this little test cleanup. When JDK-8234625: hs test serviceability/sa/ClhsdbCDSCore.java fails on macOS 10.15 [0] was reviewed, there were further comments which didn't make it into the commit. I think the check for MacOS version 10.15 should not be done, the writability check on /cores is enough. I also tried to improve some messages and added a try-with-resource around the usage of a Scanner instance. Bug: https://bugs.openjdk.java.net/browse/JDK-8235403 Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8235403.0/ Thanks Christoph [0] https://bugs.openjdk.java.net/browse/JDK-8234625 From christoph.langer at sap.com Thu Dec 5 11:24:14 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 5 Dec 2019 11:24:14 +0000 Subject: RFR(T) : 8235353 : clean up hotspot problem lists In-Reply-To: <9405A3B3-8045-4863-9BD4-FD293E2C62F9@oracle.com> References: <9405A3B3-8045-4863-9BD4-FD293E2C62F9@oracle.com> Message-ID: Hi Igor, I have added your update to our test system. I'll let you know the results by tomorrow. Best regards Christoph > -----Original Message----- > From: serviceability-dev On > Behalf Of Igor Ignatyev > Sent: Donnerstag, 5. Dezember 2019 03:08 > To: Doerr, Martin ; Lindenmaier, Goetz > > Cc: serviceability-dev ; Vladimir > Kozlov ; hotspot-dev Source Developers > > Subject: Re: RFR(T) : 8235353 : clean up hotspot problem lists > > Martin, Goetz. > > could you please check that these 9 tests still pass on PPC? > > -- Igor > > > On Dec 4, 2019, at 12:01 PM, Vladimir Kozlov > wrote: > > > > I am fine with changes but we need to ask PPC64 supporter to verify that > tests passed now. > > > > Thanks, > > Vladimir K > > > > On 12/4/19 11:52 AM, Igor Ignatyev wrote: > >> http://cr.openjdk.java.net/~iignatyev//8235353/webrev.00 > >>> 9 lines changed: 0 ins; 0 del; 9 mod; > >> Hi all, > >> could you please review this small and trivial cleanup which returns > serviceablility/sa tests back to execution on linux-ppc64. the tests were > problem listed due to 8211767[1], which is closed as a dup of resolved > 8228649[2]. > >> [1] https://bugs.openjdk.java.net/browse/JDK-8211767 > >> [2] https://bugs.openjdk.java.net/browse/JDK-8228649 > >> Thanks, > >> -- Igor From robbin.ehn at oracle.com Thu Dec 5 14:01:46 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 5 Dec 2019 15:01:46 +0100 Subject: RFR(s): 8235410: Enable handshakes on Linux x86 (32-bit) Message-ID: Hi all, please review. The flag ThreadLocalHandshakes is going to be obsolete in JDK 14. So we change the (unchangeable) default for Linux x86 to on/true. There is a follow-up which removes ThreadLocalHandshakes completely. If the platform defines THREAD_LOCAL_POLL it will use handshakes. When arm32 have implemented local polls, the plan is to remove the global poll code paths. Last time I checked with some of the affected parties there was no objections. Built and sanity test. Issue: https://bugs.openjdk.java.net/browse/JDK-8235410 Code below. Thanks, Robbin diff -r 636d71e53732 src/hotspot/cpu/x86/globals_x86.hpp --- a/src/hotspot/cpu/x86/globals_x86.hpp Wed Dec 04 10:26:32 2019 +0100 +++ b/src/hotspot/cpu/x86/globals_x86.hpp Thu Dec 05 14:13:57 2019 +0100 @@ -89,12 +89,7 @@ define_pd_global(intx, InitArrayShortSize, 8*BytesPerLong); -#if defined(_LP64) || defined(_WINDOWS) define_pd_global(bool, ThreadLocalHandshakes, true); -#else -// get_thread() is slow on linux 32 bit, therefore off by default -define_pd_global(bool, ThreadLocalHandshakes, false); -#endif #define ARCH_FLAGS(develop, \ product, \ From ioi.lam at oracle.com Thu Dec 5 16:21:17 2019 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 5 Dec 2019 08:21:17 -0800 Subject: RFR (S): 8235403: Further cleanup to test serviceability/sa/ClhsdbCDSCore.java In-Reply-To: References: Message-ID: Looks good to me. Thanks - Ioi On 12/5/19 3:06 AM, Langer, Christoph wrote: > Hi, > > please review this little test cleanup. When JDK-8234625: hs test serviceability/sa/ClhsdbCDSCore.java fails on macOS 10.15 [0] was reviewed, there were further comments which didn't make it into the commit. > > I think the check for MacOS version 10.15 should not be done, the writability check on /cores is enough. I also tried to improve some messages and added a try-with-resource around the usage of a Scanner instance. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8235403 > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8235403.0/ > > Thanks > Christoph > > [0] https://bugs.openjdk.java.net/browse/JDK-8234625 > From igor.ignatyev at oracle.com Thu Dec 5 17:33:29 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 5 Dec 2019 09:33:29 -0800 Subject: RFR (S): 8235403: Further cleanup to test serviceability/sa/ClhsdbCDSCore.java In-Reply-To: References: Message-ID: <2061211D-E61F-49F2-B1F1-614820FAD60A@oracle.com> the patch looks good to me. I have a more generic question about this test though: I see we call cleanup() at almost all places which finish the test o way or another, I don't see why it's needed as jtreg is supposed to remove these files (unless -retain's value asks to preserve them), so if cleanup() isn't needed I'd like us to remove from one places but L#79 (where it serves as a defense from non empty work dir). if there are (good) reasons for cleanup to be called before exiting the test, shouldn't it also be called at L#115 and L#119? Thanks, -- Igor > On Dec 5, 2019, at 8:21 AM, Ioi Lam wrote: > > Looks good to me. > > Thanks > - Ioi > > On 12/5/19 3:06 AM, Langer, Christoph wrote: >> Hi, >> >> please review this little test cleanup. When JDK-8234625: hs test serviceability/sa/ClhsdbCDSCore.java fails on macOS 10.15 [0] was reviewed, there were further comments which didn't make it into the commit. >> >> I think the check for MacOS version 10.15 should not be done, the writability check on /cores is enough. I also tried to improve some messages and added a try-with-resource around the usage of a Scanner instance. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8235403 >> Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8235403.0/ >> >> Thanks >> Christoph >> >> [0] https://bugs.openjdk.java.net/browse/JDK-8234625 >> > From adinn at redhat.com Thu Dec 5 19:47:41 2019 From: adinn at redhat.com (Andrew Dinn) Date: Thu, 5 Dec 2019 19:47:41 +0000 Subject: 8233948: AArch64: Incorrect mapping between OptoReg and VMReg for high 64 bits of Vector Register In-Reply-To: References: Message-ID: <96a686bd-0668-1be8-1ca0-54c3f2dd9566@redhat.com> Hi Joshua, On 03/12/2019 12:26, Joshua Zhu (Arm Technology China) wrote: >> Yes, of course. I updated webrev according to your review comments. >> http://cr.openjdk.java.net/~jzhu/8233948/webrev.01/ > >>> Also, could you report what testing you did before and after your >>> change (other than checking the dump output). You will probably need >>> to repeat it to ensure these extra changes are ok. >> >> Oh, Sorry for that. I did not mention my testing in my previous mail (My >> initial RFR mail is too long). >> I ran full jtreg tests without new failure introduced. >> I will re-run it against the new webrev and will let you know the results. >> Thanks again for your review. > > Testing: Passed hotspot_all_no_apps, jdk_core and langtools:tier1 without new failure introduced. > Please let me know if any comments. Thanks! That looks good thanks. Also, thank you for correcting my arithmetic when computing vect_words. Approved for pushing. regards, Andrew Dinn ----------- From maurizio.cimadamore at oracle.com Thu Dec 5 21:04:55 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 5 Dec 2019 21:04:55 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) Message-ID: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> Hi, as part of the effort to upstream the changes related to JEP 370 (foreign memory access API) [1], I'd like to ask for a code review for the corresponding core-libs and hotspot changes: http://cr.openjdk.java.net/~mcimadamore/panama/8234049/ A javadoc for the memory access API is also available here: http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html Note: the patch passes tier1, tier2 and tier3 testing (**) Here is a brief summary of the changes in java.base and hotspot (the remaining new files are implementation classes and tests for the new API): * ciField.cpp - this one is to trust final fields in the foreign memory access implementation (otherwise VM doesn't trust memory segment bounds) * Modules.gmk - these changes are needed to require that the incubating module is loaded by the boot loader (otherwise the above changes are useless) * library_call.cpp - this one is a JIT compiler change to treat Thread.currentThread() as a well-known constant - which helps a lot in the confinement checks (thanks Vlad!) * various Buffer-related changes; these changes are needed because the memory access API allows a memory segment to be projected into a byte buffer, for interop reasons. As such, we need to insert a liveness check in the various get/put methods. Previously we had an implementation strategy where a BB was 'decorated' by a subclass called ScopedBuffer - but doing so required some changes to the BB API (e.g. making certain methods non-final, so that we could decorate them). Here I use an approach (which I have discussed with Alan) which doesn't require any public API? changes, but needs to add a 'segment' field in Buffer - and then have constructors which keep track of this extra parameter. * FileChannel changes - these changes are required so that we can reuse the Unmapper class from the MemorySegment implementation, to deterministically deallocate a mapped memory segment. This should be a 'straight' refactoring, no change in behavior should occur here. Please double check. * VarHandles - this class now provides a factory to create memory access VarHandle - this is a bit tricky, since VarHandle cannot really be implemented outside java.base (e.g. VarForm is not public). So we do the usual trick where we define a bunch of proxy interfaces (see jdk/internal/access/foreign) have the classes in java.base refer to these - and then have the implementation classes of the memory access API implement these interfaces. * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we need to provide access to otherwise hidden functionalities - e.g. creating a new scoped buffer, or retrieving the properties of a memory access handle (e.g. offset, stride etc.), so that we can implement the memory access API in its own separate module * GensrcVarHandles.gmk - these changes are needed to enable the generation of the new memory address var handle implementations; there's an helper class per carrier (e.g. VarHandleMemoryAddressAsBytes, ...). At runtime, when a memory access var handle is needed, we dynamically spin a new VH implementation which makes use of the right carrier. We need to spin because the VH can have a variable number of access coordinates (e.g. depending on the dimensions of the array to be accessed). But, under the hood, all the generated implementation will be using the same helper class. * tests - we've tried to add fairly robust tests, often checking all possible permutations of carriers/dimensions etc. Because of that, the tests might not be the easiest to look at, but they have proven to be pretty effective at shaking out issues. I think that covers the main aspects of the implementation and where it differs from vanilla JDK. P.S. In the CSR review [2], Joe raised a fair point - which is MemoryAddress has both: offset(long) --> move address of given offset offset() --> return the offset of this address in its owning segment And this was considered suboptimal, given both methods use the same name but do something quite different (one is an accessor, another is a 'wither'). one obvious option is to rename the first to 'withOffset'. But I think that would lead to verbose code (since that is a very common operation). Other options are to: * rename offset(long) to move(long), advance(long), or something else * drop offset() - but then add an overload of MemorySegment::asSlice which takes an address instead of a plain long offset I'll leave the choice to the reviewers :-) Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, Stuart, Roger, Joe and the Panama team for the feedback provided so far, which helped to get the API in the shape it is today. Cheers Maurizio (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" - but that is unrelated to this patch, and it's a known failing test. [1] - https://openjdk.java.net/jeps/370 [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 From david.holmes at oracle.com Thu Dec 5 21:36:09 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 6 Dec 2019 07:36:09 +1000 Subject: RFR(s): 8235410: Enable handshakes on Linux x86 (32-bit) In-Reply-To: References: Message-ID: <3807ba73-5b46-7f9b-c2b2-eeaaad734518@oracle.com> Hi Robbin, Change seems fine. One query below .... On 6/12/2019 12:01 am, Robbin Ehn wrote: > Hi all, please review. > > The flag ThreadLocalHandshakes is going to be obsolete in JDK 14. > So we change the (unchangeable) default for Linux x86 to on/true. > > There is a follow-up which removes ThreadLocalHandshakes completely. > If the platform defines THREAD_LOCAL_POLL it will use handshakes. > When arm32 have implemented local polls, the plan is to remove the > global poll > code paths. > > Last time I checked with some of the affected parties there was no > objections. > > Built and sanity test. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8235410 > > Code below. > > Thanks, Robbin > > diff -r 636d71e53732 src/hotspot/cpu/x86/globals_x86.hpp > --- a/src/hotspot/cpu/x86/globals_x86.hpp??? Wed Dec 04 10:26:32 2019 +0100 > +++ b/src/hotspot/cpu/x86/globals_x86.hpp??? Thu Dec 05 14:13:57 2019 +0100 > @@ -89,12 +89,7 @@ > > ?define_pd_global(intx, InitArrayShortSize, 8*BytesPerLong); > > -#if defined(_LP64) || defined(_WINDOWS) > ?define_pd_global(bool, ThreadLocalHandshakes, true); > -#else > -// get_thread() is slow on linux 32 bit, therefore off by default Why would get_thread be slow on linux 32-bit? Have we discussed this when handshakes went in? I'm feeling surprised now to see only 32-bit Linux was disabled. Thanks, David > -define_pd_global(bool, ThreadLocalHandshakes, false); > -#endif > > ?#define ARCH_FLAGS(develop, \ > ??????????????????? product, \ From vladimir.x.ivanov at oracle.com Thu Dec 5 22:39:49 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 6 Dec 2019 01:39:49 +0300 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> Message-ID: <5c1689c5-43bf-3b4a-50e9-7376097f3087@oracle.com> Awesome work, Maurizio! Regarding hotspot changes: > * ciField.cpp - this one is to trust final fields in the foreign memory > access implementation (otherwise VM doesn't trust memory segment bounds) New packages aren't part of java.base. Considering the implementation resides in an incubator module, is it enough to consider them as trusted and well-known to the JVM? > * library_call.cpp - this one is a JIT compiler change to treat > Thread.currentThread() as a well-known constant - which helps a lot in > the confinement checks (thanks Vlad!) FTR it is tracked by JDK-8235143 [1] and the patch is under review [2]. Best regards, Vladimir Ivanov [1] https://bugs.openjdk.java.net/browse/JDK-8235143 [2] https://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-December/036295.html > * various Buffer-related changes; these changes are needed because the > memory access API allows a memory segment to be projected into a byte > buffer, for interop reasons. As such, we need to insert a liveness check > in the various get/put methods. Previously we had an implementation > strategy where a BB was 'decorated' by a subclass called ScopedBuffer - > but doing so required some changes to the BB API (e.g. making certain > methods non-final, so that we could decorate them). Here I use an > approach (which I have discussed with Alan) which doesn't require any > public API? changes, but needs to add a 'segment' field in Buffer - and > then have constructors which keep track of this extra parameter. > > * FileChannel changes - these changes are required so that we can reuse > the Unmapper class from the MemorySegment implementation, to > deterministically deallocate a mapped memory segment. This should be a > 'straight' refactoring, no change in behavior should occur here. Please > double check. > > * VarHandles - this class now provides a factory to create memory access > VarHandle - this is a bit tricky, since VarHandle cannot really be > implemented outside java.base (e.g. VarForm is not public). So we do the > usual trick where we define a bunch of proxy interfaces (see > jdk/internal/access/foreign) have the classes in java.base refer to > these - and then have the implementation classes of the memory access > API implement these interfaces. > > * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we need to > provide access to otherwise hidden functionalities - e.g. creating a new > scoped buffer, or retrieving the properties of a memory access handle > (e.g. offset, stride etc.), so that we can implement the memory access > API in its own separate module > > * GensrcVarHandles.gmk - these changes are needed to enable the > generation of the new memory address var handle implementations; there's > an helper class per carrier (e.g. VarHandleMemoryAddressAsBytes, ...). > At runtime, when a memory access var handle is needed, we dynamically > spin a new VH implementation which makes use of the right carrier. We > need to spin because the VH can have a variable number of access > coordinates (e.g. depending on the dimensions of the array to be > accessed). But, under the hood, all the generated implementation will be > using the same helper class. > > * tests - we've tried to add fairly robust tests, often checking all > possible permutations of carriers/dimensions etc. Because of that, the > tests might not be the easiest to look at, but they have proven to be > pretty effective at shaking out issues. > > I think that covers the main aspects of the implementation and where it > differs from vanilla JDK. > > P.S. > > In the CSR review [2], Joe raised a fair point - which is MemoryAddress > has both: > > offset(long) --> move address of given offset > offset() --> return the offset of this address in its owning segment > > And this was considered suboptimal, given both methods use the same name > but do something quite different (one is an accessor, another is a > 'wither'). one obvious option is to rename the first to 'withOffset'. > But I think that would lead to verbose code (since that is a very common > operation). Other options are to: > > * rename offset(long) to move(long), advance(long), or something else > * drop offset() - but then add an overload of MemorySegment::asSlice > which takes an address instead of a plain long offset > > I'll leave the choice to the reviewers :-) > > > > Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, Stuart, > Roger, Joe and the Panama team for the feedback provided so far, which > helped to get the API in the shape it is today. > > Cheers > Maurizio > > (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" - > but that is unrelated to this patch, and it's a known failing test. > > [1] - https://openjdk.java.net/jeps/370 > [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 > From maurizio.cimadamore at oracle.com Thu Dec 5 23:27:18 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 5 Dec 2019 23:27:18 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <5c1689c5-43bf-3b4a-50e9-7376097f3087@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <5c1689c5-43bf-3b4a-50e9-7376097f3087@oracle.com> Message-ID: <0dc305b6-c727-7591-469e-fedd0f16d0a0@oracle.com> On 05/12/2019 22:39, Vladimir Ivanov wrote: > Awesome work, Maurizio! > > Regarding hotspot changes: > >> * ciField.cpp - this one is to trust final fields in the foreign >> memory access implementation (otherwise VM doesn't trust memory >> segment bounds) > > New packages aren't part of java.base. Considering the implementation > resides in an incubator module, is it enough to consider them as > trusted and well-known to the JVM? This gives same performance as with -XX:+TrustFinalNonStaticFields - if we remove these changes, then memory segment bounds are never inlined. I'm happy to change this if you have other suggestions on how to get there, of course (I can run some benchmarks w/ and w/o and post some numbers if that helps). > >> * library_call.cpp - this one is a JIT compiler change to treat >> Thread.currentThread() as a well-known constant - which helps a lot >> in the confinement checks (thanks Vlad!) > > FTR it is tracked by JDK-8235143 [1] and the patch is under review [2]. Should I remove this from the patch then? Thanks Maurizio > > Best regards, > Vladimir Ivanov > > [1] https://bugs.openjdk.java.net/browse/JDK-8235143 > > [2] > https://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-December/036295.html > >> * various Buffer-related changes; these changes are needed because >> the memory access API allows a memory segment to be projected into a >> byte buffer, for interop reasons. As such, we need to insert a >> liveness check in the various get/put methods. Previously we had an >> implementation strategy where a BB was 'decorated' by a subclass >> called ScopedBuffer - but doing so required some changes to the BB >> API (e.g. making certain methods non-final, so that we could decorate >> them). Here I use an approach (which I have discussed with Alan) >> which doesn't require any public API? changes, but needs to add a >> 'segment' field in Buffer - and then have constructors which keep >> track of this extra parameter. >> >> * FileChannel changes - these changes are required so that we can >> reuse the Unmapper class from the MemorySegment implementation, to >> deterministically deallocate a mapped memory segment. This should be >> a 'straight' refactoring, no change in behavior should occur here. >> Please double check. >> >> * VarHandles - this class now provides a factory to create memory >> access VarHandle - this is a bit tricky, since VarHandle cannot >> really be implemented outside java.base (e.g. VarForm is not public). >> So we do the usual trick where we define a bunch of proxy interfaces >> (see jdk/internal/access/foreign) have the classes in java.base refer >> to these - and then have the implementation classes of the memory >> access API implement these interfaces. >> >> * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we need >> to provide access to otherwise hidden functionalities - e.g. creating >> a new scoped buffer, or retrieving the properties of a memory access >> handle (e.g. offset, stride etc.), so that we can implement the >> memory access API in its own separate module >> >> * GensrcVarHandles.gmk - these changes are needed to enable the >> generation of the new memory address var handle implementations; >> there's an helper class per carrier (e.g. >> VarHandleMemoryAddressAsBytes, ...). At runtime, when a memory access >> var handle is needed, we dynamically spin a new VH implementation >> which makes use of the right carrier. We need to spin because the VH >> can have a variable number of access coordinates (e.g. depending on >> the dimensions of the array to be accessed). But, under the hood, all >> the generated implementation will be using the same helper class. >> >> * tests - we've tried to add fairly robust tests, often checking all >> possible permutations of carriers/dimensions etc. Because of that, >> the tests might not be the easiest to look at, but they have proven >> to be pretty effective at shaking out issues. >> >> I think that covers the main aspects of the implementation and where >> it differs from vanilla JDK. >> >> P.S. >> >> In the CSR review [2], Joe raised a fair point - which is >> MemoryAddress has both: >> >> offset(long) --> move address of given offset >> offset() --> return the offset of this address in its owning segment >> >> And this was considered suboptimal, given both methods use the same >> name but do something quite different (one is an accessor, another is >> a 'wither'). one obvious option is to rename the first to >> 'withOffset'. But I think that would lead to verbose code (since that >> is a very common operation). Other options are to: >> >> * rename offset(long) to move(long), advance(long), or something else >> * drop offset() - but then add an overload of MemorySegment::asSlice >> which takes an address instead of a plain long offset >> >> I'll leave the choice to the reviewers :-) >> >> >> >> Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, >> Stuart, Roger, Joe and the Panama team for the feedback provided so >> far, which helped to get the API in the shape it is today. >> >> Cheers >> Maurizio >> >> (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" - >> but that is unrelated to this patch, and it's a known failing test. >> >> [1] - https://openjdk.java.net/jeps/370 >> [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 >> From forax at univ-mlv.fr Fri Dec 6 00:13:14 2019 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 6 Dec 2019 01:13:14 +0100 (CET) Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> Message-ID: <105628217.1955942.1575591194730.JavaMail.zimbra@u-pem.fr> Hi Maurizio, that's a huge work :) So as a guy discovering the API, i've not taken a deep look to the implementation because i've noob questions. The first sentence of the overview of GroupLayout should say that there are two types of GroupLayout struct and union instead of talking about members layouts in italic. Still in GroupLayout, there is a static method "optSize" with no documentation and you don"t use the prefix "opt" anywhere else so i think it should be dropped. hasNaturalAlignment() is protected, but you don"t allow user to implement their own GroupLayout, so it's like the method was package private but visible in the javadoc. MemoryLayout and MemoryLayout.PathElement static methods have names that are ok to use in static imports, but it's not explicit written in the javadoc. People will say that Java is verbose :) I don't see the point of having MemoryLayouts separated from MemoryLayout. MemoryAddress should have a method that does raw bytes comparison like there is MemoryAddress.copy() so simulate struct comparison. I don't understand how MemorySegment.acquire() solve the fact that you have two threads that can access to the same underlying memory area with unprotected access. When you own a memory segment, an other thread can acquire a new memory segment and then both thread can mutate the same memory location ? MemorySegment.isAccessible() is a very overloaded term, isOwnByCurrentThread() is maybe better ? In the implementation of MemoryScope, please don't use i++ or i-- to change the lambda parameter, what you want is i - 1 or i + 1. And i wander if it's not more efficient to use getAndIncrement and if it's negative (overflow), to do a supplementary volatile write to Integer.MIN_VALUE before raising the IllegalStateException. In AddressVarHandleGenerator.iConstInsn(), ICONST_M1, ICONST_0, etc are subsequent bytecodes so you can group all the case to: mv.visitInsn(ICONST_0 + i); regards, R?mi ----- Mail original ----- > De: "Maurizio Cimadamore" > ?: "core-libs-dev" , "hotspot-dev" > Envoy?: Jeudi 5 D?cembre 2019 22:04:55 > Objet: RFR JDK-8234049: Implementation of Memory Access API (Incubator) > Hi, > as part of the effort to upstream the changes related to JEP 370 > (foreign memory access API) [1], I'd like to ask for a code review for > the corresponding core-libs and hotspot changes: > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049/ > > A javadoc for the memory access API is also available here: > > http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html > > Note: the patch passes tier1, tier2 and tier3 testing (**) > > > Here is a brief summary of the changes in java.base and hotspot (the > remaining new files are implementation classes and tests for the new API): > > * ciField.cpp - this one is to trust final fields in the foreign memory > access implementation (otherwise VM doesn't trust memory segment bounds) > > * Modules.gmk - these changes are needed to require that the incubating > module is loaded by the boot loader (otherwise the above changes are > useless) > > * library_call.cpp - this one is a JIT compiler change to treat > Thread.currentThread() as a well-known constant - which helps a lot in > the confinement checks (thanks Vlad!) > > * various Buffer-related changes; these changes are needed because the > memory access API allows a memory segment to be projected into a byte > buffer, for interop reasons. As such, we need to insert a liveness check > in the various get/put methods. Previously we had an implementation > strategy where a BB was 'decorated' by a subclass called ScopedBuffer - > but doing so required some changes to the BB API (e.g. making certain > methods non-final, so that we could decorate them). Here I use an > approach (which I have discussed with Alan) which doesn't require any > public API? changes, but needs to add a 'segment' field in Buffer - and > then have constructors which keep track of this extra parameter. > > * FileChannel changes - these changes are required so that we can reuse > the Unmapper class from the MemorySegment implementation, to > deterministically deallocate a mapped memory segment. This should be a > 'straight' refactoring, no change in behavior should occur here. Please > double check. > > * VarHandles - this class now provides a factory to create memory access > VarHandle - this is a bit tricky, since VarHandle cannot really be > implemented outside java.base (e.g. VarForm is not public). So we do the > usual trick where we define a bunch of proxy interfaces (see > jdk/internal/access/foreign) have the classes in java.base refer to > these - and then have the implementation classes of the memory access > API implement these interfaces. > > * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we need to > provide access to otherwise hidden functionalities - e.g. creating a new > scoped buffer, or retrieving the properties of a memory access handle > (e.g. offset, stride etc.), so that we can implement the memory access > API in its own separate module > > * GensrcVarHandles.gmk - these changes are needed to enable the > generation of the new memory address var handle implementations; there's > an helper class per carrier (e.g. VarHandleMemoryAddressAsBytes, ...). > At runtime, when a memory access var handle is needed, we dynamically > spin a new VH implementation which makes use of the right carrier. We > need to spin because the VH can have a variable number of access > coordinates (e.g. depending on the dimensions of the array to be > accessed). But, under the hood, all the generated implementation will be > using the same helper class. > > * tests - we've tried to add fairly robust tests, often checking all > possible permutations of carriers/dimensions etc. Because of that, the > tests might not be the easiest to look at, but they have proven to be > pretty effective at shaking out issues. > > I think that covers the main aspects of the implementation and where it > differs from vanilla JDK. > > P.S. > > In the CSR review [2], Joe raised a fair point - which is MemoryAddress > has both: > > offset(long) --> move address of given offset > offset() --> return the offset of this address in its owning segment > > And this was considered suboptimal, given both methods use the same name > but do something quite different (one is an accessor, another is a > 'wither'). one obvious option is to rename the first to 'withOffset'. > But I think that would lead to verbose code (since that is a very common > operation). Other options are to: > > * rename offset(long) to move(long), advance(long), or something else > * drop offset() - but then add an overload of MemorySegment::asSlice > which takes an address instead of a plain long offset > > I'll leave the choice to the reviewers :-) > > > > Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, Stuart, > Roger, Joe and the Panama team for the feedback provided so far, which > helped to get the API in the shape it is today. > > Cheers > Maurizio > > (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" - > but that is unrelated to this patch, and it's a known failing test. > > [1] - https://openjdk.java.net/jeps/370 > [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 From maurizio.cimadamore at oracle.com Fri Dec 6 01:07:33 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 6 Dec 2019 01:07:33 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <105628217.1955942.1575591194730.JavaMail.zimbra@u-pem.fr> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <105628217.1955942.1575591194730.JavaMail.zimbra@u-pem.fr> Message-ID: <6fbd62eb-e434-c546-37af-fcc9b0744fc8@oracle.com> On 06/12/2019 00:13, Remi Forax wrote: > Hi Maurizio, > that's a huge work :) > > So as a guy discovering the API, > i've not taken a deep look to the implementation because i've noob questions. > > The first sentence of the overview of GroupLayout should say that there are two types of GroupLayout struct and union instead of talking about members layouts in italic. > Still in GroupLayout, there is a static method "optSize" with no documentation and you don"t use the prefix "opt" anywhere else so i think it should be dropped. > hasNaturalAlignment() is protected, but you don"t allow user to implement their own GroupLayout, so it's like the method was package private but visible in the javadoc. Good catch - these two methods were never meant to be exposed - they are morally package private members of a shared implementation class - I will fix that > > MemoryLayout and MemoryLayout.PathElement static methods have names that are ok to use in static imports, but it's not explicit written in the javadoc. > People will say that Java is verbose :) To give some context here - we had an earlier variant of the API which was much more concise - e.g. layout.varHandle(p -> p.groupElement("foo").sequenceElement()) This uses a builder-style idiom. The problem with this approach is that is less constant-folding friendly - which is the main reason as to why we went the current path. > > I don't see the point of having MemoryLayouts separated from MemoryLayout. Possibly - I found myself thinking that too - although, with the subsequent Panama step (ABI support) we'll be adding a ton of ABI-dependent layouts in here... (but we could address that in other ways also). If there's enough consensus one way or another I'm happy to consolidate the two classes. > > MemoryAddress should have a method that does raw bytes comparison like there is MemoryAddress.copy() so simulate struct comparison. Uhm - it's not a bad idea - but I don't immediately see why it *has* to be in the API - it will be a loop - we can't do much more than that, in terms of making it fast. > > I don't understand how MemorySegment.acquire() solve the fact that you have two threads that can access to the same underlying memory area with unprotected access. > When you own a memory segment, an other thread can acquire a new memory segment and then both thread can mutate the same memory location ? With concurrent access you have two kinds of races: - access vs. access (e.g. thread A reads while threads B write) - access vs. close (e.g. thread A reads while thread B close the segment) The first race is *not* what we want to protect you against here. The VarHandle API has plenty of primitives which allow clients to roll in their own synchronization. The really nasty issue is with the latter race - if you access a segment that has been closed, you don't just read a stale value, you can crash the VM. When you acquire a segment you obtain a new view of the same memory block which is temporally bounded by the original segment. The original segment cannot be closed until all the acquired views have been closed. This guarantees that the memory will never go away behind your back. > > MemorySegment.isAccessible() is a very overloaded term, isOwnByCurrentThread() is maybe better ? Open to suggestion - I don't particularly like yours though :-) > > In the implementation of MemoryScope, please don't use i++ or i-- to change the lambda parameter, what you want is i - 1 or i + 1. > And i wander if it's not more efficient to use getAndIncrement and if it's negative (overflow), to do a supplementary volatile write to Integer.MIN_VALUE before raising the IllegalStateException. Goo points - note that, in these cases, we don't really care about performance much - because acquiring a segment is not on the hot path (which is memory access). > > In AddressVarHandleGenerator.iConstInsn(), > ICONST_M1, ICONST_0, etc are subsequent bytecodes so you can group all the case to: mv.visitInsn(ICONST_0 + i); will do thanks! Maurizio > > regards, > R?mi > > ----- Mail original ----- >> De: "Maurizio Cimadamore" >> ?: "core-libs-dev" , "hotspot-dev" >> Envoy?: Jeudi 5 D?cembre 2019 22:04:55 >> Objet: RFR JDK-8234049: Implementation of Memory Access API (Incubator) >> Hi, >> as part of the effort to upstream the changes related to JEP 370 >> (foreign memory access API) [1], I'd like to ask for a code review for >> the corresponding core-libs and hotspot changes: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8234049/ >> >> A javadoc for the memory access API is also available here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >> >> Note: the patch passes tier1, tier2 and tier3 testing (**) >> >> >> Here is a brief summary of the changes in java.base and hotspot (the >> remaining new files are implementation classes and tests for the new API): >> >> * ciField.cpp - this one is to trust final fields in the foreign memory >> access implementation (otherwise VM doesn't trust memory segment bounds) >> >> * Modules.gmk - these changes are needed to require that the incubating >> module is loaded by the boot loader (otherwise the above changes are >> useless) >> >> * library_call.cpp - this one is a JIT compiler change to treat >> Thread.currentThread() as a well-known constant - which helps a lot in >> the confinement checks (thanks Vlad!) >> >> * various Buffer-related changes; these changes are needed because the >> memory access API allows a memory segment to be projected into a byte >> buffer, for interop reasons. As such, we need to insert a liveness check >> in the various get/put methods. Previously we had an implementation >> strategy where a BB was 'decorated' by a subclass called ScopedBuffer - >> but doing so required some changes to the BB API (e.g. making certain >> methods non-final, so that we could decorate them). Here I use an >> approach (which I have discussed with Alan) which doesn't require any >> public API? changes, but needs to add a 'segment' field in Buffer - and >> then have constructors which keep track of this extra parameter. >> >> * FileChannel changes - these changes are required so that we can reuse >> the Unmapper class from the MemorySegment implementation, to >> deterministically deallocate a mapped memory segment. This should be a >> 'straight' refactoring, no change in behavior should occur here. Please >> double check. >> >> * VarHandles - this class now provides a factory to create memory access >> VarHandle - this is a bit tricky, since VarHandle cannot really be >> implemented outside java.base (e.g. VarForm is not public). So we do the >> usual trick where we define a bunch of proxy interfaces (see >> jdk/internal/access/foreign) have the classes in java.base refer to >> these - and then have the implementation classes of the memory access >> API implement these interfaces. >> >> * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we need to >> provide access to otherwise hidden functionalities - e.g. creating a new >> scoped buffer, or retrieving the properties of a memory access handle >> (e.g. offset, stride etc.), so that we can implement the memory access >> API in its own separate module >> >> * GensrcVarHandles.gmk - these changes are needed to enable the >> generation of the new memory address var handle implementations; there's >> an helper class per carrier (e.g. VarHandleMemoryAddressAsBytes, ...). >> At runtime, when a memory access var handle is needed, we dynamically >> spin a new VH implementation which makes use of the right carrier. We >> need to spin because the VH can have a variable number of access >> coordinates (e.g. depending on the dimensions of the array to be >> accessed). But, under the hood, all the generated implementation will be >> using the same helper class. >> >> * tests - we've tried to add fairly robust tests, often checking all >> possible permutations of carriers/dimensions etc. Because of that, the >> tests might not be the easiest to look at, but they have proven to be >> pretty effective at shaking out issues. >> >> I think that covers the main aspects of the implementation and where it >> differs from vanilla JDK. >> >> P.S. >> >> In the CSR review [2], Joe raised a fair point - which is MemoryAddress >> has both: >> >> offset(long) --> move address of given offset >> offset() --> return the offset of this address in its owning segment >> >> And this was considered suboptimal, given both methods use the same name >> but do something quite different (one is an accessor, another is a >> 'wither'). one obvious option is to rename the first to 'withOffset'. >> But I think that would lead to verbose code (since that is a very common >> operation). Other options are to: >> >> * rename offset(long) to move(long), advance(long), or something else >> * drop offset() - but then add an overload of MemorySegment::asSlice >> which takes an address instead of a plain long offset >> >> I'll leave the choice to the reviewers :-) >> >> >> >> Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, Stuart, >> Roger, Joe and the Panama team for the feedback provided so far, which >> helped to get the API in the shape it is today. >> >> Cheers >> Maurizio >> >> (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" - >> but that is unrelated to this patch, and it's a known failing test. >> >> [1] - https://openjdk.java.net/jeps/370 >> [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 From fujie at loongson.cn Fri Dec 6 02:27:43 2019 From: fujie at loongson.cn (Jie Fu) Date: Fri, 6 Dec 2019 10:27:43 +0800 Subject: RFR: 8235456: Minimal VM is broken after JDK-8212160 Message-ID: <6bda7d1b-7868-97ec-2676-988d956664b4@loongson.cn> Hi all, May I get reviews for the one-line fix? JBS:??? https://bugs.openjdk.java.net/browse/JDK-8235456 Webrev: http://cr.openjdk.java.net/~jiefu/8235456/webrev.00/ Thanks a lot. Best regards, Jie From Joshua.Zhu at arm.com Fri Dec 6 02:34:54 2019 From: Joshua.Zhu at arm.com (Joshua Zhu (Arm Technology China)) Date: Fri, 6 Dec 2019 02:34:54 +0000 Subject: 8233948: AArch64: Incorrect mapping between OptoReg and VMReg for high 64 bits of Vector Register In-Reply-To: <96a686bd-0668-1be8-1ca0-54c3f2dd9566@redhat.com> References: <96a686bd-0668-1be8-1ca0-54c3f2dd9566@redhat.com> Message-ID: Hi, > > Testing: Passed hotspot_all_no_apps, jdk_core and langtools:tier1 without > new failure introduced. > > Please let me know if any comments. Thanks! > > That looks good thanks. > > Also, thank you for correcting my arithmetic when computing vect_words. > > Approved for pushing. Andrew Dinn, thanks a lot for your review and comments! Ningsheng, could you please help push this patch? Thanks! Best Regards, Joshua From ningsheng.jian at arm.com Fri Dec 6 02:44:33 2019 From: ningsheng.jian at arm.com (Ningsheng Jian) Date: Fri, 6 Dec 2019 10:44:33 +0800 Subject: 8233948: AArch64: Incorrect mapping between OptoReg and VMReg for high 64 bits of Vector Register In-Reply-To: References: <96a686bd-0668-1be8-1ca0-54c3f2dd9566@redhat.com> Message-ID: <0250139e-3795-dfb0-d33c-4ea3d8370484@arm.com> Pushed. Thanks, Ningsheng On 12/6/19 10:34 AM, Joshua Zhu (Arm Technology China) wrote: > Hi, > >>> Testing: Passed hotspot_all_no_apps, jdk_core and langtools:tier1 without >> new failure introduced. >>> Please let me know if any comments. Thanks! >> >> That looks good thanks. >> >> Also, thank you for correcting my arithmetic when computing vect_words. >> >> Approved for pushing. > > Andrew Dinn, thanks a lot for your review and comments! > Ningsheng, could you please help push this patch? Thanks! > > Best Regards, > Joshua > From david.holmes at oracle.com Fri Dec 6 02:47:52 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 6 Dec 2019 12:47:52 +1000 Subject: RFR: 8235456: Minimal VM is broken after JDK-8212160 In-Reply-To: <6bda7d1b-7868-97ec-2676-988d956664b4@loongson.cn> References: <6bda7d1b-7868-97ec-2676-988d956664b4@loongson.cn> Message-ID: Hi Jie, Looks good and trivial. Sorry about the breakage - again. Thanks, David On 6/12/2019 12:27 pm, Jie Fu wrote: > Hi all, > > May I get reviews for the one-line fix? > > JBS:??? https://bugs.openjdk.java.net/browse/JDK-8235456 > Webrev: http://cr.openjdk.java.net/~jiefu/8235456/webrev.00/ > > Thanks a lot. > Best regards, > Jie > From fujie at loongson.cn Fri Dec 6 02:51:33 2019 From: fujie at loongson.cn (Jie Fu) Date: Fri, 6 Dec 2019 10:51:33 +0800 Subject: RFR: 8235456: Minimal VM is broken after JDK-8212160 In-Reply-To: References: <6bda7d1b-7868-97ec-2676-988d956664b4@loongson.cn> Message-ID: <56d86788-12ea-f481-af70-804c75e7c69b@loongson.cn> Thanks David for your review. On 2019/12/6 ??10:47, David Holmes wrote: > Looks good and trivial. From maurizio.cimadamore at oracle.com Fri Dec 6 08:32:12 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 6 Dec 2019 08:32:12 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <105628217.1955942.1575591194730.JavaMail.zimbra@u-pem.fr> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <105628217.1955942.1575591194730.JavaMail.zimbra@u-pem.fr> Message-ID: <39931722-731d-6cdd-fa9b-c0173a8e565c@oracle.com> Couple of naming ideas: > MemorySegment.isAccessible() is a very overloaded term, isOwnByCurrentThread() is maybe better ? One solution here would be to, instead, have an accessor called ownerThread() - then clients can test doing ownerThread() != Thread.currentThread (or some other thread). After all a segment is specified to have an owner, so it kind of makes sense to have an accessor for it? > I don't understand how MemorySegment.acquire() solve the fact that you have two threads that can access to the same underlying memory area with unprotected access. > When you own a memory segment, an other thread can acquire a new memory segment and then both thread can mutate the same memory location ? I believe the name "acquire" might not describe accurately what is going on - I believe "fork" is much closer to what the method is actually doing. You can imagine segments forming a tree - if you keep forking from same segment, you create a lot of sub-segments - and so on and so forth. A segment cannot be closed if it has a non-zero number of forks. Maurizio From maurizio.cimadamore at oracle.com Fri Dec 6 08:48:49 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 6 Dec 2019 08:48:49 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <0dc305b6-c727-7591-469e-fedd0f16d0a0@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <5c1689c5-43bf-3b4a-50e9-7376097f3087@oracle.com> <0dc305b6-c727-7591-469e-fedd0f16d0a0@oracle.com> Message-ID: On 05/12/2019 23:27, Maurizio Cimadamore wrote: > This gives same performance as with -XX:+TrustFinalNonStaticFields - > if we remove these changes, then memory segment bounds are never > inlined. I'm happy to change this if you have other suggestions on how > to get there, of course (I can run some benchmarks w/ and w/o and post > some numbers if that helps). Here are some numbers for a straight VH get using a constant address; with the patch I get this (throughput - higher the better): GetSetConstantBenchmark.testMemoryHandleGet? --> 299325598.056 ? 160635.595? ops/s Without the patch I get this: GetSetConstantBenchmark.testMemoryHandleGet? -->? 163721200.469 ? 2344351.046? ops/s So, the difference is quiet big. Graal (using JVMCI) seems less sensitive to this particular issue, and I detected no differences there. That said, if we have plans to fix this another way, I'm happy to remove this workaround. Cheers Maurizio From robbin.ehn at oracle.com Fri Dec 6 09:22:20 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 6 Dec 2019 10:22:20 +0100 Subject: RFR(s): 8235410: Enable handshakes on Linux x86 (32-bit) In-Reply-To: <3807ba73-5b46-7f9b-c2b2-eeaaad734518@oracle.com> References: <3807ba73-5b46-7f9b-c2b2-eeaaad734518@oracle.com> Message-ID: Thanks David! On 2019-12-05 22:36, David Holmes wrote: >> diff -r 636d71e53732 src/hotspot/cpu/x86/globals_x86.hpp >> --- a/src/hotspot/cpu/x86/globals_x86.hpp??? Wed Dec 04 10:26:32 2019 >> +0100 >> +++ b/src/hotspot/cpu/x86/globals_x86.hpp??? Thu Dec 05 14:13:57 2019 >> +0100 >> @@ -89,12 +89,7 @@ >> >> ??define_pd_global(intx, InitArrayShortSize, 8*BytesPerLong); >> >> -#if defined(_LP64) || defined(_WINDOWS) >> ??define_pd_global(bool, ThreadLocalHandshakes, true); >> -#else >> -// get_thread() is slow on linux 32 bit, therefore off by default > > Why would get_thread be slow on linux 32-bit? Have we discussed this > when handshakes went in? I'm feeling surprised now to see only 32-bit > Linux was disabled. You actually explained it RFR for 32-bit TLH: "Thread::current() already uses compiler-based TLS. get_thread (on non-Windows x86) make a VM call to Thread::current(). If we want to get faster here we have to introduce compiler specific hooks directly into the TLS implementation (not nice to do initially or maintain). " :) /Robbin > > Thanks, > David > >> -define_pd_global(bool, ThreadLocalHandshakes, false); >> -#endif >> >> ??#define ARCH_FLAGS(develop, \ >> ???????????????????? product, \ From christoph.langer at sap.com Fri Dec 6 10:19:53 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 6 Dec 2019 10:19:53 +0000 Subject: RFR (S): 8235403: Further cleanup to test serviceability/sa/ClhsdbCDSCore.java In-Reply-To: <2061211D-E61F-49F2-B1F1-614820FAD60A@oracle.com> References: <2061211D-E61F-49F2-B1F1-614820FAD60A@oracle.com> Message-ID: Hi Igor, > the patch looks good to me. Thanks for reviewing > I have a more generic question about this test though: I see we call cleanup() > at almost all places which finish the test o way or another, I don't see why it's > needed as jtreg is supposed to remove these files (unless -retain's value asks > to preserve them), so if cleanup() isn't needed I'd like us to remove from one > places but L#79 (where it serves as a defense from non empty work dir). if > there are (good) reasons for cleanup to be called before exiting the test, > shouldn't it also be called at L#115 and L#119? Well, I think it's not too bad to actively call cleanup() , given that there might be a rather large core file lying around, occupying space. In line 115 and 119, no core file was probably created. However, I guess it would be still ok and defensive enough to call cleanup there as well - be it only to remove the shared archive file. I'll add cleanup() calls at these locations and run it through the testing. If I don't find any problems, I'll push with them... Cheers Christoph From christoph.langer at sap.com Fri Dec 6 10:20:28 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 6 Dec 2019 10:20:28 +0000 Subject: RFR (S): 8235403: Further cleanup to test serviceability/sa/ClhsdbCDSCore.java In-Reply-To: References: Message-ID: Thanks for the review, Ioi. > -----Original Message----- > From: hotspot-dev On Behalf Of > Ioi Lam > Sent: Donnerstag, 5. Dezember 2019 17:21 > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR (S): 8235403: Further cleanup to test > serviceability/sa/ClhsdbCDSCore.java > > Looks good to me. > > Thanks > - Ioi > > On 12/5/19 3:06 AM, Langer, Christoph wrote: > > Hi, > > > > please review this little test cleanup. When JDK-8234625: hs test > serviceability/sa/ClhsdbCDSCore.java fails on macOS 10.15 [0] was reviewed, > there were further comments which didn't make it into the commit. > > > > I think the check for MacOS version 10.15 should not be done, the > writability check on /cores is enough. I also tried to improve some messages > and added a try-with-resource around the usage of a Scanner instance. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8235403 > > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8235403.0/ > > > > Thanks > > Christoph > > > > [0] https://bugs.openjdk.java.net/browse/JDK-8234625 > > From maurizio.cimadamore at oracle.com Fri Dec 6 10:43:22 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 6 Dec 2019 10:43:22 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> Message-ID: <7765b06b-acf2-f11f-980e-33af4ca3934c@oracle.com> Hi, here's an updated version of the patch: http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v2/ And a delta of the changes since last version here: http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v2_delta/ The javadoc has been updated inline here: http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html Summary of changes: * fixed spurious protected methods in AbstractLayout and subclasses which leaked into API * removed library_call.cpp changes, as these are being tracked separately by Vlad * compacted ILOAD code in AddressVarHandleGenerator * replaced uses of ++i/--i with i + 1/i - 1 in MemoryScope I have made no changes to the *name* of the methods in the API. In fact, I suggest we keep a list of the names we'd like to revisit, and we address them all at once at the end of the review (once we're happy with the contents). Here's a list of the current open naming issues: * MemoryAddress::offset() vs. MemoryAddress::offset(long) -- not much distance between these two semantically different operations * MemorySegment::isAccessible() -- as the A* word is overloaded, some other name should be found? * MemorySegment::acquire() -- replace with MemorySegment::fork? Cheers Maurizio On 05/12/2019 21:04, Maurizio Cimadamore wrote: > Hi, > as part of the effort to upstream the changes related to JEP 370 > (foreign memory access API) [1], I'd like to ask for a code review for > the corresponding core-libs and hotspot changes: > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049/ > > A javadoc for the memory access API is also available here: > > http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html > > > Note: the patch passes tier1, tier2 and tier3 testing (**) > > > Here is a brief summary of the changes in java.base and hotspot (the > remaining new files are implementation classes and tests for the new > API): > > * ciField.cpp - this one is to trust final fields in the foreign > memory access implementation (otherwise VM doesn't trust memory > segment bounds) > > * Modules.gmk - these changes are needed to require that the > incubating module is loaded by the boot loader (otherwise the above > changes are useless) > > * library_call.cpp - this one is a JIT compiler change to treat > Thread.currentThread() as a well-known constant - which helps a lot in > the confinement checks (thanks Vlad!) > > * various Buffer-related changes; these changes are needed because the > memory access API allows a memory segment to be projected into a byte > buffer, for interop reasons. As such, we need to insert a liveness > check in the various get/put methods. Previously we had an > implementation strategy where a BB was 'decorated' by a subclass > called ScopedBuffer - but doing so required some changes to the BB API > (e.g. making certain methods non-final, so that we could decorate > them). Here I use an approach (which I have discussed with Alan) which > doesn't require any public API changes, but needs to add a 'segment' > field in Buffer - and then have constructors which keep track of this > extra parameter. > > * FileChannel changes - these changes are required so that we can > reuse the Unmapper class from the MemorySegment implementation, to > deterministically deallocate a mapped memory segment. This should be a > 'straight' refactoring, no change in behavior should occur here. > Please double check. > > * VarHandles - this class now provides a factory to create memory > access VarHandle - this is a bit tricky, since VarHandle cannot really > be implemented outside java.base (e.g. VarForm is not public). So we > do the usual trick where we define a bunch of proxy interfaces (see > jdk/internal/access/foreign) have the classes in java.base refer to > these - and then have the implementation classes of the memory access > API implement these interfaces. > > * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we need > to provide access to otherwise hidden functionalities - e.g. creating > a new scoped buffer, or retrieving the properties of a memory access > handle (e.g. offset, stride etc.), so that we can implement the memory > access API in its own separate module > > * GensrcVarHandles.gmk - these changes are needed to enable the > generation of the new memory address var handle implementations; > there's an helper class per carrier (e.g. > VarHandleMemoryAddressAsBytes, ...). At runtime, when a memory access > var handle is needed, we dynamically spin a new VH implementation > which makes use of the right carrier. We need to spin because the VH > can have a variable number of access coordinates (e.g. depending on > the dimensions of the array to be accessed). But, under the hood, all > the generated implementation will be using the same helper class. > > * tests - we've tried to add fairly robust tests, often checking all > possible permutations of carriers/dimensions etc. Because of that, the > tests might not be the easiest to look at, but they have proven to be > pretty effective at shaking out issues. > > I think that covers the main aspects of the implementation and where > it differs from vanilla JDK. > > P.S. > > In the CSR review [2], Joe raised a fair point - which is > MemoryAddress has both: > > offset(long) --> move address of given offset > offset() --> return the offset of this address in its owning segment > > And this was considered suboptimal, given both methods use the same > name but do something quite different (one is an accessor, another is > a 'wither'). one obvious option is to rename the first to > 'withOffset'. But I think that would lead to verbose code (since that > is a very common operation). Other options are to: > > * rename offset(long) to move(long), advance(long), or something else > * drop offset() - but then add an overload of MemorySegment::asSlice > which takes an address instead of a plain long offset > > I'll leave the choice to the reviewers :-) > > > > Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, > Stuart, Roger, Joe and the Panama team for the feedback provided so > far, which helped to get the API in the shape it is today. > > Cheers > Maurizio > > (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" - > but that is unrelated to this patch, and it's a known failing test. > > [1] - https://openjdk.java.net/jeps/370 > [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 > From martin.doerr at sap.com Fri Dec 6 10:51:11 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 6 Dec 2019 10:51:11 +0000 Subject: RFR(T) : 8235353 : clean up hotspot problem lists In-Reply-To: References: <9405A3B3-8045-4863-9BD4-FD293E2C62F9@oracle.com> Message-ID: Hi Igor and Vladimir, the tests have passed on PPC64. Change is good. Thanks for checking with us. Best regards, Martin > -----Original Message----- > From: Langer, Christoph > Sent: Donnerstag, 5. Dezember 2019 12:24 > To: Igor Ignatyev ; Doerr, Martin > ; Lindenmaier, Goetz > > Cc: serviceability-dev ; Vladimir > Kozlov ; hotspot-dev Source Developers > > Subject: RE: RFR(T) : 8235353 : clean up hotspot problem lists > > Hi Igor, > > I have added your update to our test system. I'll let you know the results by > tomorrow. > > Best regards > Christoph > > > -----Original Message----- > > From: serviceability-dev > On > > Behalf Of Igor Ignatyev > > Sent: Donnerstag, 5. Dezember 2019 03:08 > > To: Doerr, Martin ; Lindenmaier, Goetz > > > > Cc: serviceability-dev ; Vladimir > > Kozlov ; hotspot-dev Source Developers > > > > Subject: Re: RFR(T) : 8235353 : clean up hotspot problem lists > > > > Martin, Goetz. > > > > could you please check that these 9 tests still pass on PPC? > > > > -- Igor > > > > > On Dec 4, 2019, at 12:01 PM, Vladimir Kozlov > > > wrote: > > > > > > I am fine with changes but we need to ask PPC64 supporter to verify that > > tests passed now. > > > > > > Thanks, > > > Vladimir K > > > > > > On 12/4/19 11:52 AM, Igor Ignatyev wrote: > > >> http://cr.openjdk.java.net/~iignatyev//8235353/webrev.00 > > >>> 9 lines changed: 0 ins; 0 del; 9 mod; > > >> Hi all, > > >> could you please review this small and trivial cleanup which returns > > serviceablility/sa tests back to execution on linux-ppc64. the tests were > > problem listed due to 8211767[1], which is closed as a dup of resolved > > 8228649[2]. > > >> [1] https://bugs.openjdk.java.net/browse/JDK-8211767 > > >> [2] https://bugs.openjdk.java.net/browse/JDK-8228649 > > >> Thanks, > > >> -- Igor From maurizio.cimadamore at oracle.com Fri Dec 6 12:06:00 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 6 Dec 2019 12:06:00 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <83d4f431-85f7-4434-2088-d79288223686@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <7765b06b-acf2-f11f-980e-33af4ca3934c@oracle.com> <83d4f431-85f7-4434-2088-d79288223686@oracle.com> Message-ID: <96811c65-c48a-3f41-12e8-a8b855cf3894@oracle.com> On 06/12/2019 11:53, Jorn Vernee wrote: > > * drop offset() - but then add an overload of MemorySegment::asSlice > which takes an address instead of a plain long offset > > This sounds good to me, since it fits with what we're doing with > makeUncheckedSegment(MemoryAddress, length), and we added the offset() > accessor to support slicing. I don't like changing the name of > offset(long) to advance(long) since the offset can be negative as well > (and 'advance' implies only positive). Right - I'd say if we go down this path we need at least two overload: * MemorySegment::asSlice(MemoryAddress, long) * MemoryAddress::offset(MemoryAddress) And then we could be offset() free :-) > > >> I don't see the point of having MemoryLayouts separated from > MemoryLayout. > > Possibly - I found myself thinking that too - although, with the > subsequent Panama step (ABI support) we'll be adding a ton of > ABI-dependent layouts in here... (but we could address that in other > ways also). > > We had some ideas to move the ABI constants to separate classes I > remember. I.e. provide separate classes for different platform ABIs > (implementing SystemABI), and stick the constants in there. Together > with the idea to make JAVA_INT into Integer::LAYOUT later, I think the > remaining constants fit into MemoryLayout pretty naturally, and we can > remove MemoryLayouts. Yeah - that's what I was referring to when I said " (but we could address that in other ways also). " :-) Maurizio > > Jorn > > On 06/12/2019 11:43, Maurizio Cimadamore wrote: >> Hi, >> here's an updated version of the patch: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v2/ >> >> And a delta of the changes since last version here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v2_delta/ >> >> The javadoc has been updated inline here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >> >> >> Summary of changes: >> >> * fixed spurious protected methods in AbstractLayout and subclasses >> which leaked into API >> * removed library_call.cpp changes, as these are being tracked >> separately by Vlad >> * compacted ILOAD code in AddressVarHandleGenerator >> * replaced uses of ++i/--i with i + 1/i - 1 in MemoryScope >> >> I have made no changes to the *name* of the methods in the API. In >> fact, I suggest we keep a list of the names we'd like to revisit, and >> we address them all at once at the end of the review (once we're >> happy with the contents). Here's a list of the current open naming >> issues: >> >> * MemoryAddress::offset() vs. MemoryAddress::offset(long) -- not much >> distance between these two semantically different operations >> * MemorySegment::isAccessible() -- as the A* word is overloaded, some >> other name should be found? >> * MemorySegment::acquire() -- replace with MemorySegment::fork? >> >> Cheers >> Maurizio >> >> >> On 05/12/2019 21:04, Maurizio Cimadamore wrote: >>> Hi, >>> as part of the effort to upstream the changes related to JEP 370 >>> (foreign memory access API) [1], I'd like to ask for a code review >>> for the corresponding core-libs and hotspot changes: >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/8234049/ >>> >>> A javadoc for the memory access API is also available here: >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >>> >>> >>> Note: the patch passes tier1, tier2 and tier3 testing (**) >>> >>> >>> Here is a brief summary of the changes in java.base and hotspot (the >>> remaining new files are implementation classes and tests for the new >>> API): >>> >>> * ciField.cpp - this one is to trust final fields in the foreign >>> memory access implementation (otherwise VM doesn't trust memory >>> segment bounds) >>> >>> * Modules.gmk - these changes are needed to require that the >>> incubating module is loaded by the boot loader (otherwise the above >>> changes are useless) >>> >>> * library_call.cpp - this one is a JIT compiler change to treat >>> Thread.currentThread() as a well-known constant - which helps a lot >>> in the confinement checks (thanks Vlad!) >>> >>> * various Buffer-related changes; these changes are needed because >>> the memory access API allows a memory segment to be projected into a >>> byte buffer, for interop reasons. As such, we need to insert a >>> liveness check in the various get/put methods. Previously we had an >>> implementation strategy where a BB was 'decorated' by a subclass >>> called ScopedBuffer - but doing so required some changes to the BB >>> API (e.g. making certain methods non-final, so that we could >>> decorate them). Here I use an approach (which I have discussed with >>> Alan) which doesn't require any public API changes, but needs to add >>> a 'segment' field in Buffer - and then have constructors which keep >>> track of this extra parameter. >>> >>> * FileChannel changes - these changes are required so that we can >>> reuse the Unmapper class from the MemorySegment implementation, to >>> deterministically deallocate a mapped memory segment. This should be >>> a 'straight' refactoring, no change in behavior should occur here. >>> Please double check. >>> >>> * VarHandles - this class now provides a factory to create memory >>> access VarHandle - this is a bit tricky, since VarHandle cannot >>> really be implemented outside java.base (e.g. VarForm is not >>> public). So we do the usual trick where we define a bunch of proxy >>> interfaces (see jdk/internal/access/foreign) have the classes in >>> java.base refer to these - and then have the implementation classes >>> of the memory access API implement these interfaces. >>> >>> * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we >>> need to provide access to otherwise hidden functionalities - e.g. >>> creating a new scoped buffer, or retrieving the properties of a >>> memory access handle (e.g. offset, stride etc.), so that we can >>> implement the memory access API in its own separate module >>> >>> * GensrcVarHandles.gmk - these changes are needed to enable the >>> generation of the new memory address var handle implementations; >>> there's an helper class per carrier (e.g. >>> VarHandleMemoryAddressAsBytes, ...). At runtime, when a memory >>> access var handle is needed, we dynamically spin a new VH >>> implementation which makes use of the right carrier. We need to spin >>> because the VH can have a variable number of access coordinates >>> (e.g. depending on the dimensions of the array to be accessed). But, >>> under the hood, all the generated implementation will be using the >>> same helper class. >>> >>> * tests - we've tried to add fairly robust tests, often checking all >>> possible permutations of carriers/dimensions etc. Because of that, >>> the tests might not be the easiest to look at, but they have proven >>> to be pretty effective at shaking out issues. >>> >>> I think that covers the main aspects of the implementation and where >>> it differs from vanilla JDK. >>> >>> P.S. >>> >>> In the CSR review [2], Joe raised a fair point - which is >>> MemoryAddress has both: >>> >>> offset(long) --> move address of given offset >>> offset() --> return the offset of this address in its owning segment >>> >>> And this was considered suboptimal, given both methods use the same >>> name but do something quite different (one is an accessor, another >>> is a 'wither'). one obvious option is to rename the first to >>> 'withOffset'. But I think that would lead to verbose code (since >>> that is a very common operation). Other options are to: >>> >>> * rename offset(long) to move(long), advance(long), or something else >>> * drop offset() - but then add an overload of MemorySegment::asSlice >>> which takes an address instead of a plain long offset >>> >>> I'll leave the choice to the reviewers :-) >>> >>> >>> >>> Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, >>> Stuart, Roger, Joe and the Panama team for the feedback provided so >>> far, which helped to get the API in the shape it is today. >>> >>> Cheers >>> Maurizio >>> >>> (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" >>> - but that is unrelated to this patch, and it's a known failing test. >>> >>> [1] - https://openjdk.java.net/jeps/370 >>> [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 >>> From david.holmes at oracle.com Fri Dec 6 12:13:29 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 6 Dec 2019 22:13:29 +1000 Subject: RFR(s): 8235410: Enable handshakes on Linux x86 (32-bit) In-Reply-To: References: <3807ba73-5b46-7f9b-c2b2-eeaaad734518@oracle.com> Message-ID: <4991bfcb-480e-c65d-e4b8-6dde3032e9ef@oracle.com> On 6/12/2019 7:22 pm, Robbin Ehn wrote: > Thanks David! > > On 2019-12-05 22:36, David Holmes wrote: >>> diff -r 636d71e53732 src/hotspot/cpu/x86/globals_x86.hpp >>> --- a/src/hotspot/cpu/x86/globals_x86.hpp??? Wed Dec 04 10:26:32 2019 >>> +0100 >>> +++ b/src/hotspot/cpu/x86/globals_x86.hpp??? Thu Dec 05 14:13:57 2019 >>> +0100 >>> @@ -89,12 +89,7 @@ >>> >>> ??define_pd_global(intx, InitArrayShortSize, 8*BytesPerLong); >>> >>> -#if defined(_LP64) || defined(_WINDOWS) >>> ??define_pd_global(bool, ThreadLocalHandshakes, true); >>> -#else >>> -// get_thread() is slow on linux 32 bit, therefore off by default >> >> Why would get_thread be slow on linux 32-bit? Have we discussed this >> when handshakes went in? I'm feeling surprised now to see only 32-bit >> Linux was disabled. > > You actually explained it RFR for 32-bit TLH: > > "Thread::current() already uses compiler-based TLS. get_thread (on > non-Windows x86) make a VM call to Thread::current(). If we want to get > faster here we have to introduce compiler specific hooks directly into > the TLS implementation (not nice to do initially or maintain). " > > :) Sad how much one forgets :( Right get_thread in 32-bit non-Windows has to make the VM call, whereas on 64-bit we have the r15_thread register dedicated. David > /Robbin > > >> >> Thanks, >> David >> >>> -define_pd_global(bool, ThreadLocalHandshakes, false); >>> -#endif >>> >>> ??#define ARCH_FLAGS(develop, \ >>> ???????????????????? product, \ From coleen.phillimore at oracle.com Fri Dec 6 12:22:01 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 6 Dec 2019 07:22:01 -0500 Subject: RFR: 8235456: Minimal VM is broken after JDK-8212160 In-Reply-To: <6bda7d1b-7868-97ec-2676-988d956664b4@loongson.cn> References: <6bda7d1b-7868-97ec-2676-988d956664b4@loongson.cn> Message-ID: <920a5145-a512-0b79-411c-d90135a2a82c@oracle.com> Jie, Thank you for fixing this.?? I tried to build minimal yesterday and some configuration failed and said I had to install X or something. I used to be able to build the minimal vm not that long ago, so I'll pursue why it's broken again for me. I have to admit that I thought the broken code might be in jvmtiThreadState.cpp because I couldn't find how oops_do and nmethods_do() are excluded.?? Maybe the whole file is excluded somehow. Thank you for handling this right away again. Coleen On 12/5/19 9:27 PM, Jie Fu wrote: > Hi all, > > May I get reviews for the one-line fix? > > JBS:??? https://bugs.openjdk.java.net/browse/JDK-8235456 > Webrev: http://cr.openjdk.java.net/~jiefu/8235456/webrev.00/ > > Thanks a lot. > Best regards, > Jie > From vladimir.x.ivanov at oracle.com Fri Dec 6 12:28:45 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 6 Dec 2019 15:28:45 +0300 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <0dc305b6-c727-7591-469e-fedd0f16d0a0@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <5c1689c5-43bf-3b4a-50e9-7376097f3087@oracle.com> <0dc305b6-c727-7591-469e-fedd0f16d0a0@oracle.com> Message-ID: >>> * ciField.cpp - this one is to trust final fields in the foreign >>> memory access implementation (otherwise VM doesn't trust memory >>> segment bounds) >> >> New packages aren't part of java.base. Considering the implementation >> resides in an incubator module, is it enough to consider them as >> trusted and well-known to the JVM? > This gives same performance as with -XX:+TrustFinalNonStaticFields - if > we remove these changes, then memory segment bounds are never inlined. > I'm happy to change this if you have other suggestions on how to get > there, of course (I can run some benchmarks w/ and w/o and post some > numbers if that helps). I'm not questioning whether these changes are needed for the implementation or not. I wanted to attract attention that it's not just a mechanical expansion of the list we have done in the past, but a new case (java.base vs incubator) and hear from others are they fine with leaving it as is? All the options I see (either new JVM annotation or trust final fields by default) require significant engineering effort. So, I'd like to clarify first whether the effort to rewrite current solution is required or not. >>> * library_call.cpp - this one is a JIT compiler change to treat >>> Thread.currentThread() as a well-known constant - which helps a lot >>> in the confinement checks (thanks Vlad!) >> >> FTR it is tracked by JDK-8235143 [1] and the patch is under review [2]. > > Should I remove this from the patch then? Yes, please. Best regards, Vladimir Ivanov >> >> Best regards, >> Vladimir Ivanov >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8235143 >> >> [2] >> https://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-December/036295.html >> >> >>> * various Buffer-related changes; these changes are needed because >>> the memory access API allows a memory segment to be projected into a >>> byte buffer, for interop reasons. As such, we need to insert a >>> liveness check in the various get/put methods. Previously we had an >>> implementation strategy where a BB was 'decorated' by a subclass >>> called ScopedBuffer - but doing so required some changes to the BB >>> API (e.g. making certain methods non-final, so that we could decorate >>> them). Here I use an approach (which I have discussed with Alan) >>> which doesn't require any public API? changes, but needs to add a >>> 'segment' field in Buffer - and then have constructors which keep >>> track of this extra parameter. >>> >>> * FileChannel changes - these changes are required so that we can >>> reuse the Unmapper class from the MemorySegment implementation, to >>> deterministically deallocate a mapped memory segment. This should be >>> a 'straight' refactoring, no change in behavior should occur here. >>> Please double check. >>> >>> * VarHandles - this class now provides a factory to create memory >>> access VarHandle - this is a bit tricky, since VarHandle cannot >>> really be implemented outside java.base (e.g. VarForm is not public). >>> So we do the usual trick where we define a bunch of proxy interfaces >>> (see jdk/internal/access/foreign) have the classes in java.base refer >>> to these - and then have the implementation classes of the memory >>> access API implement these interfaces. >>> >>> * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we need >>> to provide access to otherwise hidden functionalities - e.g. creating >>> a new scoped buffer, or retrieving the properties of a memory access >>> handle (e.g. offset, stride etc.), so that we can implement the >>> memory access API in its own separate module >>> >>> * GensrcVarHandles.gmk - these changes are needed to enable the >>> generation of the new memory address var handle implementations; >>> there's an helper class per carrier (e.g. >>> VarHandleMemoryAddressAsBytes, ...). At runtime, when a memory access >>> var handle is needed, we dynamically spin a new VH implementation >>> which makes use of the right carrier. We need to spin because the VH >>> can have a variable number of access coordinates (e.g. depending on >>> the dimensions of the array to be accessed). But, under the hood, all >>> the generated implementation will be using the same helper class. >>> >>> * tests - we've tried to add fairly robust tests, often checking all >>> possible permutations of carriers/dimensions etc. Because of that, >>> the tests might not be the easiest to look at, but they have proven >>> to be pretty effective at shaking out issues. >>> >>> I think that covers the main aspects of the implementation and where >>> it differs from vanilla JDK. >>> >>> P.S. >>> >>> In the CSR review [2], Joe raised a fair point - which is >>> MemoryAddress has both: >>> >>> offset(long) --> move address of given offset >>> offset() --> return the offset of this address in its owning segment >>> >>> And this was considered suboptimal, given both methods use the same >>> name but do something quite different (one is an accessor, another is >>> a 'wither'). one obvious option is to rename the first to >>> 'withOffset'. But I think that would lead to verbose code (since that >>> is a very common operation). Other options are to: >>> >>> * rename offset(long) to move(long), advance(long), or something else >>> * drop offset() - but then add an overload of MemorySegment::asSlice >>> which takes an address instead of a plain long offset >>> >>> I'll leave the choice to the reviewers :-) >>> >>> >>> >>> Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, >>> Stuart, Roger, Joe and the Panama team for the feedback provided so >>> far, which helped to get the API in the shape it is today. >>> >>> Cheers >>> Maurizio >>> >>> (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" - >>> but that is unrelated to this patch, and it's a known failing test. >>> >>> [1] - https://openjdk.java.net/jeps/370 >>> [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 >>> From fujie at loongson.cn Fri Dec 6 12:49:13 2019 From: fujie at loongson.cn (Jie Fu) Date: Fri, 6 Dec 2019 20:49:13 +0800 Subject: RFR: 8235456: Minimal VM is broken after JDK-8212160 In-Reply-To: <920a5145-a512-0b79-411c-d90135a2a82c@oracle.com> References: <6bda7d1b-7868-97ec-2676-988d956664b4@loongson.cn> <920a5145-a512-0b79-411c-d90135a2a82c@oracle.com> Message-ID: <337f959f-cda5-ebf2-1775-37d5d019374e@loongson.cn> It's my pleasure. Best regards, Jie On 2019/12/6 ??8:22, coleen.phillimore at oracle.com wrote: > > Jie, > Thank you for fixing this.?? I tried to build minimal yesterday and > some configuration failed and said I had to install X or something. I > used to be able to build the minimal vm not that long ago, so I'll > pursue why it's broken again for me. > > I have to admit that I thought the broken code might be in > jvmtiThreadState.cpp because I couldn't find how oops_do and > nmethods_do() are excluded.?? Maybe the whole file is excluded somehow. > > Thank you for handling this right away again. > Coleen > > On 12/5/19 9:27 PM, Jie Fu wrote: >> Hi all, >> >> May I get reviews for the one-line fix? >> >> JBS:??? https://bugs.openjdk.java.net/browse/JDK-8235456 >> Webrev: http://cr.openjdk.java.net/~jiefu/8235456/webrev.00/ >> >> Thanks a lot. >> Best regards, >> Jie >> From maurizio.cimadamore at oracle.com Fri Dec 6 13:36:22 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 6 Dec 2019 13:36:22 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <5c1689c5-43bf-3b4a-50e9-7376097f3087@oracle.com> <0dc305b6-c727-7591-469e-fedd0f16d0a0@oracle.com> Message-ID: On 06/12/2019 12:28, Vladimir Ivanov wrote: > >>>> * ciField.cpp - this one is to trust final fields in the foreign >>>> memory access implementation (otherwise VM doesn't trust memory >>>> segment bounds) >>> >>> New packages aren't part of java.base. Considering the >>> implementation resides in an incubator module, is it enough to >>> consider them as trusted and well-known to the JVM? >> This gives same performance as with -XX:+TrustFinalNonStaticFields - >> if we remove these changes, then memory segment bounds are never >> inlined. I'm happy to change this if you have other suggestions on >> how to get there, of course (I can run some benchmarks w/ and w/o and >> post some numbers if that helps). > > I'm not questioning whether these changes are needed for the > implementation or not. I wanted to attract attention that it's not > just a mechanical expansion of the list we have done in the past, but > a new case (java.base vs incubator) and hear from others are they fine > with leaving it as is? > > All the options I see (either new JVM annotation or trust final fields > by default) require significant engineering effort. So, I'd like to > clarify first whether the effort to rewrite current solution is > required or not. Thanks for the clarification - I agree it's a bit unorthodox, but I'll leave to others (more qualified than me) to respond on that point. > >>>> * library_call.cpp - this one is a JIT compiler change to treat >>>> Thread.currentThread() as a well-known constant - which helps a lot >>>> in the confinement checks (thanks Vlad!) >>> >>> FTR it is tracked by JDK-8235143 [1] and the patch is under review [2]. >> >> Should I remove this from the patch then? > > Yes, please. Done in the recently submitted v2 Thanks Maurizio > > Best regards, > Vladimir Ivanov > >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8235143 >>> >>> [2] >>> https://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2019-December/036295.html >>> >>> >>>> * various Buffer-related changes; these changes are needed because >>>> the memory access API allows a memory segment to be projected into >>>> a byte buffer, for interop reasons. As such, we need to insert a >>>> liveness check in the various get/put methods. Previously we had an >>>> implementation strategy where a BB was 'decorated' by a subclass >>>> called ScopedBuffer - but doing so required some changes to the BB >>>> API (e.g. making certain methods non-final, so that we could >>>> decorate them). Here I use an approach (which I have discussed with >>>> Alan) which doesn't require any public API? changes, but needs to >>>> add a 'segment' field in Buffer - and then have constructors which >>>> keep track of this extra parameter. >>>> >>>> * FileChannel changes - these changes are required so that we can >>>> reuse the Unmapper class from the MemorySegment implementation, to >>>> deterministically deallocate a mapped memory segment. This should >>>> be a 'straight' refactoring, no change in behavior should occur >>>> here. Please double check. >>>> >>>> * VarHandles - this class now provides a factory to create memory >>>> access VarHandle - this is a bit tricky, since VarHandle cannot >>>> really be implemented outside java.base (e.g. VarForm is not >>>> public). So we do the usual trick where we define a bunch of proxy >>>> interfaces (see jdk/internal/access/foreign) have the classes in >>>> java.base refer to these - and then have the implementation classes >>>> of the memory access API implement these interfaces. >>>> >>>> * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we >>>> need to provide access to otherwise hidden functionalities - e.g. >>>> creating a new scoped buffer, or retrieving the properties of a >>>> memory access handle (e.g. offset, stride etc.), so that we can >>>> implement the memory access API in its own separate module >>>> >>>> * GensrcVarHandles.gmk - these changes are needed to enable the >>>> generation of the new memory address var handle implementations; >>>> there's an helper class per carrier (e.g. >>>> VarHandleMemoryAddressAsBytes, ...). At runtime, when a memory >>>> access var handle is needed, we dynamically spin a new VH >>>> implementation which makes use of the right carrier. We need to >>>> spin because the VH can have a variable number of access >>>> coordinates (e.g. depending on the dimensions of the array to be >>>> accessed). But, under the hood, all the generated implementation >>>> will be using the same helper class. >>>> >>>> * tests - we've tried to add fairly robust tests, often checking >>>> all possible permutations of carriers/dimensions etc. Because of >>>> that, the tests might not be the easiest to look at, but they have >>>> proven to be pretty effective at shaking out issues. >>>> >>>> I think that covers the main aspects of the implementation and >>>> where it differs from vanilla JDK. >>>> >>>> P.S. >>>> >>>> In the CSR review [2], Joe raised a fair point - which is >>>> MemoryAddress has both: >>>> >>>> offset(long) --> move address of given offset >>>> offset() --> return the offset of this address in its owning segment >>>> >>>> And this was considered suboptimal, given both methods use the same >>>> name but do something quite different (one is an accessor, another >>>> is a 'wither'). one obvious option is to rename the first to >>>> 'withOffset'. But I think that would lead to verbose code (since >>>> that is a very common operation). Other options are to: >>>> >>>> * rename offset(long) to move(long), advance(long), or something else >>>> * drop offset() - but then add an overload of >>>> MemorySegment::asSlice which takes an address instead of a plain >>>> long offset >>>> >>>> I'll leave the choice to the reviewers :-) >>>> >>>> >>>> >>>> Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, >>>> Stuart, Roger, Joe and the Panama team for the feedback provided so >>>> far, which helped to get the API in the shape it is today. >>>> >>>> Cheers >>>> Maurizio >>>> >>>> (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" >>>> - but that is unrelated to this patch, and it's a known failing test. >>>> >>>> [1] - https://openjdk.java.net/jeps/370 >>>> [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 >>>> From matthias.baesken at sap.com Fri Dec 6 15:17:17 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 6 Dec 2019 15:17:17 +0000 Subject: RFR [XS]: 8235489: handle return values of sscanf calls in hotspot Message-ID: Hello, I noticed there are some remaining sscanf calls in the hotspot codebase missing return value checks. Those should better be checked . Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8235489 http://cr.openjdk.java.net/~mbaesken/webrevs/8235489.0/ Thanks, Matthias From igor.ignatyev at oracle.com Fri Dec 6 17:18:16 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 6 Dec 2019 09:18:16 -0800 Subject: RFR(T) : 8235353 : clean up hotspot problem lists In-Reply-To: References: <9405A3B3-8045-4863-9BD4-FD293E2C62F9@oracle.com> Message-ID: Martin, Christoph, thanks for verifying this. pushed. -- Igor > On Dec 6, 2019, at 2:51 AM, Doerr, Martin wrote: > > Hi Igor and Vladimir, > > the tests have passed on PPC64. Change is good. Thanks for checking with us. > > Best regards, > Martin > > >> -----Original Message----- >> From: Langer, Christoph >> Sent: Donnerstag, 5. Dezember 2019 12:24 >> To: Igor Ignatyev ; Doerr, Martin >> ; Lindenmaier, Goetz >> >> Cc: serviceability-dev ; Vladimir >> Kozlov ; hotspot-dev Source Developers >> >> Subject: RE: RFR(T) : 8235353 : clean up hotspot problem lists >> >> Hi Igor, >> >> I have added your update to our test system. I'll let you know the results by >> tomorrow. >> >> Best regards >> Christoph >> >>> -----Original Message----- >>> From: serviceability-dev >> On >>> Behalf Of Igor Ignatyev >>> Sent: Donnerstag, 5. Dezember 2019 03:08 >>> To: Doerr, Martin ; Lindenmaier, Goetz >>> >>> Cc: serviceability-dev ; Vladimir >>> Kozlov ; hotspot-dev Source Developers >>> >>> Subject: Re: RFR(T) : 8235353 : clean up hotspot problem lists >>> >>> Martin, Goetz. >>> >>> could you please check that these 9 tests still pass on PPC? >>> >>> -- Igor >>> >>>> On Dec 4, 2019, at 12:01 PM, Vladimir Kozlov >> >>> wrote: >>>> >>>> I am fine with changes but we need to ask PPC64 supporter to verify that >>> tests passed now. >>>> >>>> Thanks, >>>> Vladimir K >>>> >>>> On 12/4/19 11:52 AM, Igor Ignatyev wrote: >>>>> http://cr.openjdk.java.net/~iignatyev//8235353/webrev.00 >>>>>> 9 lines changed: 0 ins; 0 del; 9 mod; >>>>> Hi all, >>>>> could you please review this small and trivial cleanup which returns >>> serviceablility/sa tests back to execution on linux-ppc64. the tests were >>> problem listed due to 8211767[1], which is closed as a dup of resolved >>> 8228649[2]. >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8211767 >>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8228649 >>>>> Thanks, >>>>> -- Igor > From joe.darcy at oracle.com Fri Dec 6 18:48:46 2019 From: joe.darcy at oracle.com (Joe Darcy) Date: Fri, 6 Dec 2019 10:48:46 -0800 Subject: JDK 14 RFR (XXS) of JDK-8235499: Change HotSpot jtreg records test to better handle JDK updates Message-ID: Hello, Several of the HotSpot tests for records need small changes to more gracefully handle JDK updates, such as JDK 14 -> 15 coming up soon! ??? http://cr.openjdk.java.net/~darcy/8235499.0/ Patch below. (Some updates will be need to make all the tests in this directory pass on 15, but these small updates are valid independently.) Thanks, -Joe --- old/test/hotspot/jtreg/runtime/records/ignoreRecordAttribute.java 2019-12-06 10:44:14.592741024 -0800 +++ new/test/hotspot/jtreg/runtime/records/ignoreRecordAttribute.java 2019-12-06 10:44:14.264741024 -0800 @@ -35,7 +35,7 @@ ?public class ignoreRecordAttribute { ???? public static void main(String[] args) throws Exception { - +??????? String MAJOR_VERSION = Integer.toString(44 + Runtime.version().feature()); ???????? ProcessBuilder pb = ProcessTools.createJavaProcessBuilder("--enable-preview", ???????????? "-Xlog:class+record", "-Xshare:off", "superNotJLRecord"); ???????? OutputAnalyzer output = new OutputAnalyzer(pb.start()); @@ -46,7 +46,7 @@ ???????????? "-Xlog:class+record", "-Xshare:off", "recordIgnoredVersion"); ???????? output = new OutputAnalyzer(pb.start()); ???????? output.shouldContain("Ignoring Record attribute"); -??????? output.shouldContain("because class file version is not 58.65535"); +??????? output.shouldContain("because class file version is not " + MAJOR_VERSION + ".65535"); ???? } ?} --- old/test/hotspot/jtreg/runtime/records/recordReflectionTest.java 2019-12-06 10:44:15.276741024 -0800 +++ new/test/hotspot/jtreg/runtime/records/recordReflectionTest.java 2019-12-06 10:44:15.036741024 -0800 @@ -23,7 +23,7 @@ ?/* ? * @test - * @compile --enable-preview --source 14 recordReflectionTest.java + * @compile --enable-preview --source ${jdk.version} recordReflectionTest.java ? * @run main/othervm --enable-preview recordReflectionTest ? */ From harold.seigel at oracle.com Fri Dec 6 19:10:08 2019 From: harold.seigel at oracle.com (Harold Seigel) Date: Fri, 6 Dec 2019 14:10:08 -0500 Subject: JDK 14 RFR (XXS) of JDK-8235499: Change HotSpot jtreg records test to better handle JDK updates In-Reply-To: References: Message-ID: <5a8acc52-4e68-de0a-04ce-722e2802170f@oracle.com> Hi Joe, Many of the Records tests will fail when the class file version is switched from 58 to 59.? This is because the JVM requires that Record classes have version 58.65535. Perhaps, once the 15 repo is open I can change the JVM to temporarily allow Records to have either version 58.65535 or 59.65535.? Then, once the class file major version is changed to 59, I can update the tests and disallow version 58.65535 for records. Harold On 12/6/2019 1:48 PM, Joe Darcy wrote: > Hello, > > Several of the HotSpot tests for records need small changes to more > gracefully handle JDK updates, such as JDK 14 -> 15 coming up soon! > > ??? http://cr.openjdk.java.net/~darcy/8235499.0/ > > Patch below. > > (Some updates will be need to make all the tests in this directory > pass on 15, but these small updates are valid independently.) > > Thanks, > > -Joe > > --- old/test/hotspot/jtreg/runtime/records/ignoreRecordAttribute.java > 2019-12-06 10:44:14.592741024 -0800 > +++ new/test/hotspot/jtreg/runtime/records/ignoreRecordAttribute.java > 2019-12-06 10:44:14.264741024 -0800 > @@ -35,7 +35,7 @@ > ?public class ignoreRecordAttribute { > > ???? public static void main(String[] args) throws Exception { > - > +??????? String MAJOR_VERSION = Integer.toString(44 + > Runtime.version().feature()); > ???????? ProcessBuilder pb = > ProcessTools.createJavaProcessBuilder("--enable-preview", > ???????????? "-Xlog:class+record", "-Xshare:off", "superNotJLRecord"); > ???????? OutputAnalyzer output = new OutputAnalyzer(pb.start()); > @@ -46,7 +46,7 @@ > ???????????? "-Xlog:class+record", "-Xshare:off", > "recordIgnoredVersion"); > ???????? output = new OutputAnalyzer(pb.start()); > ???????? output.shouldContain("Ignoring Record attribute"); > -??????? output.shouldContain("because class file version is not > 58.65535"); > +??????? output.shouldContain("because class file version is not " + > MAJOR_VERSION + ".65535"); > ???? } > > ?} > --- old/test/hotspot/jtreg/runtime/records/recordReflectionTest.java > 2019-12-06 10:44:15.276741024 -0800 > +++ new/test/hotspot/jtreg/runtime/records/recordReflectionTest.java > 2019-12-06 10:44:15.036741024 -0800 > @@ -23,7 +23,7 @@ > > ?/* > ? * @test > - * @compile --enable-preview --source 14 recordReflectionTest.java > + * @compile --enable-preview --source ${jdk.version} > recordReflectionTest.java > ? * @run main/othervm --enable-preview recordReflectionTest > ? */ > > From igor.ignatyev at oracle.com Fri Dec 6 19:12:59 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 6 Dec 2019 11:12:59 -0800 Subject: JDK 14 RFR (XXS) of JDK-8235499: Change HotSpot jtreg records test to better handle JDK updates In-Reply-To: References: Message-ID: <8CB7197C-A52C-4369-A84E-C15CA6619107@oracle.com> Hi Joe, the fix looks good to me. thanks, -- Igor > On Dec 6, 2019, at 10:48 AM, Joe Darcy wrote: > > Hello, > > Several of the HotSpot tests for records need small changes to more gracefully handle JDK updates, such as JDK 14 -> 15 coming up soon! > > http://cr.openjdk.java.net/~darcy/8235499.0/ > > Patch below. > > (Some updates will be need to make all the tests in this directory pass on 15, but these small updates are valid independently.) > > Thanks, > > -Joe > > --- old/test/hotspot/jtreg/runtime/records/ignoreRecordAttribute.java 2019-12-06 10:44:14.592741024 -0800 > +++ new/test/hotspot/jtreg/runtime/records/ignoreRecordAttribute.java 2019-12-06 10:44:14.264741024 -0800 > @@ -35,7 +35,7 @@ > public class ignoreRecordAttribute { > > public static void main(String[] args) throws Exception { > - > + String MAJOR_VERSION = Integer.toString(44 + Runtime.version().feature()); > ProcessBuilder pb = ProcessTools.createJavaProcessBuilder("--enable-preview", > "-Xlog:class+record", "-Xshare:off", "superNotJLRecord"); > OutputAnalyzer output = new OutputAnalyzer(pb.start()); > @@ -46,7 +46,7 @@ > "-Xlog:class+record", "-Xshare:off", "recordIgnoredVersion"); > output = new OutputAnalyzer(pb.start()); > output.shouldContain("Ignoring Record attribute"); > - output.shouldContain("because class file version is not 58.65535"); > + output.shouldContain("because class file version is not " + MAJOR_VERSION + ".65535"); > } > > } > --- old/test/hotspot/jtreg/runtime/records/recordReflectionTest.java 2019-12-06 10:44:15.276741024 -0800 > +++ new/test/hotspot/jtreg/runtime/records/recordReflectionTest.java 2019-12-06 10:44:15.036741024 -0800 > @@ -23,7 +23,7 @@ > > /* > * @test > - * @compile --enable-preview --source 14 recordReflectionTest.java > + * @compile --enable-preview --source ${jdk.version} recordReflectionTest.java > * @run main/othervm --enable-preview recordReflectionTest > */ > > From harold.seigel at oracle.com Fri Dec 6 19:49:33 2019 From: harold.seigel at oracle.com (Harold Seigel) Date: Fri, 6 Dec 2019 14:49:33 -0500 Subject: JDK 14 RFR (XXS) of JDK-8235499: Change HotSpot jtreg records test to better handle JDK updates In-Reply-To: <5a8acc52-4e68-de0a-04ce-722e2802170f@oracle.com> References: <5a8acc52-4e68-de0a-04ce-722e2802170f@oracle.com> Message-ID: <283de367-e4dd-d373-e9c1-74798caf8fed@oracle.com> Hi Joe, I plan to change the JVM to check for JVM_CLASSFILE_MAJOR_VERSION.65535 instead of a hardwired 58.65535.? See https://bugs.openjdk.java.net/browse/JDK-8235513 So, your change looks good and ignore my previous email. Thanks, Harold On 12/6/2019 2:10 PM, Harold Seigel wrote: > Hi Joe, > > Many of the Records tests will fail when the class file version is > switched from 58 to 59.? This is because the JVM requires that Record > classes have version 58.65535. > > Perhaps, once the 15 repo is open I can change the JVM to temporarily > allow Records to have either version 58.65535 or 59.65535.? Then, once > the class file major version is changed to 59, I can update the tests > and disallow version 58.65535 for records. > > Harold > > On 12/6/2019 1:48 PM, Joe Darcy wrote: >> Hello, >> >> Several of the HotSpot tests for records need small changes to more >> gracefully handle JDK updates, such as JDK 14 -> 15 coming up soon! >> >> ??? http://cr.openjdk.java.net/~darcy/8235499.0/ >> >> Patch below. >> >> (Some updates will be need to make all the tests in this directory >> pass on 15, but these small updates are valid independently.) >> >> Thanks, >> >> -Joe >> >> --- old/test/hotspot/jtreg/runtime/records/ignoreRecordAttribute.java >> 2019-12-06 10:44:14.592741024 -0800 >> +++ new/test/hotspot/jtreg/runtime/records/ignoreRecordAttribute.java >> 2019-12-06 10:44:14.264741024 -0800 >> @@ -35,7 +35,7 @@ >> ?public class ignoreRecordAttribute { >> >> ???? public static void main(String[] args) throws Exception { >> - >> +??????? String MAJOR_VERSION = Integer.toString(44 + >> Runtime.version().feature()); >> ???????? ProcessBuilder pb = >> ProcessTools.createJavaProcessBuilder("--enable-preview", >> ???????????? "-Xlog:class+record", "-Xshare:off", "superNotJLRecord"); >> ???????? OutputAnalyzer output = new OutputAnalyzer(pb.start()); >> @@ -46,7 +46,7 @@ >> ???????????? "-Xlog:class+record", "-Xshare:off", >> "recordIgnoredVersion"); >> ???????? output = new OutputAnalyzer(pb.start()); >> ???????? output.shouldContain("Ignoring Record attribute"); >> -??????? output.shouldContain("because class file version is not >> 58.65535"); >> +??????? output.shouldContain("because class file version is not " + >> MAJOR_VERSION + ".65535"); >> ???? } >> >> ?} >> --- old/test/hotspot/jtreg/runtime/records/recordReflectionTest.java >> 2019-12-06 10:44:15.276741024 -0800 >> +++ new/test/hotspot/jtreg/runtime/records/recordReflectionTest.java >> 2019-12-06 10:44:15.036741024 -0800 >> @@ -23,7 +23,7 @@ >> >> ?/* >> ? * @test >> - * @compile --enable-preview --source 14 recordReflectionTest.java >> + * @compile --enable-preview --source ${jdk.version} >> recordReflectionTest.java >> ? * @run main/othervm --enable-preview recordReflectionTest >> ? */ >> >> From kim.barrett at oracle.com Fri Dec 6 20:11:30 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 6 Dec 2019 15:11:30 -0500 Subject: RFR [XS]: 8235489: handle return values of sscanf calls in hotspot In-Reply-To: References: Message-ID: <17DCFAEB-8E3C-4B94-9453-0AD85D3ADC94@oracle.com> > On Dec 6, 2019, at 10:17 AM, Baesken, Matthias wrote: > > Hello, I noticed there are some remaining sscanf calls in the hotspot codebase missing return value checks. > Those should better be checked . > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8235489 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235489.0/ > > > Thanks, Matthias ------------------------------------------------------------------------------ src/hotspot/os/linux/os_linux.cpp 2085 memset(name, 0, sizeof(name)); What is this line of the change for? Is this because the pathname field in a line can be blank? That isn't sufficient. That case needs to be handled after the "if (matches < 6) continue;" line, e.g. check whether matches == 6 (e.g. a blank name field), forcing "name" to be blank if so (e.g. name[0] = '\0'). Otherwise, "name" will contain data from the most recent parsed line that contained a match for that field. ------------------------------------------------------------------------------ src/hotspot/os/linux/os_linux.cpp 2083 char device[6]; ... 2088 int matches = sscanf(line, UINT64_FORMAT_X "-" UINT64_FORMAT_X " %4s " UINT64_FORMAT_X " %5s " INT64_FORMAT " %s", Good eye, spotting the mismatch between the declared size of device and the size spec in the format string (e.g. changing "%7s" => "%5s"). ------------------------------------------------------------------------------ src/hotspot/os/linux/os_linux.cpp In the sscanf format string, there's no length spec for the pathname, e.g. it uses an unadorned "%s". PATH_MAX doesn't deal with long file names. ------------------------------------------------------------------------------ From paul.sandoz at oracle.com Fri Dec 6 21:04:57 2019 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 6 Dec 2019 13:04:57 -0800 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> Message-ID: I mostly looked at the API and implementation and not the tests. s/offset/add or plus ? add ?l? to the offset of this memory address the result of which is the offset of the returned memory address. If we ever have controlled operator overloading that?s how I would like to express it :-) Paul. VarHandles.java ? 40 static ClassValue> addressFactories = new ClassValue>() { 41 @Override 42 protected Map computeValue(Class type) { 43 return new ConcurrentHashMap<>(); 44 } 45 }; s/addressFactories/ADDRESS_FACTORIES/ Perhaps expose as ConcurrentMap to express concurrency requirements. May be useful to pre-size the map. JavaNioAccess.java ? 52 /** 53 * Constructs an heap ByteBuffer with given backing array, offset, capacity and segment. 54 */ 55 ByteBuffer newHeapByteBuffer(byte[] hb, int offset, int capacity, MemorySegmentProxy segment); 56 Formatting issue. FileChannelImpl.java ? 867 private static abstract class Unmapper 868 implements Runnable, UnmapperProxy 869 { 870 // may be required to close file 871 private static final NativeDispatcher nd = new FileDispatcherImpl(); 872 873 // keep track of mapped buffer usage 874 static volatile int count; 875 static volatile long totalSize; 876 static volatile long totalCapacity; These new field declarations are also declared on the concrete subtypes. Did you intend to remove the declarations from the latter? VarHandles ? I see you are relying on the fallback to lambda forms when linking the var handle, otherwise the explicit guard methods just explode :-) Clever generation of VH code on demand punching in live constants, while pre-cooking accessors for ?base" access. AddressVarHandleGenerator.java -- 355 private int returnInsn(Class type) { 356 switch (LambdaForm.BasicType.basicType(type)) { 357 case I_TYPE: return Opcodes.IRETURN; 358 case J_TYPE: return Opcodes.LRETURN; 359 case F_TYPE: return Opcodes.FRETURN; 360 case D_TYPE: return Opcodes.DRETURN; 361 case L_TYPE: return Opcodes.ARETURN; 362 case V_TYPE: return RETURN; 363 default: 364 throw new InternalError("unknown return type: " + type); 365 } 366 } Replace with expression form? (Same for following method too) I believe "JEP 361: Switch Expressions (Standard)? is integrated. GroupLayout.java ? 80 OptionalLong sizeof(List elems) { 81 long size = 0; 82 for (MemoryLayout elem : elems) { 83 if (AbstractLayout.optSize(elem).isPresent()) { 84 size = sizeOp.applyAsLong(size, elem.bitSize()); 85 } else { 86 return OptionalLong.empty(); 87 } 88 } 89 return OptionalLong.of(size); 90 } FWIW you can do this: OptionalLong sizeof(List elems) { return elems.stream().filter(e -> AbstractLayout.optSize(e).isPresent()).mapToLong(MemoryLayout::bitSize) .reduce(sizeOp); } It may be question why there is no way to query if a MemoryLayout has a size or not. Does it require a hasSize method? MemoryAddress.java ? Should MemoryAddress implement Comparable? MemoryHandles.java ? * As an example, consider the memory layout expressed by a {@link SequenceLayout} instance constructed as follows: * 
{@code
SequenceLayout seq = MemoryLayout.ofSequence(5,
    MemoryLayout.ofStruct(
        MemoryLayout.ofPaddingBits(32),
        MemoryLayout.ofValueBits(32).withName("value")
    ));
 * }
MemoryLayout.ofValueBits requires a byte order argument e.g.: MemoryLayout.ofValueBits(32, ByteOrder.BIG_ENDIAN) * 
{@code
VarHandle handle = MemoryHandles.varHandle(int.class); //(MemoryAddress) -> int
handle = MemoryHandles.withOffset(handle, 4); //(MemoryAddress) -> int
handle = MemoryHandles.withStride(handle, 8); //(MemoryAddress, long) -> int
 * }
MemoryHandles.varHandle requires a byte order argument e.g.: VarHandle handle = MemoryHandles.varHandle(int.class, ByteOrder.BIG_ENDIAN); //(MemoryAddress) -> int (See also SequenceLayout) 105 public static VarHandle varHandle(Class carrier, ByteOrder byteOrder) { You may need to specify what access modes are supported, as is the case for MethodHandles.byteBufferViewVarHandle, and also specify how comparison is performed for float/double with atomic update access modes i.e. copy and paste appropriate text. (Same applies to MemoryLayout.varHandle) SequenceLayout.java ? 118 @Override 119 public int hashCode() { 120 return super.hashCode() ^ elemCount.hashCode() ^ elementLayout.hashCode(); 121 } I commonly resort to using Objects.hashCode. Don?t have a strong preference here. I guess you might be concerned about efficient? package-info.java ? 30 * 
{@code
  31 static final VarHandle intHandle = MemoryHandles.varHandle(int.class);
  32 
  33 try (MemorySegment segment = MemorySegment.allocateNative(10 * 4)) {
  34    MemoryAddress base = segment.baseAddress();
  35    for (long i = 0 ; i < 10 ; i++) {
  36      intHandle.set(base.offset(i * 4), (int)i);
  37    }
  38  }
  39  * }
MemoryHandles.varHandle requires a byte order argument. (I wish we could compile code snippets in JavaDoc to surface errors.) MemoryAddressImpl ? 48 public MemoryAddressImpl(MemorySegmentImpl segment) { 49 this(segment, 0); 50 } IDE reporting this constructor is never used. MemorySegmentImpl.java ? 61 final static long NONCE = new Random().nextLong(); If a better quality NONCE is required do ?new SplittableRandom()..nextLong();? 186 private final boolean isSet(int mask) { 187 return (this.mask & mask) != 0; 188 } Unnecessary final modifier. Utils.java ? 143 MemoryScope scope = new MemoryScope(null, () -> unmapperProxy.unmap()); Replace with method ref. > On Dec 5, 2019, at 1:04 PM, Maurizio Cimadamore wrote: > > Hi, > as part of the effort to upstream the changes related to JEP 370 (foreign memory access API) [1], I'd like to ask for a code review for the corresponding core-libs and hotspot changes: > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049/ > > A javadoc for the memory access API is also available here: > > http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html > > Note: the patch passes tier1, tier2 and tier3 testing (**) > > > Here is a brief summary of the changes in java.base and hotspot (the remaining new files are implementation classes and tests for the new API): > > * ciField.cpp - this one is to trust final fields in the foreign memory access implementation (otherwise VM doesn't trust memory segment bounds) > > * Modules.gmk - these changes are needed to require that the incubating module is loaded by the boot loader (otherwise the above changes are useless) > > * library_call.cpp - this one is a JIT compiler change to treat Thread.currentThread() as a well-known constant - which helps a lot in the confinement checks (thanks Vlad!) > > * various Buffer-related changes; these changes are needed because the memory access API allows a memory segment to be projected into a byte buffer, for interop reasons. As such, we need to insert a liveness check in the various get/put methods. Previously we had an implementation strategy where a BB was 'decorated' by a subclass called ScopedBuffer - but doing so required some changes to the BB API (e.g. making certain methods non-final, so that we could decorate them). Here I use an approach (which I have discussed with Alan) which doesn't require any public API changes, but needs to add a 'segment' field in Buffer - and then have constructors which keep track of this extra parameter. > > * FileChannel changes - these changes are required so that we can reuse the Unmapper class from the MemorySegment implementation, to deterministically deallocate a mapped memory segment. This should be a 'straight' refactoring, no change in behavior should occur here. Please double check. > > * VarHandles - this class now provides a factory to create memory access VarHandle - this is a bit tricky, since VarHandle cannot really be implemented outside java.base (e.g. VarForm is not public). So we do the usual trick where we define a bunch of proxy interfaces (see jdk/internal/access/foreign) have the classes in java.base refer to these - and then have the implementation classes of the memory access API implement these interfaces. > > * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we need to provide access to otherwise hidden functionalities - e.g. creating a new scoped buffer, or retrieving the properties of a memory access handle (e.g. offset, stride etc.), so that we can implement the memory access API in its own separate module > > * GensrcVarHandles.gmk - these changes are needed to enable the generation of the new memory address var handle implementations; there's an helper class per carrier (e.g. VarHandleMemoryAddressAsBytes, ...). At runtime, when a memory access var handle is needed, we dynamically spin a new VH implementation which makes use of the right carrier. We need to spin because the VH can have a variable number of access coordinates (e.g. depending on the dimensions of the array to be accessed). But, under the hood, all the generated implementation will be using the same helper class. > > * tests - we've tried to add fairly robust tests, often checking all possible permutations of carriers/dimensions etc. Because of that, the tests might not be the easiest to look at, but they have proven to be pretty effective at shaking out issues. > > I think that covers the main aspects of the implementation and where it differs from vanilla JDK. > > P.S. > > In the CSR review [2], Joe raised a fair point - which is MemoryAddress has both: > > offset(long) --> move address of given offset > offset() --> return the offset of this address in its owning segment > > And this was considered suboptimal, given both methods use the same name but do something quite different (one is an accessor, another is a 'wither'). one obvious option is to rename the first to 'withOffset'. But I think that would lead to verbose code (since that is a very common operation). Other options are to: > > * rename offset(long) to move(long), advance(long), or something else > * drop offset() - but then add an overload of MemorySegment::asSlice which takes an address instead of a plain long offset > > I'll leave the choice to the reviewers :-) > > > > Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, Stuart, Roger, Joe and the Panama team for the feedback provided so far, which helped to get the API in the shape it is today. > > Cheers > Maurizio > > (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" - but that is unrelated to this patch, and it's a known failing test. > > [1] - https://openjdk.java.net/jeps/370 > [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 > From maurizio.cimadamore at oracle.com Fri Dec 6 23:50:21 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 6 Dec 2019 23:50:21 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> Message-ID: <858356d6-229a-716e-7717-af6aaead2368@oracle.com> Thanks Paul, I'll do a pass and fix the issues you found. Some comments inline: On 06/12/2019 21:04, Paul Sandoz wrote: > I mostly looked at the API and implementation and not the tests. > > s/offset/add or plus ? add ?l? to the offset of this memory address the result of which is the offset of the returned memory address. > > If we ever have controlled operator overloading that?s how I would like to express it :-) > > Paul. > > > VarHandles.java > ? > > 40 static ClassValue> addressFactories = new ClassValue>() { > 41 @Override > 42 protected Map computeValue(Class type) { > 43 return new ConcurrentHashMap<>(); > 44 } > 45 }; > > s/addressFactories/ADDRESS_FACTORIES/ > > Perhaps expose as ConcurrentMap to express concurrency requirements. > > May be useful to pre-size the map. > > > JavaNioAccess.java > ? > > 52 /** > 53 * Constructs an heap ByteBuffer with given backing array, offset, capacity and segment. > 54 */ > 55 ByteBuffer newHeapByteBuffer(byte[] hb, int offset, int capacity, MemorySegmentProxy segment); > 56 > > Formatting issue. > > > FileChannelImpl.java > ? > > 867 private static abstract class Unmapper > 868 implements Runnable, UnmapperProxy > 869 { > 870 // may be required to close file > 871 private static final NativeDispatcher nd = new FileDispatcherImpl(); > 872 > 873 // keep track of mapped buffer usage > 874 static volatile int count; > 875 static volatile long totalSize; > 876 static volatile long totalCapacity; > > These new field declarations are also declared on the concrete subtypes. Did you intend to remove the declarations from the latter? Uhm - I might have messed up something... but I think the fields should be removed from the superclass - it's only the concrete subclasses using them, and they are being used inside synchronized blocks for their corresponding classes - so I think they are meant to be disjoint. I think I added them by accident in Unmapper. > > > > VarHandles > ? > > I see you are relying on the fallback to lambda forms when linking the var handle, otherwise the explicit guard methods just explode :-) > > Clever generation of VH code on demand punching in live constants, while pre-cooking accessors for ?base" access. > > > AddressVarHandleGenerator.java > -- > > 355 private int returnInsn(Class type) { > 356 switch (LambdaForm.BasicType.basicType(type)) { > 357 case I_TYPE: return Opcodes.IRETURN; > 358 case J_TYPE: return Opcodes.LRETURN; > 359 case F_TYPE: return Opcodes.FRETURN; > 360 case D_TYPE: return Opcodes.DRETURN; > 361 case L_TYPE: return Opcodes.ARETURN; > 362 case V_TYPE: return RETURN; > 363 default: > 364 throw new InternalError("unknown return type: " + type); > 365 } > 366 } > > Replace with expression form? (Same for following method too) I believe "JEP 361: Switch Expressions (Standard)? is integrated. > > > GroupLayout.java > ? > > 80 OptionalLong sizeof(List elems) { > 81 long size = 0; > 82 for (MemoryLayout elem : elems) { > 83 if (AbstractLayout.optSize(elem).isPresent()) { > 84 size = sizeOp.applyAsLong(size, elem.bitSize()); > 85 } else { > 86 return OptionalLong.empty(); > 87 } > 88 } > 89 return OptionalLong.of(size); > 90 } > > FWIW you can do this: > > OptionalLong sizeof(List elems) { > return elems.stream().filter(e -> AbstractLayout.optSize(e).isPresent()).mapToLong(MemoryLayout::bitSize) > .reduce(sizeOp); > } > > It may be question why there is no way to query if a MemoryLayout has a size or not. Does it require a hasSize method? I'm on the fence on that one. Yes for completeness it should be there... but doesn't seem all that useful either. I can see that, pulling on that string, you end up with an optional size (since sometimes the size isn't there) but that means all clients pay the price for the few layouts that use unspecified sequences. I was hoping to keep them under the rug to be honest. > > > MemoryAddress.java > ? > > Should MemoryAddress implement Comparable? I don't think so - as they don't form an order - two addresses are comparable only if their segments are the same. > > > > MemoryHandles.java > ? > > * As an example, consider the memory layout expressed by a {@link SequenceLayout} instance constructed as follows: > * 
{@code
> SequenceLayout seq = MemoryLayout.ofSequence(5,
>      MemoryLayout.ofStruct(
>          MemoryLayout.ofPaddingBits(32),
>          MemoryLayout.ofValueBits(32).withName("value")
>      ));
>   * }
> > > MemoryLayout.ofValueBits requires a byte order argument e.g.: > > MemoryLayout.ofValueBits(32, ByteOrder.BIG_ENDIAN) > > > * 
{@code
> VarHandle handle = MemoryHandles.varHandle(int.class); //(MemoryAddress) -> int
> handle = MemoryHandles.withOffset(handle, 4); //(MemoryAddress) -> int
> handle = MemoryHandles.withStride(handle, 8); //(MemoryAddress, long) -> int
>   * }
> > > MemoryHandles.varHandle requires a byte order argument e.g.: > > VarHandle handle = MemoryHandles.varHandle(int.class, ByteOrder.BIG_ENDIAN); //(MemoryAddress) -> int > > (See also SequenceLayout) Thanks for catching these > > > 105 public static VarHandle varHandle(Class carrier, ByteOrder byteOrder) { > > You may need to specify what access modes are supported, as is the case for MethodHandles.byteBufferViewVarHandle, and also specify how comparison is performed for float/double with atomic update access modes i.e. copy and paste appropriate text. (Same applies to MemoryLayout.varHandle) Gotcha > > > SequenceLayout.java > ? > > 118 @Override > 119 public int hashCode() { > 120 return super.hashCode() ^ elemCount.hashCode() ^ elementLayout.hashCode(); > 121 } > > I commonly resort to using Objects.hashCode. Don?t have a strong preference here. I guess you might be concerned about efficient? Not really, I agree this should just use Objects::hashCode > > > package-info.java > ? > > 30 * 
{@code
>    31 static final VarHandle intHandle = MemoryHandles.varHandle(int.class);
>    32
>    33 try (MemorySegment segment = MemorySegment.allocateNative(10 * 4)) {
>    34    MemoryAddress base = segment.baseAddress();
>    35    for (long i = 0 ; i < 10 ; i++) {
>    36      intHandle.set(base.offset(i * 4), (int)i);
>    37    }
>    38  }
>    39  * }
> > MemoryHandles.varHandle requires a byte order argument. > > (I wish we could compile code snippets in JavaDoc to surface errors.) > > > MemoryAddressImpl > ? > > 48 public MemoryAddressImpl(MemorySegmentImpl segment) { > 49 this(segment, 0); > 50 } > > > IDE reporting this constructor is never used. > > > MemorySegmentImpl.java > ? > > 61 final static long NONCE = new Random().nextLong(); > > If a better quality NONCE is required do ?new SplittableRandom()..nextLong();? > > > 186 private final boolean isSet(int mask) { > 187 return (this.mask & mask) != 0; > 188 } > > > Unnecessary final modifier. > > > Utils.java > ? > > 143 MemoryScope scope = new MemoryScope(null, () -> unmapperProxy.unmap()); > > > Replace with method ref. > Thanks Maurizio > > >> On Dec 5, 2019, at 1:04 PM, Maurizio Cimadamore wrote: >> >> Hi, >> as part of the effort to upstream the changes related to JEP 370 (foreign memory access API) [1], I'd like to ask for a code review for the corresponding core-libs and hotspot changes: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8234049/ >> >> A javadoc for the memory access API is also available here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >> >> Note: the patch passes tier1, tier2 and tier3 testing (**) >> >> >> Here is a brief summary of the changes in java.base and hotspot (the remaining new files are implementation classes and tests for the new API): >> >> * ciField.cpp - this one is to trust final fields in the foreign memory access implementation (otherwise VM doesn't trust memory segment bounds) >> >> * Modules.gmk - these changes are needed to require that the incubating module is loaded by the boot loader (otherwise the above changes are useless) >> >> * library_call.cpp - this one is a JIT compiler change to treat Thread.currentThread() as a well-known constant - which helps a lot in the confinement checks (thanks Vlad!) >> >> * various Buffer-related changes; these changes are needed because the memory access API allows a memory segment to be projected into a byte buffer, for interop reasons. As such, we need to insert a liveness check in the various get/put methods. Previously we had an implementation strategy where a BB was 'decorated' by a subclass called ScopedBuffer - but doing so required some changes to the BB API (e.g. making certain methods non-final, so that we could decorate them). Here I use an approach (which I have discussed with Alan) which doesn't require any public API changes, but needs to add a 'segment' field in Buffer - and then have constructors which keep track of this extra parameter. >> >> * FileChannel changes - these changes are required so that we can reuse the Unmapper class from the MemorySegment implementation, to deterministically deallocate a mapped memory segment. This should be a 'straight' refactoring, no change in behavior should occur here. Please double check. >> >> * VarHandles - this class now provides a factory to create memory access VarHandle - this is a bit tricky, since VarHandle cannot really be implemented outside java.base (e.g. VarForm is not public). So we do the usual trick where we define a bunch of proxy interfaces (see jdk/internal/access/foreign) have the classes in java.base refer to these - and then have the implementation classes of the memory access API implement these interfaces. >> >> * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we need to provide access to otherwise hidden functionalities - e.g. creating a new scoped buffer, or retrieving the properties of a memory access handle (e.g. offset, stride etc.), so that we can implement the memory access API in its own separate module >> >> * GensrcVarHandles.gmk - these changes are needed to enable the generation of the new memory address var handle implementations; there's an helper class per carrier (e.g. VarHandleMemoryAddressAsBytes, ...). At runtime, when a memory access var handle is needed, we dynamically spin a new VH implementation which makes use of the right carrier. We need to spin because the VH can have a variable number of access coordinates (e.g. depending on the dimensions of the array to be accessed). But, under the hood, all the generated implementation will be using the same helper class. >> >> * tests - we've tried to add fairly robust tests, often checking all possible permutations of carriers/dimensions etc. Because of that, the tests might not be the easiest to look at, but they have proven to be pretty effective at shaking out issues. >> >> I think that covers the main aspects of the implementation and where it differs from vanilla JDK. >> >> P.S. >> >> In the CSR review [2], Joe raised a fair point - which is MemoryAddress has both: >> >> offset(long) --> move address of given offset >> offset() --> return the offset of this address in its owning segment >> >> And this was considered suboptimal, given both methods use the same name but do something quite different (one is an accessor, another is a 'wither'). one obvious option is to rename the first to 'withOffset'. But I think that would lead to verbose code (since that is a very common operation). Other options are to: >> >> * rename offset(long) to move(long), advance(long), or something else >> * drop offset() - but then add an overload of MemorySegment::asSlice which takes an address instead of a plain long offset >> >> I'll leave the choice to the reviewers :-) >> >> >> >> Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, Stuart, Roger, Joe and the Panama team for the feedback provided so far, which helped to get the API in the shape it is today. >> >> Cheers >> Maurizio >> >> (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" - but that is unrelated to this patch, and it's a known failing test. >> >> [1] - https://openjdk.java.net/jeps/370 >> [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 >> From joe.darcy at oracle.com Sat Dec 7 06:35:14 2019 From: joe.darcy at oracle.com (Joe Darcy) Date: Fri, 6 Dec 2019 22:35:14 -0800 Subject: JDK 15 RFR of JDK 15 RFR: JDK-8225361 : Start of release updates for JDK 15 Message-ID: <8e831b6d-c4a9-59b7-3bbe-bb988336bd7d@oracle.com> Hello, Please review the VM portions of the start-of-release updates for JDK 15: ??? http://cr.openjdk.java.net/~darcy/8225361.5/ Some additional adjustments to class file parsing for records may occur and get merged in before the start of 15. Thanks, -Joe From david.holmes at oracle.com Sat Dec 7 07:30:14 2019 From: david.holmes at oracle.com (David Holmes) Date: Sat, 7 Dec 2019 17:30:14 +1000 Subject: JDK 15 RFR of JDK 15 RFR: JDK-8225361 : Start of release updates for JDK 15 In-Reply-To: <8e831b6d-c4a9-59b7-3bbe-bb988336bd7d@oracle.com> References: <8e831b6d-c4a9-59b7-3bbe-bb988336bd7d@oracle.com> Message-ID: <2d5c7444-91f1-bc32-63f1-425e586587d6@oracle.com> Hi Joe, On 7/12/2019 4:35 pm, Joe Darcy wrote: > Hello, > > Please review the VM portions of the start-of-release updates for JDK 15: > > ??? http://cr.openjdk.java.net/~darcy/8225361.5/ > > Some additional adjustments to class file parsing for records may occur > and get merged in before the start of 15. Harold is handling the classfileParser.cpp changes under: "8235513: Change JVM to check for preview features using JVM_CLASSFILE_MAJOR_VERSION" which is out for review. I skimmed through all the changes: src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassFile.java ! V59(59, 0); // JDK 14 s/14/15/ Otherwise all seems fine. Thanks, David ----- > Thanks, > > -Joe > From gerard.ziemski at oracle.com Sat Dec 7 13:34:28 2019 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Sat, 7 Dec 2019 07:34:28 -0600 Subject: RFR (M) 8223261 "JDK-8189208 followup - remove JDK_GetVersionInfo0 and the supporting code" In-Reply-To: <4623455a-2d90-e697-89b3-830d5becf8f9@oracle.com> References: <4623455a-2d90-e697-89b3-830d5becf8f9@oracle.com> Message-ID: Hi all, Please review this revision 2 of a cleanup, where we remove JDK_GetVersionInfo0 and related code, since we can get build versions directly from within the VM itself. The rev2 differs from rev1 in that it's simpler due to JDK-8234185, which was just checked in yesterday. Waiting for this fix to go in first, allowed us not to have to delete the jdk_util.h file, which in turn means that we don't have to touch all those files that used that include. bug: https://bugs.openjdk.java.net/browse/JDK-8223261 webrev: http://cr.openjdk.java.net/~gziemski/8223261_rev2 tests: passes Mach5 tier1,2,3 (another tier1,2,3,4,5,6 in progress...) cheers From claes.redestad at oracle.com Sat Dec 7 13:44:02 2019 From: claes.redestad at oracle.com (Claes Redestad) Date: Sat, 7 Dec 2019 14:44:02 +0100 Subject: RFR (M) 8223261 "JDK-8189208 followup - remove JDK_GetVersionInfo0 and the supporting code" In-Reply-To: References: <4623455a-2d90-e697-89b3-830d5becf8f9@oracle.com> Message-ID: Looks good to me! /Claes On 2019-12-07 14:34, gerard ziemski wrote: > Hi all, > > Please review this revision 2 of a cleanup, where we remove > JDK_GetVersionInfo0 and related code, since we can get build versions > directly from within the VM itself. > > The rev2 differs from rev1 in that it's simpler due to JDK-8234185, > which was just checked in yesterday. Waiting for this fix to go in > first, allowed us not to have to delete the jdk_util.h file, which in > turn means that we don't have to touch all those files that used that > include. > > > bug: https://bugs.openjdk.java.net/browse/JDK-8223261 > webrev: http://cr.openjdk.java.net/~gziemski/8223261_rev2 > tests: passes Mach5 tier1,2,3 (another tier1,2,3,4,5,6 in progress...) > > > cheers > From christoph.langer at sap.com Sat Dec 7 15:15:12 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Sat, 7 Dec 2019 15:15:12 +0000 Subject: RFR (M) 8223261 "JDK-8189208 followup - remove JDK_GetVersionInfo0 and the supporting code" In-Reply-To: References: <4623455a-2d90-e697-89b3-830d5becf8f9@oracle.com> Message-ID: Hi Gerard, this looks good. Cheers Christoph > -----Original Message----- > From: gerard ziemski > Sent: Samstag, 7. Dezember 2019 14:34 > To: hotspot-dev developers ; core-libs- > dev at openjdk.java.net; Claes Redestad ; > Mandy Chung ; David Holmes > ; Sergey Bylokhov > ; Langer, Christoph > > Subject: Re: RFR (M) 8223261 "JDK-8189208 followup - remove > JDK_GetVersionInfo0 and the supporting code" > > Hi all, > > Please review this revision 2 of a cleanup, where we remove > JDK_GetVersionInfo0 and related code, since we can get build versions > directly from within the VM itself. > > The rev2 differs from rev1 in that it's simpler due to JDK-8234185, > which was just checked in yesterday. Waiting for this fix to go in > first, allowed us not to have to delete the jdk_util.h file, which in > turn means that we don't have to touch all those files that used that > include. > > > bug: https://bugs.openjdk.java.net/browse/JDK-8223261 > webrev: http://cr.openjdk.java.net/~gziemski/8223261_rev2 > tests: passes Mach5 tier1,2,3 (another tier1,2,3,4,5,6 in progress...) > > > cheers From maurizio.cimadamore at oracle.com Sat Dec 7 19:02:24 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Sat, 7 Dec 2019 19:02:24 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> Message-ID: On 06/12/2019 21:04, Paul Sandoz wrote: > GroupLayout.java > ? > > 80 OptionalLong sizeof(List elems) { > 81 long size = 0; > 82 for (MemoryLayout elem : elems) { > 83 if (AbstractLayout.optSize(elem).isPresent()) { > 84 size = sizeOp.applyAsLong(size, elem.bitSize()); > 85 } else { > 86 return OptionalLong.empty(); > 87 } > 88 } > 89 return OptionalLong.of(size); > 90 } > > FWIW you can do this: > > OptionalLong sizeof(List elems) { > return elems.stream().filter(e -> AbstractLayout.optSize(e).isPresent()).mapToLong(MemoryLayout::bitSize) > .reduce(sizeOp); Looked at this more carefully - the code you suggest has a slightly different behavior than the one I wrote - note that the original code short-circuit when the first layout member with no size is encountered. IIUC your code will just drop unsized member layouts, and compute size on the rest - which is not what I had in mind. Am I understanding correctly? Maurizio From mandy.chung at oracle.com Sat Dec 7 19:33:55 2019 From: mandy.chung at oracle.com (Mandy Chung) Date: Sat, 7 Dec 2019 11:33:55 -0800 Subject: RFR (M) 8223261 "JDK-8189208 followup - remove JDK_GetVersionInfo0 and the supporting code" In-Reply-To: References: <4623455a-2d90-e697-89b3-830d5becf8f9@oracle.com> Message-ID: <0d206768-674a-b462-cd7d-4dbbd5a21fdd@oracle.com> Looks good. Mandy On 12/7/19 5:34 AM, gerard ziemski wrote: > Hi all, > > Please review this revision 2 of a cleanup, where we remove > JDK_GetVersionInfo0 and related code, since we can get build versions > directly from within the VM itself. > > The rev2 differs from rev1 in that it's simpler due to JDK-8234185, > which was just checked in yesterday. Waiting for this fix to go in > first, allowed us not to have to delete the jdk_util.h file, which in > turn means that we don't have to touch all those files that used that > include. > > > bug: https://bugs.openjdk.java.net/browse/JDK-8223261 > webrev: http://cr.openjdk.java.net/~gziemski/8223261_rev2 > tests: passes Mach5 tier1,2,3 (another tier1,2,3,4,5,6 in progress...) > > > cheers > From john.r.rose at oracle.com Sat Dec 7 23:29:40 2019 From: john.r.rose at oracle.com (John Rose) Date: Sat, 7 Dec 2019 15:29:40 -0800 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <5c1689c5-43bf-3b4a-50e9-7376097f3087@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <5c1689c5-43bf-3b4a-50e9-7376097f3087@oracle.com> Message-ID: On Dec 5, 2019, at 2:39 PM, Vladimir Ivanov wrote: > > Regarding hotspot changes: > >> * ciField.cpp - this one is to trust final fields in the foreign memory access implementation (otherwise VM doesn't trust memory segment bounds) > > New packages aren't part of java.base. Considering the implementation resides in an incubator module, is it enough to consider them as trusted and well-known to the JVM? Yes, I think we can get away with this. IIUC, the effect you are observing is that java.base trusts something outside of itself. This is not new, and is OK as long as the ?something? can?t be spoofed by an untrusted party. ? John From david.holmes at oracle.com Sun Dec 8 00:27:51 2019 From: david.holmes at oracle.com (David Holmes) Date: Sun, 8 Dec 2019 10:27:51 +1000 Subject: RFR (M) 8223261 "JDK-8189208 followup - remove JDK_GetVersionInfo0 and the supporting code" In-Reply-To: References: <4623455a-2d90-e697-89b3-830d5becf8f9@oracle.com> Message-ID: <7b3eb0e0-0711-68b9-7a5f-dbc22d50488d@oracle.com> Looks good! Thanks, David On 7/12/2019 11:34 pm, gerard ziemski wrote: > Hi all, > > Please review this revision 2 of a cleanup, where we remove > JDK_GetVersionInfo0 and related code, since we can get build versions > directly from within the VM itself. > > The rev2 differs from rev1 in that it's simpler due to JDK-8234185, > which was just checked in yesterday. Waiting for this fix to go in > first, allowed us not to have to delete the jdk_util.h file, which in > turn means that we don't have to touch all those files that used that > include. > > > bug: https://bugs.openjdk.java.net/browse/JDK-8223261 > webrev: http://cr.openjdk.java.net/~gziemski/8223261_rev2 > tests: passes Mach5 tier1,2,3 (another tier1,2,3,4,5,6 in progress...) > > > cheers > From maurizio.cimadamore at oracle.com Sun Dec 8 01:44:56 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Sun, 8 Dec 2019 01:44:56 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <7765b06b-acf2-f11f-980e-33af4ca3934c@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <7765b06b-acf2-f11f-980e-33af4ca3934c@oracle.com> Message-ID: <14719046-4d92-d9ff-c59e-924e10212edf@oracle.com> Another update: http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v3/ And a delta of the changes since last version (v2) here: http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v3_delta/ The javadoc has been updated inline here: http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html Summary of changes: * fixed some (cosmetic) issues in FileChannelmpl following offline discussion with Alan * changed tests group definition, so that now jdk_foreign is part of tier1_part3 * updated javadoc in various places to fix code samples (as per Paul comments) * updated javadoc in MemoryHandles to talk about access modes (as per Paul comments) - I added a new section on top, and then referred to in from all relevant places * some changes to use Objects.hash (as per Paul comments), and some minor refactor in the AddreddGenerator (to use switch expression) * tightened javadoc for MemoryAddress::copy - the method now is specified to throw IAE if the range of source/dest addresses overlap - I've fixed the impl and added a test (as per Raffaello comments) * tightened byte buffer VarHandle view - if the view is created from a byte buffer obtained from a segment (!!!) we should do a segment check - added tests And here's a list of the pending API-related issues. Since these are IMHO minor issues, I suggest we defer them to a followup minor cleanup, so that we can move ahead with finalization of the CSR with the current API. Here's the list: * Should MemoryAddress implement Comparable? I think the answer is "probably not" - an address doesn't have a 'length' so you can't really do a range comparison (maybe memory segments though?). For now, users can project to byte buffer and take it from there (since ByteBuffer <: Comparable) * should we have a? way to ask a Layout if its size is specified ? (Paul) * MemoryAddress::offset() vs. MemoryAddress::offset(long) -- not much distance between these two semantically different operations (Paul suggested using 'add' - which I'm not enthusiastic about because it's not adding two addresses - it's adding a long to an address...) * MemorySegment::isAccessible() -- as the A* word is overloaded, some other name should be found? * MemorySegment::acquire() -- replace with MemorySegment::fork? Thanks Maurizio On 06/12/2019 10:43, Maurizio Cimadamore wrote: > Hi, > here's an updated version of the patch: > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v2/ > > And a delta of the changes since last version here: > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v2_delta/ > > The javadoc has been updated inline here: > > http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html > > > Summary of changes: > > * fixed spurious protected methods in AbstractLayout and subclasses > which leaked into API > * removed library_call.cpp changes, as these are being tracked > separately by Vlad > * compacted ILOAD code in AddressVarHandleGenerator > * replaced uses of ++i/--i with i + 1/i - 1 in MemoryScope > > I have made no changes to the *name* of the methods in the API. In > fact, I suggest we keep a list of the names we'd like to revisit, and > we address them all at once at the end of the review (once we're happy > with the contents). Here's a list of the current open naming issues: > > * MemoryAddress::offset() vs. MemoryAddress::offset(long) -- not much > distance between these two semantically different operations > * MemorySegment::isAccessible() -- as the A* word is overloaded, some > other name should be found? > * MemorySegment::acquire() -- replace with MemorySegment::fork? > > Cheers > Maurizio > > > On 05/12/2019 21:04, Maurizio Cimadamore wrote: >> Hi, >> as part of the effort to upstream the changes related to JEP 370 >> (foreign memory access API) [1], I'd like to ask for a code review >> for the corresponding core-libs and hotspot changes: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8234049/ >> >> A javadoc for the memory access API is also available here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >> >> >> Note: the patch passes tier1, tier2 and tier3 testing (**) >> >> >> Here is a brief summary of the changes in java.base and hotspot (the >> remaining new files are implementation classes and tests for the new >> API): >> >> * ciField.cpp - this one is to trust final fields in the foreign >> memory access implementation (otherwise VM doesn't trust memory >> segment bounds) >> >> * Modules.gmk - these changes are needed to require that the >> incubating module is loaded by the boot loader (otherwise the above >> changes are useless) >> >> * library_call.cpp - this one is a JIT compiler change to treat >> Thread.currentThread() as a well-known constant - which helps a lot >> in the confinement checks (thanks Vlad!) >> >> * various Buffer-related changes; these changes are needed because >> the memory access API allows a memory segment to be projected into a >> byte buffer, for interop reasons. As such, we need to insert a >> liveness check in the various get/put methods. Previously we had an >> implementation strategy where a BB was 'decorated' by a subclass >> called ScopedBuffer - but doing so required some changes to the BB >> API (e.g. making certain methods non-final, so that we could decorate >> them). Here I use an approach (which I have discussed with Alan) >> which doesn't require any public API changes, but needs to add a >> 'segment' field in Buffer - and then have constructors which keep >> track of this extra parameter. >> >> * FileChannel changes - these changes are required so that we can >> reuse the Unmapper class from the MemorySegment implementation, to >> deterministically deallocate a mapped memory segment. This should be >> a 'straight' refactoring, no change in behavior should occur here. >> Please double check. >> >> * VarHandles - this class now provides a factory to create memory >> access VarHandle - this is a bit tricky, since VarHandle cannot >> really be implemented outside java.base (e.g. VarForm is not public). >> So we do the usual trick where we define a bunch of proxy interfaces >> (see jdk/internal/access/foreign) have the classes in java.base refer >> to these - and then have the implementation classes of the memory >> access API implement these interfaces. >> >> * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we need >> to provide access to otherwise hidden functionalities - e.g. creating >> a new scoped buffer, or retrieving the properties of a memory access >> handle (e.g. offset, stride etc.), so that we can implement the >> memory access API in its own separate module >> >> * GensrcVarHandles.gmk - these changes are needed to enable the >> generation of the new memory address var handle implementations; >> there's an helper class per carrier (e.g. >> VarHandleMemoryAddressAsBytes, ...). At runtime, when a memory access >> var handle is needed, we dynamically spin a new VH implementation >> which makes use of the right carrier. We need to spin because the VH >> can have a variable number of access coordinates (e.g. depending on >> the dimensions of the array to be accessed). But, under the hood, all >> the generated implementation will be using the same helper class. >> >> * tests - we've tried to add fairly robust tests, often checking all >> possible permutations of carriers/dimensions etc. Because of that, >> the tests might not be the easiest to look at, but they have proven >> to be pretty effective at shaking out issues. >> >> I think that covers the main aspects of the implementation and where >> it differs from vanilla JDK. >> >> P.S. >> >> In the CSR review [2], Joe raised a fair point - which is >> MemoryAddress has both: >> >> offset(long) --> move address of given offset >> offset() --> return the offset of this address in its owning segment >> >> And this was considered suboptimal, given both methods use the same >> name but do something quite different (one is an accessor, another is >> a 'wither'). one obvious option is to rename the first to >> 'withOffset'. But I think that would lead to verbose code (since that >> is a very common operation). Other options are to: >> >> * rename offset(long) to move(long), advance(long), or something else >> * drop offset() - but then add an overload of MemorySegment::asSlice >> which takes an address instead of a plain long offset >> >> I'll leave the choice to the reviewers :-) >> >> >> >> Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, >> Stuart, Roger, Joe and the Panama team for the feedback provided so >> far, which helped to get the API in the shape it is today. >> >> Cheers >> Maurizio >> >> (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" - >> but that is unrelated to this patch, and it's a known failing test. >> >> [1] - https://openjdk.java.net/jeps/370 >> [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 >> From claes.redestad at oracle.com Sun Dec 8 21:54:05 2019 From: claes.redestad at oracle.com (Claes Redestad) Date: Sun, 8 Dec 2019 22:54:05 +0100 Subject: RFR: 8235551: BitMap::count_one_bits should use population_count Message-ID: Hi, with some adjustments, BitMap::count_one_bits can be rewritten to use the population_count utility method introduced in JDK-8217519 Bug: https://bugs.openjdk.java.net/browse/JDK-8235551 Webrev: http://cr.openjdk.java.net/~redestad/8235551/open.00/ This generalizes population_count to accept any unsigned integer, to accommodate that BitMap will use 32- or 64-bit integers depending on platform. Testing: tier1-3 Thanks! /Claes From Alan.Bateman at oracle.com Mon Dec 9 07:48:14 2019 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 9 Dec 2019 07:48:14 +0000 Subject: JDK 15 RFR of JDK 15 RFR: JDK-8225361 : Start of release updates for JDK 15 In-Reply-To: <2d5c7444-91f1-bc32-63f1-425e586587d6@oracle.com> References: <8e831b6d-c4a9-59b7-3bbe-bb988336bd7d@oracle.com> <2d5c7444-91f1-bc32-63f1-425e586587d6@oracle.com> Message-ID: <963f8a94-dae1-1e94-d686-76a01b467b8d@oracle.com> On 07/12/2019 07:30, David Holmes wrote: > > Harold is handling the classfileParser.cpp changes under: > > "8235513: Change JVM to check for preview features using > JVM_CLASSFILE_MAJOR_VERSION" > > which is out for review. > > I skimmed through all the changes: > > src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassFile.java > > !???????? V59(59, 0);?? // JDK 14 > > s/14/15/ > > Otherwise all seems fine. I looked through the patch file and didn't see any other issues. At some point I think we can change the ClasFileVersionsTests to not require an update at each class file version bump. -Alan From erik.osterlund at oracle.com Mon Dec 9 09:16:44 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 9 Dec 2019 10:16:44 +0100 Subject: RFR: ZGC: Add support for JFR leak profiler In-Reply-To: <6e08f198-6555-8d24-0be2-b9b6b76874ce@oracle.com> References: <27862ac1-e86b-406a-7819-85a13fbb24a3@oracle.com> <6e08f198-6555-8d24-0be2-b9b6b76874ce@oracle.com> Message-ID: <9b66f11f-0e17-ffa4-9f31-f811dfcc693f@oracle.com> Hi, I renamed some functions from oops_do to weak_oops_do to follow our naming conventions, and changed the filtering in the (now) weak_oops_do function to check if the raw oop is not null, instead of is_dead() which now has a new meaning since last webrev. Incremental: http://cr.openjdk.java.net/~eosterlund/8235174/webrev.01_02/ Full: http://cr.openjdk.java.net/~eosterlund/8235174/webrev.02/ This version has passed tier1-5. Thanks, /Erik On 2019-12-04 17:16, Erik ?sterlund wrote: > Hi Stefan, > > Thanks for the review. > > On 2019-12-04 14:55, Stefan Karlsson wrote: >> Hi Erik, >> >> This looks good to me! > > Thanks! > >> I think it would be good for the code to limit the number of >> UnifedOopRef::addr calls, since those are unsafe ways into the the >> addr and _could_ be abused to go around the UnifiedOopRef >> encapsulation. Maybe something for the future. > > Yeah. I wrote that function because I noticed that the address was > used with literally every single > address looking type I could imagine, in different places (uintptr_t, > address, char*, oop*, void*, > you name it). So I just wrote the addr function to be able to > perform this change as mechanically > as possible. > >> Just a small nit: >> >> https://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/src/hotspot/share/jfr/leakprofiler/checkpoint/eventEmitter.cpp.frames.html >> >> >> Now that you've limited the use of object_addr, maybe you could also >> limit its scope by moving >> >> 113?? const oop* object_addr = sample->object_addr(); >> >> to around: >> >> 123 edge = >> edge_store->put(UnifiedOopRef::encode_in_native(object_addr)); > > Absolutely. > > Here is a new webrev with your proposed change, and with some changes > from Markus (big thanks to Markus for looking at this in great detail): > http://cr.openjdk.java.net/~eosterlund/8235174/webrev.01/ > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00_01/ > > His changes mostly massage some types and order some includes. For > example Markus prefers passing around UnifiedOopRef > as non-const (now that it is a POD, it's like passing around an integer). > > The most important change though, was fixing a bug in my patch by > making sure that ObjectSampler::weak_oops_do > actually clears the oops that are dead. I thought it did, like the > other weak_oops_do functions. Turns out > that it has not been needed until now. The oops were not cleared, yet > the ObjectSample is reused as some > kind of cached object (presumably to call malloc less). The > consequence was that when the ObjectSample gets subsequently reused > with a dead oop in it, set_object() now using > NativeAccess::oop_store > with my patch crashes things, because the G1 barrier backend SATB > enqueues what was there in memory before > (which is dead garbage memory). > > Now one might argue that it's really silly of G1 barriers to SATB > enqueue on non-strong stores (because the very > snapshot that SATB tracks is the strong graph, which is the graph > being marked concurrently, nothing else). But I > would like to change that heuristic weirdness in a separate change. In > this patch I am happy with the object simply > being cleared in weak_oops_do, like it is in all other weak_oops_do > functions. That fixes the issue. > > Before there was a separate _dead boolean to track that the > ObjectSample is dead. Now that the oop is cleared > by the GC when it dies, the boolean is no longer needed, because the > is_dead() function can simply look at the oop. > > Thanks, > /Erik > >> Thanks, >> StefanK >> >> On 2019-12-02 11:09, erik.osterlund at oracle.com wrote: >>> Hi, >>> >>> The JFR leak profiler is currently not supported on ZGC, because it >>> has not been accessorized appropriately. >>> This patch adds the required Access API sprinkling necessary, to >>> enable this for ZGC. I did the following: >>> >>> 1) Renamed UnifiedOop to UnifiedOopRef, because it describes the >>> reference to an oop, not the oop itself. >>> 2) Changed UnifiedOopRef to be a POD, instead of an AllStatic that >>> passes around its encoded data as >>> ?? const oop* (even though it might be a tagged pointer to a narrow >>> oop), improving type safty. >>> 3) Added another tag bit to UnifiedOopRef, allowing it to keep track >>> of whether the described location is >>> ?? IN_NATIVE or IN_HEAP. The dereference function performs the >>> appropriate NativeAccess/HeapAccess with the >>> ?? AS_NO_KEEPALIVE decotator, which is fine because the leak >>> profiler does not create any edges to oops >>> ?? found during traversal. >>> 4) Removed code blob oop iteration, because it can't be supported >>> appropriately by the current UnifiedOopRef >>> ?? mechanism (because oops are misaligned, and UnifiedOopRef >>> requires tag bits - I believe this has never >>> ?? worked properly) >>> 6) Accessorized loads on strong roots to >>> NativeAccess::oop_load. JFR leak profiler never >>> ?? creates new references to oops that are visited, so it does not >>> need to keep things alive >>> 7) Explicitly told the oop iteration closures to not visit >>> referents. The default referent strategy used >>> ?? before is DO_DISCOVERY, which doesn't seem right. >>> 8) Fixed a pre-existing issue where the array index reported in the >>> leak chain was calculated incorrectly >>> ?? as the byte offset from the array base. It should be scaled by >>> the element length of the array. >>> >>> Ran some JFR leak profiler tests, and some ad-hoc leak profiling in >>> various applications. Also checked that >>> there were no observable perf regressions in the iterations, and >>> there were not. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8235174 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/ >>> >>> Thanks, >>> /Erik >> > From matthias.baesken at sap.com Mon Dec 9 11:22:49 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 9 Dec 2019 11:22:49 +0000 Subject: RFR [XS]: 8235489: handle return values of sscanf calls in hotspot In-Reply-To: <17DCFAEB-8E3C-4B94-9453-0AD85D3ADC94@oracle.com> References: <17DCFAEB-8E3C-4B94-9453-0AD85D3ADC94@oracle.com> Message-ID: Hi Kim, new webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8235489.2/ regarding the initialization of "name" - this is indeed for lines without a name entry - those lines exist in /proc/self/maps . I adjusted the initialization following your recommendation ( handle matches == 6). I also changed the unadorned "%s to one with an int-stringsize-parameter . Best regards, Matthias > > > On Dec 6, 2019, at 10:17 AM, Baesken, Matthias > wrote: > > > > Hello, I noticed there are some remaining sscanf calls in the hotspot > codebase missing return value checks. > > Those should better be checked . > > > > > > Bug/webrev : > > > > https://bugs.openjdk.java.net/browse/JDK-8235489 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235489.0/ > > > > > > Thanks, Matthias > > ------------------------------------------------------------------------------ > src/hotspot/os/linux/os_linux.cpp > 2085 memset(name, 0, sizeof(name)); > > What is this line of the change for? Is this because the pathname > field in a line can be blank? That isn't sufficient. That case needs > to be handled after the "if (matches < 6) continue;" line, e.g. check > whether matches == 6 (e.g. a blank name field), forcing "name" to be > blank if so (e.g. name[0] = '\0'). Otherwise, "name" will contain > data from the most recent parsed line that contained a match for that > field. > > ------------------------------------------------------------------------------ > src/hotspot/os/linux/os_linux.cpp > 2083 char device[6]; > ... > 2088 int matches = sscanf(line, UINT64_FORMAT_X "-" > UINT64_FORMAT_X " %4s " UINT64_FORMAT_X " %5s " INT64_FORMAT " > %s", > > Good eye, spotting the mismatch between the declared size of device > and the size spec in the format string (e.g. changing "%7s" => "%5s"). > > ------------------------------------------------------------------------------ > src/hotspot/os/linux/os_linux.cpp > > In the sscanf format string, there's no length spec for the pathname, > e.g. it uses an unadorned "%s". PATH_MAX doesn't deal with long file > names. > > ------------------------------------------------------------------------------ From matthias.baesken at sap.com Mon Dec 9 11:36:43 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 9 Dec 2019 11:36:43 +0000 Subject: RFR [XS] : 8235478: [jdk11] handle VS2017 15.9 and VS2019 in vm_version.cpp Message-ID: Hello, please review this small jdk11 change . It handles VS2017 15.9 and VS2019 in vm_version.cpp . More or less it is a jdk11 - downport of what 8235243 + 8235325 do , but in jdk/jdk . Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8235478 http://cr.openjdk.java.net/~mbaesken/webrevs/8235478.0/ Thanks, Matthias From peter.levart at gmail.com Mon Dec 9 18:16:47 2019 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 9 Dec 2019 19:16:47 +0100 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> Message-ID: <55613fbc-a8b2-b4bb-d4f1-5b08c2b91014@gmail.com> Hi Maurizio, Nice work! I see the API prefixes each independent access to memory with a call to the following MemorySegmentImpl method: ?157???? @Override ?158???? public final void checkValidState() { ?159???????? if (owner != Thread.currentThread()) { ?160???????????? throw new IllegalStateException("Attempt to access segment outside owning thread"); ?161???????? } ?162???????? scope.checkAlive(); ?163???? } The 1st check is making sure the method is invoked in the "owner" thread, while the second is making sure that scope is still alive. The 2nd check is only correct if it is performed in the "owner" thread which is guaranteed. These two are checks of mutable fields in two places (objects). May I suggest a scheme that combines both checks and only tests for one value in one object on the hot path: - drop Thread owner field from MemorySegmentImpl - drop boolean isAlive field from MemoryScope - add Thread owner field to MemoryScope - delegate MemorySegment.isAlive() and MemorySegmentImpl.checkValidState() to MemoryScope: ??? final boolean isAlive() { ??????? return activeCount.get() >= UNACQUIRED; ??? } ??? final boolean isAccessible() { ??????? return Thread.currentThread() == owner; ??? } ??? final void checkAlive() { ??????? if (!isAlive()) { ??????????? throw new IllegalStateException("Segment is not alive"); ??????? } ??? } ??? final void checkValidState() { ??????? if (Thread.currentThread() != owner) { ??????????? checkAlive(); ??????????? throw new IllegalStateException("Attempt to access segment outside owning thread"); ??????? } ??? } - make MemoryScope.close() be the following: ??? void close() { ??????? // assert Thread.currentThread() == MemorySegmentImpl.owner ??????? if (!activeCount.compareAndSet(UNACQUIRED, CLOSED)) { ??????????? throw new IllegalStateException("Cannot close a segment that has active acquired views"); ??????? } ??????? owner = null; ??????? if (cleanupAction != null) { ??????????? cleanupAction.run(); ??????? } ??? } - make other arrangements that pass Thread.currentThread() to MemoryScope instead of to MemorySegmentImpl constructor. Voila, only (Thread.currentThread() != owner) check on the hot path! Notice that .isAlive() is now also thread-safe. Previously it wasn't. What do you think? Might this be better performance wise? Regards, Peter On 12/5/19 10:04 PM, Maurizio Cimadamore wrote: > Hi, > as part of the effort to upstream the changes related to JEP 370 > (foreign memory access API) [1], I'd like to ask for a code review for > the corresponding core-libs and hotspot changes: > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049/ > > A javadoc for the memory access API is also available here: > > http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html > > > Note: the patch passes tier1, tier2 and tier3 testing (**) > > > Here is a brief summary of the changes in java.base and hotspot (the > remaining new files are implementation classes and tests for the new > API): > > * ciField.cpp - this one is to trust final fields in the foreign > memory access implementation (otherwise VM doesn't trust memory > segment bounds) > > * Modules.gmk - these changes are needed to require that the > incubating module is loaded by the boot loader (otherwise the above > changes are useless) > > * library_call.cpp - this one is a JIT compiler change to treat > Thread.currentThread() as a well-known constant - which helps a lot in > the confinement checks (thanks Vlad!) > > * various Buffer-related changes; these changes are needed because the > memory access API allows a memory segment to be projected into a byte > buffer, for interop reasons. As such, we need to insert a liveness > check in the various get/put methods. Previously we had an > implementation strategy where a BB was 'decorated' by a subclass > called ScopedBuffer - but doing so required some changes to the BB API > (e.g. making certain methods non-final, so that we could decorate > them). Here I use an approach (which I have discussed with Alan) which > doesn't require any public API changes, but needs to add a 'segment' > field in Buffer - and then have constructors which keep track of this > extra parameter. > > * FileChannel changes - these changes are required so that we can > reuse the Unmapper class from the MemorySegment implementation, to > deterministically deallocate a mapped memory segment. This should be a > 'straight' refactoring, no change in behavior should occur here. > Please double check. > > * VarHandles - this class now provides a factory to create memory > access VarHandle - this is a bit tricky, since VarHandle cannot really > be implemented outside java.base (e.g. VarForm is not public). So we > do the usual trick where we define a bunch of proxy interfaces (see > jdk/internal/access/foreign) have the classes in java.base refer to > these - and then have the implementation classes of the memory access > API implement these interfaces. > > * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we need > to provide access to otherwise hidden functionalities - e.g. creating > a new scoped buffer, or retrieving the properties of a memory access > handle (e.g. offset, stride etc.), so that we can implement the memory > access API in its own separate module > > * GensrcVarHandles.gmk - these changes are needed to enable the > generation of the new memory address var handle implementations; > there's an helper class per carrier (e.g. > VarHandleMemoryAddressAsBytes, ...). At runtime, when a memory access > var handle is needed, we dynamically spin a new VH implementation > which makes use of the right carrier. We need to spin because the VH > can have a variable number of access coordinates (e.g. depending on > the dimensions of the array to be accessed). But, under the hood, all > the generated implementation will be using the same helper class. > > * tests - we've tried to add fairly robust tests, often checking all > possible permutations of carriers/dimensions etc. Because of that, the > tests might not be the easiest to look at, but they have proven to be > pretty effective at shaking out issues. > > I think that covers the main aspects of the implementation and where > it differs from vanilla JDK. > > P.S. > > In the CSR review [2], Joe raised a fair point - which is > MemoryAddress has both: > > offset(long) --> move address of given offset > offset() --> return the offset of this address in its owning segment > > And this was considered suboptimal, given both methods use the same > name but do something quite different (one is an accessor, another is > a 'wither'). one obvious option is to rename the first to > 'withOffset'. But I think that would lead to verbose code (since that > is a very common operation). Other options are to: > > * rename offset(long) to move(long), advance(long), or something else > * drop offset() - but then add an overload of MemorySegment::asSlice > which takes an address instead of a plain long offset > > I'll leave the choice to the reviewers :-) > > > > Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, > Stuart, Roger, Joe and the Panama team for the feedback provided so > far, which helped to get the API in the shape it is today. > > Cheers > Maurizio > > (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" - > but that is unrelated to this patch, and it's a known failing test. > > [1] - https://openjdk.java.net/jeps/370 > [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 > From maurizio.cimadamore at oracle.com Mon Dec 9 18:31:06 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 9 Dec 2019 18:31:06 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <55613fbc-a8b2-b4bb-d4f1-5b08c2b91014@gmail.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <55613fbc-a8b2-b4bb-d4f1-5b08c2b91014@gmail.com> Message-ID: <15774b62-48b5-3614-0fd3-b62060b9825a@oracle.com> Hi Peter, this looks like a clever optimization which basically takes advantage of the fact that you can only access a segment if you are the same thread as the segment owner. I think after we integrate, I'd love to give your approach a try and see what happens performance-wise. Right now we don't pay much for confinement check because the Thread owner is a final constant - so these checks are optimized away nicely. But we pay something for the liveness check, because the 'isAlive' field is mutable. I believe your approach won't change a lot here - it folds the two checks, but we were already paying zero for one of the two, so... Anyway - all this needs to be validated by adequate JMH-ing :-) Thanks very much for your comments! Maurizio On 09/12/2019 18:16, Peter Levart wrote: > Hi Maurizio, > > Nice work! > > I see the API prefixes each independent access to memory with a call > to the following MemorySegmentImpl method: > > ?157???? @Override > ?158???? public final void checkValidState() { > ?159???????? if (owner != Thread.currentThread()) { > ?160???????????? throw new IllegalStateException("Attempt to access > segment outside owning thread"); > ?161???????? } > ?162???????? scope.checkAlive(); > ?163???? } > > The 1st check is making sure the method is invoked in the "owner" > thread, while the second is making sure that scope is still alive. The > 2nd check is only correct if it is performed in the "owner" thread > which is guaranteed. These two are checks of mutable fields in two > places (objects). > > May I suggest a scheme that combines both checks and only tests for > one value in one object on the hot path: > > - drop Thread owner field from MemorySegmentImpl > - drop boolean isAlive field from MemoryScope > - add Thread owner field to MemoryScope > - delegate MemorySegment.isAlive() and > MemorySegmentImpl.checkValidState() to MemoryScope: > > ??? final boolean isAlive() { > ??????? return activeCount.get() >= UNACQUIRED; > ??? } > > ??? final boolean isAccessible() { > ??????? return Thread.currentThread() == owner; > ??? } > > ??? final void checkAlive() { > ??????? if (!isAlive()) { > ??????????? throw new IllegalStateException("Segment is not alive"); > ??????? } > ??? } > > ??? final void checkValidState() { > ??????? if (Thread.currentThread() != owner) { > ??????????? checkAlive(); > ??????????? throw new IllegalStateException("Attempt to access segment > outside owning thread"); > ??????? } > ??? } > > - make MemoryScope.close() be the following: > > ??? void close() { > ??????? // assert Thread.currentThread() == MemorySegmentImpl.owner > ??????? if (!activeCount.compareAndSet(UNACQUIRED, CLOSED)) { > ??????????? throw new IllegalStateException("Cannot close a segment > that has active acquired views"); > ??????? } > ??????? owner = null; > ??????? if (cleanupAction != null) { > ??????????? cleanupAction.run(); > ??????? } > ??? } > > - make other arrangements that pass Thread.currentThread() to > MemoryScope instead of to MemorySegmentImpl constructor. > > > Voila, only (Thread.currentThread() != owner) check on the hot path! > > Notice that .isAlive() is now also thread-safe. Previously it wasn't. > > > What do you think? Might this be better performance wise? > > Regards, Peter > > > On 12/5/19 10:04 PM, Maurizio Cimadamore wrote: >> Hi, >> as part of the effort to upstream the changes related to JEP 370 >> (foreign memory access API) [1], I'd like to ask for a code review >> for the corresponding core-libs and hotspot changes: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8234049/ >> >> A javadoc for the memory access API is also available here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >> >> >> Note: the patch passes tier1, tier2 and tier3 testing (**) >> >> >> Here is a brief summary of the changes in java.base and hotspot (the >> remaining new files are implementation classes and tests for the new >> API): >> >> * ciField.cpp - this one is to trust final fields in the foreign >> memory access implementation (otherwise VM doesn't trust memory >> segment bounds) >> >> * Modules.gmk - these changes are needed to require that the >> incubating module is loaded by the boot loader (otherwise the above >> changes are useless) >> >> * library_call.cpp - this one is a JIT compiler change to treat >> Thread.currentThread() as a well-known constant - which helps a lot >> in the confinement checks (thanks Vlad!) >> >> * various Buffer-related changes; these changes are needed because >> the memory access API allows a memory segment to be projected into a >> byte buffer, for interop reasons. As such, we need to insert a >> liveness check in the various get/put methods. Previously we had an >> implementation strategy where a BB was 'decorated' by a subclass >> called ScopedBuffer - but doing so required some changes to the BB >> API (e.g. making certain methods non-final, so that we could decorate >> them). Here I use an approach (which I have discussed with Alan) >> which doesn't require any public API changes, but needs to add a >> 'segment' field in Buffer - and then have constructors which keep >> track of this extra parameter. >> >> * FileChannel changes - these changes are required so that we can >> reuse the Unmapper class from the MemorySegment implementation, to >> deterministically deallocate a mapped memory segment. This should be >> a 'straight' refactoring, no change in behavior should occur here. >> Please double check. >> >> * VarHandles - this class now provides a factory to create memory >> access VarHandle - this is a bit tricky, since VarHandle cannot >> really be implemented outside java.base (e.g. VarForm is not public). >> So we do the usual trick where we define a bunch of proxy interfaces >> (see jdk/internal/access/foreign) have the classes in java.base refer >> to these - and then have the implementation classes of the memory >> access API implement these interfaces. >> >> * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we need >> to provide access to otherwise hidden functionalities - e.g. creating >> a new scoped buffer, or retrieving the properties of a memory access >> handle (e.g. offset, stride etc.), so that we can implement the >> memory access API in its own separate module >> >> * GensrcVarHandles.gmk - these changes are needed to enable the >> generation of the new memory address var handle implementations; >> there's an helper class per carrier (e.g. >> VarHandleMemoryAddressAsBytes, ...). At runtime, when a memory access >> var handle is needed, we dynamically spin a new VH implementation >> which makes use of the right carrier. We need to spin because the VH >> can have a variable number of access coordinates (e.g. depending on >> the dimensions of the array to be accessed). But, under the hood, all >> the generated implementation will be using the same helper class. >> >> * tests - we've tried to add fairly robust tests, often checking all >> possible permutations of carriers/dimensions etc. Because of that, >> the tests might not be the easiest to look at, but they have proven >> to be pretty effective at shaking out issues. >> >> I think that covers the main aspects of the implementation and where >> it differs from vanilla JDK. >> >> P.S. >> >> In the CSR review [2], Joe raised a fair point - which is >> MemoryAddress has both: >> >> offset(long) --> move address of given offset >> offset() --> return the offset of this address in its owning segment >> >> And this was considered suboptimal, given both methods use the same >> name but do something quite different (one is an accessor, another is >> a 'wither'). one obvious option is to rename the first to >> 'withOffset'. But I think that would lead to verbose code (since that >> is a very common operation). Other options are to: >> >> * rename offset(long) to move(long), advance(long), or something else >> * drop offset() - but then add an overload of MemorySegment::asSlice >> which takes an address instead of a plain long offset >> >> I'll leave the choice to the reviewers :-) >> >> >> >> Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, >> Stuart, Roger, Joe and the Panama team for the feedback provided so >> far, which helped to get the API in the shape it is today. >> >> Cheers >> Maurizio >> >> (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" - >> but that is unrelated to this patch, and it's a known failing test. >> >> [1] - https://openjdk.java.net/jeps/370 >> [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 >> > From paul.sandoz at oracle.com Mon Dec 9 18:36:58 2019 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 9 Dec 2019 10:36:58 -0800 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> Message-ID: Oops, my bad, in my haste I misread the original code. (Effectively a reduction on OptionalLong where empty dominates, but the code to write that is I think less clear than the imperative loop.) Paul. > On Dec 7, 2019, at 11:02 AM, Maurizio Cimadamore wrote: > > > On 06/12/2019 21:04, Paul Sandoz wrote: >> GroupLayout.java >> ? >> >> 80 OptionalLong sizeof(List elems) { >> 81 long size = 0; >> 82 for (MemoryLayout elem : elems) { >> 83 if (AbstractLayout.optSize(elem).isPresent()) { >> 84 size = sizeOp.applyAsLong(size, elem.bitSize()); >> 85 } else { >> 86 return OptionalLong.empty(); >> 87 } >> 88 } >> 89 return OptionalLong.of(size); >> 90 } >> >> FWIW you can do this: >> >> OptionalLong sizeof(List elems) { >> return elems.stream().filter(e -> AbstractLayout.optSize(e).isPresent()).mapToLong(MemoryLayout::bitSize) >> .reduce(sizeOp); > > > Looked at this more carefully - the code you suggest has a slightly different behavior than the one I wrote - note that the original code short-circuit when the first layout member with no size is encountered. IIUC your code will just drop unsized member layouts, and compute size on the rest - which is not what I had in mind. Am I understanding correctly? > > Maurizio > From maurizio.cimadamore at oracle.com Mon Dec 9 19:23:36 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 9 Dec 2019 19:23:36 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <14719046-4d92-d9ff-c59e-924e10212edf@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <7765b06b-acf2-f11f-980e-33af4ca3934c@oracle.com> <14719046-4d92-d9ff-c59e-924e10212edf@oracle.com> Message-ID: <24fdb620-2b26-0647-524f-71a40b50d9d3@oracle.com> Another iteration: http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v4/ Delta of the changes since last version (v3) here: http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v4_delta/ The javadoc has been updated inline here: http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html Summary of changes: * Improved javadoc regarding alignment and access modes in MemoryHandles * related to the above, the memory access varhandle now checks for low-level VM alignment to for access other than get/set (as happens for regular byte buffer view var handle) * fixed MemoryHandles, JavaLangInvokeAccess, VarHandles, MethodHandleImpl to speak about alignmentMask, rather than alignment (as per Roger comments) * added some more javadoc to internal classes MemoryAddressProxy, VarHandleMemoryAddressBase, JavaNioAccess.java and JavaLangInvokeAccess.java? (as per Roger comments) * fixed mising {code} block around alignment param A in MemoryLayout::bitAlignment/bytesAlignment (as per offline comments from Daniel) * added positive test to TestMemoryCopy Cheers Maurizio On 08/12/2019 01:44, Maurizio Cimadamore wrote: > Another update: > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v3/ > > And a delta of the changes since last version (v2) here: > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v3_delta/ > > The javadoc has been updated inline here: > > http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html > > > Summary of changes: > > * fixed some (cosmetic) issues in FileChannelmpl following offline > discussion with Alan > * changed tests group definition, so that now jdk_foreign is part of > tier1_part3 > * updated javadoc in various places to fix code samples (as per Paul > comments) > * updated javadoc in MemoryHandles to talk about access modes (as per > Paul comments) - I added a new section on top, and then referred to in > from all relevant places > * some changes to use Objects.hash (as per Paul comments), and some > minor refactor in the AddreddGenerator (to use switch expression) > * tightened javadoc for MemoryAddress::copy - the method now is > specified to throw IAE if the range of source/dest addresses overlap - > I've fixed the impl and added a test (as per Raffaello comments) > * tightened byte buffer VarHandle view - if the view is created from a > byte buffer obtained from a segment (!!!) we should do a segment check > - added tests > > > And here's a list of the pending API-related issues. Since these are > IMHO minor issues, I suggest we defer them to a followup minor > cleanup, so that we can move ahead with finalization of the CSR with > the current API. Here's the list: > > * Should MemoryAddress implement Comparable? I think the answer is > "probably not" - an address doesn't have a 'length' so you can't > really do a range comparison (maybe memory segments though?). For now, > users can project to byte buffer and take it from there (since > ByteBuffer <: Comparable) > * should we have a? way to ask a Layout if its size is specified ? (Paul) > * MemoryAddress::offset() vs. MemoryAddress::offset(long) -- not much > distance between these two semantically different operations (Paul > suggested using 'add' - which I'm not enthusiastic about because it's > not adding two addresses - it's adding a long to an address...) > * MemorySegment::isAccessible() -- as the A* word is overloaded, some > other name should be found? > * MemorySegment::acquire() -- replace with MemorySegment::fork? > > Thanks > Maurizio > > On 06/12/2019 10:43, Maurizio Cimadamore wrote: >> Hi, >> here's an updated version of the patch: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v2/ >> >> And a delta of the changes since last version here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v2_delta/ >> >> The javadoc has been updated inline here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >> >> >> Summary of changes: >> >> * fixed spurious protected methods in AbstractLayout and subclasses >> which leaked into API >> * removed library_call.cpp changes, as these are being tracked >> separately by Vlad >> * compacted ILOAD code in AddressVarHandleGenerator >> * replaced uses of ++i/--i with i + 1/i - 1 in MemoryScope >> >> I have made no changes to the *name* of the methods in the API. In >> fact, I suggest we keep a list of the names we'd like to revisit, and >> we address them all at once at the end of the review (once we're >> happy with the contents). Here's a list of the current open naming >> issues: >> >> * MemoryAddress::offset() vs. MemoryAddress::offset(long) -- not much >> distance between these two semantically different operations >> * MemorySegment::isAccessible() -- as the A* word is overloaded, some >> other name should be found? >> * MemorySegment::acquire() -- replace with MemorySegment::fork? >> >> Cheers >> Maurizio >> >> >> On 05/12/2019 21:04, Maurizio Cimadamore wrote: >>> Hi, >>> as part of the effort to upstream the changes related to JEP 370 >>> (foreign memory access API) [1], I'd like to ask for a code review >>> for the corresponding core-libs and hotspot changes: >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/8234049/ >>> >>> A javadoc for the memory access API is also available here: >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >>> >>> >>> Note: the patch passes tier1, tier2 and tier3 testing (**) >>> >>> >>> Here is a brief summary of the changes in java.base and hotspot (the >>> remaining new files are implementation classes and tests for the new >>> API): >>> >>> * ciField.cpp - this one is to trust final fields in the foreign >>> memory access implementation (otherwise VM doesn't trust memory >>> segment bounds) >>> >>> * Modules.gmk - these changes are needed to require that the >>> incubating module is loaded by the boot loader (otherwise the above >>> changes are useless) >>> >>> * library_call.cpp - this one is a JIT compiler change to treat >>> Thread.currentThread() as a well-known constant - which helps a lot >>> in the confinement checks (thanks Vlad!) >>> >>> * various Buffer-related changes; these changes are needed because >>> the memory access API allows a memory segment to be projected into a >>> byte buffer, for interop reasons. As such, we need to insert a >>> liveness check in the various get/put methods. Previously we had an >>> implementation strategy where a BB was 'decorated' by a subclass >>> called ScopedBuffer - but doing so required some changes to the BB >>> API (e.g. making certain methods non-final, so that we could >>> decorate them). Here I use an approach (which I have discussed with >>> Alan) which doesn't require any public API changes, but needs to add >>> a 'segment' field in Buffer - and then have constructors which keep >>> track of this extra parameter. >>> >>> * FileChannel changes - these changes are required so that we can >>> reuse the Unmapper class from the MemorySegment implementation, to >>> deterministically deallocate a mapped memory segment. This should be >>> a 'straight' refactoring, no change in behavior should occur here. >>> Please double check. >>> >>> * VarHandles - this class now provides a factory to create memory >>> access VarHandle - this is a bit tricky, since VarHandle cannot >>> really be implemented outside java.base (e.g. VarForm is not >>> public). So we do the usual trick where we define a bunch of proxy >>> interfaces (see jdk/internal/access/foreign) have the classes in >>> java.base refer to these - and then have the implementation classes >>> of the memory access API implement these interfaces. >>> >>> * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we >>> need to provide access to otherwise hidden functionalities - e.g. >>> creating a new scoped buffer, or retrieving the properties of a >>> memory access handle (e.g. offset, stride etc.), so that we can >>> implement the memory access API in its own separate module >>> >>> * GensrcVarHandles.gmk - these changes are needed to enable the >>> generation of the new memory address var handle implementations; >>> there's an helper class per carrier (e.g. >>> VarHandleMemoryAddressAsBytes, ...). At runtime, when a memory >>> access var handle is needed, we dynamically spin a new VH >>> implementation which makes use of the right carrier. We need to spin >>> because the VH can have a variable number of access coordinates >>> (e.g. depending on the dimensions of the array to be accessed). But, >>> under the hood, all the generated implementation will be using the >>> same helper class. >>> >>> * tests - we've tried to add fairly robust tests, often checking all >>> possible permutations of carriers/dimensions etc. Because of that, >>> the tests might not be the easiest to look at, but they have proven >>> to be pretty effective at shaking out issues. >>> >>> I think that covers the main aspects of the implementation and where >>> it differs from vanilla JDK. >>> >>> P.S. >>> >>> In the CSR review [2], Joe raised a fair point - which is >>> MemoryAddress has both: >>> >>> offset(long) --> move address of given offset >>> offset() --> return the offset of this address in its owning segment >>> >>> And this was considered suboptimal, given both methods use the same >>> name but do something quite different (one is an accessor, another >>> is a 'wither'). one obvious option is to rename the first to >>> 'withOffset'. But I think that would lead to verbose code (since >>> that is a very common operation). Other options are to: >>> >>> * rename offset(long) to move(long), advance(long), or something else >>> * drop offset() - but then add an overload of MemorySegment::asSlice >>> which takes an address instead of a plain long offset >>> >>> I'll leave the choice to the reviewers :-) >>> >>> >>> >>> Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, >>> Stuart, Roger, Joe and the Panama team for the feedback provided so >>> far, which helped to get the API in the shape it is today. >>> >>> Cheers >>> Maurizio >>> >>> (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" >>> - but that is unrelated to this patch, and it's a known failing test. >>> >>> [1] - https://openjdk.java.net/jeps/370 >>> [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 >>> From kim.barrett at oracle.com Mon Dec 9 20:30:43 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 9 Dec 2019 15:30:43 -0500 Subject: RFR [XS]: 8235489: handle return values of sscanf calls in hotspot In-Reply-To: References: <17DCFAEB-8E3C-4B94-9453-0AD85D3ADC94@oracle.com> Message-ID: <27E66F9B-6AA5-44BE-ABC5-EA1B6792E02F@oracle.com> > On Dec 9, 2019, at 6:22 AM, Baesken, Matthias wrote: > > Hi Kim, new webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235489.2/ > > > regarding the initialization of "name" - this is indeed for lines without a name entry - those lines exist in /proc/self/maps . > I adjusted the initialization following your recommendation ( handle matches == 6). > > I also changed the unadorned "%s to one with an int-stringsize-parameter . ------------------------------------------------------------------------------ src/hotspot/os/linux/os_linux.cpp 2084 char name[4097]; // was PATH_MAX + 1 Please stay with the original, using PATH_MAX + 1. I'm assuming this change was so in the string parsing argument you could use "%4096s" to limit the amount of data read into name. That can still be done with the size of name being based on PATH_MAX by using a variable field width for the string conversion, e.g. "%*s" with an argument of PATH_MAX for the ("*") field width, before the name argument. ------------------------------------------------------------------------------ From paul.sandoz at oracle.com Mon Dec 9 20:32:15 2019 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 9 Dec 2019 12:32:15 -0800 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <24fdb620-2b26-0647-524f-71a40b50d9d3@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <7765b06b-acf2-f11f-980e-33af4ca3934c@oracle.com> <14719046-4d92-d9ff-c59e-924e10212edf@oracle.com> <24fdb620-2b26-0647-524f-71a40b50d9d3@oracle.com> Message-ID: +1 Paul. > On Dec 9, 2019, at 11:23 AM, Maurizio Cimadamore wrote: > > Another iteration: > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v4/ > > Delta of the changes since last version (v3) here: > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v4_delta/ > > The javadoc has been updated inline here: > > http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html > > Summary of changes: > > * Improved javadoc regarding alignment and access modes in MemoryHandles > * related to the above, the memory access varhandle now checks for low-level VM alignment to for access other than get/set (as happens for regular byte buffer view var handle) > * fixed MemoryHandles, JavaLangInvokeAccess, VarHandles, MethodHandleImpl to speak about alignmentMask, rather than alignment (as per Roger comments) > * added some more javadoc to internal classes MemoryAddressProxy, VarHandleMemoryAddressBase, JavaNioAccess.java and JavaLangInvokeAccess.java (as per Roger comments) > * fixed mising {code} block around alignment param A in MemoryLayout::bitAlignment/bytesAlignment (as per offline comments from Daniel) > * added positive test to TestMemoryCopy > > Cheers > Maurizio > > On 08/12/2019 01:44, Maurizio Cimadamore wrote: >> Another update: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v3/ >> >> And a delta of the changes since last version (v2) here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v3_delta/ >> >> The javadoc has been updated inline here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >> >> Summary of changes: >> >> * fixed some (cosmetic) issues in FileChannelmpl following offline discussion with Alan >> * changed tests group definition, so that now jdk_foreign is part of tier1_part3 >> * updated javadoc in various places to fix code samples (as per Paul comments) >> * updated javadoc in MemoryHandles to talk about access modes (as per Paul comments) - I added a new section on top, and then referred to in from all relevant places >> * some changes to use Objects.hash (as per Paul comments), and some minor refactor in the AddreddGenerator (to use switch expression) >> * tightened javadoc for MemoryAddress::copy - the method now is specified to throw IAE if the range of source/dest addresses overlap - I've fixed the impl and added a test (as per Raffaello comments) >> * tightened byte buffer VarHandle view - if the view is created from a byte buffer obtained from a segment (!!!) we should do a segment check - added tests >> >> >> And here's a list of the pending API-related issues. Since these are IMHO minor issues, I suggest we defer them to a followup minor cleanup, so that we can move ahead with finalization of the CSR with the current API. Here's the list: >> >> * Should MemoryAddress implement Comparable? I think the answer is "probably not" - an address doesn't have a 'length' so you can't really do a range comparison (maybe memory segments though?). For now, users can project to byte buffer and take it from there (since ByteBuffer <: Comparable) >> * should we have a way to ask a Layout if its size is specified ? (Paul) >> * MemoryAddress::offset() vs. MemoryAddress::offset(long) -- not much distance between these two semantically different operations (Paul suggested using 'add' - which I'm not enthusiastic about because it's not adding two addresses - it's adding a long to an address...) >> * MemorySegment::isAccessible() -- as the A* word is overloaded, some other name should be found? >> * MemorySegment::acquire() -- replace with MemorySegment::fork? >> >> Thanks >> Maurizio >> >> On 06/12/2019 10:43, Maurizio Cimadamore wrote: >>> Hi, >>> here's an updated version of the patch: >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v2/ >>> >>> And a delta of the changes since last version here: >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v2_delta/ >>> >>> The javadoc has been updated inline here: >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >>> >>> Summary of changes: >>> >>> * fixed spurious protected methods in AbstractLayout and subclasses which leaked into API >>> * removed library_call.cpp changes, as these are being tracked separately by Vlad >>> * compacted ILOAD code in AddressVarHandleGenerator >>> * replaced uses of ++i/--i with i + 1/i - 1 in MemoryScope >>> >>> I have made no changes to the *name* of the methods in the API. In fact, I suggest we keep a list of the names we'd like to revisit, and we address them all at once at the end of the review (once we're happy with the contents). Here's a list of the current open naming issues: >>> >>> * MemoryAddress::offset() vs. MemoryAddress::offset(long) -- not much distance between these two semantically different operations >>> * MemorySegment::isAccessible() -- as the A* word is overloaded, some other name should be found? >>> * MemorySegment::acquire() -- replace with MemorySegment::fork? >>> >>> Cheers >>> Maurizio >>> >>> >>> On 05/12/2019 21:04, Maurizio Cimadamore wrote: >>>> Hi, >>>> as part of the effort to upstream the changes related to JEP 370 (foreign memory access API) [1], I'd like to ask for a code review for the corresponding core-libs and hotspot changes: >>>> >>>> http://cr.openjdk.java.net/~mcimadamore/panama/8234049/ >>>> >>>> A javadoc for the memory access API is also available here: >>>> >>>> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >>>> >>>> Note: the patch passes tier1, tier2 and tier3 testing (**) >>>> >>>> >>>> Here is a brief summary of the changes in java.base and hotspot (the remaining new files are implementation classes and tests for the new API): >>>> >>>> * ciField.cpp - this one is to trust final fields in the foreign memory access implementation (otherwise VM doesn't trust memory segment bounds) >>>> >>>> * Modules.gmk - these changes are needed to require that the incubating module is loaded by the boot loader (otherwise the above changes are useless) >>>> >>>> * library_call.cpp - this one is a JIT compiler change to treat Thread.currentThread() as a well-known constant - which helps a lot in the confinement checks (thanks Vlad!) >>>> >>>> * various Buffer-related changes; these changes are needed because the memory access API allows a memory segment to be projected into a byte buffer, for interop reasons. As such, we need to insert a liveness check in the various get/put methods. Previously we had an implementation strategy where a BB was 'decorated' by a subclass called ScopedBuffer - but doing so required some changes to the BB API (e.g. making certain methods non-final, so that we could decorate them). Here I use an approach (which I have discussed with Alan) which doesn't require any public API changes, but needs to add a 'segment' field in Buffer - and then have constructors which keep track of this extra parameter. >>>> >>>> * FileChannel changes - these changes are required so that we can reuse the Unmapper class from the MemorySegment implementation, to deterministically deallocate a mapped memory segment. This should be a 'straight' refactoring, no change in behavior should occur here. Please double check. >>>> >>>> * VarHandles - this class now provides a factory to create memory access VarHandle - this is a bit tricky, since VarHandle cannot really be implemented outside java.base (e.g. VarForm is not public). So we do the usual trick where we define a bunch of proxy interfaces (see jdk/internal/access/foreign) have the classes in java.base refer to these - and then have the implementation classes of the memory access API implement these interfaces. >>>> >>>> * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we need to provide access to otherwise hidden functionalities - e.g. creating a new scoped buffer, or retrieving the properties of a memory access handle (e.g. offset, stride etc.), so that we can implement the memory access API in its own separate module >>>> >>>> * GensrcVarHandles.gmk - these changes are needed to enable the generation of the new memory address var handle implementations; there's an helper class per carrier (e.g. VarHandleMemoryAddressAsBytes, ...). At runtime, when a memory access var handle is needed, we dynamically spin a new VH implementation which makes use of the right carrier. We need to spin because the VH can have a variable number of access coordinates (e.g. depending on the dimensions of the array to be accessed). But, under the hood, all the generated implementation will be using the same helper class. >>>> >>>> * tests - we've tried to add fairly robust tests, often checking all possible permutations of carriers/dimensions etc. Because of that, the tests might not be the easiest to look at, but they have proven to be pretty effective at shaking out issues. >>>> >>>> I think that covers the main aspects of the implementation and where it differs from vanilla JDK. >>>> >>>> P.S. >>>> >>>> In the CSR review [2], Joe raised a fair point - which is MemoryAddress has both: >>>> >>>> offset(long) --> move address of given offset >>>> offset() --> return the offset of this address in its owning segment >>>> >>>> And this was considered suboptimal, given both methods use the same name but do something quite different (one is an accessor, another is a 'wither'). one obvious option is to rename the first to 'withOffset'. But I think that would lead to verbose code (since that is a very common operation). Other options are to: >>>> >>>> * rename offset(long) to move(long), advance(long), or something else >>>> * drop offset() - but then add an overload of MemorySegment::asSlice which takes an address instead of a plain long offset >>>> >>>> I'll leave the choice to the reviewers :-) >>>> >>>> >>>> >>>> Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, Stuart, Roger, Joe and the Panama team for the feedback provided so far, which helped to get the API in the shape it is today. >>>> >>>> Cheers >>>> Maurizio >>>> >>>> (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" - but that is unrelated to this patch, and it's a known failing test. >>>> >>>> [1] - https://openjdk.java.net/jeps/370 >>>> [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 >>>> From kim.barrett at oracle.com Mon Dec 9 21:19:53 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 9 Dec 2019 16:19:53 -0500 Subject: RFR [XS]: 8235489: handle return values of sscanf calls in hotspot In-Reply-To: <27E66F9B-6AA5-44BE-ABC5-EA1B6792E02F@oracle.com> References: <17DCFAEB-8E3C-4B94-9453-0AD85D3ADC94@oracle.com> <27E66F9B-6AA5-44BE-ABC5-EA1B6792E02F@oracle.com> Message-ID: <53B24471-1455-4D0E-B197-4A34E55DB0C7@oracle.com> > On Dec 9, 2019, at 3:30 PM, Kim Barrett wrote: > >> On Dec 9, 2019, at 6:22 AM, Baesken, Matthias wrote: >> >> Hi Kim, new webrev : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8235489.2/ >> >> >> regarding the initialization of "name" - this is indeed for lines without a name entry - those lines exist in /proc/self/maps . >> I adjusted the initialization following your recommendation ( handle matches == 6). >> >> I also changed the unadorned "%s to one with an int-stringsize-parameter . > > ------------------------------------------------------------------------------ > src/hotspot/os/linux/os_linux.cpp > 2084 char name[4097]; // was PATH_MAX + 1 > > Please stay with the original, using PATH_MAX + 1. I'm assuming this > change was so in the string parsing argument you could use "%4096s" to > limit the amount of data read into name. That can still be done with > the size of name being based on PATH_MAX by using a variable field > width for the string conversion, e.g. "%*s" with an argument of > PATH_MAX for the ("*") field width, before the name argument. > > ------------------------------------------------------------------------------ Never mind. I forgot that the scanf family interprets ?%*? as ?assignment-suppression?. I think a better approach is to use a ?%n? specifier to capture the number of characters consumed thus far, in an int variable. Something like (untested) int name_index; const char* name; matches = sscanf(line, ?? %n?, ?, &name_index); if (matches != 6) continue; name = &line[name_index]; From robbin.ehn at oracle.com Tue Dec 10 07:44:28 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 10 Dec 2019 08:44:28 +0100 Subject: RFR(s): 8235410: Enable handshakes on Linux x86 (32-bit) In-Reply-To: References: Message-ID: <05d9a7af-2444-01bc-a5b0-05b84ef3b922@oracle.com> Hi Andrew, Could you have a look please? Thanks, Robbin On 2019-12-05 15:01, Robbin Ehn wrote: > Hi all, please review. > > The flag ThreadLocalHandshakes is going to be obsolete in JDK 14. > So we change the (unchangeable) default for Linux x86 to on/true. > > There is a follow-up which removes ThreadLocalHandshakes completely. > If the platform defines THREAD_LOCAL_POLL it will use handshakes. > When arm32 have implemented local polls, the plan is to remove the global poll > code paths. > > Last time I checked with some of the affected parties there was no objections. > > Built and sanity test. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8235410 > > Code below. > > Thanks, Robbin > > diff -r 636d71e53732 src/hotspot/cpu/x86/globals_x86.hpp > --- a/src/hotspot/cpu/x86/globals_x86.hpp??? Wed Dec 04 10:26:32 2019 +0100 > +++ b/src/hotspot/cpu/x86/globals_x86.hpp??? Thu Dec 05 14:13:57 2019 +0100 > @@ -89,12 +89,7 @@ > > ?define_pd_global(intx, InitArrayShortSize, 8*BytesPerLong); > > -#if defined(_LP64) || defined(_WINDOWS) > ?define_pd_global(bool, ThreadLocalHandshakes, true); > -#else > -// get_thread() is slow on linux 32 bit, therefore off by default > -define_pd_global(bool, ThreadLocalHandshakes, false); > -#endif > > ?#define ARCH_FLAGS(develop, \ > ??????????????????? product, \ From robbin.ehn at oracle.com Tue Dec 10 09:43:40 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 10 Dec 2019 10:43:40 +0100 Subject: RFR: 8220049: Remove -XX:-ThreadLocalHandshakes Message-ID: Hi all, please review, ThreadLocalHandshakes is obsolete in JDK 14 and this changeset removes the flag. All platforms except arm32 uses local poll. (after https://bugs.openjdk.java.net/browse/JDK-8235410, Enable handshakes on Linux x86 (32-bit) , which this patch goes on top of) When arm32 also have local poll we remove the global poll code. Changeset: http://cr.openjdk.java.net/~rehn/8220049/v1/webrev/ Issue: https://bugs.openjdk.java.net/browse/JDK-8220049 Passes t1-7, built arm32, aarch64, sparc-solaris, linux x86/x64, windows x64, osx, linux s390x, linux ppc64le (note that on x86 there is missn?ng include in src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp, --- a/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp Tue Dec 10 08:47:38 2019 +0100 +++ b/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp Tue Dec 10 10:24:14 2019 +0100 @@ -32,0 +33,1 @@ +#include "gc/shared/barrierSetAssembler_x86.hpp" ) Martin please have a look on aix changes. Thanks, Robbin From adinn at redhat.com Tue Dec 10 10:03:01 2019 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 10 Dec 2019 10:03:01 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <24fdb620-2b26-0647-524f-71a40b50d9d3@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <7765b06b-acf2-f11f-980e-33af4ca3934c@oracle.com> <14719046-4d92-d9ff-c59e-924e10212edf@oracle.com> <24fdb620-2b26-0647-524f-71a40b50d9d3@oracle.com> Message-ID: <432d8c39-58a0-4fde-fff0-e58d2a1e3049@redhat.com> Hi Maurizio, It's nice to see this squeezing into jdk14. On 09/12/2019 19:23, Maurizio Cimadamore wrote: > Another iteration: > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v4/ I eyeballed all the changes to the Buffer classes and saw no issues. Also, apologies for the mixed brace/nobrace else-if cascade in FileChannelImpl.java spotted by Roger which was my fault -- I should really have introduced braces uniformly across the whole if/else-if/else sequence. Also, I reran pmem tests to verify that your buffer changes don't disrupt the behaviour of sync mapped byte buffers and all is well. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill From maurizio.cimadamore at oracle.com Tue Dec 10 10:04:54 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 10 Dec 2019 10:04:54 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <432d8c39-58a0-4fde-fff0-e58d2a1e3049@redhat.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <7765b06b-acf2-f11f-980e-33af4ca3934c@oracle.com> <14719046-4d92-d9ff-c59e-924e10212edf@oracle.com> <24fdb620-2b26-0647-524f-71a40b50d9d3@oracle.com> <432d8c39-58a0-4fde-fff0-e58d2a1e3049@redhat.com> Message-ID: <9971153b-f07e-e6df-76e2-b01ca9a8fac3@oracle.com> Thanks for the extra testing! Maurizio On 10/12/2019 10:03, Andrew Dinn wrote: > Hi Maurizio, > > It's nice to see this squeezing into jdk14. > > On 09/12/2019 19:23, Maurizio Cimadamore wrote: >> Another iteration: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v4/ > I eyeballed all the changes to the Buffer classes and saw no issues. > Also, apologies for the mixed brace/nobrace else-if cascade in > FileChannelImpl.java spotted by Roger which was my fault -- I should > really have introduced braces uniformly across the whole if/else-if/else > sequence. > > Also, I reran pmem tests to verify that your buffer changes don't > disrupt the behaviour of sync mapped byte buffers and all is well. > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill > From david.holmes at oracle.com Tue Dec 10 10:08:13 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 10 Dec 2019 20:08:13 +1000 Subject: RFR: 8220049: Remove -XX:-ThreadLocalHandshakes In-Reply-To: References: Message-ID: <947010d5-c131-85d1-b939-1733066bc7e1@oracle.com> Hi Robbin, This all looks fine to me. Just one nit on terminology, can we change the bug synopsis to "Obsolete ThreadLocalHandshakes" please. Actual removal will be done in 15 en-masse for all expired flags. Thanks, David On 10/12/2019 7:43 pm, Robbin Ehn wrote: > Hi all, please review, > > ThreadLocalHandshakes is obsolete in JDK 14 and this changeset removes > the flag. > All platforms except arm32 uses local poll. > (after https://bugs.openjdk.java.net/browse/JDK-8235410, Enable > handshakes on Linux x86 (32-bit) > , which this patch goes on top of) > When arm32 also have local poll we remove the global poll code. > > Changeset: > http://cr.openjdk.java.net/~rehn/8220049/v1/webrev/ > Issue: > https://bugs.openjdk.java.net/browse/JDK-8220049 > > Passes t1-7, built arm32, aarch64, sparc-solaris, linux x86/x64, windows > x64, osx, linux s390x, linux ppc64le > > (note that on x86 there is missn?ng include in > src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp, > --- a/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp????? Tue Dec 10 > 08:47:38 2019 +0100 > +++ b/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp????? Tue Dec 10 > 10:24:14 2019 +0100 > @@ -32,0 +33,1 @@ > +#include "gc/shared/barrierSetAssembler_x86.hpp" > ) > > Martin please have a look on aix changes. > > Thanks, Robbin From per.liden at oracle.com Tue Dec 10 10:23:12 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 10 Dec 2019 11:23:12 +0100 Subject: RFR: 8220049: Remove -XX:-ThreadLocalHandshakes In-Reply-To: References: Message-ID: <100d4403-5424-7731-3d2a-0d684aa25440@oracle.com> Hi Robbin, Not a complete review, but the ZGC change looks good. Just one minor nit, the added include in zMark.cpp looks incorrectly sorted. @@ -44,10 +44,11 @@ #include "oops/oop.inline.hpp" #include "runtime/atomic.hpp" #include "runtime/handshake.hpp" #include "runtime/prefetch.inline.hpp" #include "runtime/thread.hpp" +#include "runtime/safepointMechanism.hpp" #include "utilities/align.hpp" #include "utilities/globalDefinitions.hpp" #include "utilities/ticks.hpp" cheers, /Per On 12/10/19 10:43 AM, Robbin Ehn wrote: > Hi all, please review, > > ThreadLocalHandshakes is obsolete in JDK 14 and this changeset removes > the flag. > All platforms except arm32 uses local poll. > (after https://bugs.openjdk.java.net/browse/JDK-8235410, Enable > handshakes on Linux x86 (32-bit) > , which this patch goes on top of) > When arm32 also have local poll we remove the global poll code. > > Changeset: > http://cr.openjdk.java.net/~rehn/8220049/v1/webrev/ > Issue: > https://bugs.openjdk.java.net/browse/JDK-8220049 > > Passes t1-7, built arm32, aarch64, sparc-solaris, linux x86/x64, windows > x64, osx, linux s390x, linux ppc64le > > (note that on x86 there is missn?ng include in > src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp, > --- a/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp????? Tue Dec 10 > 08:47:38 2019 +0100 > +++ b/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp????? Tue Dec 10 > 10:24:14 2019 +0100 > @@ -32,0 +33,1 @@ > +#include "gc/shared/barrierSetAssembler_x86.hpp" > ) > > Martin please have a look on aix changes. > > Thanks, Robbin From robbin.ehn at oracle.com Tue Dec 10 10:48:42 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 10 Dec 2019 11:48:42 +0100 Subject: RFR: 8220049: Remove -XX:-ThreadLocalHandshakes In-Reply-To: <100d4403-5424-7731-3d2a-0d684aa25440@oracle.com> References: <100d4403-5424-7731-3d2a-0d684aa25440@oracle.com> Message-ID: Hi Per, On 2019-12-10 11:23, Per Liden wrote: > Hi Robbin, > > Not a complete review, but the ZGC change looks good. Just one minor > nit, the added include in zMark.cpp looks incorrectly sorted. Yes, thanks, fixed. /Robbin > > @@ -44,10 +44,11 @@ > ?#include "oops/oop.inline.hpp" > ?#include "runtime/atomic.hpp" > ?#include "runtime/handshake.hpp" > ?#include "runtime/prefetch.inline.hpp" > ?#include "runtime/thread.hpp" > +#include "runtime/safepointMechanism.hpp" > ?#include "utilities/align.hpp" > ?#include "utilities/globalDefinitions.hpp" > ?#include "utilities/ticks.hpp" > > cheers, > /Per > > On 12/10/19 10:43 AM, Robbin Ehn wrote: >> Hi all, please review, >> >> ThreadLocalHandshakes is obsolete in JDK 14 and this changeset >> removes the flag. >> All platforms except arm32 uses local poll. >> (after https://bugs.openjdk.java.net/browse/JDK-8235410, Enable >> handshakes on Linux x86 (32-bit) >> , which this patch goes on top of) >> When arm32 also have local poll we remove the global poll code. >> >> Changeset: >> http://cr.openjdk.java.net/~rehn/8220049/v1/webrev/ >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8220049 >> >> Passes t1-7, built arm32, aarch64, sparc-solaris, linux x86/x64, >> windows x64, osx, linux s390x, linux ppc64le >> >> (note that on x86 there is missn?ng include in >> src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp, >> --- a/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp????? Tue Dec 10 >> 08:47:38 2019 +0100 >> +++ b/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp????? Tue Dec 10 >> 10:24:14 2019 +0100 >> @@ -32,0 +33,1 @@ >> +#include "gc/shared/barrierSetAssembler_x86.hpp" >> ) >> >> Martin please have a look on aix changes. >> >> Thanks, Robbin From robbin.ehn at oracle.com Tue Dec 10 10:52:57 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 10 Dec 2019 11:52:57 +0100 Subject: RFR: 8220049: Obsolete ThreadLocalHandshakes In-Reply-To: <947010d5-c131-85d1-b939-1733066bc7e1@oracle.com> References: <947010d5-c131-85d1-b939-1733066bc7e1@oracle.com> Message-ID: <68c3b8dc-fe79-2101-0769-1b0d4f76f44f@oracle.com> Hi David, On 2019-12-10 11:08, David Holmes wrote: > Hi Robbin, > > This all looks fine to me. > > Just one nit on terminology, can we change the bug synopsis to > "Obsolete ThreadLocalHandshakes" please. Actual removal will be done > in 15 en-masse for all expired flags. Yes, thanks, fixed. /Robbin > > Thanks, > David > > On 10/12/2019 7:43 pm, Robbin Ehn wrote: >> Hi all, please review, >> >> ThreadLocalHandshakes is obsolete in JDK 14 and this changeset >> removes the flag. >> All platforms except arm32 uses local poll. >> (after https://bugs.openjdk.java.net/browse/JDK-8235410, Enable >> handshakes on Linux x86 (32-bit) >> , which this patch goes on top of) >> When arm32 also have local poll we remove the global poll code. >> >> Changeset: >> http://cr.openjdk.java.net/~rehn/8220049/v1/webrev/ >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8220049 >> >> Passes t1-7, built arm32, aarch64, sparc-solaris, linux x86/x64, >> windows x64, osx, linux s390x, linux ppc64le >> >> (note that on x86 there is missn?ng include in >> src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp, >> --- a/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp????? Tue Dec 10 >> 08:47:38 2019 +0100 >> +++ b/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp????? Tue Dec 10 >> 10:24:14 2019 +0100 >> @@ -32,0 +33,1 @@ >> +#include "gc/shared/barrierSetAssembler_x86.hpp" >> ) >> >> Martin please have a look on aix changes. >> >> Thanks, Robbin From matthias.baesken at sap.com Tue Dec 10 11:22:40 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 10 Dec 2019 11:22:40 +0000 Subject: RFR [XS]: 8235489: handle return values of sscanf calls in hotspot In-Reply-To: <53B24471-1455-4D0E-B197-4A34E55DB0C7@oracle.com> References: <17DCFAEB-8E3C-4B94-9453-0AD85D3ADC94@oracle.com> <27E66F9B-6AA5-44BE-ABC5-EA1B6792E02F@oracle.com> <53B24471-1455-4D0E-B197-4A34E55DB0C7@oracle.com> Message-ID: Hi Kim, in the sscanf - call we read from array 'line' . So I think an easy solution for the potential overflow issue is to make 'name' (at least) as large as 'line' . Then we can safely use just %s . New webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8235489.3/ Best regards, Matthias > > > On Dec 9, 2019, at 3:30 PM, Kim Barrett wrote: > > > >> On Dec 9, 2019, at 6:22 AM, Baesken, Matthias > wrote: > >> > >> Hi Kim, new webrev : > >> > >> http://cr.openjdk.java.net/~mbaesken/webrevs/8235489.2/ > >> > >> > >> regarding the initialization of "name" - this is indeed for lines without a > name entry - those lines exist in /proc/self/maps . > >> I adjusted the initialization following your recommendation ( handle > matches == 6). > >> > >> I also changed the unadorned "%s to one with an int-stringsize-parameter > . > > > > ------------------------------------------------------------------------------ > > src/hotspot/os/linux/os_linux.cpp > > 2084 char name[4097]; // was PATH_MAX + 1 > > > > Please stay with the original, using PATH_MAX + 1. I'm assuming this > > change was so in the string parsing argument you could use "%4096s" to > > limit the amount of data read into name. That can still be done with > > the size of name being based on PATH_MAX by using a variable field > > width for the string conversion, e.g. "%*s" with an argument of > > PATH_MAX for the ("*") field width, before the name argument. > > > > ------------------------------------------------------------------------------ > > Never mind. I forgot that the scanf family interprets ?%*? as ?assignment- > suppression?. > > I think a better approach is to use a ?%n? specifier to capture the number of > characters > consumed thus far, in an int variable. Something like (untested) > > int name_index; > const char* name; > matches = sscanf(line, ?? %n?, ?, &name_index); > if (matches != 6) continue; > name = &line[name_index]; From stefan.karlsson at oracle.com Tue Dec 10 11:46:28 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 10 Dec 2019 12:46:28 +0100 Subject: RFR: ZGC: Add support for JFR leak profiler In-Reply-To: <9b66f11f-0e17-ffa4-9f31-f811dfcc693f@oracle.com> References: <27862ac1-e86b-406a-7819-85a13fbb24a3@oracle.com> <6e08f198-6555-8d24-0be2-b9b6b76874ce@oracle.com> <9b66f11f-0e17-ffa4-9f31-f811dfcc693f@oracle.com> Message-ID: <0f26235c-da38-c9f5-1a0d-ecb46be4efe0@oracle.com> Looks good. I think it would be nice to retain/add a comment that we don't follow code cache roots: https://cr.openjdk.java.net/~eosterlund/8235174/webrev.02/src/hotspot/share/jfr/leakprofiler/chains/rootSetClosure.cpp.udiff.html We could probably include the misaligned code cache oops by creating temporary, aligned "surrogate" oop*s in, for example, a growable array. Maybe create a bug/rfe for that? Thanks, StefanK On 2019-12-09 10:16, Erik ?sterlund wrote: > Hi, > > I renamed some functions from oops_do to weak_oops_do to follow our > naming conventions, and changed the filtering > in the (now) weak_oops_do function to check if the raw oop is not null, > instead of is_dead() which now has a new > meaning since last webrev. > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8235174/webrev.01_02/ > > Full: > http://cr.openjdk.java.net/~eosterlund/8235174/webrev.02/ > > This version has passed tier1-5. > > Thanks, > /Erik > > On 2019-12-04 17:16, Erik ?sterlund wrote: >> Hi Stefan, >> >> Thanks for the review. >> >> On 2019-12-04 14:55, Stefan Karlsson wrote: >>> Hi Erik, >>> >>> This looks good to me! >> >> Thanks! >> >>> I think it would be good for the code to limit the number of >>> UnifedOopRef::addr calls, since those are unsafe ways into the the >>> addr and _could_ be abused to go around the UnifiedOopRef >>> encapsulation. Maybe something for the future. >> >> Yeah. I wrote that function because I noticed that the address was >> used with literally every single >> address looking type I could imagine, in different places (uintptr_t, >> address, char*, oop*, void*, >> you name it). So I just wrote the addr function to be able to >> perform this change as mechanically >> as possible. >> >>> Just a small nit: >>> >>> https://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/src/hotspot/share/jfr/leakprofiler/checkpoint/eventEmitter.cpp.frames.html >>> >>> >>> Now that you've limited the use of object_addr, maybe you could also >>> limit its scope by moving >>> >>> 113?? const oop* object_addr = sample->object_addr(); >>> >>> to around: >>> >>> 123 edge = >>> edge_store->put(UnifiedOopRef::encode_in_native(object_addr)); >> >> Absolutely. >> >> Here is a new webrev with your proposed change, and with some changes >> from Markus (big thanks to Markus for looking at this in great detail): >> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.01/ >> >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00_01/ >> >> His changes mostly massage some types and order some includes. For >> example Markus prefers passing around UnifiedOopRef >> as non-const (now that it is a POD, it's like passing around an integer). >> >> The most important change though, was fixing a bug in my patch by >> making sure that ObjectSampler::weak_oops_do >> actually clears the oops that are dead. I thought it did, like the >> other weak_oops_do functions. Turns out >> that it has not been needed until now. The oops were not cleared, yet >> the ObjectSample is reused as some >> kind of cached object (presumably to call malloc less). The >> consequence was that when the ObjectSample gets subsequently reused >> with a dead oop in it, set_object() now using >> NativeAccess::oop_store >> with my patch crashes things, because the G1 barrier backend SATB >> enqueues what was there in memory before >> (which is dead garbage memory). >> >> Now one might argue that it's really silly of G1 barriers to SATB >> enqueue on non-strong stores (because the very >> snapshot that SATB tracks is the strong graph, which is the graph >> being marked concurrently, nothing else). But I >> would like to change that heuristic weirdness in a separate change. In >> this patch I am happy with the object simply >> being cleared in weak_oops_do, like it is in all other weak_oops_do >> functions. That fixes the issue. >> >> Before there was a separate _dead boolean to track that the >> ObjectSample is dead. Now that the oop is cleared >> by the GC when it dies, the boolean is no longer needed, because the >> is_dead() function can simply look at the oop. >> >> Thanks, >> /Erik >> >>> Thanks, >>> StefanK >>> >>> On 2019-12-02 11:09, erik.osterlund at oracle.com wrote: >>>> Hi, >>>> >>>> The JFR leak profiler is currently not supported on ZGC, because it >>>> has not been accessorized appropriately. >>>> This patch adds the required Access API sprinkling necessary, to >>>> enable this for ZGC. I did the following: >>>> >>>> 1) Renamed UnifiedOop to UnifiedOopRef, because it describes the >>>> reference to an oop, not the oop itself. >>>> 2) Changed UnifiedOopRef to be a POD, instead of an AllStatic that >>>> passes around its encoded data as >>>> ?? const oop* (even though it might be a tagged pointer to a narrow >>>> oop), improving type safty. >>>> 3) Added another tag bit to UnifiedOopRef, allowing it to keep track >>>> of whether the described location is >>>> ?? IN_NATIVE or IN_HEAP. The dereference function performs the >>>> appropriate NativeAccess/HeapAccess with the >>>> ?? AS_NO_KEEPALIVE decotator, which is fine because the leak >>>> profiler does not create any edges to oops >>>> ?? found during traversal. >>>> 4) Removed code blob oop iteration, because it can't be supported >>>> appropriately by the current UnifiedOopRef >>>> ?? mechanism (because oops are misaligned, and UnifiedOopRef >>>> requires tag bits - I believe this has never >>>> ?? worked properly) >>>> 6) Accessorized loads on strong roots to >>>> NativeAccess::oop_load. JFR leak profiler never >>>> ?? creates new references to oops that are visited, so it does not >>>> need to keep things alive >>>> 7) Explicitly told the oop iteration closures to not visit >>>> referents. The default referent strategy used >>>> ?? before is DO_DISCOVERY, which doesn't seem right. >>>> 8) Fixed a pre-existing issue where the array index reported in the >>>> leak chain was calculated incorrectly >>>> ?? as the byte offset from the array base. It should be scaled by >>>> the element length of the array. >>>> >>>> Ran some JFR leak profiler tests, and some ad-hoc leak profiling in >>>> various applications. Also checked that >>>> there were no observable perf regressions in the iterations, and >>>> there were not. >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8235174 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/ >>>> >>>> Thanks, >>>> /Erik >>> >> > From markus.gronlund at oracle.com Tue Dec 10 11:47:03 2019 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Tue, 10 Dec 2019 03:47:03 -0800 (PST) Subject: RFR: ZGC: Add support for JFR leak profiler In-Reply-To: <9b66f11f-0e17-ffa4-9f31-f811dfcc693f@oracle.com> References: <27862ac1-e86b-406a-7819-85a13fbb24a3@oracle.com> <6e08f198-6555-8d24-0be2-b9b6b76874ce@oracle.com> <9b66f11f-0e17-ffa4-9f31-f811dfcc693f@oracle.com> Message-ID: <0764b522-d09e-40b1-aaea-03ff975896cb@default> Hi again Erik, Looks good. Thanks Markus -----Original Message----- From: Erik ?sterlund Sent: den 9 december 2019 10:17 To: Stefan Karlsson ; hotspot-dev developers Cc: Markus Gronlund ; Erik Gahlin Subject: Re: RFR: ZGC: Add support for JFR leak profiler Hi, I renamed some functions from oops_do to weak_oops_do to follow our naming conventions, and changed the filtering in the (now) weak_oops_do function to check if the raw oop is not null, instead of is_dead() which now has a new meaning since last webrev. Incremental: http://cr.openjdk.java.net/~eosterlund/8235174/webrev.01_02/ Full: http://cr.openjdk.java.net/~eosterlund/8235174/webrev.02/ This version has passed tier1-5. Thanks, /Erik On 2019-12-04 17:16, Erik ?sterlund wrote: > Hi Stefan, > > Thanks for the review. > > On 2019-12-04 14:55, Stefan Karlsson wrote: >> Hi Erik, >> >> This looks good to me! > > Thanks! > >> I think it would be good for the code to limit the number of >> UnifedOopRef::addr calls, since those are unsafe ways into the the >> addr and _could_ be abused to go around the UnifiedOopRef >> encapsulation. Maybe something for the future. > > Yeah. I wrote that function because I noticed that the address was > used with literally every single address looking type I could imagine, > in different places (uintptr_t, address, char*, oop*, void*, you name > it). So I just wrote the addr function to be able to perform this > change as mechanically as possible. > >> Just a small nit: >> >> https://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/src/hotspot >> /share/jfr/leakprofiler/checkpoint/eventEmitter.cpp.frames.html >> >> >> Now that you've limited the use of object_addr, maybe you could also >> limit its scope by moving >> >> 113?? const oop* object_addr = sample->object_addr(); >> >> to around: >> >> 123 edge = >> edge_store->put(UnifiedOopRef::encode_in_native(object_addr)); > > Absolutely. > > Here is a new webrev with your proposed change, and with some changes > from Markus (big thanks to Markus for looking at this in great detail): > http://cr.openjdk.java.net/~eosterlund/8235174/webrev.01/ > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00_01/ > > His changes mostly massage some types and order some includes. For > example Markus prefers passing around UnifiedOopRef as non-const (now > that it is a POD, it's like passing around an integer). > > The most important change though, was fixing a bug in my patch by > making sure that ObjectSampler::weak_oops_do actually clears the oops > that are dead. I thought it did, like the other weak_oops_do > functions. Turns out that it has not been needed until now. The oops > were not cleared, yet the ObjectSample is reused as some kind of > cached object (presumably to call malloc less). The consequence was > that when the ObjectSample gets subsequently reused with a dead oop in > it, set_object() now using NativeAccess::oop_store > with my patch crashes things, because the G1 barrier backend SATB > enqueues what was there in memory before (which is dead garbage > memory). > > Now one might argue that it's really silly of G1 barriers to SATB > enqueue on non-strong stores (because the very snapshot that SATB > tracks is the strong graph, which is the graph being marked > concurrently, nothing else). But I would like to change that heuristic > weirdness in a separate change. In this patch I am happy with the > object simply being cleared in weak_oops_do, like it is in all other > weak_oops_do functions. That fixes the issue. > > Before there was a separate _dead boolean to track that the > ObjectSample is dead. Now that the oop is cleared by the GC when it > dies, the boolean is no longer needed, because the > is_dead() function can simply look at the oop. > > Thanks, > /Erik > >> Thanks, >> StefanK >> >> On 2019-12-02 11:09, erik.osterlund at oracle.com wrote: >>> Hi, >>> >>> The JFR leak profiler is currently not supported on ZGC, because it >>> has not been accessorized appropriately. >>> This patch adds the required Access API sprinkling necessary, to >>> enable this for ZGC. I did the following: >>> >>> 1) Renamed UnifiedOop to UnifiedOopRef, because it describes the >>> reference to an oop, not the oop itself. >>> 2) Changed UnifiedOopRef to be a POD, instead of an AllStatic that >>> passes around its encoded data as >>> ?? const oop* (even though it might be a tagged pointer to a narrow >>> oop), improving type safty. >>> 3) Added another tag bit to UnifiedOopRef, allowing it to keep track >>> of whether the described location is >>> ?? IN_NATIVE or IN_HEAP. The dereference function performs the >>> appropriate NativeAccess/HeapAccess with the >>> ?? AS_NO_KEEPALIVE decotator, which is fine because the leak >>> profiler does not create any edges to oops >>> ?? found during traversal. >>> 4) Removed code blob oop iteration, because it can't be supported >>> appropriately by the current UnifiedOopRef >>> ?? mechanism (because oops are misaligned, and UnifiedOopRef >>> requires tag bits - I believe this has never >>> ?? worked properly) >>> 6) Accessorized loads on strong roots to >>> NativeAccess::oop_load. JFR leak profiler never >>> ?? creates new references to oops that are visited, so it does not >>> need to keep things alive >>> 7) Explicitly told the oop iteration closures to not visit >>> referents. The default referent strategy used >>> ?? before is DO_DISCOVERY, which doesn't seem right. >>> 8) Fixed a pre-existing issue where the array index reported in the >>> leak chain was calculated incorrectly >>> ?? as the byte offset from the array base. It should be scaled by >>> the element length of the array. >>> >>> Ran some JFR leak profiler tests, and some ad-hoc leak profiling in >>> various applications. Also checked that there were no observable >>> perf regressions in the iterations, and there were not. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8235174 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/ >>> >>> Thanks, >>> /Erik >> > From martin.doerr at sap.com Tue Dec 10 12:10:02 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 10 Dec 2019 12:10:02 +0000 Subject: RFR: 8220049: Remove -XX:-ThreadLocalHandshakes In-Reply-To: References: Message-ID: Hi Robbin, thanks for taking care of AIX. Please also remove set_uses_thread_local_poll (see patch below). Other than that, AIX change looks good. Thanks, Martin diff -r 6f4ed629c1e1 src/hotspot/os/aix/safepointMechanism_aix.cpp --- a/src/hotspot/os/aix/safepointMechanism_aix.cpp Tue Dec 10 11:02:48 2019 +0100 +++ b/src/hotspot/os/aix/safepointMechanism_aix.cpp Tue Dec 10 13:01:38 2019 +0100 @@ -100,7 +100,6 @@ MemTracker::record_virtual_memory_reserve_and_commit(map_address, map_size, CALLER_PC, mtSafepoint); // Use same page for thread local handshakes without SIGTRAP - set_uses_thread_local_poll(); os::make_polling_page_unreadable(); intptr_t bad_page_val = reinterpret_cast(map_address), good_page_val = bad_page_val + os::vm_page_size(); > -----Original Message----- > From: Robbin Ehn > Sent: Dienstag, 10. Dezember 2019 10:44 > To: hotspot-dev developers ; Doerr, > Martin > Subject: RFR: 8220049: Remove -XX:-ThreadLocalHandshakes > > Hi all, please review, > > ThreadLocalHandshakes is obsolete in JDK 14 and this changeset removes the > flag. > All platforms except arm32 uses local poll. > (after https://bugs.openjdk.java.net/browse/JDK-8235410, Enable > handshakes on Linux x86 (32-bit) > , which this patch goes on top of) > When arm32 also have local poll we remove the global poll code. > > Changeset: > http://cr.openjdk.java.net/~rehn/8220049/v1/webrev/ > Issue: > https://bugs.openjdk.java.net/browse/JDK-8220049 > > Passes t1-7, built arm32, aarch64, sparc-solaris, linux x86/x64, windows x64, > osx, linux s390x, linux ppc64le > > (note that on x86 there is missn?ng include in > src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp, > --- a/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp Tue Dec 10 > 08:47:38 2019 +0100 > +++ b/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp Tue Dec 10 > 10:24:14 2019 +0100 > @@ -32,0 +33,1 @@ > +#include "gc/shared/barrierSetAssembler_x86.hpp" > ) > > Martin please have a look on aix changes. > > Thanks, Robbin From maurizio.cimadamore at oracle.com Tue Dec 10 12:51:21 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 10 Dec 2019 12:51:21 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <24fdb620-2b26-0647-524f-71a40b50d9d3@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <7765b06b-acf2-f11f-980e-33af4ca3934c@oracle.com> <14719046-4d92-d9ff-c59e-924e10212edf@oracle.com> <24fdb620-2b26-0647-524f-71a40b50d9d3@oracle.com> Message-ID: <515c1a7a-19d7-ba0a-fc93-024fe828bedb@oracle.com> And, another, small iteration http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v5 Delta compared to previous version (v4): http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v5_delta Javadoc updated in place: http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html Summary of changes: * Peter pointed out that the logic for checking overflow in MemoryScope::acquire was bogus, as once we had an overflow, the segment was rendered useless (as the count stayed in negative overflowed state) - separately, Paul pointed out that it would be better to roll in our atomic loops, and just use a VarHandle instead of atomic integer - I've rewritten MemoryScope quite a bit, to implement these changes; I think the result is much cleaner than before. Note that, in the new code, the boolean flag is gone, and we just use the int count for everything (thanks to Jorn for the nice observation). * Peter also pointer out that the isAlive() check is not thread-safe; I now have named the liveness-check-related methods explicitly (and added some javadoc), so that it's clear when they can be called safely * removed call to checkAlive from MemorySegment::acquire - MemoryScope;:acquire will check that anyway * fixed rough javadoc comment on MemoryLayout::withName * dropped non-overlapping restriction on MemoryAddress::copy (and fixed test accordingly) Maurizio On 09/12/2019 19:23, Maurizio Cimadamore wrote: > Another iteration: > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v4/ > > Delta of the changes since last version (v3) here: > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v4_delta/ > > The javadoc has been updated inline here: > > http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html > > > Summary of changes: > > * Improved javadoc regarding alignment and access modes in MemoryHandles > * related to the above, the memory access varhandle now checks for > low-level VM alignment to for access other than get/set (as happens > for regular byte buffer view var handle) > * fixed MemoryHandles, JavaLangInvokeAccess, VarHandles, > MethodHandleImpl to speak about alignmentMask, rather than alignment > (as per Roger comments) > * added some more javadoc to internal classes MemoryAddressProxy, > VarHandleMemoryAddressBase, JavaNioAccess.java and > JavaLangInvokeAccess.java? (as per Roger comments) > * fixed mising {code} block around alignment param A in > MemoryLayout::bitAlignment/bytesAlignment (as per offline comments > from Daniel) > * added positive test to TestMemoryCopy > > Cheers > Maurizio > > On 08/12/2019 01:44, Maurizio Cimadamore wrote: >> Another update: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v3/ >> >> And a delta of the changes since last version (v2) here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v3_delta/ >> >> The javadoc has been updated inline here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >> >> >> Summary of changes: >> >> * fixed some (cosmetic) issues in FileChannelmpl following offline >> discussion with Alan >> * changed tests group definition, so that now jdk_foreign is part of >> tier1_part3 >> * updated javadoc in various places to fix code samples (as per Paul >> comments) >> * updated javadoc in MemoryHandles to talk about access modes (as per >> Paul comments) - I added a new section on top, and then referred to >> in from all relevant places >> * some changes to use Objects.hash (as per Paul comments), and some >> minor refactor in the AddreddGenerator (to use switch expression) >> * tightened javadoc for MemoryAddress::copy - the method now is >> specified to throw IAE if the range of source/dest addresses overlap >> - I've fixed the impl and added a test (as per Raffaello comments) >> * tightened byte buffer VarHandle view - if the view is created from >> a byte buffer obtained from a segment (!!!) we should do a segment >> check - added tests >> >> >> And here's a list of the pending API-related issues. Since these are >> IMHO minor issues, I suggest we defer them to a followup minor >> cleanup, so that we can move ahead with finalization of the CSR with >> the current API. Here's the list: >> >> * Should MemoryAddress implement Comparable? I think the answer is >> "probably not" - an address doesn't have a 'length' so you can't >> really do a range comparison (maybe memory segments though?). For >> now, users can project to byte buffer and take it from there (since >> ByteBuffer <: Comparable) >> * should we have a? way to ask a Layout if its size is specified ? >> (Paul) >> * MemoryAddress::offset() vs. MemoryAddress::offset(long) -- not much >> distance between these two semantically different operations (Paul >> suggested using 'add' - which I'm not enthusiastic about because it's >> not adding two addresses - it's adding a long to an address...) >> * MemorySegment::isAccessible() -- as the A* word is overloaded, some >> other name should be found? >> * MemorySegment::acquire() -- replace with MemorySegment::fork? >> >> Thanks >> Maurizio >> >> On 06/12/2019 10:43, Maurizio Cimadamore wrote: >>> Hi, >>> here's an updated version of the patch: >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v2/ >>> >>> And a delta of the changes since last version here: >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v2_delta/ >>> >>> The javadoc has been updated inline here: >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >>> >>> >>> Summary of changes: >>> >>> * fixed spurious protected methods in AbstractLayout and subclasses >>> which leaked into API >>> * removed library_call.cpp changes, as these are being tracked >>> separately by Vlad >>> * compacted ILOAD code in AddressVarHandleGenerator >>> * replaced uses of ++i/--i with i + 1/i - 1 in MemoryScope >>> >>> I have made no changes to the *name* of the methods in the API. In >>> fact, I suggest we keep a list of the names we'd like to revisit, >>> and we address them all at once at the end of the review (once we're >>> happy with the contents). Here's a list of the current open naming >>> issues: >>> >>> * MemoryAddress::offset() vs. MemoryAddress::offset(long) -- not >>> much distance between these two semantically different operations >>> * MemorySegment::isAccessible() -- as the A* word is overloaded, >>> some other name should be found? >>> * MemorySegment::acquire() -- replace with MemorySegment::fork? >>> >>> Cheers >>> Maurizio >>> >>> >>> On 05/12/2019 21:04, Maurizio Cimadamore wrote: >>>> Hi, >>>> as part of the effort to upstream the changes related to JEP 370 >>>> (foreign memory access API) [1], I'd like to ask for a code review >>>> for the corresponding core-libs and hotspot changes: >>>> >>>> http://cr.openjdk.java.net/~mcimadamore/panama/8234049/ >>>> >>>> A javadoc for the memory access API is also available here: >>>> >>>> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >>>> >>>> >>>> Note: the patch passes tier1, tier2 and tier3 testing (**) >>>> >>>> >>>> Here is a brief summary of the changes in java.base and hotspot >>>> (the remaining new files are implementation classes and tests for >>>> the new API): >>>> >>>> * ciField.cpp - this one is to trust final fields in the foreign >>>> memory access implementation (otherwise VM doesn't trust memory >>>> segment bounds) >>>> >>>> * Modules.gmk - these changes are needed to require that the >>>> incubating module is loaded by the boot loader (otherwise the above >>>> changes are useless) >>>> >>>> * library_call.cpp - this one is a JIT compiler change to treat >>>> Thread.currentThread() as a well-known constant - which helps a lot >>>> in the confinement checks (thanks Vlad!) >>>> >>>> * various Buffer-related changes; these changes are needed because >>>> the memory access API allows a memory segment to be projected into >>>> a byte buffer, for interop reasons. As such, we need to insert a >>>> liveness check in the various get/put methods. Previously we had an >>>> implementation strategy where a BB was 'decorated' by a subclass >>>> called ScopedBuffer - but doing so required some changes to the BB >>>> API (e.g. making certain methods non-final, so that we could >>>> decorate them). Here I use an approach (which I have discussed with >>>> Alan) which doesn't require any public API changes, but needs to >>>> add a 'segment' field in Buffer - and then have constructors which >>>> keep track of this extra parameter. >>>> >>>> * FileChannel changes - these changes are required so that we can >>>> reuse the Unmapper class from the MemorySegment implementation, to >>>> deterministically deallocate a mapped memory segment. This should >>>> be a 'straight' refactoring, no change in behavior should occur >>>> here. Please double check. >>>> >>>> * VarHandles - this class now provides a factory to create memory >>>> access VarHandle - this is a bit tricky, since VarHandle cannot >>>> really be implemented outside java.base (e.g. VarForm is not >>>> public). So we do the usual trick where we define a bunch of proxy >>>> interfaces (see jdk/internal/access/foreign) have the classes in >>>> java.base refer to these - and then have the implementation classes >>>> of the memory access API implement these interfaces. >>>> >>>> * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we >>>> need to provide access to otherwise hidden functionalities - e.g. >>>> creating a new scoped buffer, or retrieving the properties of a >>>> memory access handle (e.g. offset, stride etc.), so that we can >>>> implement the memory access API in its own separate module >>>> >>>> * GensrcVarHandles.gmk - these changes are needed to enable the >>>> generation of the new memory address var handle implementations; >>>> there's an helper class per carrier (e.g. >>>> VarHandleMemoryAddressAsBytes, ...). At runtime, when a memory >>>> access var handle is needed, we dynamically spin a new VH >>>> implementation which makes use of the right carrier. We need to >>>> spin because the VH can have a variable number of access >>>> coordinates (e.g. depending on the dimensions of the array to be >>>> accessed). But, under the hood, all the generated implementation >>>> will be using the same helper class. >>>> >>>> * tests - we've tried to add fairly robust tests, often checking >>>> all possible permutations of carriers/dimensions etc. Because of >>>> that, the tests might not be the easiest to look at, but they have >>>> proven to be pretty effective at shaking out issues. >>>> >>>> I think that covers the main aspects of the implementation and >>>> where it differs from vanilla JDK. >>>> >>>> P.S. >>>> >>>> In the CSR review [2], Joe raised a fair point - which is >>>> MemoryAddress has both: >>>> >>>> offset(long) --> move address of given offset >>>> offset() --> return the offset of this address in its owning segment >>>> >>>> And this was considered suboptimal, given both methods use the same >>>> name but do something quite different (one is an accessor, another >>>> is a 'wither'). one obvious option is to rename the first to >>>> 'withOffset'. But I think that would lead to verbose code (since >>>> that is a very common operation). Other options are to: >>>> >>>> * rename offset(long) to move(long), advance(long), or something else >>>> * drop offset() - but then add an overload of >>>> MemorySegment::asSlice which takes an address instead of a plain >>>> long offset >>>> >>>> I'll leave the choice to the reviewers :-) >>>> >>>> >>>> >>>> Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, >>>> Stuart, Roger, Joe and the Panama team for the feedback provided so >>>> far, which helped to get the API in the shape it is today. >>>> >>>> Cheers >>>> Maurizio >>>> >>>> (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" >>>> - but that is unrelated to this patch, and it's a known failing test. >>>> >>>> [1] - https://openjdk.java.net/jeps/370 >>>> [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 >>>> From erik.osterlund at oracle.com Tue Dec 10 13:05:59 2019 From: erik.osterlund at oracle.com (erik.osterlund at oracle.com) Date: Tue, 10 Dec 2019 14:05:59 +0100 Subject: RFR: ZGC: Add support for JFR leak profiler In-Reply-To: <0f26235c-da38-c9f5-1a0d-ecb46be4efe0@oracle.com> References: <27862ac1-e86b-406a-7819-85a13fbb24a3@oracle.com> <6e08f198-6555-8d24-0be2-b9b6b76874ce@oracle.com> <9b66f11f-0e17-ffa4-9f31-f811dfcc693f@oracle.com> <0f26235c-da38-c9f5-1a0d-ecb46be4efe0@oracle.com> Message-ID: <3f513d8f-02a0-7feb-e4a3-8e105598e8cf@oracle.com> Hi Stefan, On 12/10/19 12:46 PM, Stefan Karlsson wrote: > Looks good. > > I think it would be nice to retain/add a comment that we don't follow > code cache roots: > https://cr.openjdk.java.net/~eosterlund/8235174/webrev.02/src/hotspot/share/jfr/leakprofiler/chains/rootSetClosure.cpp.udiff.html > I will add a comment before pushing. > We could probably include the misaligned code cache oops by creating > temporary, aligned "surrogate" oop*s in, for example, a growable > array. Maybe create a bug/rfe for that? Sure, I'll file an RFE. Thanks for the review! /Erik > Thanks, > StefanK > > On 2019-12-09 10:16, Erik ?sterlund wrote: >> Hi, >> >> I renamed some functions from oops_do to weak_oops_do to follow our >> naming conventions, and changed the filtering >> in the (now) weak_oops_do function to check if the raw oop is not >> null, instead of is_dead() which now has a new >> meaning since last webrev. >> >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.01_02/ >> >> Full: >> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.02/ >> >> This version has passed tier1-5. >> >> Thanks, >> /Erik >> >> On 2019-12-04 17:16, Erik ?sterlund wrote: >>> Hi Stefan, >>> >>> Thanks for the review. >>> >>> On 2019-12-04 14:55, Stefan Karlsson wrote: >>>> Hi Erik, >>>> >>>> This looks good to me! >>> >>> Thanks! >>> >>>> I think it would be good for the code to limit the number of >>>> UnifedOopRef::addr calls, since those are unsafe ways into the the >>>> addr and _could_ be abused to go around the UnifiedOopRef >>>> encapsulation. Maybe something for the future. >>> >>> Yeah. I wrote that function because I noticed that the address was >>> used with literally every single >>> address looking type I could imagine, in different places >>> (uintptr_t, address, char*, oop*, void*, >>> you name it). So I just wrote the addr function to be able to >>> perform this change as mechanically >>> as possible. >>> >>>> Just a small nit: >>>> >>>> https://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/src/hotspot/share/jfr/leakprofiler/checkpoint/eventEmitter.cpp.frames.html >>>> >>>> >>>> Now that you've limited the use of object_addr, maybe you could >>>> also limit its scope by moving >>>> >>>> 113?? const oop* object_addr = sample->object_addr(); >>>> >>>> to around: >>>> >>>> 123 edge = >>>> edge_store->put(UnifiedOopRef::encode_in_native(object_addr)); >>> >>> Absolutely. >>> >>> Here is a new webrev with your proposed change, and with some >>> changes from Markus (big thanks to Markus for looking at this in >>> great detail): >>> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.01/ >>> >>> Incremental: >>> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00_01/ >>> >>> His changes mostly massage some types and order some includes. For >>> example Markus prefers passing around UnifiedOopRef >>> as non-const (now that it is a POD, it's like passing around an >>> integer). >>> >>> The most important change though, was fixing a bug in my patch by >>> making sure that ObjectSampler::weak_oops_do >>> actually clears the oops that are dead. I thought it did, like the >>> other weak_oops_do functions. Turns out >>> that it has not been needed until now. The oops were not cleared, >>> yet the ObjectSample is reused as some >>> kind of cached object (presumably to call malloc less). The >>> consequence was that when the ObjectSample gets subsequently reused >>> with a dead oop in it, set_object() now using >>> NativeAccess::oop_store >>> with my patch crashes things, because the G1 barrier backend SATB >>> enqueues what was there in memory before >>> (which is dead garbage memory). >>> >>> Now one might argue that it's really silly of G1 barriers to SATB >>> enqueue on non-strong stores (because the very >>> snapshot that SATB tracks is the strong graph, which is the graph >>> being marked concurrently, nothing else). But I >>> would like to change that heuristic weirdness in a separate change. >>> In this patch I am happy with the object simply >>> being cleared in weak_oops_do, like it is in all other weak_oops_do >>> functions. That fixes the issue. >>> >>> Before there was a separate _dead boolean to track that the >>> ObjectSample is dead. Now that the oop is cleared >>> by the GC when it dies, the boolean is no longer needed, because the >>> is_dead() function can simply look at the oop. >>> >>> Thanks, >>> /Erik >>> >>>> Thanks, >>>> StefanK >>>> >>>> On 2019-12-02 11:09, erik.osterlund at oracle.com wrote: >>>>> Hi, >>>>> >>>>> The JFR leak profiler is currently not supported on ZGC, because >>>>> it has not been accessorized appropriately. >>>>> This patch adds the required Access API sprinkling necessary, to >>>>> enable this for ZGC. I did the following: >>>>> >>>>> 1) Renamed UnifiedOop to UnifiedOopRef, because it describes the >>>>> reference to an oop, not the oop itself. >>>>> 2) Changed UnifiedOopRef to be a POD, instead of an AllStatic that >>>>> passes around its encoded data as >>>>> ?? const oop* (even though it might be a tagged pointer to a >>>>> narrow oop), improving type safty. >>>>> 3) Added another tag bit to UnifiedOopRef, allowing it to keep >>>>> track of whether the described location is >>>>> ?? IN_NATIVE or IN_HEAP. The dereference function performs the >>>>> appropriate NativeAccess/HeapAccess with the >>>>> ?? AS_NO_KEEPALIVE decotator, which is fine because the leak >>>>> profiler does not create any edges to oops >>>>> ?? found during traversal. >>>>> 4) Removed code blob oop iteration, because it can't be supported >>>>> appropriately by the current UnifiedOopRef >>>>> ?? mechanism (because oops are misaligned, and UnifiedOopRef >>>>> requires tag bits - I believe this has never >>>>> ?? worked properly) >>>>> 6) Accessorized loads on strong roots to >>>>> NativeAccess::oop_load. JFR leak profiler never >>>>> ?? creates new references to oops that are visited, so it does not >>>>> need to keep things alive >>>>> 7) Explicitly told the oop iteration closures to not visit >>>>> referents. The default referent strategy used >>>>> ?? before is DO_DISCOVERY, which doesn't seem right. >>>>> 8) Fixed a pre-existing issue where the array index reported in >>>>> the leak chain was calculated incorrectly >>>>> ?? as the byte offset from the array base. It should be scaled by >>>>> the element length of the array. >>>>> >>>>> Ran some JFR leak profiler tests, and some ad-hoc leak profiling >>>>> in various applications. Also checked that >>>>> there were no observable perf regressions in the iterations, and >>>>> there were not. >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8235174 >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/ >>>>> >>>>> Thanks, >>>>> /Erik >>>> >>> >> From erik.osterlund at oracle.com Tue Dec 10 13:06:27 2019 From: erik.osterlund at oracle.com (erik.osterlund at oracle.com) Date: Tue, 10 Dec 2019 14:06:27 +0100 Subject: RFR: ZGC: Add support for JFR leak profiler In-Reply-To: <0764b522-d09e-40b1-aaea-03ff975896cb@default> References: <27862ac1-e86b-406a-7819-85a13fbb24a3@oracle.com> <6e08f198-6555-8d24-0be2-b9b6b76874ce@oracle.com> <9b66f11f-0e17-ffa4-9f31-f811dfcc693f@oracle.com> <0764b522-d09e-40b1-aaea-03ff975896cb@default> Message-ID: <188ca5df-4dcf-e262-c230-b6fe29382a3b@oracle.com> Hi Markus, Thanks for the review. /Erik On 12/10/19 12:47 PM, Markus Gronlund wrote: > Hi again Erik, > > Looks good. > > Thanks > Markus > > -----Original Message----- > From: Erik ?sterlund > Sent: den 9 december 2019 10:17 > To: Stefan Karlsson ; hotspot-dev developers > Cc: Markus Gronlund ; Erik Gahlin > Subject: Re: RFR: ZGC: Add support for JFR leak profiler > > Hi, > > I renamed some functions from oops_do to weak_oops_do to follow our naming conventions, and changed the filtering in the (now) weak_oops_do function to check if the raw oop is not null, instead of is_dead() which now has a new meaning since last webrev. > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8235174/webrev.01_02/ > > Full: > http://cr.openjdk.java.net/~eosterlund/8235174/webrev.02/ > > This version has passed tier1-5. > > Thanks, > /Erik > > On 2019-12-04 17:16, Erik ?sterlund wrote: >> Hi Stefan, >> >> Thanks for the review. >> >> On 2019-12-04 14:55, Stefan Karlsson wrote: >>> Hi Erik, >>> >>> This looks good to me! >> Thanks! >> >>> I think it would be good for the code to limit the number of >>> UnifedOopRef::addr calls, since those are unsafe ways into the the >>> addr and _could_ be abused to go around the UnifiedOopRef >>> encapsulation. Maybe something for the future. >> Yeah. I wrote that function because I noticed that the address was >> used with literally every single address looking type I could imagine, >> in different places (uintptr_t, address, char*, oop*, void*, you name >> it). So I just wrote the addr function to be able to perform this >> change as mechanically as possible. >> >>> Just a small nit: >>> >>> https://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/src/hotspot >>> /share/jfr/leakprofiler/checkpoint/eventEmitter.cpp.frames.html >>> >>> >>> Now that you've limited the use of object_addr, maybe you could also >>> limit its scope by moving >>> >>> 113?? const oop* object_addr = sample->object_addr(); >>> >>> to around: >>> >>> 123 edge = >>> edge_store->put(UnifiedOopRef::encode_in_native(object_addr)); >> Absolutely. >> >> Here is a new webrev with your proposed change, and with some changes >> from Markus (big thanks to Markus for looking at this in great detail): >> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.01/ >> >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00_01/ >> >> His changes mostly massage some types and order some includes. For >> example Markus prefers passing around UnifiedOopRef as non-const (now >> that it is a POD, it's like passing around an integer). >> >> The most important change though, was fixing a bug in my patch by >> making sure that ObjectSampler::weak_oops_do actually clears the oops >> that are dead. I thought it did, like the other weak_oops_do >> functions. Turns out that it has not been needed until now. The oops >> were not cleared, yet the ObjectSample is reused as some kind of >> cached object (presumably to call malloc less). The consequence was >> that when the ObjectSample gets subsequently reused with a dead oop in >> it, set_object() now using NativeAccess::oop_store >> with my patch crashes things, because the G1 barrier backend SATB >> enqueues what was there in memory before (which is dead garbage >> memory). >> >> Now one might argue that it's really silly of G1 barriers to SATB >> enqueue on non-strong stores (because the very snapshot that SATB >> tracks is the strong graph, which is the graph being marked >> concurrently, nothing else). But I would like to change that heuristic >> weirdness in a separate change. In this patch I am happy with the >> object simply being cleared in weak_oops_do, like it is in all other >> weak_oops_do functions. That fixes the issue. >> >> Before there was a separate _dead boolean to track that the >> ObjectSample is dead. Now that the oop is cleared by the GC when it >> dies, the boolean is no longer needed, because the >> is_dead() function can simply look at the oop. >> >> Thanks, >> /Erik >> >>> Thanks, >>> StefanK >>> >>> On 2019-12-02 11:09, erik.osterlund at oracle.com wrote: >>>> Hi, >>>> >>>> The JFR leak profiler is currently not supported on ZGC, because it >>>> has not been accessorized appropriately. >>>> This patch adds the required Access API sprinkling necessary, to >>>> enable this for ZGC. I did the following: >>>> >>>> 1) Renamed UnifiedOop to UnifiedOopRef, because it describes the >>>> reference to an oop, not the oop itself. >>>> 2) Changed UnifiedOopRef to be a POD, instead of an AllStatic that >>>> passes around its encoded data as >>>> ?? const oop* (even though it might be a tagged pointer to a narrow >>>> oop), improving type safty. >>>> 3) Added another tag bit to UnifiedOopRef, allowing it to keep track >>>> of whether the described location is >>>> ?? IN_NATIVE or IN_HEAP. The dereference function performs the >>>> appropriate NativeAccess/HeapAccess with the >>>> ?? AS_NO_KEEPALIVE decotator, which is fine because the leak >>>> profiler does not create any edges to oops >>>> ?? found during traversal. >>>> 4) Removed code blob oop iteration, because it can't be supported >>>> appropriately by the current UnifiedOopRef >>>> ?? mechanism (because oops are misaligned, and UnifiedOopRef >>>> requires tag bits - I believe this has never >>>> ?? worked properly) >>>> 6) Accessorized loads on strong roots to >>>> NativeAccess::oop_load. JFR leak profiler never >>>> ?? creates new references to oops that are visited, so it does not >>>> need to keep things alive >>>> 7) Explicitly told the oop iteration closures to not visit >>>> referents. The default referent strategy used >>>> ?? before is DO_DISCOVERY, which doesn't seem right. >>>> 8) Fixed a pre-existing issue where the array index reported in the >>>> leak chain was calculated incorrectly >>>> ?? as the byte offset from the array base. It should be scaled by >>>> the element length of the array. >>>> >>>> Ran some JFR leak profiler tests, and some ad-hoc leak profiling in >>>> various applications. Also checked that there were no observable >>>> perf regressions in the iterations, and there were not. >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8235174 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8235174/webrev.00/ >>>> >>>> Thanks, >>>> /Erik From robbin.ehn at oracle.com Tue Dec 10 13:32:35 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 10 Dec 2019 14:32:35 +0100 Subject: RFR: 8220049: Remove -XX:-ThreadLocalHandshakes In-Reply-To: References: Message-ID: Hi Martin, On 2019-12-10 13:10, Doerr, Martin wrote: > Hi Robbin, > > thanks for taking care of AIX. Please also remove set_uses_thread_local_poll (see patch below). > Other than that, AIX change looks good. Thanks, applied! /Robbin > > Thanks, > Martin > > > diff -r 6f4ed629c1e1 src/hotspot/os/aix/safepointMechanism_aix.cpp > --- a/src/hotspot/os/aix/safepointMechanism_aix.cpp Tue Dec 10 11:02:48 2019 +0100 > +++ b/src/hotspot/os/aix/safepointMechanism_aix.cpp Tue Dec 10 13:01:38 2019 +0100 > @@ -100,7 +100,6 @@ > MemTracker::record_virtual_memory_reserve_and_commit(map_address, map_size, CALLER_PC, mtSafepoint); > > // Use same page for thread local handshakes without SIGTRAP > - set_uses_thread_local_poll(); > os::make_polling_page_unreadable(); > intptr_t bad_page_val = reinterpret_cast(map_address), > good_page_val = bad_page_val + os::vm_page_size(); > > > >> -----Original Message----- >> From: Robbin Ehn >> Sent: Dienstag, 10. Dezember 2019 10:44 >> To: hotspot-dev developers ; Doerr, >> Martin >> Subject: RFR: 8220049: Remove -XX:-ThreadLocalHandshakes >> >> Hi all, please review, >> >> ThreadLocalHandshakes is obsolete in JDK 14 and this changeset removes the >> flag. >> All platforms except arm32 uses local poll. >> (after https://bugs.openjdk.java.net/browse/JDK-8235410, Enable >> handshakes on Linux x86 (32-bit) >> , which this patch goes on top of) >> When arm32 also have local poll we remove the global poll code. >> >> Changeset: >> http://cr.openjdk.java.net/~rehn/8220049/v1/webrev/ >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8220049 >> >> Passes t1-7, built arm32, aarch64, sparc-solaris, linux x86/x64, windows x64, >> osx, linux s390x, linux ppc64le >> >> (note that on x86 there is missn?ng include in >> src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp, >> --- a/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp Tue Dec 10 >> 08:47:38 2019 +0100 >> +++ b/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp Tue Dec 10 >> 10:24:14 2019 +0100 >> @@ -32,0 +33,1 @@ >> +#include "gc/shared/barrierSetAssembler_x86.hpp" >> ) >> >> Martin please have a look on aix changes. >> >> Thanks, Robbin From harold.seigel at oracle.com Tue Dec 10 15:25:35 2019 From: harold.seigel at oracle.com (Harold Seigel) Date: Tue, 10 Dec 2019 10:25:35 -0500 Subject: JDK 15 RFR of JDK 15 RFR: JDK-8225361 : Start of release updates for JDK 15 In-Reply-To: <8e831b6d-c4a9-59b7-3bbe-bb988336bd7d@oracle.com> References: <8e831b6d-c4a9-59b7-3bbe-bb988336bd7d@oracle.com> Message-ID: Hi Joe, Once you've included the recent JDK-14 fix for JDK-8235513 into your changes then I think the only classFileParser.cpp changes that are needed are: +#define JAVA_15_VERSION 59 Also, there's no need to change test test/hotspot/jtreg/runtime/records/oldRecordAttribute.jcod Thanks for changing the other .jcod test files. Thanks, Harold On 12/7/2019 1:35 AM, Joe Darcy wrote: > Hello, > > Please review the VM portions of the start-of-release updates for JDK 15: > > ??? http://cr.openjdk.java.net/~darcy/8225361.5/ > > Some additional adjustments to class file parsing for records may > occur and get merged in before the start of 15. > > Thanks, > > -Joe > From erik.osterlund at oracle.com Tue Dec 10 16:02:16 2019 From: erik.osterlund at oracle.com (erik.osterlund at oracle.com) Date: Tue, 10 Dec 2019 17:02:16 +0100 Subject: RFR: 8235669: G1: Stack walking API can expose AS_NO_KEEPALIVE oops Message-ID: <3b2a6c47-b958-5e41-d7c3-a4d25000b17e@oracle.com> Hi, When the stack is walked and e.g. locals are turned into StackValues, it might be that said local was made a constant oop by the JIT. In such cases, it is read from the nmethod using ON_STRONG_OOP_REF | AS_NO_KEEPALIVE. However, these oops need to be kept alive when concurrent marking is ongoing. While I haven't seen crashes obviously linked to this yet, I don't think we should wait until we do, because it certainly will eventually. Bug: https://bugs.openjdk.java.net/browse/JDK-8235669 Webrev: http://cr.openjdk.java.net/~eosterlund/8235669/webrev.00/ Thanks, /Erik From joe.darcy at oracle.com Tue Dec 10 18:26:45 2019 From: joe.darcy at oracle.com (Joe Darcy) Date: Tue, 10 Dec 2019 10:26:45 -0800 Subject: JDK 15 RFR of JDK 15 RFR: JDK-8225361 : Start of release updates for JDK 15 In-Reply-To: References: <8e831b6d-c4a9-59b7-3bbe-bb988336bd7d@oracle.com> Message-ID: <5d6a45b2-c87a-e55c-f4b4-3343ccba09a3@oracle.com> Hi Harold, On 12/10/2019 7:25 AM, Harold Seigel wrote: > > Hi Joe, > > Once you've included the recent JDK-14 fix for JDK-8235513 > into your changes > then I think the only classFileParser.cpp changes that are needed are: > > +#define JAVA_15_VERSION 59 > Got it; I've merged in your changes into my working repo and I'll submit another test job with the JAVA_15_VERSION definition to verify all is well. > Also, there's no need to change test > test/hotspot/jtreg/runtime/records/oldRecordAttribute.jcod > Right; it is not strictly necessary to change the version number in that file, but as I was changing all the other version of the jcod files, I thought it was a slightly better as a test to use the immediately prior major version in the file. > Thanks for changing the other .jcod test files. > For future work, it would be helpful to have the ability to have the effect of filling in the version number as a variable. Thanks, -Joe > Thanks, Harold > > On 12/7/2019 1:35 AM, Joe Darcy wrote: >> Hello, >> >> Please review the VM portions of the start-of-release updates for JDK >> 15: >> >> http://cr.openjdk.java.net/~darcy/8225361.5/ >> >> Some additional adjustments to class file parsing for records may >> occur and get merged in before the start of 15. >> >> Thanks, >> >> -Joe >> From paul.sandoz at oracle.com Tue Dec 10 18:45:35 2019 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 10 Dec 2019 10:45:35 -0800 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <515c1a7a-19d7-ba0a-fc93-024fe828bedb@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <7765b06b-acf2-f11f-980e-33af4ca3934c@oracle.com> <14719046-4d92-d9ff-c59e-924e10212edf@oracle.com> <24fdb620-2b26-0647-524f-71a40b50d9d3@oracle.com> <515c1a7a-19d7-ba0a-fc93-024fe828bedb@oracle.com> Message-ID: <12784CB4-7357-4F19-B307-1AE8354D40F7@oracle.com> > On Dec 10, 2019, at 4:51 AM, Maurizio Cimadamore wrote: > > And, another, small iteration > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v5 > > Delta compared to previous version (v4): > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v5_delta > > Javadoc updated in place: > > http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html > > > Summary of changes: > > * Peter pointed out that the logic for checking overflow in MemoryScope::acquire was bogus, as once we had an overflow, the segment was rendered useless (as the count stayed in negative overflowed state) - separately, Paul pointed out that it would be better to roll in our atomic loops, and just use a VarHandle instead of atomic integer - I've rewritten MemoryScope quite a bit, to implement these changes; I think the result is much cleaner than before. Note that, in the new code, the boolean flag is gone, and we just use the int count for everything (thanks to Jorn for the nice observation). MemoryScope changes look good. In j.u.concurrent we use ExceptionInInitializerError in the static blocks when there is an error looking up the VH, FWIW close can also fail because it has already been closed but the exception message does not distinguish between the two states (active or already closed). Paul. > > * Peter also pointer out that the isAlive() check is not thread-safe; I now have named the liveness-check-related methods explicitly (and added some javadoc), so that it's clear when they can be called safely > > * removed call to checkAlive from MemorySegment::acquire - MemoryScope;:acquire will check that anyway > > * fixed rough javadoc comment on MemoryLayout::withName > > * dropped non-overlapping restriction on MemoryAddress::copy (and fixed test accordingly) > > Maurizio > > > On 09/12/2019 19:23, Maurizio Cimadamore wrote: >> Another iteration: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v4/ >> >> Delta of the changes since last version (v3) here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v4_delta/ >> >> The javadoc has been updated inline here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >> >> Summary of changes: >> >> * Improved javadoc regarding alignment and access modes in MemoryHandles >> * related to the above, the memory access varhandle now checks for low-level VM alignment to for access other than get/set (as happens for regular byte buffer view var handle) >> * fixed MemoryHandles, JavaLangInvokeAccess, VarHandles, MethodHandleImpl to speak about alignmentMask, rather than alignment (as per Roger comments) >> * added some more javadoc to internal classes MemoryAddressProxy, VarHandleMemoryAddressBase, JavaNioAccess.java and JavaLangInvokeAccess.java (as per Roger comments) >> * fixed mising {code} block around alignment param A in MemoryLayout::bitAlignment/bytesAlignment (as per offline comments from Daniel) >> * added positive test to TestMemoryCopy >> >> Cheers >> Maurizio >> >> On 08/12/2019 01:44, Maurizio Cimadamore wrote: >>> Another update: >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v3/ >>> >>> And a delta of the changes since last version (v2) here: >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v3_delta/ >>> >>> The javadoc has been updated inline here: >>> >>> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >>> >>> Summary of changes: >>> >>> * fixed some (cosmetic) issues in FileChannelmpl following offline discussion with Alan >>> * changed tests group definition, so that now jdk_foreign is part of tier1_part3 >>> * updated javadoc in various places to fix code samples (as per Paul comments) >>> * updated javadoc in MemoryHandles to talk about access modes (as per Paul comments) - I added a new section on top, and then referred to in from all relevant places >>> * some changes to use Objects.hash (as per Paul comments), and some minor refactor in the AddreddGenerator (to use switch expression) >>> * tightened javadoc for MemoryAddress::copy - the method now is specified to throw IAE if the range of source/dest addresses overlap - I've fixed the impl and added a test (as per Raffaello comments) >>> * tightened byte buffer VarHandle view - if the view is created from a byte buffer obtained from a segment (!!!) we should do a segment check - added tests >>> >>> >>> And here's a list of the pending API-related issues. Since these are IMHO minor issues, I suggest we defer them to a followup minor cleanup, so that we can move ahead with finalization of the CSR with the current API. Here's the list: >>> >>> * Should MemoryAddress implement Comparable? I think the answer is "probably not" - an address doesn't have a 'length' so you can't really do a range comparison (maybe memory segments though?). For now, users can project to byte buffer and take it from there (since ByteBuffer <: Comparable) >>> * should we have a way to ask a Layout if its size is specified ? (Paul) >>> * MemoryAddress::offset() vs. MemoryAddress::offset(long) -- not much distance between these two semantically different operations (Paul suggested using 'add' - which I'm not enthusiastic about because it's not adding two addresses - it's adding a long to an address...) >>> * MemorySegment::isAccessible() -- as the A* word is overloaded, some other name should be found? >>> * MemorySegment::acquire() -- replace with MemorySegment::fork? >>> >>> Thanks >>> Maurizio >>> >>> On 06/12/2019 10:43, Maurizio Cimadamore wrote: >>>> Hi, >>>> here's an updated version of the patch: >>>> >>>> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v2/ >>>> >>>> And a delta of the changes since last version here: >>>> >>>> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v2_delta/ >>>> >>>> The javadoc has been updated inline here: >>>> >>>> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >>>> >>>> Summary of changes: >>>> >>>> * fixed spurious protected methods in AbstractLayout and subclasses which leaked into API >>>> * removed library_call.cpp changes, as these are being tracked separately by Vlad >>>> * compacted ILOAD code in AddressVarHandleGenerator >>>> * replaced uses of ++i/--i with i + 1/i - 1 in MemoryScope >>>> >>>> I have made no changes to the *name* of the methods in the API. In fact, I suggest we keep a list of the names we'd like to revisit, and we address them all at once at the end of the review (once we're happy with the contents). Here's a list of the current open naming issues: >>>> >>>> * MemoryAddress::offset() vs. MemoryAddress::offset(long) -- not much distance between these two semantically different operations >>>> * MemorySegment::isAccessible() -- as the A* word is overloaded, some other name should be found? >>>> * MemorySegment::acquire() -- replace with MemorySegment::fork? >>>> >>>> Cheers >>>> Maurizio >>>> >>>> >>>> On 05/12/2019 21:04, Maurizio Cimadamore wrote: >>>>> Hi, >>>>> as part of the effort to upstream the changes related to JEP 370 (foreign memory access API) [1], I'd like to ask for a code review for the corresponding core-libs and hotspot changes: >>>>> >>>>> http://cr.openjdk.java.net/~mcimadamore/panama/8234049/ >>>>> >>>>> A javadoc for the memory access API is also available here: >>>>> >>>>> http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html >>>>> >>>>> Note: the patch passes tier1, tier2 and tier3 testing (**) >>>>> >>>>> >>>>> Here is a brief summary of the changes in java.base and hotspot (the remaining new files are implementation classes and tests for the new API): >>>>> >>>>> * ciField.cpp - this one is to trust final fields in the foreign memory access implementation (otherwise VM doesn't trust memory segment bounds) >>>>> >>>>> * Modules.gmk - these changes are needed to require that the incubating module is loaded by the boot loader (otherwise the above changes are useless) >>>>> >>>>> * library_call.cpp - this one is a JIT compiler change to treat Thread.currentThread() as a well-known constant - which helps a lot in the confinement checks (thanks Vlad!) >>>>> >>>>> * various Buffer-related changes; these changes are needed because the memory access API allows a memory segment to be projected into a byte buffer, for interop reasons. As such, we need to insert a liveness check in the various get/put methods. Previously we had an implementation strategy where a BB was 'decorated' by a subclass called ScopedBuffer - but doing so required some changes to the BB API (e.g. making certain methods non-final, so that we could decorate them). Here I use an approach (which I have discussed with Alan) which doesn't require any public API changes, but needs to add a 'segment' field in Buffer - and then have constructors which keep track of this extra parameter. >>>>> >>>>> * FileChannel changes - these changes are required so that we can reuse the Unmapper class from the MemorySegment implementation, to deterministically deallocate a mapped memory segment. This should be a 'straight' refactoring, no change in behavior should occur here. Please double check. >>>>> >>>>> * VarHandles - this class now provides a factory to create memory access VarHandle - this is a bit tricky, since VarHandle cannot really be implemented outside java.base (e.g. VarForm is not public). So we do the usual trick where we define a bunch of proxy interfaces (see jdk/internal/access/foreign) have the classes in java.base refer to these - and then have the implementation classes of the memory access API implement these interfaces. >>>>> >>>>> * JavaNIOAccess, JavaLangInvokeAccess - because of the above, we need to provide access to otherwise hidden functionalities - e.g. creating a new scoped buffer, or retrieving the properties of a memory access handle (e.g. offset, stride etc.), so that we can implement the memory access API in its own separate module >>>>> >>>>> * GensrcVarHandles.gmk - these changes are needed to enable the generation of the new memory address var handle implementations; there's an helper class per carrier (e.g. VarHandleMemoryAddressAsBytes, ...). At runtime, when a memory access var handle is needed, we dynamically spin a new VH implementation which makes use of the right carrier. We need to spin because the VH can have a variable number of access coordinates (e.g. depending on the dimensions of the array to be accessed). But, under the hood, all the generated implementation will be using the same helper class. >>>>> >>>>> * tests - we've tried to add fairly robust tests, often checking all possible permutations of carriers/dimensions etc. Because of that, the tests might not be the easiest to look at, but they have proven to be pretty effective at shaking out issues. >>>>> >>>>> I think that covers the main aspects of the implementation and where it differs from vanilla JDK. >>>>> >>>>> P.S. >>>>> >>>>> In the CSR review [2], Joe raised a fair point - which is MemoryAddress has both: >>>>> >>>>> offset(long) --> move address of given offset >>>>> offset() --> return the offset of this address in its owning segment >>>>> >>>>> And this was considered suboptimal, given both methods use the same name but do something quite different (one is an accessor, another is a 'wither'). one obvious option is to rename the first to 'withOffset'. But I think that would lead to verbose code (since that is a very common operation). Other options are to: >>>>> >>>>> * rename offset(long) to move(long), advance(long), or something else >>>>> * drop offset() - but then add an overload of MemorySegment::asSlice which takes an address instead of a plain long offset >>>>> >>>>> I'll leave the choice to the reviewers :-) >>>>> >>>>> >>>>> >>>>> Finally, I'd like to thank Mark, Brian, John, Alan, Paul, Vlad, Stuart, Roger, Joe and the Panama team for the feedback provided so far, which helped to get the API in the shape it is today. >>>>> >>>>> Cheers >>>>> Maurizio >>>>> >>>>> (**) There is one failure, for "java/util/TimeZone/Bug6329116.java" - but that is unrelated to this patch, and it's a known failing test. >>>>> >>>>> [1] - https://openjdk.java.net/jeps/370 >>>>> [2] - https://bugs.openjdk.java.net/browse/JDK-8234050 >>>>> From daniel.daugherty at oracle.com Tue Dec 10 20:19:21 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 10 Dec 2019 15:19:21 -0500 Subject: RFR: 8220049: Remove -XX:-ThreadLocalHandshakes In-Reply-To: References: Message-ID: <6c2aa4f9-1c91-efee-71a8-f60acd47d005@oracle.com> On 12/10/19 4:43 AM, Robbin Ehn wrote: > Hi all, please review, > > ThreadLocalHandshakes is obsolete in JDK 14 and this changeset removes > the flag. > All platforms except arm32 uses local poll. > (after https://bugs.openjdk.java.net/browse/JDK-8235410, Enable > handshakes on Linux x86 (32-bit) > , which this patch goes on top of) > When arm32 also have local poll we remove the global poll code. > > Changeset: > http://cr.openjdk.java.net/~rehn/8220049/v1/webrev/ src/hotspot/cpu/aarch64/globals_aarch64.hpp src/hotspot/cpu/arm/globals_arm.hpp src/hotspot/cpu/ppc/globals_ppc.hpp src/hotspot/cpu/s390/globals_s390.hpp src/hotspot/cpu/sparc/globals_sparc.hpp src/hotspot/cpu/x86/globals_x86.hpp src/hotspot/cpu/zero/globals_zero.hpp ??? No comments on the above flag deletions. src/hotspot/os/aix/safepointMechanism_aix.cpp ??? nit - Needs a copyright year update. src/hotspot/share/aot/aotCodeHeap.cpp ??? nit - Needs a copyright year update. src/hotspot/share/aot/aotCodeHeap.hpp ??? No comments. src/hotspot/share/gc/z/zMark.cpp ??? No comments. src/hotspot/share/runtime/arguments.cpp ??? No comments. src/hotspot/share/runtime/biasedLocking.cpp ? L38: #include "runtime/handshake.hpp" ? L39: #include "runtime/task.hpp" ? L40: #include "runtime/threadSMR.hpp" ? L41: #include "runtime/safepointMechanism.hpp" ????? nit - New include is out of order; should be after handshake.hpp. ????? No content comments. src/hotspot/share/runtime/flags/jvmFlagConstraintsRuntime.cpp ??? nit - Needs a copyright year update. src/hotspot/share/runtime/flags/jvmFlagConstraintsRuntime.hpp ??? No comments. src/hotspot/share/runtime/globals.hpp ??? No comments. src/hotspot/share/runtime/handshake.cpp ??? No comments. src/hotspot/share/runtime/safepoint.cpp ??? No comments. src/hotspot/share/runtime/safepointMechanism.cpp ??? No comments. src/hotspot/share/runtime/safepointMechanism.hpp ??? No comments. src/hotspot/share/runtime/sweeper.cpp ??? No comments. src/jdk.aot/share/classes/jdk.tools.jaotc.binformat/src/jdk/tools/jaotc/binformat/BinaryContainer.java src/jdk.aot/share/classes/jdk.tools.jaotc/src/jdk/tools/jaotc/MarkProcessor.java ??? nit - Needs a copyright year update (MarkProcessor.java). ??? No content comments on the above AOT changes. src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.aarch64/src/org/graalvm/compiler/hotspot/aarch64/AArch64HotSpotSafepointOp.java src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.amd64/src/org/graalvm/compiler/hotspot/amd64/AMD64HotSpotSafepointOp.java src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.sparc.test/src/org/graalvm/compiler/hotspot/sparc/test/SPARCAllocatorTest.java src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.sparc/src/org/graalvm/compiler/hotspot/sparc/SPARCHotSpotSafepointOp.java src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfig.java ??? nit - Needs a copyright year update (most of the Graal files). ??? No comments on the above Graal changes. Thumbs up! I only have nit comments above and I don't need to see a new webrev if you fix that. Dan > Issue: > https://bugs.openjdk.java.net/browse/JDK-8220049 > > Passes t1-7, built arm32, aarch64, sparc-solaris, linux x86/x64, > windows x64, osx, linux s390x, linux ppc64le > > (note that on x86 there is missn?ng include in > src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp, > --- a/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp????? Tue Dec 10 > 08:47:38 2019 +0100 > +++ b/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp????? Tue Dec 10 > 10:24:14 2019 +0100 > @@ -32,0 +33,1 @@ > +#include "gc/shared/barrierSetAssembler_x86.hpp" > ) > > Martin please have a look on aix changes. > > Thanks, Robbin From kim.barrett at oracle.com Tue Dec 10 20:27:27 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 10 Dec 2019 15:27:27 -0500 Subject: RFR [XS]: 8235489: handle return values of sscanf calls in hotspot In-Reply-To: References: <17DCFAEB-8E3C-4B94-9453-0AD85D3ADC94@oracle.com> <27E66F9B-6AA5-44BE-ABC5-EA1B6792E02F@oracle.com> <53B24471-1455-4D0E-B197-4A34E55DB0C7@oracle.com> Message-ID: <29127B7D-BBB5-4A3E-A6F7-84130EF90CE0@oracle.com> > On Dec 10, 2019, at 6:22 AM, Baesken, Matthias wrote: > > Hi Kim, in the sscanf - call we read from array 'line' . > So I think an easy solution for the potential overflow issue is to make 'name' (at least) as large as 'line' . > Then we can safely use just %s . > > New webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235489.3/ I have a mild preference for the "%n" approach, but this alternative works too, so okay. Just one thing; please use "char name[sizeof(line)]" rather than copying the size expression. Other than that, looks good. From kim.barrett at oracle.com Tue Dec 10 22:23:00 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 10 Dec 2019 17:23:00 -0500 Subject: RFR: 8235551: BitMap::count_one_bits should use population_count In-Reply-To: References: Message-ID: <00C9FA92-418E-4EE7-985F-86DB64FBFBDD@oracle.com> > On Dec 8, 2019, at 4:54 PM, Claes Redestad wrote: > > Hi, > > with some adjustments, BitMap::count_one_bits can be rewritten to use > the population_count utility method introduced in JDK-8217519 > > Bug: https://bugs.openjdk.java.net/browse/JDK-8235551 > Webrev: http://cr.openjdk.java.net/~redestad/8235551/open.00/ > > This generalizes population_count to accept any unsigned integer, to > accommodate that BitMap will use 32- or 64-bit integers depending on > platform. Looks good. This comment is a bit misleading: src/hotspot/share/utilities/population_count.hpp 37 // Adapted from Hacker's Delight, 2nd Edition, Figure 5-2. The implementation being used is a combination of that in Figure 5-2 and another version a bit later in the associated text. I suggest changing the comment to say Adapted From Hacker's Delight, 2nd Edition, Figure 5-2 and the text that follows. That way, someone looking up the reference won't just go "Wait, what? That isn't what's in 5-2." There are also a couple of pre-existing issues with BitMap::count_one_bits which are not being addressed here: https://bugs.openjdk.java.net/browse/JDK-8235714 https://bugs.openjdk.java.net/browse/JDK-8235715 From maurizio.cimadamore at oracle.com Wed Dec 11 00:13:39 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 11 Dec 2019 00:13:39 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <12784CB4-7357-4F19-B307-1AE8354D40F7@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <7765b06b-acf2-f11f-980e-33af4ca3934c@oracle.com> <14719046-4d92-d9ff-c59e-924e10212edf@oracle.com> <24fdb620-2b26-0647-524f-71a40b50d9d3@oracle.com> <515c1a7a-19d7-ba0a-fc93-024fe828bedb@oracle.com> <12784CB4-7357-4F19-B307-1AE8354D40F7@oracle.com> Message-ID: <523d08f4-5604-3a2d-7245-733a5fdc028b@oracle.com> On 10/12/2019 18:45, Paul Sandoz wrote: > MemoryScope changes look good. > > In j.u.concurrent we use ExceptionInInitializerError in the static blocks when there is an error looking up the VH, > > FWIW close can also fail because it has already been closed but the exception message does not distinguish between the two states (active or already closed). > > Paul. > Hi Paul, would something like this be satisfactory? diff -r 50ef46599c56 src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/MemoryScope.java --- a/src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/MemoryScope.java Tue Dec 10 16:28:03 2019 +0000 +++ b/src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/MemoryScope.java Wed Dec 11 00:12:20 2019 +0000 @@ -48,7 +48,7 @@ ???????? try { ???????????? COUNT_HANDLE = MethodHandles.lookup().findVarHandle(MemoryScope.class, "activeCount", int.class); ???????? } catch (Throwable ex) { -??????????? throw new IllegalStateException(ex); +??????????? throw new ExceptionInInitializerError(ex); ???????? } ???? } @@ -107,6 +107,9 @@ ???? void close() { ???????? if (!COUNT_HANDLE.compareAndSet(this, UNACQUIRED, CLOSED)) { +??????????? //first check if already closed... +??????????? checkAliveConfined(); +??????????? //...if not, then we have acquired views that are still active ???????????? throw new IllegalStateException("Cannot close a segment that has active acquired views"); ???????? } ???????? if (cleanupAction != null) { If you need a full webrev please let me know. Thanks Maurizio From matthias.baesken at sap.com Wed Dec 11 08:05:54 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 11 Dec 2019 08:05:54 +0000 Subject: RFR [XS]: 8235671: enhance print_rlimit_info in os_posix Message-ID: Hello, please review this small change . It adds 2 rlimit infos to os::Posix::print_rlimit_info . One is cross posix : RLIMIT_CPU http://man7.org/linux/man-pages/man2/getrlimit.2.html RLIMIT_CPU This is a limit, in seconds, on the amount of CPU time that the process can consume. When the process reaches the soft limit, it is sent a SIGXCPU signal. ... One is AIX-only : RLIMIT_THREADS Manpage : RLIMIT_THREADS The maximum number of threads each process can create. This limit is enforced by the kernel and the pthread library. Bug/web : https://bugs.openjdk.java.net/browse/JDK-8235671 http://cr.openjdk.java.net/~mbaesken/webrevs/8235671.0/ Thanks, Matthias From matthias.baesken at sap.com Wed Dec 11 09:21:39 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 11 Dec 2019 09:21:39 +0000 Subject: RFR [XS]: 8235489: handle return values of sscanf calls in hotspot In-Reply-To: <29127B7D-BBB5-4A3E-A6F7-84130EF90CE0@oracle.com> References: <17DCFAEB-8E3C-4B94-9453-0AD85D3ADC94@oracle.com> <27E66F9B-6AA5-44BE-ABC5-EA1B6792E02F@oracle.com> <53B24471-1455-4D0E-B197-4A34E55DB0C7@oracle.com> <29127B7D-BBB5-4A3E-A6F7-84130EF90CE0@oracle.com> Message-ID: > > please use "char name[sizeof(line)]" rather than copying the size expression. > Hi Kim , new webrev using sizeof(line) : http://cr.openjdk.java.net/~mbaesken/webrevs/8235489.4/ In case I hear no objections, I will push this as XS . Best regards, Matthias > > > On Dec 10, 2019, at 6:22 AM, Baesken, Matthias > wrote: > > > > Hi Kim, in the sscanf - call we read from array 'line' . > > So I think an easy solution for the potential overflow issue is to make > 'name' (at least) as large as 'line' . > > Then we can safely use just %s . > > > > New webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235489.3/ > > I have a mild preference for the "%n" approach, but this alternative > works too, so okay. Just one thing; please use "char name[sizeof(line)]" > rather than copying the size expression. > > Other than that, looks good. From robbin.ehn at oracle.com Wed Dec 11 10:11:16 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 11 Dec 2019 11:11:16 +0100 Subject: RFR: 8220049: Remove -XX:-ThreadLocalHandshakes In-Reply-To: <6c2aa4f9-1c91-efee-71a8-f60acd47d005@oracle.com> References: <6c2aa4f9-1c91-efee-71a8-f60acd47d005@oracle.com> Message-ID: <07406cde-7fcc-3674-55f5-953e95d7041d@oracle.com> Thanks Dan. Fixed below! /Robbin On 2019-12-10 21:19, Daniel D. Daugherty wrote: > On 12/10/19 4:43 AM, Robbin Ehn wrote: >> Hi all, please review, >> >> ThreadLocalHandshakes is obsolete in JDK 14 and this changeset >> removes the flag. >> All platforms except arm32 uses local poll. >> (after https://bugs.openjdk.java.net/browse/JDK-8235410, Enable >> handshakes on Linux x86 (32-bit) >> , which this patch goes on top of) >> When arm32 also have local poll we remove the global poll code. >> >> Changeset: >> http://cr.openjdk.java.net/~rehn/8220049/v1/webrev/ > > src/hotspot/cpu/aarch64/globals_aarch64.hpp > src/hotspot/cpu/arm/globals_arm.hpp > src/hotspot/cpu/ppc/globals_ppc.hpp > src/hotspot/cpu/s390/globals_s390.hpp > src/hotspot/cpu/sparc/globals_sparc.hpp > src/hotspot/cpu/x86/globals_x86.hpp > src/hotspot/cpu/zero/globals_zero.hpp > ??? No comments on the above flag deletions. > > src/hotspot/os/aix/safepointMechanism_aix.cpp > ??? nit - Needs a copyright year update. > > src/hotspot/share/aot/aotCodeHeap.cpp > ??? nit - Needs a copyright year update. > > src/hotspot/share/aot/aotCodeHeap.hpp > ??? No comments. > > src/hotspot/share/gc/z/zMark.cpp > ??? No comments. > > src/hotspot/share/runtime/arguments.cpp > ??? No comments. > > src/hotspot/share/runtime/biasedLocking.cpp > ? L38: #include "runtime/handshake.hpp" > ? L39: #include "runtime/task.hpp" > ? L40: #include "runtime/threadSMR.hpp" > ? L41: #include "runtime/safepointMechanism.hpp" > ????? nit - New include is out of order; should be after handshake.hpp. > ????? No content comments. > > src/hotspot/share/runtime/flags/jvmFlagConstraintsRuntime.cpp > ??? nit - Needs a copyright year update. > > src/hotspot/share/runtime/flags/jvmFlagConstraintsRuntime.hpp > ??? No comments. > > src/hotspot/share/runtime/globals.hpp > ??? No comments. > > src/hotspot/share/runtime/handshake.cpp > ??? No comments. > > src/hotspot/share/runtime/safepoint.cpp > ??? No comments. > > src/hotspot/share/runtime/safepointMechanism.cpp > ??? No comments. > > src/hotspot/share/runtime/safepointMechanism.hpp > ??? No comments. > > src/hotspot/share/runtime/sweeper.cpp > ??? No comments. > > src/jdk.aot/share/classes/jdk.tools.jaotc.binformat/src/jdk/tools/jaotc/binformat/BinaryContainer.java > > src/jdk.aot/share/classes/jdk.tools.jaotc/src/jdk/tools/jaotc/MarkProcessor.java > > ??? nit - Needs a copyright year update (MarkProcessor.java). > ??? No content comments on the above AOT changes. > > src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.aarch64/src/org/graalvm/compiler/hotspot/aarch64/AArch64HotSpotSafepointOp.java > > src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.amd64/src/org/graalvm/compiler/hotspot/amd64/AMD64HotSpotSafepointOp.java > > src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.sparc.test/src/org/graalvm/compiler/hotspot/sparc/test/SPARCAllocatorTest.java > > src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.sparc/src/org/graalvm/compiler/hotspot/sparc/SPARCHotSpotSafepointOp.java > > src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfig.java > > ??? nit - Needs a copyright year update (most of the Graal files). > ??? No comments on the above Graal changes. > > Thumbs up! I only have nit comments above and I don't need > to see a new webrev if you fix that. > > Dan > > >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8220049 >> >> Passes t1-7, built arm32, aarch64, sparc-solaris, linux x86/x64, >> windows x64, osx, linux s390x, linux ppc64le >> >> (note that on x86 there is missn?ng include in >> src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp, >> --- a/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp????? Tue Dec 10 >> 08:47:38 2019 +0100 >> +++ b/src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp????? Tue Dec 10 >> 10:24:14 2019 +0100 >> @@ -32,0 +33,1 @@ >> +#include "gc/shared/barrierSetAssembler_x86.hpp" >> ) >> >> Martin please have a look on aix changes. >> >> Thanks, Robbin > From kim.barrett at oracle.com Wed Dec 11 10:13:33 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 11 Dec 2019 05:13:33 -0500 Subject: RFR [XS]: 8235489: handle return values of sscanf calls in hotspot In-Reply-To: References: <17DCFAEB-8E3C-4B94-9453-0AD85D3ADC94@oracle.com> <27E66F9B-6AA5-44BE-ABC5-EA1B6792E02F@oracle.com> <53B24471-1455-4D0E-B197-4A34E55DB0C7@oracle.com> <29127B7D-BBB5-4A3E-A6F7-84130EF90CE0@oracle.com> Message-ID: > On Dec 11, 2019, at 4:21 AM, Baesken, Matthias wrote: > >> >> please use "char name[sizeof(line)]" rather than copying the size expression. >> > > Hi Kim , new webrev using sizeof(line) : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235489.4/ This version looks good to me. > In case I hear no objections, I will push this as XS . The term used in the HotSpot How-To [0] is ?trivial?. This change doesn?t look like that, esp. with the amount of discussion it engendered. [0] https://wiki.openjdk.java.net/display/HotSpot/HotSpot+How+To From christoph.langer at sap.com Wed Dec 11 11:16:26 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 11 Dec 2019 11:16:26 +0000 Subject: RFR [XS]: 8235489: handle return values of sscanf calls in hotspot In-Reply-To: References: <17DCFAEB-8E3C-4B94-9453-0AD85D3ADC94@oracle.com> <27E66F9B-6AA5-44BE-ABC5-EA1B6792E02F@oracle.com> <53B24471-1455-4D0E-B197-4A34E55DB0C7@oracle.com> <29127B7D-BBB5-4A3E-A6F7-84130EF90CE0@oracle.com> Message-ID: Hi Matthias, your latest webrev looks fine to me. Can you run this patch through our regression system once more before pushing? (I can see that there is an older version of the patch at the moment...) Thanks & Best regards Christoph > -----Original Message----- > From: hotspot-dev On Behalf Of > Kim Barrett > Sent: Mittwoch, 11. Dezember 2019 11:14 > To: Baesken, Matthias > Cc: hotspot-dev at openjdk.java.net > Subject: Re: RFR [XS]: 8235489: handle return values of sscanf calls in hotspot > > > On Dec 11, 2019, at 4:21 AM, Baesken, Matthias > wrote: > > > >> > >> please use "char name[sizeof(line)]" rather than copying the size > expression. > >> > > > > Hi Kim , new webrev using sizeof(line) : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235489.4/ > > This version looks good to me. > > > In case I hear no objections, I will push this as XS . > > The term used in the HotSpot How-To [0] is ?trivial?. This change doesn?t > look like that, > esp. with the amount of discussion it engendered. > > > [0] https://wiki.openjdk.java.net/display/HotSpot/HotSpot+How+To From nils.eliasson at oracle.com Wed Dec 11 14:02:22 2019 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Wed, 11 Dec 2019 15:02:22 +0100 Subject: RFR: 8235551: BitMap::count_one_bits should use population_count In-Reply-To: References: Message-ID: <7e9d1bd0-1021-eeee-ad25-1e59a994892c@oracle.com> Hi Claes, Looks good. Regards, Nils On 2019-12-08 22:54, Claes Redestad wrote: > Hi, > > with some adjustments, BitMap::count_one_bits can be rewritten to use > the population_count utility method introduced in JDK-8217519 > > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8235551 > Webrev: http://cr.openjdk.java.net/~redestad/8235551/open.00/ > > This generalizes population_count to accept any unsigned integer, to > accommodate that BitMap will use 32- or 64-bit integers depending on > platform. > > Testing: tier1-3 > > Thanks! > > /Claes From daniel.daugherty at oracle.com Wed Dec 11 14:13:04 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 11 Dec 2019 09:13:04 -0500 Subject: RFR(s): 8235410: Enable handshakes on Linux x86 (32-bit) In-Reply-To: References: Message-ID: Thumbs up. One left after this: > src/hotspot/cpu/arm/globals_arm.hpp:define_pd_global(bool, ThreadLocalHandshakes, false); but that one should be take care of with the fix for: 8229971 Arm32: implementation for Thread-local handshakes that Boris sent out for review... Almost there... Dan On 12/5/19 9:01 AM, Robbin Ehn wrote: > Hi all, please review. > > The flag ThreadLocalHandshakes is going to be obsolete in JDK 14. > So we change the (unchangeable) default for Linux x86 to on/true. > > There is a follow-up which removes ThreadLocalHandshakes completely. > If the platform defines THREAD_LOCAL_POLL it will use handshakes. > When arm32 have implemented local polls, the plan is to remove the > global poll > code paths. > > Last time I checked with some of the affected parties there was no > objections. > > Built and sanity test. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8235410 > > Code below. > > Thanks, Robbin > > diff -r 636d71e53732 src/hotspot/cpu/x86/globals_x86.hpp > --- a/src/hotspot/cpu/x86/globals_x86.hpp??? Wed Dec 04 10:26:32 2019 > +0100 > +++ b/src/hotspot/cpu/x86/globals_x86.hpp??? Thu Dec 05 14:13:57 2019 > +0100 > @@ -89,12 +89,7 @@ > > ?define_pd_global(intx, InitArrayShortSize, 8*BytesPerLong); > > -#if defined(_LP64) || defined(_WINDOWS) > ?define_pd_global(bool, ThreadLocalHandshakes, true); > -#else > -// get_thread() is slow on linux 32 bit, therefore off by default > -define_pd_global(bool, ThreadLocalHandshakes, false); > -#endif > > ?#define ARCH_FLAGS(develop, \ > ??????????????????? product, \ > From robbin.ehn at oracle.com Wed Dec 11 14:23:22 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 11 Dec 2019 15:23:22 +0100 Subject: RFR(s): 8235410: Enable handshakes on Linux x86 (32-bit) In-Reply-To: References: Message-ID: <51ad11a9-b423-d53c-6c17-9b2eb1c22790@oracle.com> Thanks Dan! /Robbin On 2019-12-11 15:13, Daniel D. Daugherty wrote: > Thumbs up. > > One left after this: > > > src/hotspot/cpu/arm/globals_arm.hpp:define_pd_global(bool, > ThreadLocalHandshakes, false); > > but that one should be take care of with the fix for: > > 8229971 Arm32: implementation for Thread-local handshakes > > that Boris sent out for review... > > Almost there... > > Dan > > > On 12/5/19 9:01 AM, Robbin Ehn wrote: >> Hi all, please review. >> >> The flag ThreadLocalHandshakes is going to be obsolete in JDK 14. >> So we change the (unchangeable) default for Linux x86 to on/true. >> >> There is a follow-up which removes ThreadLocalHandshakes completely. >> If the platform defines THREAD_LOCAL_POLL it will use handshakes. >> When arm32 have implemented local polls, the plan is to remove the >> global poll >> code paths. >> >> Last time I checked with some of the affected parties there was no >> objections. >> >> Built and sanity test. >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8235410 >> >> Code below. >> >> Thanks, Robbin >> >> diff -r 636d71e53732 src/hotspot/cpu/x86/globals_x86.hpp >> --- a/src/hotspot/cpu/x86/globals_x86.hpp??? Wed Dec 04 10:26:32 2019 >> +0100 >> +++ b/src/hotspot/cpu/x86/globals_x86.hpp??? Thu Dec 05 14:13:57 2019 >> +0100 >> @@ -89,12 +89,7 @@ >> >> ?define_pd_global(intx, InitArrayShortSize, 8*BytesPerLong); >> >> -#if defined(_LP64) || defined(_WINDOWS) >> ?define_pd_global(bool, ThreadLocalHandshakes, true); >> -#else >> -// get_thread() is slow on linux 32 bit, therefore off by default >> -define_pd_global(bool, ThreadLocalHandshakes, false); >> -#endif >> >> ?#define ARCH_FLAGS(develop, \ >> ??????????????????? product, \ >> > From claes.redestad at oracle.com Wed Dec 11 14:33:29 2019 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 11 Dec 2019 15:33:29 +0100 Subject: RFR: 8235551: BitMap::count_one_bits should use population_count In-Reply-To: <00C9FA92-418E-4EE7-985F-86DB64FBFBDD@oracle.com> References: <00C9FA92-418E-4EE7-985F-86DB64FBFBDD@oracle.com> Message-ID: <0b8c8b7e-922a-000c-f345-fdb9e32b6312@oracle.com> On 2019-12-10 23:23, Kim Barrett wrote: > > Looks good. Thanks! > > This comment is a bit misleading: > > src/hotspot/share/utilities/population_count.hpp > 37 // Adapted from Hacker's Delight, 2nd Edition, Figure 5-2. > > The implementation being used is a combination of that in Figure 5-2 > and another version a bit later in the associated text. I suggest > changing the comment to say > > Adapted From Hacker's Delight, 2nd Edition, Figure 5-2 and the text > that follows. > > That way, someone looking up the reference won't just go "Wait, what? > That isn't what's in 5-2." > Will fix before push. /Claes From claes.redestad at oracle.com Wed Dec 11 15:19:22 2019 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 11 Dec 2019 16:19:22 +0100 Subject: RFR: 8235551: BitMap::count_one_bits should use population_count In-Reply-To: <7e9d1bd0-1021-eeee-ad25-1e59a994892c@oracle.com> References: <7e9d1bd0-1021-eeee-ad25-1e59a994892c@oracle.com> Message-ID: <92132cfb-1001-42bc-9208-638aec3f8c55@oracle.com> Thanks for reviewing, Nils! /Claes On 2019-12-11 15:02, Nils Eliasson wrote: > Hi Claes, > > Looks good. > > Regards, > Nils > > > On 2019-12-08 22:54, Claes Redestad wrote: >> Hi, >> >> with some adjustments, BitMap::count_one_bits can be rewritten to use >> the population_count utility method introduced in JDK-8217519 >> >> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8235551 >> Webrev: http://cr.openjdk.java.net/~redestad/8235551/open.00/ >> >> This generalizes population_count to accept any unsigned integer, to >> accommodate that BitMap will use 32- or 64-bit integers depending on >> platform. >> >> Testing: tier1-3 >> >> Thanks! >> >> /Claes From maurizio.cimadamore at oracle.com Wed Dec 11 15:39:51 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 11 Dec 2019 15:39:51 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <523d08f4-5604-3a2d-7245-733a5fdc028b@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <7765b06b-acf2-f11f-980e-33af4ca3934c@oracle.com> <14719046-4d92-d9ff-c59e-924e10212edf@oracle.com> <24fdb620-2b26-0647-524f-71a40b50d9d3@oracle.com> <515c1a7a-19d7-ba0a-fc93-024fe828bedb@oracle.com> <12784CB4-7357-4F19-B307-1AE8354D40F7@oracle.com> <523d08f4-5604-3a2d-7245-733a5fdc028b@oracle.com> Message-ID: I went ahead and created a new revision: http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v6/ Delta from latest (v5) here: http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v6_delta/ No javadoc chnages here. Summary of changes: * addressed Paul feedback on MemoryScope * on further testing, I found two issues on var handle construction from layouts: ?? - PathElement.sequenceElement(long) does not add the sequence element offset to the new path element offset ?? - the layout path machiner does not check that strides are a multiple of alignment - this allow creation of 'bad' var handles which are guaranteed not to work in for all indices - consider this layout: var seq = MemoryLayout.ofSequence(JAVA_SHORT.withBitAlignment(32)); The above layout has an issue: the element requests 32-bit alignent, but the elements are only 16 bits apart. So, for odd access indices, there will be an alignment exception. If you create a VarHandle with combinators, strides are checked against alignment, so it is not possible to obtain a VarHandle for the above layout. But with the Layout API is possible, because the layout path machinery does not enforce same restriction. I've now fixed the code in LayoutPath (most of the changes are due to a rename from 'scale' to 'stride') to take care of this. Bonus point: since offsets must satisfy alignment, and all strides must also satisfy same alignment - it follows that whetever offset can come out of the VH index machinery, it will be aligned. This means that, in the VarHandle code we can just check the base address. In other words, the memory location accessed by the VarHandle will be something like (below N denotes alignment, s1..sn are strides, O1..Om are offsets): address = base + offset where: offset = N * s1 * i1 + N * s2 * i2 ... + N * sn * in + N * O1 + N * O2 + ... N * Om = N * (s1 * i1 + s2 * i2 ... + sn * in + O1 + O2 + ... Om) = N * K So, in order to show that the "address" is aligned, we only have to show that "base" is a multiple of N. This helps a lot, as e.g. when looping through a native array, the base array stays the same, and only the offset part changes, which means C2 will have no troubles hoisting out the alignment check out of the loop. As things appear stable, I do not plan to make any other changes to the implementation/API ahead of integration, unless there's something critical which reviewers think should be fixed. Thanks Maurizio On 11/12/2019 00:13, Maurizio Cimadamore wrote: > > On 10/12/2019 18:45, Paul Sandoz wrote: >> MemoryScope changes look good. >> >> In j.u.concurrent we use ExceptionInInitializerError in the static >> blocks when there is an error looking up the VH, >> >> FWIW close can also fail because it has already been closed but the >> exception message does not distinguish between the two states (active >> or already closed). >> ? Paul. >> > Hi Paul, > > would something like this be satisfactory? > > > diff -r 50ef46599c56 > src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/MemoryScope.java > --- > a/src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/MemoryScope.java > Tue Dec 10 16:28:03 2019 +0000 > +++ > b/src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/MemoryScope.java > Wed Dec 11 00:12:20 2019 +0000 > @@ -48,7 +48,7 @@ > ???????? try { > ???????????? COUNT_HANDLE = > MethodHandles.lookup().findVarHandle(MemoryScope.class, "activeCount", > int.class); > ???????? } catch (Throwable ex) { > -??????????? throw new IllegalStateException(ex); > +??????????? throw new ExceptionInInitializerError(ex); > ???????? } > ???? } > > @@ -107,6 +107,9 @@ > > ???? void close() { > ???????? if (!COUNT_HANDLE.compareAndSet(this, UNACQUIRED, CLOSED)) { > +??????????? //first check if already closed... > +??????????? checkAliveConfined(); > +??????????? //...if not, then we have acquired views that are still > active > ???????????? throw new IllegalStateException("Cannot close a segment > that has active acquired views"); > ???????? } > ???????? if (cleanupAction != null) { > > > > If you need a full webrev please let me know. > > Thanks > Maurizio > From matthias.baesken at sap.com Wed Dec 11 15:47:23 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 11 Dec 2019 15:47:23 +0000 Subject: RFR [XS]: 8235489: handle return values of sscanf calls in hotspot In-Reply-To: References: <17DCFAEB-8E3C-4B94-9453-0AD85D3ADC94@oracle.com> <27E66F9B-6AA5-44BE-ABC5-EA1B6792E02F@oracle.com> <53B24471-1455-4D0E-B197-4A34E55DB0C7@oracle.com> <29127B7D-BBB5-4A3E-A6F7-84130EF90CE0@oracle.com> Message-ID: Hi Christoph, thanks for the review . > > Hi Matthias, > > your latest webrev looks fine to me. > > Can you run this patch through our regression system once more before > pushing? (I can see that there is an older version of the patch at the > moment...) > Yes I'll put in the latest revision . Best regards, Matthias From matthias.baesken at sap.com Wed Dec 11 16:02:05 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 11 Dec 2019 16:02:05 +0000 Subject: RFR [XS] : 8235478: [jdk11] handle VS2017 15.9 and VS2019 in vm_version.cpp Message-ID: Ping ... any reviews please ? Best regards, Matthias From: Baesken, Matthias Sent: Montag, 9. Dezember 2019 12:37 To: jdk-updates-dev at openjdk.java.net Cc: 'hotspot-dev at openjdk.java.net' Subject: RFR [XS] : 8235478: [jdk11] handle VS2017 15.9 and VS2019 in vm_version.cpp Hello, please review this small jdk11 change . It handles VS2017 15.9 and VS2019 in vm_version.cpp . More or less it is a jdk11 - downport of what 8235243 + 8235325 do , but in jdk/jdk . Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8235478 http://cr.openjdk.java.net/~mbaesken/webrevs/8235478.0/ Thanks, Matthias From paul.sandoz at oracle.com Wed Dec 11 16:43:17 2019 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 11 Dec 2019 08:43:17 -0800 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <7765b06b-acf2-f11f-980e-33af4ca3934c@oracle.com> <14719046-4d92-d9ff-c59e-924e10212edf@oracle.com> <24fdb620-2b26-0647-524f-71a40b50d9d3@oracle.com> <515c1a7a-19d7-ba0a-fc93-024fe828bedb@oracle.com> <12784CB4-7357-4F19-B307-1AE8354D40F7@oracle.com> <523d08f4-5604-3a2d-7245-733a5fdc028b@oracle.com> Message-ID: <6C8BC7CC-6210-4DD3-BBAB-A7F52C1F8A07@oracle.com> Looks very good, Paul. > On Dec 11, 2019, at 7:39 AM, Maurizio Cimadamore wrote: > > I went ahead and created a new revision: > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v6/ > Delta from latest (v5) here: > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v6_delta/ > No javadoc chnages here. > > Summary of changes: > > * addressed Paul feedback on MemoryScope > * on further testing, I found two issues on var handle construction from layouts: > - PathElement.sequenceElement(long) does not add the sequence element offset to the new path element offset > - the layout path machiner does not check that strides are a multiple of alignment - this allow creation of 'bad' var handles which are guaranteed not to work in for all indices - consider this layout: > > var seq = MemoryLayout.ofSequence(JAVA_SHORT.withBitAlignment(32)); > > The above layout has an issue: the element requests 32-bit alignent, but the elements are only 16 bits apart. So, for odd access indices, there will be an alignment exception. > > If you create a VarHandle with combinators, strides are checked against alignment, so it is not possible to obtain a VarHandle for the above layout. But with the Layout API is possible, because the layout path machinery does not enforce same restriction. > > I've now fixed the code in LayoutPath (most of the changes are due to a rename from 'scale' to 'stride') to take care of this. > > Bonus point: since offsets must satisfy alignment, and all strides must also satisfy same alignment - it follows that whetever offset can come out of the VH index machinery, it will be aligned. This means that, in the VarHandle code we can just check the base address. In other words, the memory location accessed by the VarHandle will be something like (below N denotes alignment, s1..sn are strides, O1..Om are offsets): > > > > address = base + offset > > where: > > offset = N * s1 * i1 + N * s2 * i2 ... + N * sn * in + N * O1 + N * O2 + ... N * Om > = N * (s1 * i1 + s2 * i2 ... + sn * in + O1 + O2 + ... Om) > = N * K > > > > So, in order to show that the "address" is aligned, we only have to show that "base" is a multiple of N. This helps a lot, as e.g. when looping through a native array, the base array stays the same, and only the offset part changes, which means C2 will have no troubles hoisting out the alignment check out of the loop. > > > As things appear stable, I do not plan to make any other changes to the implementation/API ahead of integration, unless there's something critical which reviewers think should be fixed. > > Thanks > Maurizio > > > > > On 11/12/2019 00:13, Maurizio Cimadamore wrote: >> >> On 10/12/2019 18:45, Paul Sandoz wrote: >>> MemoryScope changes look good. >>> >>> In j.u.concurrent we use ExceptionInInitializerError in the static blocks when there is an error looking up the VH, >>> >>> FWIW close can also fail because it has already been closed but the exception message does not distinguish between the two states (active or already closed). >>> Paul. >>> >> Hi Paul, >> >> would something like this be satisfactory? >> >> >> diff -r 50ef46599c56 src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/MemoryScope.java >> --- a/src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/MemoryScope.java Tue Dec 10 16:28:03 2019 +0000 >> +++ b/src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/MemoryScope.java Wed Dec 11 00:12:20 2019 +0000 >> @@ -48,7 +48,7 @@ >> try { >> COUNT_HANDLE = MethodHandles.lookup().findVarHandle(MemoryScope.class, "activeCount", int.class); >> } catch (Throwable ex) { >> - throw new IllegalStateException(ex); >> + throw new ExceptionInInitializerError(ex); >> } >> } >> >> @@ -107,6 +107,9 @@ >> >> void close() { >> if (!COUNT_HANDLE.compareAndSet(this, UNACQUIRED, CLOSED)) { >> + //first check if already closed... >> + checkAliveConfined(); >> + //...if not, then we have acquired views that are still active >> throw new IllegalStateException("Cannot close a segment that has active acquired views"); >> } >> if (cleanupAction != null) { >> >> >> >> If you need a full webrev please let me know. >> >> Thanks >> Maurizio >> From christoph.langer at sap.com Thu Dec 12 10:46:36 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 12 Dec 2019 10:46:36 +0000 Subject: RFR [XS] : 8235478: [jdk11] handle VS2017 15.9 and VS2019 in vm_version.cpp In-Reply-To: References: Message-ID: Hi Matthias, the change itself looks good. However, your approach to create a new bug which is actually a backport of two other bugs is not the right thing to do. I converted JDK-8235478 to be a backport of JDK-8235243. Now, please request backport approval in JDK-8235243 and JDK-8235325. Once it's approved, please push your change with a commit message that resolves both items, that is: 8235243: handle VS2017 15.9 and VS2019 in abstract_vm_version 8235325: build failure on Linux after 8235243 Reviewed-by: dholmes, mdoerr This will pick up and resolve JDK-8235478 as backport of 8235243 and create/resolve another backport item for 8235325. Best regards Christoph > -----Original Message----- > From: hotspot-dev On Behalf Of > Baesken, Matthias > Sent: Montag, 9. Dezember 2019 12:37 > To: jdk-updates-dev at openjdk.java.net > Cc: 'hotspot-dev at openjdk.java.net' > Subject: RFR [XS] : 8235478: [jdk11] handle VS2017 15.9 and VS2019 in > vm_version.cpp > > Hello, please review this small jdk11 change . > It handles VS2017 15.9 and VS2019 in vm_version.cpp . > > More or less it is a jdk11 - downport of what 8235243 + 8235325 do , but in > jdk/jdk . > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8235478 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235478.0/ > > > Thanks, Matthias From matthias.baesken at sap.com Thu Dec 12 11:19:01 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 12 Dec 2019 11:19:01 +0000 Subject: RFR [XS] : 8235478: [jdk11] handle VS2017 15.9 and VS2019 in vm_version.cpp In-Reply-To: References: Message-ID: Hi Christoph, thanks . > , please request backport approval in JDK- > 8235243 and JDK-8235325. I added the jdk11u-fix-request label to both bugs . Best regards, Matthias > Hi Matthias, > > the change itself looks good. > > However, your approach to create a new bug which is actually a backport of > two other bugs is not the right thing to do. I converted JDK-8235478 to be a > backport of JDK-8235243. Now, please request backport approval in JDK- > 8235243 and JDK-8235325. Once it's approved, please push your change with > a commit message that resolves both items, that is: > > 8235243: handle VS2017 15.9 and VS2019 in abstract_vm_version > 8235325: build failure on Linux after 8235243 > Reviewed-by: dholmes, mdoerr > > This will pick up and resolve JDK-8235478 as backport of 8235243 and > create/resolve another backport item for 8235325. > > Best regards > Christoph > > > > -----Original Message----- > > From: hotspot-dev On Behalf > Of > > Baesken, Matthias > > Sent: Montag, 9. Dezember 2019 12:37 > > To: jdk-updates-dev at openjdk.java.net > > Cc: 'hotspot-dev at openjdk.java.net' > > Subject: RFR [XS] : 8235478: [jdk11] handle VS2017 15.9 and VS2019 in > > vm_version.cpp > > > > Hello, please review this small jdk11 change . > > It handles VS2017 15.9 and VS2019 in vm_version.cpp . > > > > More or less it is a jdk11 - downport of what 8235243 + 8235325 do , but in > > jdk/jdk . > > > > Bug/webrev : > > > > https://bugs.openjdk.java.net/browse/JDK-8235478 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235478.0/ > > > > > > Thanks, Matthias From chris.hegarty at oracle.com Thu Dec 12 15:46:41 2019 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 12 Dec 2019 15:46:41 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <6C8BC7CC-6210-4DD3-BBAB-A7F52C1F8A07@oracle.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <7765b06b-acf2-f11f-980e-33af4ca3934c@oracle.com> <14719046-4d92-d9ff-c59e-924e10212edf@oracle.com> <24fdb620-2b26-0647-524f-71a40b50d9d3@oracle.com> <515c1a7a-19d7-ba0a-fc93-024fe828bedb@oracle.com> <12784CB4-7357-4F19-B307-1AE8354D40F7@oracle.com> <523d08f4-5604-3a2d-7245-733a5fdc028b@oracle.com> <6C8BC7CC-6210-4DD3-BBAB-A7F52C1F8A07@oracle.com> Message-ID: <54E4ADE2-0FCF-4FE6-9F92-7DC3366E5AB1@oracle.com> > On 11 Dec 2019, at 16:43, Paul Sandoz wrote: > > Looks very good, +1 This looks very good to me too. Nice work. -Chris. > Paul. > >> On Dec 11, 2019, at 7:39 AM, Maurizio Cimadamore wrote: >> >> I went ahead and created a new revision: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v6/ >> Delta from latest (v5) here: >> >> http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v6_delta/ >> No javadoc chnages here. >> >> Summary of changes: >> >> * addressed Paul feedback on MemoryScope >> * on further testing, I found two issues on var handle construction from layouts: >> - PathElement.sequenceElement(long) does not add the sequence element offset to the new path element offset >> - the layout path machiner does not check that strides are a multiple of alignment - this allow creation of 'bad' var handles which are guaranteed not to work in for all indices - consider this layout: >> >> var seq = MemoryLayout.ofSequence(JAVA_SHORT.withBitAlignment(32)); >> >> The above layout has an issue: the element requests 32-bit alignent, but the elements are only 16 bits apart. So, for odd access indices, there will be an alignment exception. >> >> If you create a VarHandle with combinators, strides are checked against alignment, so it is not possible to obtain a VarHandle for the above layout. But with the Layout API is possible, because the layout path machinery does not enforce same restriction. >> >> I've now fixed the code in LayoutPath (most of the changes are due to a rename from 'scale' to 'stride') to take care of this. >> >> Bonus point: since offsets must satisfy alignment, and all strides must also satisfy same alignment - it follows that whetever offset can come out of the VH index machinery, it will be aligned. This means that, in the VarHandle code we can just check the base address. In other words, the memory location accessed by the VarHandle will be something like (below N denotes alignment, s1..sn are strides, O1..Om are offsets): >> >> >> >> address = base + offset >> >> where: >> >> offset = N * s1 * i1 + N * s2 * i2 ... + N * sn * in + N * O1 + N * O2 + ... N * Om >> = N * (s1 * i1 + s2 * i2 ... + sn * in + O1 + O2 + ... Om) >> = N * K >> >> >> >> So, in order to show that the "address" is aligned, we only have to show that "base" is a multiple of N. This helps a lot, as e.g. when looping through a native array, the base array stays the same, and only the offset part changes, which means C2 will have no troubles hoisting out the alignment check out of the loop. >> >> >> As things appear stable, I do not plan to make any other changes to the implementation/API ahead of integration, unless there's something critical which reviewers think should be fixed. >> >> Thanks >> Maurizio >> >> >> >> >> On 11/12/2019 00:13, Maurizio Cimadamore wrote: >>> >>> On 10/12/2019 18:45, Paul Sandoz wrote: >>>> MemoryScope changes look good. >>>> >>>> In j.u.concurrent we use ExceptionInInitializerError in the static blocks when there is an error looking up the VH, >>>> >>>> FWIW close can also fail because it has already been closed but the exception message does not distinguish between the two states (active or already closed). >>>> Paul. >>>> >>> Hi Paul, >>> >>> would something like this be satisfactory? >>> >>> >>> diff -r 50ef46599c56 src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/MemoryScope.java >>> --- a/src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/MemoryScope.java Tue Dec 10 16:28:03 2019 +0000 >>> +++ b/src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/MemoryScope.java Wed Dec 11 00:12:20 2019 +0000 >>> @@ -48,7 +48,7 @@ >>> try { >>> COUNT_HANDLE = MethodHandles.lookup().findVarHandle(MemoryScope.class, "activeCount", int.class); >>> } catch (Throwable ex) { >>> - throw new IllegalStateException(ex); >>> + throw new ExceptionInInitializerError(ex); >>> } >>> } >>> >>> @@ -107,6 +107,9 @@ >>> >>> void close() { >>> if (!COUNT_HANDLE.compareAndSet(this, UNACQUIRED, CLOSED)) { >>> + //first check if already closed... >>> + checkAliveConfined(); >>> + //...if not, then we have acquired views that are still active >>> throw new IllegalStateException("Cannot close a segment that has active acquired views"); >>> } >>> if (cleanupAction != null) { >>> >>> >>> >>> If you need a full webrev please let me know. >>> >>> Thanks >>> Maurizio >>> > From maurizio.cimadamore at oracle.com Thu Dec 12 23:05:38 2019 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 12 Dec 2019 23:05:38 +0000 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <7765b06b-acf2-f11f-980e-33af4ca3934c@oracle.com> <14719046-4d92-d9ff-c59e-924e10212edf@oracle.com> <24fdb620-2b26-0647-524f-71a40b50d9d3@oracle.com> <515c1a7a-19d7-ba0a-fc93-024fe828bedb@oracle.com> <12784CB4-7357-4F19-B307-1AE8354D40F7@oracle.com> <523d08f4-5604-3a2d-7245-733a5fdc028b@oracle.com> Message-ID: I've just pushed this patch to jdk/jdk14. I'd like to thank all the reviewers for the great (and quick) feedback. Cheers Maurizio On 11/12/2019 15:39, Maurizio Cimadamore wrote: > I went ahead and created a new revision: > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v6/ > > Delta from latest (v5) here: > > http://cr.openjdk.java.net/~mcimadamore/panama/8234049_v6_delta/ > > No javadoc chnages here. > > Summary of changes: > > * addressed Paul feedback on MemoryScope > * on further testing, I found two issues on var handle construction > from layouts: > ?? - PathElement.sequenceElement(long) does not add the sequence > element offset to the new path element offset > ?? - the layout path machiner does not check that strides are a > multiple of alignment - this allow creation of 'bad' var handles which > are guaranteed not to work in for all indices - consider this layout: > > var seq = MemoryLayout.ofSequence(JAVA_SHORT.withBitAlignment(32)); > > The above layout has an issue: the element requests 32-bit alignent, > but the elements are only 16 bits apart. So, for odd access indices, > there will be an alignment exception. > > If you create a VarHandle with combinators, strides are checked > against alignment, so it is not possible to obtain a VarHandle for the > above layout. But with the Layout API is possible, because the layout > path machinery does not enforce same restriction. > > I've now fixed the code in LayoutPath (most of the changes are due to > a rename from 'scale' to 'stride') to take care of this. > > Bonus point: since offsets must satisfy alignment, and all strides > must also satisfy same alignment - it follows that whetever offset can > come out of the VH index machinery, it will be aligned. This means > that, in the VarHandle code we can just check the base address. In > other words, the memory location accessed by the VarHandle will be > something like (below N denotes alignment, s1..sn are strides, O1..Om > are offsets): > > > address = base + offset > > where: > > offset = N * s1 * i1 + N * s2 * i2 ... + N * sn * in + N * O1 + N * O2 > + ... N * Om > = N * (s1 * i1 + s2 * i2 ... + sn * in + O1 + O2 + ... Om) > = N * K > > > So, in order to show that the "address" is aligned, we only have to > show that "base" is a multiple of N. This helps a lot, as e.g. when > looping through a native array, the base array stays the same, and > only the offset part changes, which means C2 will have no troubles > hoisting out the alignment check out of the loop. > > > As things appear stable, I do not plan to make any other changes to > the implementation/API ahead of integration, unless there's something > critical which reviewers think should be fixed. > > Thanks > Maurizio > > > On 11/12/2019 00:13, Maurizio Cimadamore wrote: >> >> On 10/12/2019 18:45, Paul Sandoz wrote: >>> MemoryScope changes look good. >>> >>> In j.u.concurrent we use ExceptionInInitializerError in the static >>> blocks when there is an error looking up the VH, >>> >>> FWIW close can also fail because it has already been closed but the >>> exception message does not distinguish between the two states >>> (active or already closed). >>> ? Paul. >>> >> Hi Paul, >> >> would something like this be satisfactory? >> >> >> diff -r 50ef46599c56 >> src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/MemoryScope.java >> --- >> a/src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/MemoryScope.java >> Tue Dec 10 16:28:03 2019 +0000 >> +++ >> b/src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/MemoryScope.java >> Wed Dec 11 00:12:20 2019 +0000 >> @@ -48,7 +48,7 @@ >> ???????? try { >> ???????????? COUNT_HANDLE = >> MethodHandles.lookup().findVarHandle(MemoryScope.class, >> "activeCount", int.class); >> ???????? } catch (Throwable ex) { >> -??????????? throw new IllegalStateException(ex); >> +??????????? throw new ExceptionInInitializerError(ex); >> ???????? } >> ???? } >> >> @@ -107,6 +107,9 @@ >> >> ???? void close() { >> ???????? if (!COUNT_HANDLE.compareAndSet(this, UNACQUIRED, CLOSED)) { >> +??????????? //first check if already closed... >> +??????????? checkAliveConfined(); >> +??????????? //...if not, then we have acquired views that are still >> active >> ???????????? throw new IllegalStateException("Cannot close a segment >> that has active acquired views"); >> ???????? } >> ???????? if (cleanupAction != null) { >> >> >> >> If you need a full webrev please let me know. >> >> Thanks >> Maurizio >> From john.r.rose at oracle.com Fri Dec 13 00:22:18 2019 From: john.r.rose at oracle.com (John Rose) Date: Thu, 12 Dec 2019 16:22:18 -0800 Subject: RFR JDK-8234049: Implementation of Memory Access API (Incubator) In-Reply-To: <55613fbc-a8b2-b4bb-d4f1-5b08c2b91014@gmail.com> References: <05c71944-cd1d-d5c7-3e0d-704fdfb5f5c3@oracle.com> <55613fbc-a8b2-b4bb-d4f1-5b08c2b91014@gmail.com> Message-ID: On Dec 9, 2019, at 10:16 AM, Peter Levart wrote: > What do you think? Might this be better performance wise? Yes, a fast positive proof that the access is allowed can be implemented along the lines you mention. The owner token could be overloaded in additional ways, if and when we do more patterns of sharing. The case of a permanently shareable heap-allocated segment could be handled by a similar fast check, if some constant sentinel value ALL_THREADS is stored in the owner instead Thread.current(). The case of a temporarily shareable segment, held jointly by several threads, could be handled by an owner token which was suitably tied to those threads. This suggests that ?owner == currentThread()? could be at some point generalized to ?owner.accepts(currentThread())? assuming that the fast paths of OwnerSet::accepts could be optimized well. ? John From david.holmes at oracle.com Fri Dec 13 04:51:33 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 13 Dec 2019 14:51:33 +1000 Subject: RFR 15 (S) 8231559: Remove expired flags in JDK 15 Message-ID: <79bfba32-bd63-fa64-265b-233e9d35d28e@oracle.com> It is that time of the release again. All the expired entries in the flag table can be deleted. Bug: https://bugs.openjdk.java.net/browse/JDK-8231559 webrev: http://cr.openjdk.java.net/~dholmes/8231559/webrev/ It's mostly all the old CMS flags. Thanks, David From igor.ignatyev at oracle.com Fri Dec 13 05:19:43 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 12 Dec 2019 21:19:43 -0800 Subject: [14] RFR(S/T) : 8235866 : bump jtreg requiredVersion to 4.2b16 Message-ID: <764BB5FF-AC59-40E2-AB5A-F63A745E9AEE@oracle.com> http://cr.openjdk.java.net/~iignatyev//8235866/webrev.00 > 5 lines changed: 0 ins; 0 del; 5 mod Hi all, could you please review this small patch which updates TEST.ROOT in all test suites to require jtreg4.2b16? 8230067[1] updated profiles to use jtreg4.2b16, but didn't update TEST.ROOT files. this patch forces all users to have jtreg4.2b16, so there will be no difference in how tests are executed locally and in automated testing infrastructures. webrev: http://cr.openjdk.java.net/~iignatyev//8235866/webrev.00 JBS: https://bugs.openjdk.java.net/browse/JDK-8235866 [1] https://bugs.openjdk.java.net/browse/JDK-8230067 Thanks, -- Igor From kim.barrett at oracle.com Fri Dec 13 06:08:34 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 13 Dec 2019 01:08:34 -0500 Subject: RFR 15 (S) 8231559: Remove expired flags in JDK 15 In-Reply-To: <79bfba32-bd63-fa64-265b-233e9d35d28e@oracle.com> References: <79bfba32-bd63-fa64-265b-233e9d35d28e@oracle.com> Message-ID: <4729AB83-E2B3-4D7D-B749-9E48777C3EBD@oracle.com> > On Dec 12, 2019, at 11:51 PM, David Holmes wrote: > > It is that time of the release again. All the expired entries in the flag table can be deleted. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8231559 > webrev: http://cr.openjdk.java.net/~dholmes/8231559/webrev/ > > It's mostly all the old CMS flags. > > Thanks, > David Looks good. This change also fixes https://bugs.openjdk.java.net/browse/JDK-8232739. From david.holmes at oracle.com Fri Dec 13 06:12:00 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 13 Dec 2019 16:12:00 +1000 Subject: RFR 15 (S) 8231559: Remove expired flags in JDK 15 In-Reply-To: <4729AB83-E2B3-4D7D-B749-9E48777C3EBD@oracle.com> References: <79bfba32-bd63-fa64-265b-233e9d35d28e@oracle.com> <4729AB83-E2B3-4D7D-B749-9E48777C3EBD@oracle.com> Message-ID: <9eb2eb7b-b22b-9188-59aa-3d2366ef2ded@oracle.com> On 13/12/2019 4:08 pm, Kim Barrett wrote: >> On Dec 12, 2019, at 11:51 PM, David Holmes wrote: >> >> It is that time of the release again. All the expired entries in the flag table can be deleted. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231559 >> webrev: http://cr.openjdk.java.net/~dholmes/8231559/webrev/ >> >> It's mostly all the old CMS flags. >> >> Thanks, >> David > > Looks good. Thanks Kim. > This change also fixes https://bugs.openjdk.java.net/browse/JDK-8232739. Thanks for pointing that out (my search failed to find it somehow). I've closed it as a dup of this bug. Thanks, David From david.holmes at oracle.com Fri Dec 13 06:21:24 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 13 Dec 2019 16:21:24 +1000 Subject: [14] RFR(S/T) : 8235866 : bump jtreg requiredVersion to 4.2b16 In-Reply-To: <764BB5FF-AC59-40E2-AB5A-F63A745E9AEE@oracle.com> References: <764BB5FF-AC59-40E2-AB5A-F63A745E9AEE@oracle.com> Message-ID: Hi Igor, On 13/12/2019 3:19 pm, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8235866/webrev.00 >> 5 lines changed: 0 ins; 0 del; 5 mod > > Hi all, > > could you please review this small patch which updates TEST.ROOT in all test suites to require jtreg4.2b16? > 8230067[1] updated profiles to use jtreg4.2b16, but didn't update TEST.ROOT files. this patch forces all users to have jtreg4.2b16, so there will be no difference in how tests are executed locally and in automated testing infrastructures. > > webrev: http://cr.openjdk.java.net/~iignatyev//8235866/webrev.00 So you changed everyone to b14 under 8219417 but then jaxp was reverted back to b13 under https://bugs.openjdk.java.net/browse/JDK-8219705 just a few weeks later. Does that mean jaxp has an issue with later jtreg builds? Or was that a mistake? Otherwise changes seem fine to me. When was b16 released? Thanks, David > JBS: https://bugs.openjdk.java.net/browse/JDK-8235866 > [1] https://bugs.openjdk.java.net/browse/JDK-8230067 > > Thanks, > -- Igor > > > From igor.ignatyev at oracle.com Fri Dec 13 06:46:02 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 12 Dec 2019 22:46:02 -0800 Subject: [14] RFR(S/T) : 8235866 : bump jtreg requiredVersion to 4.2b16 In-Reply-To: References: <764BB5FF-AC59-40E2-AB5A-F63A745E9AEE@oracle.com> Message-ID: <7522383C-1C26-4422-83CA-BB4E795CCF6B@oracle.com> > On Dec 12, 2019, at 10:21 PM, David Holmes wrote: > > Hi Igor, > > On 13/12/2019 3:19 pm, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8235866/webrev.00 >>> 5 lines changed: 0 ins; 0 del; 5 mod >> Hi all, >> could you please review this small patch which updates TEST.ROOT in all test suites to require jtreg4.2b16? >> 8230067[1] updated profiles to use jtreg4.2b16, but didn't update TEST.ROOT files. this patch forces all users to have jtreg4.2b16, so there will be no difference in how tests are executed locally and in automated testing infrastructures. >> webrev: http://cr.openjdk.java.net/~iignatyev//8235866/webrev.00 > > So you changed everyone to b14 under 8219417 but then jaxp was reverted back to b13 under > > https://bugs.openjdk.java.net/browse/JDK-8219705 > > just a few weeks later. Does that mean jaxp has an issue with later jtreg builds? Or was that a mistake? I don't see why 8219705 can require to change requiredVersion, nor do I see it being discussed in RFR (https://mail.openjdk.java.net/pipermail/core-libs-dev/2019-March/058923.html ), so I assume it's just unrelated local changes which got integrated. but just to be safe, @Joe/Lance, were there any reasons why you needed to switch jtreg's requiredVersion back to 4.2b13 in jaxp? are there any reasons which prevent jaxp from switching to jtreg4.2b16 now? > > Otherwise changes seem fine to me. > > When was b16 released? the tag was added by Jon on Dec 03 2019. so I believe there was enough time for usual suspects to build/adjust, and I'm aware about at least one publicly available place where people can get the latest build of jtreg, if they have problems w/ building it themselves. Thanks, -- Igor > > Thanks, > David > >> JBS: https://bugs.openjdk.java.net/browse/JDK-8235866 >> [1] https://bugs.openjdk.java.net/browse/JDK-8230067 >> Thanks, >> -- Igor >> From vladimir.kozlov at oracle.com Fri Dec 13 06:54:52 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 12 Dec 2019 22:54:52 -0800 Subject: RFR 15 (S) 8231559: Remove expired flags in JDK 15 In-Reply-To: <79bfba32-bd63-fa64-265b-233e9d35d28e@oracle.com> References: <79bfba32-bd63-fa64-265b-233e9d35d28e@oracle.com> Message-ID: Good. Thanks Vladimir > On Dec 12, 2019, at 8:51 PM, David Holmes wrote: > > It is that time of the release again. All the expired entries in the flag table can be deleted. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8231559 > webrev: http://cr.openjdk.java.net/~dholmes/8231559/webrev/ > > It's mostly all the old CMS flags. > > Thanks, > David From david.holmes at oracle.com Fri Dec 13 07:50:40 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 13 Dec 2019 17:50:40 +1000 Subject: RFR 15 (S) 8231559: Remove expired flags in JDK 15 In-Reply-To: References: <79bfba32-bd63-fa64-265b-233e9d35d28e@oracle.com> Message-ID: <5c5e40de-c007-9682-b6e4-af6d1fd8fbef@oracle.com> Thanks Vladimir! David On 13/12/2019 4:54 pm, Vladimir Kozlov wrote: > Good. > > Thanks > Vladimir > >> On Dec 12, 2019, at 8:51 PM, David Holmes wrote: >> >> It is that time of the release again. All the expired entries in the flag table can be deleted. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231559 >> webrev: http://cr.openjdk.java.net/~dholmes/8231559/webrev/ >> >> It's mostly all the old CMS flags. >> >> Thanks, >> David > From kusrinivasan at vmware.com Fri Dec 13 14:28:26 2019 From: kusrinivasan at vmware.com (Kumar Srinivasan) Date: Fri, 13 Dec 2019 14:28:26 +0000 Subject: [14] RFR(S/T) : 8235866 : bump jtreg requiredVersion to 4.2b16 In-Reply-To: <7522383C-1C26-4422-83CA-BB4E795CCF6B@oracle.com> References: <764BB5FF-AC59-40E2-AB5A-F63A745E9AEE@oracle.com> <7522383C-1C26-4422-83CA-BB4E795CCF6B@oracle.com> Message-ID: <7AC57A49-095F-4A2C-A1A3-4E2390702BB7@vmware.com> Hey Igor ? please read inlined comment below?.. [cc?ed Jon] On Dec 12, 2019, at 10:46 PM, Igor Ignatyev > wrote: On Dec 12, 2019, at 10:21 PM, David Holmes > wrote: Hi Igor, On 13/12/2019 3:19 pm, Igor Ignatyev wrote: https://nam04.safelinks.protection.outlook.com/?url=http:%2F%2Fcr.openjdk.java.net%2F~iignatyev%2F%2F8235866%2Fwebrev.00&data=02%7C01%7Ckusrinivasan%40vmware.com%7C8e1128e05e8a441dfd8e08d77f983396%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637118164005266666&sdata=e5sMrba8xFcOfBlFXXSSI9zRcZcSIyCQvfmfBtUN8bQ%3D&reserved=0 5 lines changed: 0 ins; 0 del; 5 mod Hi all, could you please review this small patch which updates TEST.ROOT in all test suites to require jtreg4.2b16? 8230067[1] updated profiles to use jtreg4.2b16, but didn't update TEST.ROOT files. this patch forces all users to have jtreg4.2b16, so there will be no difference in how tests are executed locally and in automated testing infrastructures. webrev: https://nam04.safelinks.protection.outlook.com/?url=http:%2F%2Fcr.openjdk.java.net%2F~iignatyev%2F%2F8235866%2Fwebrev.00&data=02%7C01%7Ckusrinivasan%40vmware.com%7C8e1128e05e8a441dfd8e08d77f983396%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637118164005266666&sdata=e5sMrba8xFcOfBlFXXSSI9zRcZcSIyCQvfmfBtUN8bQ%3D&reserved=0 So you changed everyone to b14 under 8219417 but then jaxp was reverted back to b13 under https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.openjdk.java.net%2Fbrowse%2FJDK-8219705&data=02%7C01%7Ckusrinivasan%40vmware.com%7C8e1128e05e8a441dfd8e08d77f983396%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637118164005266666&sdata=SaI%2Bta5Ug%2BkyKP%2BSJ3iV6HsZHFtZXLU6gTWHXXn%2FyOo%3D&reserved=0 just a few weeks later. Does that mean jaxp has an issue with later jtreg builds? Or was that a mistake? I don't see why 8219705 can require to change requiredVersion, nor do I see it being discussed in RFR (https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openjdk.java.net%2Fpipermail%2Fcore-libs-dev%2F2019-March%2F058923.html&data=02%7C01%7Ckusrinivasan%40vmware.com%7C8e1128e05e8a441dfd8e08d77f983396%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637118164005266666&sdata=tJ74fk0nGaqZ6ROHDYgI96CGeirjKeViQ%2BkkOhWRjN4%3D&reserved=0 ), so I assume it's just unrelated local changes which got integrated. but just to be safe, @Joe/Lance, were there any reasons why you needed to switch jtreg's requiredVersion back to 4.2b13 in jaxp? are there any reasons which prevent jaxp from switching to jtreg4.2b16 now? Otherwise changes seem fine to me. When was b16 released? the tag was added by Jon on Dec 03 2019. so I believe there was enough time for usual suspects to build/adjust, and I'm aware about at least one publicly available place where people can get the latest build of jtreg, if they have problems w/ building it themselves. This is broken since Dec 3. Do you have a contact ? you may want to let them know. Likely cause: The jerkins build is invoking the build script from the wrong directory. https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/ Kumar Thanks, -- Igor Thanks, David JBS: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.openjdk.java.net%2Fbrowse%2FJDK-8235866&data=02%7C01%7Ckusrinivasan%40vmware.com%7C8e1128e05e8a441dfd8e08d77f983396%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637118164005266666&sdata=BIhDQmqlS8eb1i70LoIFYWl5xgLxf9TmsDhLCtkxabw%3D&reserved=0 [1] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.openjdk.java.net%2Fbrowse%2FJDK-8230067&data=02%7C01%7Ckusrinivasan%40vmware.com%7C8e1128e05e8a441dfd8e08d77f983396%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637118164005266666&sdata=XA0hRPuPrvzK%2B0S25ALE2w1JWUVsUR%2B0ncwbgzpdn3c%3D&reserved=0 Thanks, -- Igor From igor.ignatyev at oracle.com Fri Dec 13 16:36:50 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 13 Dec 2019 08:36:50 -0800 Subject: [14] RFR(S/T) : 8235866 : bump jtreg requiredVersion to 4.2b16 In-Reply-To: <7AC57A49-095F-4A2C-A1A3-4E2390702BB7@vmware.com> References: <764BB5FF-AC59-40E2-AB5A-F63A745E9AEE@oracle.com> <7522383C-1C26-4422-83CA-BB4E795CCF6B@oracle.com> <7AC57A49-095F-4A2C-A1A3-4E2390702BB7@vmware.com> Message-ID: <57E4CACC-AF3A-40C8-A36E-5C8DF57D8613@oracle.com> Hi Kumar, I haven't heard adoptopenjdk people bringing it up. I agree w/ your hypothesis, this job started to fail after https://hg.openjdk.java.net/code-tools/jtreg/rev/deee95d5d8ff which checks that '.hg' directory exists in the current directory. -- Igor > On Dec 13, 2019, at 6:28 AM, Kumar Srinivasan wrote: > > This is broken since Dec 3. Do you have a contact ? you may want to let them know. > Likely cause: The jerkins build is invoking the build script from the wrong directory. > https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/ From jonathan.gibbons at oracle.com Fri Dec 13 16:47:26 2019 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 13 Dec 2019 08:47:26 -0800 Subject: [14] RFR(S/T) : 8235866 : bump jtreg requiredVersion to 4.2b16 In-Reply-To: <57E4CACC-AF3A-40C8-A36E-5C8DF57D8613@oracle.com> References: <764BB5FF-AC59-40E2-AB5A-F63A745E9AEE@oracle.com> <7522383C-1C26-4422-83CA-BB4E795CCF6B@oracle.com> <7AC57A49-095F-4A2C-A1A3-4E2390702BB7@vmware.com> <57E4CACC-AF3A-40C8-A36E-5C8DF57D8613@oracle.com> Message-ID: <1b5732f4-8520-1eaa-e688-4d4a0080f65d@oracle.com> I've been trying to contact AdoptOpenJDK folk about this and other related issues. I've not heard back yet. In the meantime, I note that these days it is reasonably easy for folk to build jtreg if they do not have easy access to the latest prebuilt binaries. There's a README in the root of the jtreg repo with details. http://hg.openjdk.java.net/code-tools/jtreg/file/c82fc1196801/README -- Jon On 12/13/19 8:36 AM, Igor Ignatyev wrote: > Hi Kumar, > > I haven't heard adoptopenjdk people bringing it up. I agree w/ your > hypothesis, this job started to fail after > https://hg.openjdk.java.net/code-tools/jtreg/rev/deee95d5d8ff?which > checks that '.hg' directory exists in the current directory. > > -- Igor > >> On Dec 13, 2019, at 6:28 AM, Kumar Srinivasan >> > wrote: >> >> This is broken since Dec 3. Do you have a contact ? you may want to >> let them know. >> Likely cause: The jerkins build is ?invoking the build script from >> the wrong directory. >> https://ci.adoptopenjdk.net/view/Dependencies/job/jtreg/ > From ioi.lam at oracle.com Sat Dec 14 23:21:11 2019 From: ioi.lam at oracle.com (Ioi Lam) Date: Sat, 14 Dec 2019 15:21:11 -0800 Subject: RFR(S) 8199290 [TESTBUG] sun.hotspot.WhiteBox$WhiteBoxPermission is not copied Message-ID: <5c84da2d-452f-21fe-2427-e783da804c14@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8199290 http://cr.openjdk.java.net/~iklam/jdk15/8199290-copy-whitebox-inner-cls.v01/ We have many cases of * @run driver ClassFileInstaller sun.hotspot.WhiteBox which should have been * @run driver ClassFileInstaller sun.hotspot.WhiteBox sun.hotspot.WhiteBox$WhiteBoxPermission It will be too tedious to modify these cases. So let's just fix ClassFileInstaller to automatically the sun.hotspot.WhiteBox$WhiteBoxPermission when WhiteBox is installed. The JarBuilder class used by the AppCDS tests is also modified similarly. Thanks - Ioi From igor.ignatyev at oracle.com Sat Dec 14 23:40:46 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Sat, 14 Dec 2019 15:40:46 -0800 Subject: RFR(S) 8199290 [TESTBUG] sun.hotspot.WhiteBox$WhiteBoxPermission is not copied In-Reply-To: <5c84da2d-452f-21fe-2427-e783da804c14@oracle.com> References: <5c84da2d-452f-21fe-2427-e783da804c14@oracle.com> Message-ID: <066C7A48-ECFD-490C-A103-BB889FEFF382@oracle.com> Hi Ioi, looks good to me, one nit though: wouldn't it be better to use switch instead of if-elseif at ClassFileInstaller.java:L#115-119 and JarBuilder.java:L140-144? Thanks, -- Igor > On Dec 14, 2019, at 3:21 PM, Ioi Lam wrote: > > https://bugs.openjdk.java.net/browse/JDK-8199290 > http://cr.openjdk.java.net/~iklam/jdk15/8199290-copy-whitebox-inner-cls.v01/ > > We have many cases of > > * @run driver ClassFileInstaller sun.hotspot.WhiteBox > > which should have been > > * @run driver ClassFileInstaller sun.hotspot.WhiteBox sun.hotspot.WhiteBox$WhiteBoxPermission > > It will be too tedious to modify these cases. So let's just fix ClassFileInstaller > to automatically the sun.hotspot.WhiteBox$WhiteBoxPermission when WhiteBox is installed. > > The JarBuilder class used by the AppCDS tests is also modified similarly. > > Thanks > - Ioi From glaubitz at physik.fu-berlin.de Sun Dec 15 15:20:57 2019 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Sun, 15 Dec 2019 16:20:57 +0100 Subject: RFH: Debugging Zero crashing on Linux/SPARC Message-ID: <629a9af0-e79c-7d0b-46f9-1cd3c479f29d@physik.fu-berlin.de> Hi! Since we are limited to Zero on Linux/SPARC in the foreseeable future, I would like to make sure that Zero works properly again on Linux/SPARC. Unfortunately, there is still an issue with Zero crashing on SPARC when built with gcc-9. I have so far not figured out what the problem is, but the gdb backtrace looks like below. Can anyone give a pointer where to poke? Thanks, Adrian glaubitz at gcc202:~/jdk$ gdb /home/glaubitz/jdk/build/linux-sparcv9-zero-release/jdk/bin/java GNU gdb (Debian 8.3.1-1) 8.3.1 Copyright (C) 2019 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "sparc64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /home/glaubitz/jdk/build/linux-sparcv9-zero-release/jdk/bin/java... (No debugging symbols found in /home/glaubitz/jdk/build/linux-sparcv9-zero-release/jdk/bin/java) (gdb) r Starting program: /home/glaubitz/jdk/build/linux-sparcv9-zero-release/jdk/bin/java [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/sparc64-linux-gnu/libthread_db.so.1". [New Thread 0xfff80001012f9900 (LWP 212322)] Thread 2 "java" received signal SIGBUS, Bus error. [Switching to Thread 0xfff80001012f9900 (LWP 212322)] 0xfff8000100bf506c in SharedRuntime::generate_stubs () at /home/glaubitz/jdk/src/hotspot/share/runtime/sharedRuntime.cpp:106 106 _resolve_static_call_blob = generate_resolve_blob(CAST_FROM_FN_PTR(address, SharedRuntime::resolve_static_call_C), "resolve_static_call"); (gdb) bt #0 0xfff8000100bf506c in SharedRuntime::generate_stubs () at /home/glaubitz/jdk/src/hotspot/share/runtime/sharedRuntime.cpp:106 #1 0xfff80001009e4bfc in init_globals () at /home/glaubitz/jdk/src/hotspot/share/runtime/init.cpp:129 #2 0xfff8000100c431a4 in Threads::create_vm (args=, canTryAgain=0xfff80001012f8b7f) at /home/glaubitz/jdk/src/hotspot/share/runtime/thread.cpp:3887 #3 0xfff8000100a28198 in JNI_CreateJavaVM_inner (args=0xfff80001012f8ca8, penv=0xfff80001012f8ca0, vm=0xfff80001012f8c98) at /home/glaubitz/jdk/src/hotspot/share/prims/jni.cpp:3852 #4 JNI_CreateJavaVM (vm=0xfff80001012f8c98, penv=0xfff80001012f8ca0, args=0xfff80001012f8ca8) at /home/glaubitz/jdk/src/hotspot/share/prims/jni.cpp:3935 #5 0xfff800010024bb3c in InitializeJVM (ifn=, penv=0xfff80001012f8ca0, pvm=0xfff80001012f8c98) at /home/glaubitz/jdk/src/java.base/share/native/libjli/java.c:1538 #6 JavaMain (_args=) at /home/glaubitz/jdk/src/java.base/share/native/libjli/java.c:417 #7 0xfff8000100250d64 in ThreadJavaMain (args=0x7feffffae60) at /home/glaubitz/jdk/src/java.base/unix/native/libjli/java_md_solinux.c:734 #8 0xfff8000100365010 in start_thread (arg=0xfff80001012f9900) at pthread_create.c:486 #9 0xfff8000100672154 in __thread_start () at ../sysdeps/unix/sysv/linux/sparc/sparc64/clone.S:78 Backtrace stopped: previous frame identical to this frame (corrupt stack?) (gdb) -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From david.holmes at oracle.com Sun Dec 15 22:40:24 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 16 Dec 2019 08:40:24 +1000 Subject: RFR(S) 8199290 [TESTBUG] sun.hotspot.WhiteBox$WhiteBoxPermission is not copied In-Reply-To: <5c84da2d-452f-21fe-2427-e783da804c14@oracle.com> References: <5c84da2d-452f-21fe-2427-e783da804c14@oracle.com> Message-ID: Hi Ioi, This seems okay to me. I thought perhaps you would "simply" copy all inner classes for a given outer class, but perhaps we don't have a way to actually determine that set of classes? I'm also not sure how this will look if we added another pair ... I don't see how to generalize the logic. But for now to solve this particular problem (not that we seem to have been bitten by it for a while) this seems okay. I assume this is a debugging leftover though: + Thread.dumpStack(); Also style nit: i https://bugs.openjdk.java.net/browse/JDK-8199290 > http://cr.openjdk.java.net/~iklam/jdk15/8199290-copy-whitebox-inner-cls.v01/ > > > We have many cases of > > * @run driver ClassFileInstaller sun.hotspot.WhiteBox > > which should have been > > * @run driver ClassFileInstaller sun.hotspot.WhiteBox > sun.hotspot.WhiteBox$WhiteBoxPermission > > It will be too tedious to modify these cases. So let's just fix > ClassFileInstaller > to automatically the sun.hotspot.WhiteBox$WhiteBoxPermission when > WhiteBox is installed. > > The JarBuilder class used by the AppCDS tests is also modified similarly. > > Thanks > - Ioi From david.holmes at oracle.com Mon Dec 16 05:16:09 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 16 Dec 2019 15:16:09 +1000 Subject: RFR: 8235966: Process obsolete flags less aggressively Message-ID: Bug: https://bugs.openjdk.java.net/browse/JDK-8235966 webrev: http://cr.openjdk.java.net/~dholmes/8235966/webrev/ When a flag is marked as obsolete in the special-flags table we will ignore it and issue a warning that it is being ignored, as soon as we bump the version of the JDK. That means that any tests still using the obsolete flag may start to fail, leading to a surge of test failures at the start of a release cycle. For example for JDK 15 we have a whole bunch of JFR tests that fail because they still try to work with UseParallelOldGC. In another case runtime/cds/appcds/FieldLayoutFlags.java passes only be accident. When a flag is marked as obsolete for a release, all code involving that flag (including tests that use it) must be updated within that release and the flag itself removed. Whilst this is typically scheduled early in a release cycle it isn't reasonable to expect it to all occur within the first couple of days of the release cycle, nor do we want to have to ProblemList a bunch of tests when they start failing. What I propose is to instead allow an obsolete flag to continue to be processed as long as that code removal has not actually occurred - with an adjusted warning. The change I propose: - only treats an obsolete flag as obsolete if the flag cannot be found - added a new flag verification rule that disallows obsoletion in an undefined version, but expiration in a specific version i.e. we must always explicitly obsolete a flag before we expire it. The only downside here is that if we actually forget to file an issue for the actual obsoletion work we may not notice via testing. Of course whenever a change is made to the flags table to add an entry then the issue to do the obsoletion should be filed at the same time. Thanks, David ----- From ioi.lam at oracle.com Mon Dec 16 05:18:59 2019 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 15 Dec 2019 21:18:59 -0800 Subject: RFR(S) 8199290 [TESTBUG] sun.hotspot.WhiteBox$WhiteBoxPermission is not copied In-Reply-To: References: <5c84da2d-452f-21fe-2427-e783da804c14@oracle.com> Message-ID: Hi David, I had an earlier version (see comments in the JBS page) that discovers all the inner classes and copies them. However, some tests may actually want to leave out some inner classes (perhaps to test edge cases in inner class handling??). I'd rather leave that for a future RFE. I'll fix the issues you identified below. Thanks - Ioi On 12/15/19 2:40 PM, David Holmes wrote: > Hi Ioi, > > This seems okay to me. I thought perhaps you would "simply" copy all > inner classes for a given outer class, but perhaps we don't have a way > to actually determine that set of classes? I'm also not sure how this > will look if we added another pair ... I don't see how to generalize > the logic. But for now to solve this particular problem (not that we > seem to have been bitten by it for a while) this seems okay. > > I assume this is a debugging leftover though: > > +???????????? Thread.dumpStack(); > > Also style nit: > > i > spaces around < > > Thanks, > David > > On 15/12/2019 9:21 am, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8199290 >> http://cr.openjdk.java.net/~iklam/jdk15/8199290-copy-whitebox-inner-cls.v01/ >> >> >> We have many cases of >> >> * @run driver ClassFileInstaller sun.hotspot.WhiteBox >> >> which should have been >> >> * @run driver ClassFileInstaller sun.hotspot.WhiteBox >> sun.hotspot.WhiteBox$WhiteBoxPermission >> >> It will be too tedious to modify these cases. So let's just fix >> ClassFileInstaller >> to automatically the sun.hotspot.WhiteBox$WhiteBoxPermission when >> WhiteBox is installed. >> >> The JarBuilder class used by the AppCDS tests is also modified >> similarly. >> >> Thanks >> - Ioi From david.holmes at oracle.com Mon Dec 16 05:22:27 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 16 Dec 2019 15:22:27 +1000 Subject: RFR(S) 8199290 [TESTBUG] sun.hotspot.WhiteBox$WhiteBoxPermission is not copied In-Reply-To: References: <5c84da2d-452f-21fe-2427-e783da804c14@oracle.com> Message-ID: On 16/12/2019 3:18 pm, Ioi Lam wrote: > Hi David, > > I had an earlier version (see comments in the JBS page) that discovers > all the inner classes and copies them. However, some tests may actually > want to leave out some inner classes (perhaps to test edge cases in > inner class handling??). I'd rather leave that for a future RFE. Sure. Using a white-list is not terrible and until we have more usecases it isn't clear the best way to handle it. Cheers, David > I'll fix the issues you identified below. > > Thanks > - Ioi > > On 12/15/19 2:40 PM, David Holmes wrote: >> Hi Ioi, >> >> This seems okay to me. I thought perhaps you would "simply" copy all >> inner classes for a given outer class, but perhaps we don't have a way >> to actually determine that set of classes? I'm also not sure how this >> will look if we added another pair ... I don't see how to generalize >> the logic. But for now to solve this particular problem (not that we >> seem to have been bitten by it for a while) this seems okay. >> >> I assume this is a debugging leftover though: >> >> +???????????? Thread.dumpStack(); >> >> Also style nit: >> >> i> >> spaces around < >> >> Thanks, >> David >> >> On 15/12/2019 9:21 am, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8199290 >>> http://cr.openjdk.java.net/~iklam/jdk15/8199290-copy-whitebox-inner-cls.v01/ >>> >>> >>> We have many cases of >>> >>> * @run driver ClassFileInstaller sun.hotspot.WhiteBox >>> >>> which should have been >>> >>> * @run driver ClassFileInstaller sun.hotspot.WhiteBox >>> sun.hotspot.WhiteBox$WhiteBoxPermission >>> >>> It will be too tedious to modify these cases. So let's just fix >>> ClassFileInstaller >>> to automatically the sun.hotspot.WhiteBox$WhiteBoxPermission when >>> WhiteBox is installed. >>> >>> The JarBuilder class used by the AppCDS tests is also modified >>> similarly. >>> >>> Thanks >>> - Ioi > From ioi.lam at oracle.com Mon Dec 16 05:36:22 2019 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 15 Dec 2019 21:36:22 -0800 Subject: RFR(S) 8199290 [TESTBUG] sun.hotspot.WhiteBox$WhiteBoxPermission is not copied In-Reply-To: <066C7A48-ECFD-490C-A103-BB889FEFF382@oracle.com> References: <5c84da2d-452f-21fe-2427-e783da804c14@oracle.com> <066C7A48-ECFD-490C-A103-BB889FEFF382@oracle.com> Message-ID: <69f6fbaa-2b84-dcb1-c3e5-e377e8dc43ab@oracle.com> Hi Igor, Thanks for the review. Here's an updated version that uses "switch" as you suggested. It also fixes the problems identified by David: http://cr.openjdk.java.net/~iklam/jdk15/8199290-copy-whitebox-inner-cls.v02/ Thanks - Ioi On 12/14/19 3:40 PM, Igor Ignatyev wrote: > Hi Ioi, > > looks good to me, one nit though: wouldn't it be better to use switch instead of if-elseif at ClassFileInstaller.java:L#115-119 and JarBuilder.java:L140-144? > > Thanks, > -- Igor > >> On Dec 14, 2019, at 3:21 PM, Ioi Lam wrote: >> >> https://bugs.openjdk.java.net/browse/JDK-8199290 >> http://cr.openjdk.java.net/~iklam/jdk15/8199290-copy-whitebox-inner-cls.v01/ >> >> We have many cases of >> >> * @run driver ClassFileInstaller sun.hotspot.WhiteBox >> >> which should have been >> >> * @run driver ClassFileInstaller sun.hotspot.WhiteBox sun.hotspot.WhiteBox$WhiteBoxPermission >> >> It will be too tedious to modify these cases. So let's just fix ClassFileInstaller >> to automatically the sun.hotspot.WhiteBox$WhiteBoxPermission when WhiteBox is installed. >> >> The JarBuilder class used by the AppCDS tests is also modified similarly. >> >> Thanks >> - Ioi From martin.doerr at sap.com Mon Dec 16 10:09:23 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 16 Dec 2019 10:09:23 +0000 Subject: RFR [XS]: 8235671: enhance print_rlimit_info in os_posix In-Reply-To: References: Message-ID: Hi Matthias, looks good to me. Best regards, Martin > -----Original Message----- > From: hotspot-dev On Behalf Of > Baesken, Matthias > Sent: Mittwoch, 11. Dezember 2019 09:06 > To: 'hotspot-dev at openjdk.java.net' > Subject: RFR [XS]: 8235671: enhance print_rlimit_info in os_posix > > Hello, please review this small change . > > It adds 2 rlimit infos to os::Posix::print_rlimit_info . > > > One is cross posix : RLIMIT_CPU > > http://man7.org/linux/man-pages/man2/getrlimit.2.html > > RLIMIT_CPU > This is a limit, in seconds, on the amount of CPU time that > the process can consume. When the process reaches the soft > limit, it is sent a SIGXCPU signal. ... > > > One is AIX-only : RLIMIT_THREADS > > Manpage : > > RLIMIT_THREADS > The maximum number of threads each process can create. This limit > is enforced by the kernel and > the pthread library. > > > > Bug/web : > > https://bugs.openjdk.java.net/browse/JDK-8235671 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235671.0/ > > > Thanks, Matthias From matthias.baesken at sap.com Mon Dec 16 10:11:41 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 16 Dec 2019 10:11:41 +0000 Subject: RFR [XS]: 8235671: enhance print_rlimit_info in os_posix In-Reply-To: References: Message-ID: Thanks ! A second review would be great . Best regards, Matthias > > Hi Matthias, > > looks good to me. > > Best regards, > Martin > > > > -----Original Message----- > > From: hotspot-dev On Behalf > Of > > Baesken, Matthias > > Sent: Mittwoch, 11. Dezember 2019 09:06 > > To: 'hotspot-dev at openjdk.java.net' > > Subject: RFR [XS]: 8235671: enhance print_rlimit_info in os_posix > > > > Hello, please review this small change . > > > > It adds 2 rlimit infos to os::Posix::print_rlimit_info . > > > > > > One is cross posix : RLIMIT_CPU > > > > http://man7.org/linux/man-pages/man2/getrlimit.2.html > > > > RLIMIT_CPU > > This is a limit, in seconds, on the amount of CPU time that > > the process can consume. When the process reaches the soft > > limit, it is sent a SIGXCPU signal. ... > > > > > > One is AIX-only : RLIMIT_THREADS > > > > Manpage : > > > > RLIMIT_THREADS > > The maximum number of threads each process can create. This > limit > > is enforced by the kernel and > > the pthread library. > > > > > > > > Bug/web : > > > > https://bugs.openjdk.java.net/browse/JDK-8235671 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235671.0/ > > > > > > Thanks, Matthias From coleen.phillimore at oracle.com Mon Dec 16 11:41:05 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 16 Dec 2019 06:41:05 -0500 Subject: [14] RFR 8235829: graal crashes with Zombie.java test Message-ID: Summary: Start ServiceThread before compiler threads, and run nmethod barriers for zgc before adding to the service thread queue, or posting the events on the java thread queue. See bug for description of the problems found with the new Zombie.java test. open webrev at http://cr.openjdk.java.net/~coleenp/2019/8235829.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8235829 Ran tier1 all platforms, and tier2-8 testing, as well as rerunning original test failure from bug https://bugs.openjdk.java.net/browse/JDK-8173361. Thanks, Coleen From david.holmes at oracle.com Mon Dec 16 13:04:34 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 16 Dec 2019 23:04:34 +1000 Subject: [14] RFR 8235829: graal crashes with Zombie.java test In-Reply-To: References: Message-ID: <447990f9-d276-56c3-aebe-fa84d0cfcc27@oracle.com> Hi Coleen, On 16/12/2019 9:41 pm, coleen.phillimore at oracle.com wrote: > Summary: Start ServiceThread before compiler threads, and run nmethod > barriers for zgc before adding to the service thread queue, or posting > the events on the java thread queue. I can't comment on most of this but the earlier starting of the service thread has some concerns: - there is a lot of JDK level initialization which now will not have happened before the service thread is started and it is far from obvious that all possible initialization dependencies will be satisfied - current starting of the service thread in Management::initialize is guarded by "#if INCLUDE_MANAGEMENT", but now you are starting the service thread unconditionally for all builds. Hmm just saw your latest comment to the bug report - so the service thread is now (for quite some time?) being used for other than management tasks and so should always be present even if INCLUDE_MANAGEMENT is not enabled. Is that sufficient or are there likely to be other changes needed to actually ensure that all works correctly e.g. any code the service thread executes that is only defined for INCLUDE_MANAGEMENT will need to be compiled out explicitly. - the service thread and the notification thread are (were?) closely related but now started at completely different times The bug report states the problem as: "The graal crash is because compiled_method_load events are added to the ServiceThread's deferred event queue before the ServiceThread is created so are not walked to keep them from being zombied." so why isn't the solution to ensure the deferred event queue is walked? I'm not clear how starting the service thread relates to walking the queue. Thanks, David > See bug for description of the problems found with the new Zombie.java > test. > > open webrev at http://cr.openjdk.java.net/~coleenp/2019/8235829.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8235829 > > Ran tier1 all platforms, and tier2-8 testing, as well as rerunning > original test failure from bug > https://bugs.openjdk.java.net/browse/JDK-8173361. > > Thanks, > Coleen From sgehwolf at redhat.com Mon Dec 16 13:16:21 2019 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 16 Dec 2019 14:16:21 +0100 Subject: [14] RFR(S/T) : 8235866 : bump jtreg requiredVersion to 4.2b16 In-Reply-To: <1b5732f4-8520-1eaa-e688-4d4a0080f65d@oracle.com> References: <764BB5FF-AC59-40E2-AB5A-F63A745E9AEE@oracle.com> <7522383C-1C26-4422-83CA-BB4E795CCF6B@oracle.com> <7AC57A49-095F-4A2C-A1A3-4E2390702BB7@vmware.com> <57E4CACC-AF3A-40C8-A36E-5C8DF57D8613@oracle.com> <1b5732f4-8520-1eaa-e688-4d4a0080f65d@oracle.com> Message-ID: Hi Jon, On Fri, 2019-12-13 at 08:47 -0800, Jonathan Gibbons wrote: > I've been trying to contact AdoptOpenJDK folk about this and other > related issues. I've not heard back yet. It's tracked at AdoptOpenJDK via: https://github.com/AdoptOpenJDK/openjdk-build/issues/1445 Thanks, Severin From coleen.phillimore at oracle.com Mon Dec 16 13:26:58 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 16 Dec 2019 08:26:58 -0500 Subject: [14] RFR 8235829: graal crashes with Zombie.java test In-Reply-To: <447990f9-d276-56c3-aebe-fa84d0cfcc27@oracle.com> References: <447990f9-d276-56c3-aebe-fa84d0cfcc27@oracle.com> Message-ID: <975b6e46-95c1-8a37-b160-b6a57a9633a8@oracle.com> On 12/16/19 8:04 AM, David Holmes wrote: > Hi Coleen, > > On 16/12/2019 9:41 pm, coleen.phillimore at oracle.com wrote: >> Summary: Start ServiceThread before compiler threads, and run nmethod >> barriers for zgc before adding to the service thread queue, or >> posting the events on the java thread queue. > > I can't comment on most of this but the earlier starting of the > service thread has some concerns: > > - there is a lot of JDK level initialization which now will not have > happened before the service thread is started and it is far from > obvious that all possible initialization dependencies will be satisfied I agree that the order of initialization is very sensitive.? From the actions that the service thread does, the one that I found was a problem was that events were posted before the LIVE phase (see comment in has_events()), which could have happened with the existing code, but the window for the race is a lot smaller. ? The other actions can be run if there's a GC before initialization but would be a bug in the initialization code, and I didn't find these bugs in all my testing.? There are some ordering dependencies that do have odd side effects (between the compiler thread startup and initialization jsr292 classes) which have comments.? This patch doesn't touch those. > > - current starting of the service thread in Management::initialize is > guarded by "#if INCLUDE_MANAGEMENT", but now you are starting the > service thread unconditionally for all builds. Hmm just saw your > latest comment to the bug report - so the service thread is now (for > quite some time?) being used for other than management tasks and so > should always be present even if INCLUDE_MANAGEMENT is not enabled. Is > that sufficient or are there likely to be other changes needed to > actually ensure that all works correctly? e.g. any code the service > thread executes that is only defined for INCLUDE_MANAGEMENT will need > to be compiled out explicitly. > I asked Jie offline to check the minimal build.? I don't think there are other INCLUDE_MANAGEMENT actions in the service thread and I'm not sure why it was initialized there in the first place.? The minimal vm would have been broken ie. hashtables would not have been cleaned up, etc, but I'm not sure how well that is tested or if one would notice. > - the service thread and the notification thread are (were?) closely > related but now started at completely different times The notification thread is limited to "services" so it makes sense where it is.? The ServiceThread does lots of other things.? Maybe it needs renaming in 15. > > The bug report states the problem as: > > "The graal crash is because compiled_method_load events are added to > the ServiceThread's deferred event queue before the ServiceThread is > created so are not walked to keep them from being zombied." > > so why isn't the solution to ensure the deferred event queue is > walked? I'm not clear how starting the service thread relates to > walking the queue. > The service thread is responsible for walking the deferred event queue.?? See ServiceThread::oops_do/nmethods_do.?? The design could be changed to have some global walk somewhere of this queue, but essentially this queue is processed by the service thread. I had an additional change to make the queue non-static but want to limit the change at this point. Thanks, Coleen > Thanks, > David > >> See bug for description of the problems found with the new >> Zombie.java test. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2019/8235829.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8235829 >> >> Ran tier1 all platforms, and tier2-8 testing, as well as rerunning >> original test failure from bug >> https://bugs.openjdk.java.net/browse/JDK-8173361. >> >> Thanks, >> Coleen From christoph.langer at sap.com Mon Dec 16 13:51:22 2019 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 16 Dec 2019 13:51:22 +0000 Subject: RFR [XS]: 8235671: enhance print_rlimit_info in os_posix In-Reply-To: References: Message-ID: Hi Matthias, +1 ?? Cheers Christoph > -----Original Message----- > From: hotspot-dev On Behalf Of > Baesken, Matthias > Sent: Montag, 16. Dezember 2019 11:12 > To: Doerr, Martin ; 'hotspot- > dev at openjdk.java.net' > Subject: RE: RFR [XS]: 8235671: enhance print_rlimit_info in os_posix > > Thanks ! > > A second review would be great . > > Best regards, Matthias > > > > > > Hi Matthias, > > > > looks good to me. > > > > Best regards, > > Martin > > > > > > > -----Original Message----- > > > From: hotspot-dev On Behalf > > Of > > > Baesken, Matthias > > > Sent: Mittwoch, 11. Dezember 2019 09:06 > > > To: 'hotspot-dev at openjdk.java.net' > > > Subject: RFR [XS]: 8235671: enhance print_rlimit_info in os_posix > > > > > > Hello, please review this small change . > > > > > > It adds 2 rlimit infos to os::Posix::print_rlimit_info . > > > > > > > > > One is cross posix : RLIMIT_CPU > > > > > > http://man7.org/linux/man-pages/man2/getrlimit.2.html > > > > > > RLIMIT_CPU > > > This is a limit, in seconds, on the amount of CPU time that > > > the process can consume. When the process reaches the soft > > > limit, it is sent a SIGXCPU signal. ... > > > > > > > > > One is AIX-only : RLIMIT_THREADS > > > > > > Manpage : > > > > > > RLIMIT_THREADS > > > The maximum number of threads each process can create. This > > limit > > > is enforced by the kernel and > > > the pthread library. > > > > > > > > > > > > Bug/web : > > > > > > https://bugs.openjdk.java.net/browse/JDK-8235671 > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8235671.0/ > > > > > > > > > Thanks, Matthias From huizhe.wang at oracle.com Mon Dec 16 17:54:50 2019 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 16 Dec 2019 09:54:50 -0800 Subject: [14] RFR(S/T) : 8235866 : bump jtreg requiredVersion to 4.2b16 In-Reply-To: <7522383C-1C26-4422-83CA-BB4E795CCF6B@oracle.com> References: <764BB5FF-AC59-40E2-AB5A-F63A745E9AEE@oracle.com> <7522383C-1C26-4422-83CA-BB4E795CCF6B@oracle.com> Message-ID: <6e0dc2e5-607c-b991-dad3-62fec79cb43d@oracle.com> On 12/12/19 10:46 PM, Igor Ignatyev wrote: > @Joe/Lance, > were there any reasons why you needed to switch jtreg's > requiredVersion back to 4.2b13 in jaxp? are there any reasons which > prevent jaxp from switching to jtreg4.2b16 now? > I don't recall any specific reason, if any. It would be fine to switch to jtreg4.2b16. Thanks, Joe From ioi.lam at oracle.com Mon Dec 16 22:50:44 2019 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 16 Dec 2019 14:50:44 -0800 Subject: RFR(T) [TESTBUG] MismatchedWhiteBox test fails with missing WhiteBox$WhiteBoxPermission.class Message-ID: <12121a1d-44ed-fd21-c27d-a1c4de61e365@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8236045 After JDK-8199290, ClassFileInstaller is hard-coded to install WhiteBox$WhiteBoxPermission.class alongside WhiteBox.class. So in the MismatchedWhiteBox test, which creates a different version of the WhiteBox.class, we should have the inner class as well. diff -r 520a513294c3 test/hotspot/jtreg/sanity/MismatchedWhiteBox/WhiteBox.java --- a/test/hotspot/jtreg/sanity/MismatchedWhiteBox/WhiteBox.java Mon Dec 16 14:30:12 2019 -0800 +++ b/test/hotspot/jtreg/sanity/MismatchedWhiteBox/WhiteBox.java Mon Dec 16 14:48:30 2019 -0800 @@ -36,6 +36,15 @@ ?package sun.hotspot; ?public class WhiteBox { +??? @SuppressWarnings("serial") +??? public static class WhiteBoxPermission extends java.security.BasicPermission { +??????? // ClassFileInstaller is hard-coded to copy WhiteBox$WhiteBoxPermission, so let's +??????? // make a fake one here as well. +??????? public WhiteBoxPermission(String s) { +??????????? super(s); +??????? } +??? } + ???? private static native void registerNatives(); ???? static { registerNatives(); } ???? public native int notExistedMethod(); Thanks - Ioi From david.holmes at oracle.com Mon Dec 16 22:51:00 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 17 Dec 2019 08:51:00 +1000 Subject: [14] RFR 8235829: graal crashes with Zombie.java test In-Reply-To: <975b6e46-95c1-8a37-b160-b6a57a9633a8@oracle.com> References: <447990f9-d276-56c3-aebe-fa84d0cfcc27@oracle.com> <975b6e46-95c1-8a37-b160-b6a57a9633a8@oracle.com> Message-ID: <06d6e9f4-b919-3062-7d18-1245e66b27b2@oracle.com> Hi Coleen, A quick initial response ... On 16/12/2019 11:26 pm, coleen.phillimore at oracle.com wrote: > > > On 12/16/19 8:04 AM, David Holmes wrote: >> Hi Coleen, >> >> On 16/12/2019 9:41 pm, coleen.phillimore at oracle.com wrote: >>> Summary: Start ServiceThread before compiler threads, and run nmethod >>> barriers for zgc before adding to the service thread queue, or >>> posting the events on the java thread queue. >> >> I can't comment on most of this but the earlier starting of the >> service thread has some concerns: >> >> - there is a lot of JDK level initialization which now will not have >> happened before the service thread is started and it is far from >> obvious that all possible initialization dependencies will be satisfied > > I agree that the order of initialization is very sensitive.? From the > actions that the service thread does, the one that I found was a problem > was that events were posted before the LIVE phase (see comment in > has_events()), which could have happened with the existing code, but the > window for the race is a lot smaller. ? The other actions can be run if > there's a GC before initialization but would be a bug in the > initialization code, and I didn't find these bugs in all my testing. > There are some ordering dependencies that do have odd side effects > (between the compiler thread startup and initialization jsr292 classes) > which have comments.? This patch doesn't touch those. > >> >> - current starting of the service thread in Management::initialize is >> guarded by "#if INCLUDE_MANAGEMENT", but now you are starting the >> service thread unconditionally for all builds. Hmm just saw your >> latest comment to the bug report - so the service thread is now (for >> quite some time?) being used for other than management tasks and so >> should always be present even if INCLUDE_MANAGEMENT is not enabled. Is >> that sufficient or are there likely to be other changes needed to >> actually ensure that all works correctly? e.g. any code the service >> thread executes that is only defined for INCLUDE_MANAGEMENT will need >> to be compiled out explicitly. >> > > I asked Jie offline to check the minimal build.? I don't think there are > other INCLUDE_MANAGEMENT actions in the service thread and I'm not sure > why it was initialized there in the first place.? The minimal vm would > have been broken ie. hashtables would not have been cleaned up, etc, but > I'm not sure how well that is tested or if one would notice. >> - the service thread and the notification thread are (were?) closely >> related but now started at completely different times > > The notification thread is limited to "services" so it makes sense where > it is.? The ServiceThread does lots of other things.? Maybe it needs > renaming in 15. >> >> The bug report states the problem as: >> >> "The graal crash is because compiled_method_load events are added to >> the ServiceThread's deferred event queue before the ServiceThread is >> created so are not walked to keep them from being zombied." >> >> so why isn't the solution to ensure the deferred event queue is >> walked? I'm not clear how starting the service thread relates to >> walking the queue. >> > > The service thread is responsible for walking the deferred event > queue.?? See ServiceThread::oops_do/nmethods_do.?? The design could be > changed to have some global walk somewhere of this queue, but > essentially this queue is processed by the service thread. Sorry I don't follow. I thought "oops_do" and friends are for the GC threads and/or VMThread to call to process oops when GC updates them. David ----- > I had an additional change to make the queue non-static but want to > limit the change at this point. > > Thanks, > Coleen >> Thanks, >> David >> >>> See bug for description of the problems found with the new >>> Zombie.java test. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2019/8235829.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8235829 >>> >>> Ran tier1 all platforms, and tier2-8 testing, as well as rerunning >>> original test failure from bug >>> https://bugs.openjdk.java.net/browse/JDK-8173361. >>> >>> Thanks, >>> Coleen > From calvin.cheung at oracle.com Mon Dec 16 23:13:33 2019 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Mon, 16 Dec 2019 15:13:33 -0800 Subject: RFR(T) [TESTBUG] MismatchedWhiteBox test fails with missing WhiteBox$WhiteBoxPermission.class In-Reply-To: <12121a1d-44ed-fd21-c27d-a1c4de61e365@oracle.com> References: <12121a1d-44ed-fd21-c27d-a1c4de61e365@oracle.com> Message-ID: Looks good. thanks, Calvin On 12/16/19 2:50 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8236045 > > > After JDK-8199290, ClassFileInstaller is hard-coded to install > WhiteBox$WhiteBoxPermission.class alongside WhiteBox.class. So in the > MismatchedWhiteBox test, which creates a different version of the > WhiteBox.class, we should have the inner class as well. > > > diff -r 520a513294c3 > test/hotspot/jtreg/sanity/MismatchedWhiteBox/WhiteBox.java > --- a/test/hotspot/jtreg/sanity/MismatchedWhiteBox/WhiteBox.java Mon > Dec 16 14:30:12 2019 -0800 > +++ b/test/hotspot/jtreg/sanity/MismatchedWhiteBox/WhiteBox.java Mon > Dec 16 14:48:30 2019 -0800 > @@ -36,6 +36,15 @@ > ?package sun.hotspot; > > ?public class WhiteBox { > +??? @SuppressWarnings("serial") > +??? public static class WhiteBoxPermission extends > java.security.BasicPermission { > +??????? // ClassFileInstaller is hard-coded to copy > WhiteBox$WhiteBoxPermission, so let's > +??????? // make a fake one here as well. > +??????? public WhiteBoxPermission(String s) { > +??????????? super(s); > +??????? } > +??? } > + > ???? private static native void registerNatives(); > ???? static { registerNatives(); } > ???? public native int notExistedMethod(); > > > > Thanks > - Ioi From igor.ignatyev at oracle.com Mon Dec 16 23:17:01 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 16 Dec 2019 15:17:01 -0800 Subject: RFR(T) [TESTBUG] MismatchedWhiteBox test fails with missing WhiteBox$WhiteBoxPermission.class In-Reply-To: <12121a1d-44ed-fd21-c27d-a1c4de61e365@oracle.com> References: <12121a1d-44ed-fd21-c27d-a1c4de61e365@oracle.com> Message-ID: <2B7E262D-9B44-448C-A850-048A45A37D9F@oracle.com> LGTM. -- Igor > On Dec 16, 2019, at 2:50 PM, Ioi Lam wrote: > > https://bugs.openjdk.java.net/browse/JDK-8236045 > > > After JDK-8199290, ClassFileInstaller is hard-coded to install WhiteBox$WhiteBoxPermission.class alongside WhiteBox.class. So in the MismatchedWhiteBox test, which creates a different version of the WhiteBox.class, we should have the inner class as well. > > > diff -r 520a513294c3 test/hotspot/jtreg/sanity/MismatchedWhiteBox/WhiteBox.java > --- a/test/hotspot/jtreg/sanity/MismatchedWhiteBox/WhiteBox.java Mon Dec 16 14:30:12 2019 -0800 > +++ b/test/hotspot/jtreg/sanity/MismatchedWhiteBox/WhiteBox.java Mon Dec 16 14:48:30 2019 -0800 > @@ -36,6 +36,15 @@ > package sun.hotspot; > > public class WhiteBox { > + @SuppressWarnings("serial") > + public static class WhiteBoxPermission extends java.security.BasicPermission { > + // ClassFileInstaller is hard-coded to copy WhiteBox$WhiteBoxPermission, so let's > + // make a fake one here as well. > + public WhiteBoxPermission(String s) { > + super(s); > + } > + } > + > private static native void registerNatives(); > static { registerNatives(); } > public native int notExistedMethod(); > > > > Thanks > - Ioi From igor.ignatyev at oracle.com Mon Dec 16 23:21:03 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 16 Dec 2019 15:21:03 -0800 Subject: [14] RFR(S/T) : 8235866 : bump jtreg requiredVersion to 4.2b16 In-Reply-To: <6e0dc2e5-607c-b991-dad3-62fec79cb43d@oracle.com> References: <764BB5FF-AC59-40E2-AB5A-F63A745E9AEE@oracle.com> <7522383C-1C26-4422-83CA-BB4E795CCF6B@oracle.com> <6e0dc2e5-607c-b991-dad3-62fec79cb43d@oracle.com> Message-ID: <50D6341B-2351-4975-ACEE-24BC29CFEDD5@oracle.com> thanks for confirming that Joe. -- Igor > On Dec 16, 2019, at 9:54 AM, Joe Wang wrote: > > > > On 12/12/19 10:46 PM, Igor Ignatyev wrote: >> @Joe/Lance, >> were there any reasons why you needed to switch jtreg's requiredVersion back to 4.2b13 in jaxp? are there any reasons which prevent jaxp from switching to jtreg4.2b16 now? >> > > I don't recall any specific reason, if any. It would be fine to switch to jtreg4.2b16. > > Thanks, > Joe From ioi.lam at oracle.com Mon Dec 16 23:22:36 2019 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 16 Dec 2019 15:22:36 -0800 Subject: RFR(T) [TESTBUG] MismatchedWhiteBox test fails with missing WhiteBox$WhiteBoxPermission.class In-Reply-To: <2B7E262D-9B44-448C-A850-048A45A37D9F@oracle.com> References: <12121a1d-44ed-fd21-c27d-a1c4de61e365@oracle.com> <2B7E262D-9B44-448C-A850-048A45A37D9F@oracle.com> Message-ID: <4c307f74-2e15-1cdb-f137-e11e6fa6ea08@oracle.com> Thanks for the review. Pushed. - Ioi On 12/16/19 3:17 PM, Igor Ignatyev wrote: > LGTM. > -- Igor > >> On Dec 16, 2019, at 2:50 PM, Ioi Lam wrote: >> >> https://bugs.openjdk.java.net/browse/JDK-8236045 >> >> >> After JDK-8199290, ClassFileInstaller is hard-coded to install WhiteBox$WhiteBoxPermission.class alongside WhiteBox.class. So in the MismatchedWhiteBox test, which creates a different version of the WhiteBox.class, we should have the inner class as well. >> >> >> diff -r 520a513294c3 test/hotspot/jtreg/sanity/MismatchedWhiteBox/WhiteBox.java >> --- a/test/hotspot/jtreg/sanity/MismatchedWhiteBox/WhiteBox.java Mon Dec 16 14:30:12 2019 -0800 >> +++ b/test/hotspot/jtreg/sanity/MismatchedWhiteBox/WhiteBox.java Mon Dec 16 14:48:30 2019 -0800 >> @@ -36,6 +36,15 @@ >> package sun.hotspot; >> >> public class WhiteBox { >> + @SuppressWarnings("serial") >> + public static class WhiteBoxPermission extends java.security.BasicPermission { >> + // ClassFileInstaller is hard-coded to copy WhiteBox$WhiteBoxPermission, so let's >> + // make a fake one here as well. >> + public WhiteBoxPermission(String s) { >> + super(s); >> + } >> + } >> + >> private static native void registerNatives(); >> static { registerNatives(); } >> public native int notExistedMethod(); >> >> >> >> Thanks >> - Ioi From coleen.phillimore at oracle.com Tue Dec 17 02:40:57 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 16 Dec 2019 21:40:57 -0500 Subject: [14] RFR 8235829: graal crashes with Zombie.java test In-Reply-To: <06d6e9f4-b919-3062-7d18-1245e66b27b2@oracle.com> References: <447990f9-d276-56c3-aebe-fa84d0cfcc27@oracle.com> <975b6e46-95c1-8a37-b160-b6a57a9633a8@oracle.com> <06d6e9f4-b919-3062-7d18-1245e66b27b2@oracle.com> Message-ID: <39716b03-dd5e-580e-2c91-004da64bcc19@oracle.com> Short answer below. On 12/16/19 5:51 PM, David Holmes wrote: > Hi Coleen, > > A quick initial response ... > > On 16/12/2019 11:26 pm, coleen.phillimore at oracle.com wrote: >> >> >> On 12/16/19 8:04 AM, David Holmes wrote: >>> Hi Coleen, >>> >>> On 16/12/2019 9:41 pm, coleen.phillimore at oracle.com wrote: >>>> Summary: Start ServiceThread before compiler threads, and run >>>> nmethod barriers for zgc before adding to the service thread queue, >>>> or posting the events on the java thread queue. >>> >>> I can't comment on most of this but the earlier starting of the >>> service thread has some concerns: >>> >>> - there is a lot of JDK level initialization which now will not have >>> happened before the service thread is started and it is far from >>> obvious that all possible initialization dependencies will be satisfied >> >> I agree that the order of initialization is very sensitive. From the >> actions that the service thread does, the one that I found was a >> problem was that events were posted before the LIVE phase (see >> comment in has_events()), which could have happened with the existing >> code, but the window for the race is a lot smaller. ? The other >> actions can be run if there's a GC before initialization but would be >> a bug in the initialization code, and I didn't find these bugs in all >> my testing. There are some ordering dependencies that do have odd >> side effects (between the compiler thread startup and initialization >> jsr292 classes) which have comments.? This patch doesn't touch those. >> >>> >>> - current starting of the service thread in Management::initialize >>> is guarded by "#if INCLUDE_MANAGEMENT", but now you are starting the >>> service thread unconditionally for all builds. Hmm just saw your >>> latest comment to the bug report - so the service thread is now (for >>> quite some time?) being used for other than management tasks and so >>> should always be present even if INCLUDE_MANAGEMENT is not enabled. >>> Is that sufficient or are there likely to be other changes needed to >>> actually ensure that all works correctly? e.g. any code the service >>> thread executes that is only defined for INCLUDE_MANAGEMENT will >>> need to be compiled out explicitly. >>> >> >> I asked Jie offline to check the minimal build.? I don't think there >> are other INCLUDE_MANAGEMENT actions in the service thread and I'm >> not sure why it was initialized there in the first place.? The >> minimal vm would have been broken ie. hashtables would not have been >> cleaned up, etc, but I'm not sure how well that is tested or if one >> would notice. >>> - the service thread and the notification thread are (were?) closely >>> related but now started at completely different times >> >> The notification thread is limited to "services" so it makes sense >> where it is.? The ServiceThread does lots of other things.? Maybe it >> needs renaming in 15. >>> >>> The bug report states the problem as: >>> >>> "The graal crash is because compiled_method_load events are added to >>> the ServiceThread's deferred event queue before the ServiceThread is >>> created so are not walked to keep them from being zombied." >>> >>> so why isn't the solution to ensure the deferred event queue is >>> walked? I'm not clear how starting the service thread relates to >>> walking the queue. >>> >> >> The service thread is responsible for walking the deferred event >> queue.?? See ServiceThread::oops_do/nmethods_do.?? The design could >> be changed to have some global walk somewhere of this queue, but >> essentially this queue is processed by the service thread. > > Sorry I don't follow. I thought "oops_do" and friends are for the GC > threads and/or VMThread to call to process oops when GC updates them. The oops_do and nmethods_do() can be called by a thread walk in handshakes (by the sweeper thread) and by parallel GC thread walks. There isn't a single entry to do the thread-specific closures that we need to do for these deferred event queues.?? I tried a version that walked the queues with a static call but missed some places where it would be needed to make this call (didn't work).? Keeping this associated with the ServiceThread simplifies a lot. thanks, Coleen > > David > ----- > >> I had an additional change to make the queue non-static but want to >> limit the change at this point. >> >> Thanks, >> Coleen >>> Thanks, >>> David >>> >>>> See bug for description of the problems found with the new >>>> Zombie.java test. >>>> >>>> open webrev at >>>> http://cr.openjdk.java.net/~coleenp/2019/8235829.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8235829 >>>> >>>> Ran tier1 all platforms, and tier2-8 testing, as well as rerunning >>>> original test failure from bug >>>> https://bugs.openjdk.java.net/browse/JDK-8173361. >>>> >>>> Thanks, >>>> Coleen >> From david.holmes at oracle.com Tue Dec 17 04:04:16 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 17 Dec 2019 14:04:16 +1000 Subject: [14] RFR 8235829: graal crashes with Zombie.java test In-Reply-To: <39716b03-dd5e-580e-2c91-004da64bcc19@oracle.com> References: <447990f9-d276-56c3-aebe-fa84d0cfcc27@oracle.com> <975b6e46-95c1-8a37-b160-b6a57a9633a8@oracle.com> <06d6e9f4-b919-3062-7d18-1245e66b27b2@oracle.com> <39716b03-dd5e-580e-2c91-004da64bcc19@oracle.com> Message-ID: Clarification ... On 17/12/2019 12:40 pm, coleen.phillimore at oracle.com wrote: > > Short answer below. > > On 12/16/19 5:51 PM, David Holmes wrote: >> Hi Coleen, >> >> A quick initial response ... >> >> On 16/12/2019 11:26 pm, coleen.phillimore at oracle.com wrote: >>> >>> >>> On 12/16/19 8:04 AM, David Holmes wrote: >>>> Hi Coleen, >>>> >>>> On 16/12/2019 9:41 pm, coleen.phillimore at oracle.com wrote: >>>>> Summary: Start ServiceThread before compiler threads, and run >>>>> nmethod barriers for zgc before adding to the service thread queue, >>>>> or posting the events on the java thread queue. >>>> >>>> I can't comment on most of this but the earlier starting of the >>>> service thread has some concerns: >>>> >>>> - there is a lot of JDK level initialization which now will not have >>>> happened before the service thread is started and it is far from >>>> obvious that all possible initialization dependencies will be satisfied >>> >>> I agree that the order of initialization is very sensitive. From the >>> actions that the service thread does, the one that I found was a >>> problem was that events were posted before the LIVE phase (see >>> comment in has_events()), which could have happened with the existing >>> code, but the window for the race is a lot smaller. ? The other >>> actions can be run if there's a GC before initialization but would be >>> a bug in the initialization code, and I didn't find these bugs in all >>> my testing. There are some ordering dependencies that do have odd >>> side effects (between the compiler thread startup and initialization >>> jsr292 classes) which have comments.? This patch doesn't touch those. >>> >>>> >>>> - current starting of the service thread in Management::initialize >>>> is guarded by "#if INCLUDE_MANAGEMENT", but now you are starting the >>>> service thread unconditionally for all builds. Hmm just saw your >>>> latest comment to the bug report - so the service thread is now (for >>>> quite some time?) being used for other than management tasks and so >>>> should always be present even if INCLUDE_MANAGEMENT is not enabled. >>>> Is that sufficient or are there likely to be other changes needed to >>>> actually ensure that all works correctly? e.g. any code the service >>>> thread executes that is only defined for INCLUDE_MANAGEMENT will >>>> need to be compiled out explicitly. >>>> >>> >>> I asked Jie offline to check the minimal build.? I don't think there >>> are other INCLUDE_MANAGEMENT actions in the service thread and I'm >>> not sure why it was initialized there in the first place.? The >>> minimal vm would have been broken ie. hashtables would not have been >>> cleaned up, etc, but I'm not sure how well that is tested or if one >>> would notice. >>>> - the service thread and the notification thread are (were?) closely >>>> related but now started at completely different times >>> >>> The notification thread is limited to "services" so it makes sense >>> where it is.? The ServiceThread does lots of other things.? Maybe it >>> needs renaming in 15. >>>> >>>> The bug report states the problem as: >>>> >>>> "The graal crash is because compiled_method_load events are added to >>>> the ServiceThread's deferred event queue before the ServiceThread is >>>> created so are not walked to keep them from being zombied." >>>> >>>> so why isn't the solution to ensure the deferred event queue is >>>> walked? I'm not clear how starting the service thread relates to >>>> walking the queue. >>>> >>> >>> The service thread is responsible for walking the deferred event >>> queue.?? See ServiceThread::oops_do/nmethods_do.?? The design could >>> be changed to have some global walk somewhere of this queue, but >>> essentially this queue is processed by the service thread. >> >> Sorry I don't follow. I thought "oops_do" and friends are for the GC >> threads and/or VMThread to call to process oops when GC updates them. > > The oops_do and nmethods_do() can be called by a thread walk in > handshakes (by the sweeper thread) and by parallel GC thread walks. > There isn't a single entry to do the thread-specific closures that we > need to do for these deferred event queues.?? I tried a version that > walked the queues with a static call but missed some places where it > would be needed to make this call (didn't work).? Keeping this > associated with the ServiceThread simplifies a lot. Just to clarify that further, the thread walk requires the thread appears in ALL_JAVA_THREADS but that only happens after the ServiceThread has been started. So in essence we don't really need the ServiceThread to have commenced execution earlier, but we need it to have been created. Those two steps are combined in practice. Cheers, David > thanks, > Coleen > >> >> David >> ----- >> >>> I had an additional change to make the queue non-static but want to >>> limit the change at this point. >>> >>> Thanks, >>> Coleen >>>> Thanks, >>>> David >>>> >>>>> See bug for description of the problems found with the new >>>>> Zombie.java test. >>>>> >>>>> open webrev at >>>>> http://cr.openjdk.java.net/~coleenp/2019/8235829.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8235829 >>>>> >>>>> Ran tier1 all platforms, and tier2-8 testing, as well as rerunning >>>>> original test failure from bug >>>>> https://bugs.openjdk.java.net/browse/JDK-8173361. >>>>> >>>>> Thanks, >>>>> Coleen >>> > From per.liden at oracle.com Tue Dec 17 08:14:37 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 17 Dec 2019 09:14:37 +0100 Subject: [14] RFR 8235829: graal crashes with Zombie.java test In-Reply-To: References: Message-ID: <7535d8d8-f245-78a0-fe58-f0625af43b3a@oracle.com> Hi Coleen, The "nmethod entry barrier"-part looks good to me. Just one minor nit, maybe JvmtiDeferredEventQueue::run_nmethod_entry_barrier should have an "s" on it (i.e. JvmtiDeferredEventQueue::run_nmethod_entry_barriers) since it loops over all entries in the queue? But I don't dare to comment on the ServiceThread initialization order. cheers, Per On 12/16/19 12:41 PM, coleen.phillimore at oracle.com wrote: > Summary: Start ServiceThread before compiler threads, and run nmethod > barriers for zgc before adding to the service thread queue, or posting > the events on the java thread queue. > > See bug for description of the problems found with the new Zombie.java > test. > > open webrev at http://cr.openjdk.java.net/~coleenp/2019/8235829.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8235829 > > Ran tier1 all platforms, and tier2-8 testing, as well as rerunning > original test failure from bug > https://bugs.openjdk.java.net/browse/JDK-8173361. > > Thanks, > Coleen From vladimir.x.ivanov at oracle.com Tue Dec 17 12:50:13 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 17 Dec 2019 15:50:13 +0300 Subject: [15] RFR (L): 7175279: Don't use x87 FPU on x86-64 Message-ID: <02b93959-9ec3-45c1-9ba7-ebd5682548e4@oracle.com> http://cr.openjdk.java.net/~vlivanov/7175279/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-7175279 There was a major rewrite of math intrinsics which in 9 time frame which almost completely eliminated x87 code in x86-64 code base. Proposed patch removes the rest and makes x86-64 code x87-free. The main motivation for the patch is to completely eliminate non-strictfp behaving code in order to prepare the JVM for JEP 306 [1] and related enhancements [2]. Most of the changes are in C1, but there is one case in template interpreter (java_lang_math_abs) which now uses StubRoutines::x86::double_sign_mask(). It forces its initialization to be moved to StubRoutines::initialize1(). x87 instructions are made available only on x86-32. C1 changes involve removing FPU support on x86-64 and effectively make x86-specific support in linear scan allocator [2] x86-32-only. Testing: tier1-6, build on x86-23, linux-aarch64, linux-arm32. Best regards, Vladimir Ivanov [1] https://bugs.openjdk.java.net/browse/JDK-8175916 [2] https://bugs.openjdk.java.net/browse/JDK-8136414 [3] http://hg.openjdk.java.net/jdk/jdk/file/ff7cd49f2aef/src/hotspot/cpu/x86/c1_LinearScan_x86.cpp From coleen.phillimore at oracle.com Tue Dec 17 14:08:21 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 17 Dec 2019 09:08:21 -0500 Subject: [14] RFR 8235829: graal crashes with Zombie.java test In-Reply-To: <7535d8d8-f245-78a0-fe58-f0625af43b3a@oracle.com> References: <7535d8d8-f245-78a0-fe58-f0625af43b3a@oracle.com> Message-ID: On 12/17/19 3:14 AM, Per Liden wrote: > Hi Coleen, > > The "nmethod entry barrier"-part looks good to me. Just one minor nit, > maybe JvmtiDeferredEventQueue::run_nmethod_entry_barrier should have > an "s" on it (i.e. > JvmtiDeferredEventQueue::run_nmethod_entry_barriers) since it loops > over all entries in the queue? I changed both entries in jvmtiImpl.hpp to run_nmethod_entry_barriers to avoid confusion on my part.? It then calls the nmethod version that is singular.?? Thanks for the suggestion of names. > > But I don't dare to comment on the ServiceThread initialization order. I moved it before the compiler and jvmti initialization, and jvmti won't post events until the LIVE phase.? I tried to be very careful! Thanks, Coleen > > cheers, > Per > > On 12/16/19 12:41 PM, coleen.phillimore at oracle.com wrote: >> Summary: Start ServiceThread before compiler threads, and run nmethod >> barriers for zgc before adding to the service thread queue, or >> posting the events on the java thread queue. >> >> See bug for description of the problems found with the new >> Zombie.java test. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2019/8235829.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8235829 >> >> Ran tier1 all platforms, and tier2-8 testing, as well as rerunning >> original test failure from bug >> https://bugs.openjdk.java.net/browse/JDK-8173361. >> >> Thanks, >> Coleen From jesper.wilhelmsson at oracle.com Tue Dec 17 14:26:29 2019 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Tue, 17 Dec 2019 15:26:29 +0100 Subject: [15] RFR (L): 7175279: Don't use x87 FPU on x86-64 In-Reply-To: <02b93959-9ec3-45c1-9ba7-ebd5682548e4@oracle.com> References: <02b93959-9ec3-45c1-9ba7-ebd5682548e4@oracle.com> Message-ID: <1A324EE8-6965-4F76-B55B-D16184DE2ED9@oracle.com> Hi, This is a fairly large (wide spread) change. Is there any risk for conflicts with remaining work in JDK 14? In the interest of keeping forwardports as conflict free as possible, would it make sense to hold this change until the number of changes in 14 has dropped? Thanks, /Jesper > On 17 Dec 2019, at 13:50, Vladimir Ivanov wrote: > > http://cr.openjdk.java.net/~vlivanov/7175279/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-7175279 > > There was a major rewrite of math intrinsics which in 9 time frame which almost completely eliminated x87 code in x86-64 code base. > > Proposed patch removes the rest and makes x86-64 code x87-free. > > The main motivation for the patch is to completely eliminate non-strictfp behaving code in order to prepare the JVM for JEP 306 [1] and related enhancements [2]. > > Most of the changes are in C1, but there is one case in template interpreter (java_lang_math_abs) which now uses StubRoutines::x86::double_sign_mask(). It forces its initialization to be moved to StubRoutines::initialize1(). > > x87 instructions are made available only on x86-32. > > C1 changes involve removing FPU support on x86-64 and effectively make x86-specific support in linear scan allocator [2] x86-32-only. > > Testing: tier1-6, build on x86-23, linux-aarch64, linux-arm32. > > Best regards, > Vladimir Ivanov > > [1] https://bugs.openjdk.java.net/browse/JDK-8175916 > > [2] https://bugs.openjdk.java.net/browse/JDK-8136414 > > [3] http://hg.openjdk.java.net/jdk/jdk/file/ff7cd49f2aef/src/hotspot/cpu/x86/c1_LinearScan_x86.cpp From vladimir.x.ivanov at oracle.com Tue Dec 17 15:02:38 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 17 Dec 2019 18:02:38 +0300 Subject: [15] RFR (L): 7175279: Don't use x87 FPU on x86-64 In-Reply-To: <1A324EE8-6965-4F76-B55B-D16184DE2ED9@oracle.com> References: <02b93959-9ec3-45c1-9ba7-ebd5682548e4@oracle.com> <1A324EE8-6965-4F76-B55B-D16184DE2ED9@oracle.com> Message-ID: Hi Jesper, > This is a fairly large (wide spread) change. Is there any risk for conflicts with remaining work in JDK 14? > In the interest of keeping forwardports as conflict free as possible, would it make sense to hold this change until the number of changes in 14 has dropped? I consider the risk of merge conflicts as low, since the bulk of changes touch C1 (and it isn't usually changed much). But I'm fine with waiting until the rate of fixes in JDK 14 drops (after RDP2?). Best regards, Vladimir Ivanov >> On 17 Dec 2019, at 13:50, Vladimir Ivanov wrote: >> >> http://cr.openjdk.java.net/~vlivanov/7175279/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-7175279 >> >> There was a major rewrite of math intrinsics which in 9 time frame which almost completely eliminated x87 code in x86-64 code base. >> >> Proposed patch removes the rest and makes x86-64 code x87-free. >> >> The main motivation for the patch is to completely eliminate non-strictfp behaving code in order to prepare the JVM for JEP 306 [1] and related enhancements [2]. >> >> Most of the changes are in C1, but there is one case in template interpreter (java_lang_math_abs) which now uses StubRoutines::x86::double_sign_mask(). It forces its initialization to be moved to StubRoutines::initialize1(). >> >> x87 instructions are made available only on x86-32. >> >> C1 changes involve removing FPU support on x86-64 and effectively make x86-specific support in linear scan allocator [2] x86-32-only. >> >> Testing: tier1-6, build on x86-23, linux-aarch64, linux-arm32. >> >> Best regards, >> Vladimir Ivanov >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8175916 >> >> [2] https://bugs.openjdk.java.net/browse/JDK-8136414 >> >> [3] http://hg.openjdk.java.net/jdk/jdk/file/ff7cd49f2aef/src/hotspot/cpu/x86/c1_LinearScan_x86.cpp > From coleen.phillimore at oracle.com Tue Dec 17 15:27:08 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 17 Dec 2019 10:27:08 -0500 Subject: [14] RFR 8235829: graal crashes with Zombie.java test In-Reply-To: References: <447990f9-d276-56c3-aebe-fa84d0cfcc27@oracle.com> <975b6e46-95c1-8a37-b160-b6a57a9633a8@oracle.com> <06d6e9f4-b919-3062-7d18-1245e66b27b2@oracle.com> <39716b03-dd5e-580e-2c91-004da64bcc19@oracle.com> Message-ID: <53208d1a-b969-d2e7-8657-d95f2d2d22e8@oracle.com> On 12/16/19 11:04 PM, David Holmes wrote: > Clarification ... > > On 17/12/2019 12:40 pm, coleen.phillimore at oracle.com wrote: >> >> Short answer below. >> >> On 12/16/19 5:51 PM, David Holmes wrote: >>> Hi Coleen, >>> >>> A quick initial response ... >>> >>> On 16/12/2019 11:26 pm, coleen.phillimore at oracle.com wrote: >>>> >>>> >>>> On 12/16/19 8:04 AM, David Holmes wrote: >>>>> Hi Coleen, >>>>> >>>>> On 16/12/2019 9:41 pm, coleen.phillimore at oracle.com wrote: >>>>>> Summary: Start ServiceThread before compiler threads, and run >>>>>> nmethod barriers for zgc before adding to the service thread >>>>>> queue, or posting the events on the java thread queue. >>>>> >>>>> I can't comment on most of this but the earlier starting of the >>>>> service thread has some concerns: >>>>> >>>>> - there is a lot of JDK level initialization which now will not >>>>> have happened before the service thread is started and it is far >>>>> from obvious that all possible initialization dependencies will be >>>>> satisfied >>>> >>>> I agree that the order of initialization is very sensitive. From >>>> the actions that the service thread does, the one that I found was >>>> a problem was that events were posted before the LIVE phase (see >>>> comment in has_events()), which could have happened with the >>>> existing code, but the window for the race is a lot smaller. ? The >>>> other actions can be run if there's a GC before initialization but >>>> would be a bug in the initialization code, and I didn't find these >>>> bugs in all my testing. There are some ordering dependencies that >>>> do have odd side effects (between the compiler thread startup and >>>> initialization jsr292 classes) which have comments.? This patch >>>> doesn't touch those. >>>> >>>>> >>>>> - current starting of the service thread in Management::initialize >>>>> is guarded by "#if INCLUDE_MANAGEMENT", but now you are starting >>>>> the service thread unconditionally for all builds. Hmm just saw >>>>> your latest comment to the bug report - so the service thread is >>>>> now (for quite some time?) being used for other than management >>>>> tasks and so should always be present even if INCLUDE_MANAGEMENT >>>>> is not enabled. Is that sufficient or are there likely to be other >>>>> changes needed to actually ensure that all works correctly? e.g. >>>>> any code the service thread executes that is only defined for >>>>> INCLUDE_MANAGEMENT will need to be compiled out explicitly. >>>>> >>>> >>>> I asked Jie offline to check the minimal build.? I don't think >>>> there are other INCLUDE_MANAGEMENT actions in the service thread >>>> and I'm not sure why it was initialized there in the first place.? >>>> The minimal vm would have been broken ie. hashtables would not have >>>> been cleaned up, etc, but I'm not sure how well that is tested or >>>> if one would notice. >>>>> - the service thread and the notification thread are (were?) >>>>> closely related but now started at completely different times >>>> >>>> The notification thread is limited to "services" so it makes sense >>>> where it is.? The ServiceThread does lots of other things.? Maybe >>>> it needs renaming in 15. >>>>> >>>>> The bug report states the problem as: >>>>> >>>>> "The graal crash is because compiled_method_load events are added >>>>> to the ServiceThread's deferred event queue before the >>>>> ServiceThread is created so are not walked to keep them from being >>>>> zombied." >>>>> >>>>> so why isn't the solution to ensure the deferred event queue is >>>>> walked? I'm not clear how starting the service thread relates to >>>>> walking the queue. >>>>> >>>> >>>> The service thread is responsible for walking the deferred event >>>> queue.?? See ServiceThread::oops_do/nmethods_do.?? The design could >>>> be changed to have some global walk somewhere of this queue, but >>>> essentially this queue is processed by the service thread. >>> >>> Sorry I don't follow. I thought "oops_do" and friends are for the GC >>> threads and/or VMThread to call to process oops when GC updates them. >> >> The oops_do and nmethods_do() can be called by a thread walk in >> handshakes (by the sweeper thread) and by parallel GC thread walks. >> There isn't a single entry to do the thread-specific closures that we >> need to do for these deferred event queues.?? I tried a version that >> walked the queues with a static call but missed some places where it >> would be needed to make this call (didn't work).? Keeping this >> associated with the ServiceThread simplifies a lot. > > Just to clarify that further, the thread walk requires the thread > appears in ALL_JAVA_THREADS but that only happens after the > ServiceThread has been started. So in essence we don't really need the > ServiceThread to have commenced execution earlier, but we need it to > have been created. Those two steps are combined in practice. Yes.? Then the ServiceThread waits on the Service_lock until notified by these events: ????? while (((sensors_changed = (!UseNotificationThread && LowMemoryDetector::has_pending_requests())) | ????????????? (has_jvmti_events = _jvmti_service_queue.has_events()) | ????????????? (has_gc_notification_event = (!UseNotificationThread && GCNotifier::has_event())) | ????????????? (has_dcmd_notification_event = (!UseNotificationThread && DCmdFactory::has_pending_jmx_notification())) | ????????????? (stringtable_work = StringTable::has_work()) | ????????????? (symboltable_work = SymbolTable::has_work()) | ????????????? (resolved_method_table_work = ResolvedMethodTable::has_work()) | ????????????? (thread_id_table_work = ThreadIdTable::has_work()) | ????????????? (protection_domain_table_work = SystemDictionary::pd_cache_table()->has_work()) | ????????????? (oopstorage_work = OopStorage::has_cleanup_work_and_reset()) ???????????? ) == 0) { The first, third and fourth events are from management.cpp events that were initialized after the ServiceThread was started. The second event I have changed, to wait until LIVE phase to return true. The stringtable, symboltable, resolved_method_table, thread_id and pd table have static _has_work variables initialized to false. The oopstorage_work has similar, but may update a time-based counter a bit earlier with the service thread starting earlier.? I think this is harmless. It is possible that after the service thread starts and before the compiler thread starts, there could be a GC that notifies the stringtable to clean up.? This seems like a good thing that the GC would clean up these tables with this order.? I ran the tier4 graal tests and there were no failures. Thanks, Coleen > > Cheers, > David > >> thanks, >> Coleen >> >>> >>> David >>> ----- >>> >>>> I had an additional change to make the queue non-static but want to >>>> limit the change at this point. >>>> >>>> Thanks, >>>> Coleen >>>>> Thanks, >>>>> David >>>>> >>>>>> See bug for description of the problems found with the new >>>>>> Zombie.java test. >>>>>> >>>>>> open webrev at >>>>>> http://cr.openjdk.java.net/~coleenp/2019/8235829.01/webrev >>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8235829 >>>>>> >>>>>> Ran tier1 all platforms, and tier2-8 testing, as well as >>>>>> rerunning original test failure from bug >>>>>> https://bugs.openjdk.java.net/browse/JDK-8173361. >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>> >> From jesper.wilhelmsson at oracle.com Tue Dec 17 15:38:55 2019 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Tue, 17 Dec 2019 16:38:55 +0100 Subject: [15] RFR (L): 7175279: Don't use x87 FPU on x86-64 In-Reply-To: References: <02b93959-9ec3-45c1-9ba7-ebd5682548e4@oracle.com> <1A324EE8-6965-4F76-B55B-D16184DE2ED9@oracle.com> Message-ID: <722D0395-9B89-490C-A6C4-5604AFFDA94B@oracle.com> > On 17 Dec 2019, at 16:02, Vladimir Ivanov wrote: > > Hi Jesper, > >> This is a fairly large (wide spread) change. Is there any risk for conflicts with remaining work in JDK 14? >> In the interest of keeping forwardports as conflict free as possible, would it make sense to hold this change until the number of changes in 14 has dropped? > > I consider the risk of merge conflicts as low, since the bulk of changes touch C1 (and it isn't usually changed much). > > But I'm fine with waiting until the rate of fixes in JDK 14 drops (after RDP2?). Yes, once we enter RDP2 there shouldn't be many changes left for JDK 14. Thank you! /Jesper > > Best regards, > Vladimir Ivanov > >>> On 17 Dec 2019, at 13:50, Vladimir Ivanov wrote: >>> >>> http://cr.openjdk.java.net/~vlivanov/7175279/webrev.00/ >>> https://bugs.openjdk.java.net/browse/JDK-7175279 >>> >>> There was a major rewrite of math intrinsics which in 9 time frame which almost completely eliminated x87 code in x86-64 code base. >>> >>> Proposed patch removes the rest and makes x86-64 code x87-free. >>> >>> The main motivation for the patch is to completely eliminate non-strictfp behaving code in order to prepare the JVM for JEP 306 [1] and related enhancements [2]. >>> >>> Most of the changes are in C1, but there is one case in template interpreter (java_lang_math_abs) which now uses StubRoutines::x86::double_sign_mask(). It forces its initialization to be moved to StubRoutines::initialize1(). >>> >>> x87 instructions are made available only on x86-32. >>> >>> C1 changes involve removing FPU support on x86-64 and effectively make x86-specific support in linear scan allocator [2] x86-32-only. >>> >>> Testing: tier1-6, build on x86-23, linux-aarch64, linux-arm32. >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8175916 >>> >>> [2] https://bugs.openjdk.java.net/browse/JDK-8136414 >>> >>> [3] http://hg.openjdk.java.net/jdk/jdk/file/ff7cd49f2aef/src/hotspot/cpu/x86/c1_LinearScan_x86.cpp From daniel.daugherty at oracle.com Tue Dec 17 15:47:09 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 17 Dec 2019 10:47:09 -0500 Subject: RFR: 8235966: Process obsolete flags less aggressively In-Reply-To: References: Message-ID: <674e4f0d-563f-3bb4-dee0-74aaacb9b270@oracle.com> On 12/16/19 12:16 AM, David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8235966 > webrev: http://cr.openjdk.java.net/~dholmes/8235966/webrev/ src/hotspot/share/runtime/arguments.cpp ??? L745: ????? // if flag has become obsolete it should not have a "globals" flag defined anymore. ??? L746: ????? if (!version_less_than(JDK_Version::current(), flag.obsolete_in)) { ??? L747: ??????? if (JVMFlag::find_declared_flag(flag.name) != NULL) { ??? L748: ????????? // Temporarily disable the warning: 8196739 ??? L749: ????????? // warning("Global variable for obsolete special flag entry \"%s\" should be removed", flag.name); ??? L750: ??????? } ??? L751: ????? } ??????? It seems like we've been down a similar road before: ??????? JDK-8196739 Disable obsolete/expired VM flag transitional warnings ??????? https://bugs.openjdk.java.net/browse/JDK-8196739 ??????? This one may ring a bell... Fixed by dholmes in jdk11-b01... :-) ??????? And this followup sub-task to re-enable that warning: ??????? JDK-8196741 Re-enable obsolete/expired VM flag transitional warnings ??????? https://bugs.openjdk.java.net/browse/JDK-8196741 ??????? was closed as "Won't fix" on 2019.08.02. ??????? So the obvious questions: ??????? - Why is the new warning less problematic to tests that don't ????????? tolerate unexpected output? ??????? - If you move forward with this fix, then I think think code ????????? block needs to be removed or modified or am I missing something? ??????? There's a similar commented out check on L757-L765, but that one ??????? is for an expired flag... You might want to adjust/delete it also? ??? L753: ??????? warning("Special flag entry \"%s\" must be explicitly obsoleted before expired.", flag.name); ??? L754: ??????? success = false; ??????? nit - s/before expired/before being expired/ ??????? Update: I now see that "style" is in several places in this ??????????? function. I'm not sure what to think here... it grates, ? ? ? ? ? ? but I can live with it. ??????? nit - L75[34] indented too much by two spaces. ??? L962: ????????? return real_name; ??????? nit - indented too much by two spaces. Trying to understand the modified logic in argument processing is making my head spin... - You've added a call to is_obsolete_flag() in ? handle_aliases_and_deprecation() and is_obsolete_flag() ? is where the new warning is output: ??? warning("Temporarily processing option %s; support is scheduled for removal in %s" ? handle_aliases_and_deprecation() is called from six different places, ? but the call sites are different based on the argument pattern so I ? have (mostly) convinced myself that there should not be any duplicate ? warning lines. So I now understand the new logic that allows an obsoleted option to be specified with a warning as long as the option still exists. I'm good with the technical change, but... I'm worried about tests that don't tolerate the new warning mesg, i.e., why wouldn't this become an issue again: JDK-8196739 Disable obsolete/expired VM flag transitional warnings Dan > > When a flag is marked as obsolete in the special-flags table we will > ignore it and issue a warning that it is being ignored, as soon as we > bump the version of the JDK. That means that any tests still using the > obsolete flag may start to fail, leading to a surge of test failures > at the start of a release cycle. For example for JDK 15 we have a > whole bunch of JFR tests that fail because they still try to work with > UseParallelOldGC. In another case > runtime/cds/appcds/FieldLayoutFlags.java passes only be accident. > > When a flag is marked as obsolete for a release, all code involving > that flag (including tests that use it) must be updated within that > release and the flag itself removed. Whilst this is typically > scheduled early in a release cycle it isn't reasonable to expect it to > all occur within the first couple of days of the release cycle, nor do > we want to have to ProblemList a bunch of tests when they start failing. > > What I propose is to instead allow an obsolete flag to continue to be > processed as long as that code removal has not actually occurred - > with an adjusted warning. The change I propose: > > - only treats an obsolete flag as obsolete if the flag cannot be found > - added a new flag verification rule that disallows obsoletion in an > undefined version, but expiration in a specific version i.e. we must > always explicitly obsolete a flag before we expire it. > > The only downside here is that if we actually forget to file an issue > for the actual obsoletion work we may not notice via testing. Of > course whenever a change is made to the flags table to add an entry > then the issue to do the obsoletion should be filed at the same time. > > Thanks, > David > ----- > From matthias.baesken at sap.com Tue Dec 17 16:17:32 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 17 Dec 2019 16:17:32 +0000 Subject: building libjvm with -Os for space optimization - was : RE: RFR: 8234525: enable link-time section-gc for linux s390x to remove unused code In-Reply-To: <84420dee-9889-b320-253b-f00551a2c9da@oracle.com> References: <3bffe1cf-4567-0cf6-4bfb-ad79bd0b9596@oracle.com> <84420dee-9889-b320-253b-f00551a2c9da@oracle.com> Message-ID: Hello, a small update from my side . I had (on Linux) for a couple of days our jdk11u tests + benchmarks running with -Os instead of the usual settings (-O3) . With have (independent of -Os / -O3) a bit of variance in our benchmarks ( jbb15 , jvm2008, jvm98 ) . However the benchmark results of -Os vs. -O3 were comparable . >we discussed doing the opposite for Mac OS X recently, where builds are >currently set to -Os by default. -O3 helped various networking >(micro)benchmarks by up to 20%. What microbenchmarks do you have in mind that show much better performance with -O3 compared to -Os ? Best regards, Matthias > > Hi Martin, > > On 2019-11-27 19:03, Doerr, Martin wrote: > > Hi Claes, > > > > that kind of surprises me. I'd expect files which rather benefit from -O3 to > be far less than those which benefit from -Os. > > Most performance critical code lives inside the code cache and is not > dependent on C++ compiler optimizations. > > I'd expect GC code, C2's register allocation and a few runtime files to be the > most performance critical C++ code. > > So the list of files for -Os may become long. > > that might very well be the end result, and once/if we've gone down this > path long enough to see that -O3 becomes the exception, we can re- > examine the default. Changing the default and then trying to recuperate > would be hard/impossible to do incrementally. > > > > > Yeah, I think we should use native profiling information to find out what's > really going on > > > Your idea to change file by file and check for performance regression > makes sense to me, though > Hopefully we don't have to do one RFE per file.. :-) > > /Claes From vladimir.kozlov at oracle.com Tue Dec 17 17:45:18 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 17 Dec 2019 09:45:18 -0800 Subject: [15] RFR (L): 7175279: Don't use x87 FPU on x86-64 In-Reply-To: <02b93959-9ec3-45c1-9ba7-ebd5682548e4@oracle.com> References: <02b93959-9ec3-45c1-9ba7-ebd5682548e4@oracle.com> Message-ID: Finally! Very good cleanup. Few notes. c1_CodeStubs.hpp - I think it should be stronger than assert to catch it in product too (we can do check in product because it is not performance critical code). c1_LinearScan.cpp - I think IA64 is used for Itanium. For 64-bit x86 we use AMD64: https://hg.openjdk.java.net/jdk/jdk/file/cfaa2457a60a/src/hotspot/share/utilities/macros.hpp#l456 Thanks, Vladimir On 12/17/19 4:50 AM, Vladimir Ivanov wrote: > http://cr.openjdk.java.net/~vlivanov/7175279/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-7175279 > > There was a major rewrite of math intrinsics which in 9 time frame which almost completely eliminated x87 code in x86-64 > code base. > > Proposed patch removes the rest and makes x86-64 code x87-free. > > The main motivation for the patch is to completely eliminate non-strictfp behaving code in order to prepare the JVM for > JEP 306 [1] and related enhancements [2]. > > Most of the changes are in C1, but there is one case in template interpreter (java_lang_math_abs) which now uses > StubRoutines::x86::double_sign_mask(). It forces its initialization to be moved to StubRoutines::initialize1(). > > x87 instructions are made available only on x86-32. > > C1 changes involve removing FPU support on x86-64 and effectively make x86-specific support in linear scan allocator [2] > x86-32-only. > > Testing: tier1-6, build on x86-23, linux-aarch64, linux-arm32. > > Best regards, > Vladimir Ivanov > > [1] https://bugs.openjdk.java.net/browse/JDK-8175916 > > [2] https://bugs.openjdk.java.net/browse/JDK-8136414 > > [3] http://hg.openjdk.java.net/jdk/jdk/file/ff7cd49f2aef/src/hotspot/cpu/x86/c1_LinearScan_x86.cpp From igor.ignatyev at oracle.com Tue Dec 17 19:30:26 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 17 Dec 2019 11:30:26 -0800 Subject: RFR(S): 8236111 : narrow allowSmartActionArgs disabling Message-ID: http://cr.openjdk.java.net/~iignatyev/8236111/webrev.00/ > 31 lines changed: 20 ins; 11 del; 0 mod; Hi all, could you please review this small patch which enables allowSmartActionArgs in hotspot and jdk test suites and disables them in a small number of test directories? the patch also removes TEST.properties files which enabled allowSmartActionArgs as they aren't needed anymore. from JBS: > currently, allowSmartActionArgs is disabled for the whole hotspot and jdk test suites and enabled just in few places. this makes it a bit harder for people to use smart action arguments in these test suites as they have to not to forget to enable them. and given in all the other test suites, smart action arguments are enabled, it can be confusing and frustrating. testing: tier1-5 JBS: https://bugs.openjdk.java.net/browse/JDK-8236111 webrev: http://cr.openjdk.java.net/~iignatyev/8236111/webrev.00/ Thanks, -- Igor From david.holmes at oracle.com Tue Dec 17 22:03:10 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 18 Dec 2019 08:03:10 +1000 Subject: RFR: 8235966: Process obsolete flags less aggressively In-Reply-To: <674e4f0d-563f-3bb4-dee0-74aaacb9b270@oracle.com> References: <674e4f0d-563f-3bb4-dee0-74aaacb9b270@oracle.com> Message-ID: Hi Dan, Thanks for taking a look. Updated webrev: http://cr.openjdk.java.net/~dholmes/8235966/webrev.v2/ Discussion below. On 18/12/2019 1:47 am, Daniel D. Daugherty wrote: > On 12/16/19 12:16 AM, David Holmes wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8235966 >> webrev: http://cr.openjdk.java.net/~dholmes/8235966/webrev/ > > src/hotspot/share/runtime/arguments.cpp > ??? L745: ????? // if flag has become obsolete it should not have a > "globals" flag defined anymore. > ??? L746: ????? if (!version_less_than(JDK_Version::current(), > flag.obsolete_in)) { > ??? L747: ??????? if (JVMFlag::find_declared_flag(flag.name) != NULL) { > ??? L748: ????????? // Temporarily disable the warning: 8196739 > ??? L749: ????????? // warning("Global variable for obsolete special > flag entry \"%s\" should be removed", flag.name); > ??? L750: ??????? } > ??? L751: ????? } > ??????? It seems like we've been down a similar road before: > > ??????? JDK-8196739 Disable obsolete/expired VM flag transitional warnings > ??????? https://bugs.openjdk.java.net/browse/JDK-8196739 > > ??????? This one may ring a bell... Fixed by dholmes in jdk11-b01... :-) > > ??????? And this followup sub-task to re-enable that warning: > > ??????? JDK-8196741 Re-enable obsolete/expired VM flag transitional > warnings > ??????? https://bugs.openjdk.java.net/browse/JDK-8196741 > > ??????? was closed as "Won't fix" on 2019.08.02. > > ??????? So the obvious questions: > > ??????? - Why is the new warning less problematic to tests that don't > ????????? tolerate unexpected output? Two different situations. The commented out warning happens unconditionally when you run the VM and it finds any flag marked obsolete that hasn't been removed. Hence every single test will encounter this warning. The situation I am modifying is when a test uses a flag that is marked for obsoletion. In the majority of cases the flag is already deprecated and so already issuing a deprecation warning that the test has to handle. Without my change there would still be an obsoletion warning, so this test is in for a warning no matter what. Also note that for hotspot at least we have strived to make tests tolerate unexpected output. The reason JDK-8196741 was closed as "won't fix" was because other areas wouldn't commit to doing that. > ??????? - If you move forward with this fix, then I think think code > ????????? block needs to be removed or modified or am I missing something? I've rewritten the comment at the head of verify_special_jvm_flags to explain why we can't issue a warning, and have deleted the block. > ??????? There's a similar commented out check on L757-L765, but that one > ??????? is for an expired flag... You might want to adjust/delete it also? Deleted. > ??? L753: ??????? warning("Special flag entry \"%s\" must be explicitly > obsoleted before expired.", flag.name); > ??? L754: ??????? success = false; > ??????? nit - s/before expired/before being expired/ > ??????? Update: I now see that "style" is in several places in this > ??????????? function. I'm not sure what to think here... it grates, > ? ? ? ? ? ? but I can live with it. > > ??????? nit - L75[34] indented too much by two spaces. Fixed. > ??? L962: ????????? return real_name; > ??????? nit - indented too much by two spaces. Fixed. > > Trying to understand the modified logic in argument processing is > making my head spin... Mine too. It took a few attempts to put the logic in the right place and make adjustments so that it all works as expected for a correctly specified flag and an erroneous one. > - You've added a call to is_obsolete_flag() in > ? handle_aliases_and_deprecation() and is_obsolete_flag() > ? is where the new warning is output: > > ??? warning("Temporarily processing option %s; support is scheduled for > removal in %s" > > ? handle_aliases_and_deprecation() is called from six different places, > ? but the call sites are different based on the argument pattern so I > ? have (mostly) convinced myself that there should not be any duplicate > ? warning lines. Right - handle_aliases_and_deprecation is only called for a syntactically correct flag based on those patterns. It normally filters out obsoleted/expired flags and lets them fall through to later error processing (in process_argument after parse_arg returns false). That error processing is where the normal obsoletion check is performed. So I had to not filter the flag in handle_aliases_and_deprecation in this case, but still produce the warning for a malformed flag. E.g. java -XX:+UseParallelOldGC -version Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing option UseParallelOldGC; support is scheduled for removal in 15.0 java version "15-internal" 2020-09-15 java -XX:UseParallelOldGC -version Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing option UseParallelOldGC; support is scheduled for removal in 15.0 Missing +/- setting for VM option 'UseParallelOldGC' Error: Could not create the Java Virtual Machine. > So I now understand the new logic that allows an obsoleted option > to be specified with a warning as long as the option still exists. > I'm good with the technical change, but... > > I'm worried about tests that don't tolerate the new warning mesg, > i.e., why wouldn't this become an issue again: > > JDK-8196739 Disable obsolete/expired VM flag transitional warnings Explained above. Thanks, David > Dan > > >> >> When a flag is marked as obsolete in the special-flags table we will >> ignore it and issue a warning that it is being ignored, as soon as we >> bump the version of the JDK. That means that any tests still using the >> obsolete flag may start to fail, leading to a surge of test failures >> at the start of a release cycle. For example for JDK 15 we have a >> whole bunch of JFR tests that fail because they still try to work with >> UseParallelOldGC. In another case >> runtime/cds/appcds/FieldLayoutFlags.java passes only be accident. >> >> When a flag is marked as obsolete for a release, all code involving >> that flag (including tests that use it) must be updated within that >> release and the flag itself removed. Whilst this is typically >> scheduled early in a release cycle it isn't reasonable to expect it to >> all occur within the first couple of days of the release cycle, nor do >> we want to have to ProblemList a bunch of tests when they start failing. >> >> What I propose is to instead allow an obsolete flag to continue to be >> processed as long as that code removal has not actually occurred - >> with an adjusted warning. The change I propose: >> >> - only treats an obsolete flag as obsolete if the flag cannot be found >> - added a new flag verification rule that disallows obsoletion in an >> undefined version, but expiration in a specific version i.e. we must >> always explicitly obsolete a flag before we expire it. >> >> The only downside here is that if we actually forget to file an issue >> for the actual obsoletion work we may not notice via testing. Of >> course whenever a change is made to the flags table to add an entry >> then the issue to do the obsoletion should be filed at the same time. >> >> Thanks, >> David >> ----- >> > From augustnagro at gmail.com Tue Dec 17 23:05:40 2019 From: augustnagro at gmail.com (August Nagro) Date: Tue, 17 Dec 2019 17:05:40 -0600 Subject: building libjvm with -Os for space optimization - was : RE: RFR: 8234525: enable link-time section-gc for linux s390x to remove unused code Message-ID: I published some benchmarks of OpenJDK on Mac with Ofast and O3 [1]. Some microbenchmarks like Netty?s HttpObjectEncoder experienced >100% speedup with O3, and the more real-world Dacapo suite was ~15% improvement over O2 (which is exactly the same as Os). I did include a few other flags, however the speedup was primarily due to optimization level. Building with Os is the old wisdom. It used to be the case that many programs would be faster with the smaller binary size, but this is almost never the case nowadays. - August [1]: http://august.nagro.us/optimized-openjdk.html From matthias.baesken at sap.com Wed Dec 18 07:58:09 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 18 Dec 2019 07:58:09 +0000 Subject: building libjvm with -Os for space optimization - was : RE: RFR: 8234525: enable link-time section-gc for linux s390x to remove unused code In-Reply-To: References: Message-ID: Hi August , thanks for pointing to your webpage, very interesting ! We did our builds+tests/benchmarks with gcc 7.4.0 , what compiler+version did you use? Probably I should look a bit more into Dacapo (we used that one in the past too sometimes). Best regards, Matthias > > I published some benchmarks of OpenJDK on Mac with Ofast and O3 [1]. > Some microbenchmarks like Netty?s HttpObjectEncoder experienced >100% > speedup with O3, and the more real-world Dacapo suite was ~15% > improvement over O2 (which is exactly the same as Os). I did include a few > other flags, however the speedup was primarily due to optimization level. > > Building with Os is the old wisdom. It used to be the case that many programs > would be faster with the smaller binary size, but this is almost never the case > nowadays. > > - August > > [1]: http://august.nagro.us/optimized-openjdk.html From augustnagro at gmail.com Wed Dec 18 13:41:48 2019 From: augustnagro at gmail.com (August Nagro) Date: Wed, 18 Dec 2019 07:41:48 -0600 Subject: building libjvm with -Os for space optimization - was : RE: RFR: 8234525: enable link-time section-gc for linux s390x to remove unused code In-Reply-To: References: Message-ID: I compiled with clang since I'm on Mac. The Renaissance benchmark suite is also a good one that I learned about recently. My opinion is that there are probably more compelling alternatives if reducing binary size is the goal. Even if the tests show that Os/O2 is no different than O3, who knows if this will be true in the future. Regards, - August On Wed, Dec 18, 2019, 1:58 AM Baesken, Matthias wrote: > Hi August , thanks for pointing to your webpage, very interesting ! > > We did our builds+tests/benchmarks with gcc 7.4.0 , what > compiler+version did you use? > > Probably I should look a bit more into Dacapo (we used that one in the > past too sometimes). > > Best regards, Matthias > > > > > > I published some benchmarks of OpenJDK on Mac with Ofast and O3 [1]. > > Some microbenchmarks like Netty?s HttpObjectEncoder experienced >100% > > speedup with O3, and the more real-world Dacapo suite was ~15% > > improvement over O2 (which is exactly the same as Os). I did include a > few > > other flags, however the speedup was primarily due to optimization level. > > > > Building with Os is the old wisdom. It used to be the case that many > programs > > would be faster with the smaller binary size, but this is almost never > the case > > nowadays. > > > > - August > > > > [1]: http://august.nagro.us/optimized-openjdk.html > From david.holmes at oracle.com Wed Dec 18 13:45:25 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 18 Dec 2019 23:45:25 +1000 Subject: [14] RFR 8235829: graal crashes with Zombie.java test In-Reply-To: <53208d1a-b969-d2e7-8657-d95f2d2d22e8@oracle.com> References: <447990f9-d276-56c3-aebe-fa84d0cfcc27@oracle.com> <975b6e46-95c1-8a37-b160-b6a57a9633a8@oracle.com> <06d6e9f4-b919-3062-7d18-1245e66b27b2@oracle.com> <39716b03-dd5e-580e-2c91-004da64bcc19@oracle.com> <53208d1a-b969-d2e7-8657-d95f2d2d22e8@oracle.com> Message-ID: Thanks for the additional info Coleen! Just to add a bit more to the initialization history. The ServiceThread is a generalization of the LowMemoryDetectorThread that was part of the management API, and so it was initialized in Management::initialize. When it turned into the ServiceThread - to process JVMTI deferred events in addition to the low-memory-detector events - the initialization placement remained the same. Then later the INCLUDE_MANAGEMENT guards were added (JDK-7189254, October 2012). Later still we started adding other items of work for the ServiceThread. The earliest was the AllocationContextService notification in September 2014 but as that no longer exists I can't tell if that was the first non-management related use. Then the StringTable use was added 18 months ago - which definitely was outside the realm of the management API. So that is when the MinimalVM was first "broken". So it is good that is fixed. With regard to the placement in the initialization order, my remaining concern was with JVMTI event processing that might happen via events generated very early in the init sequence. But you have now modified things so that we will only process events in the LIVE phase, which only activates after all the class library initialization is complete. So overall I'm no longer significantly concerned about the change to the initialization order as I think you have it all covered. Thanks for bearing with me and all the off-list discussion. Cheers, David ----- On 18/12/2019 1:27 am, coleen.phillimore at oracle.com wrote: > > > On 12/16/19 11:04 PM, David Holmes wrote: >> Clarification ... >> >> On 17/12/2019 12:40 pm, coleen.phillimore at oracle.com wrote: >>> >>> Short answer below. >>> >>> On 12/16/19 5:51 PM, David Holmes wrote: >>>> Hi Coleen, >>>> >>>> A quick initial response ... >>>> >>>> On 16/12/2019 11:26 pm, coleen.phillimore at oracle.com wrote: >>>>> >>>>> >>>>> On 12/16/19 8:04 AM, David Holmes wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> On 16/12/2019 9:41 pm, coleen.phillimore at oracle.com wrote: >>>>>>> Summary: Start ServiceThread before compiler threads, and run >>>>>>> nmethod barriers for zgc before adding to the service thread >>>>>>> queue, or posting the events on the java thread queue. >>>>>> >>>>>> I can't comment on most of this but the earlier starting of the >>>>>> service thread has some concerns: >>>>>> >>>>>> - there is a lot of JDK level initialization which now will not >>>>>> have happened before the service thread is started and it is far >>>>>> from obvious that all possible initialization dependencies will be >>>>>> satisfied >>>>> >>>>> I agree that the order of initialization is very sensitive. From >>>>> the actions that the service thread does, the one that I found was >>>>> a problem was that events were posted before the LIVE phase (see >>>>> comment in has_events()), which could have happened with the >>>>> existing code, but the window for the race is a lot smaller. ? The >>>>> other actions can be run if there's a GC before initialization but >>>>> would be a bug in the initialization code, and I didn't find these >>>>> bugs in all my testing. There are some ordering dependencies that >>>>> do have odd side effects (between the compiler thread startup and >>>>> initialization jsr292 classes) which have comments.? This patch >>>>> doesn't touch those. >>>>> >>>>>> >>>>>> - current starting of the service thread in Management::initialize >>>>>> is guarded by "#if INCLUDE_MANAGEMENT", but now you are starting >>>>>> the service thread unconditionally for all builds. Hmm just saw >>>>>> your latest comment to the bug report - so the service thread is >>>>>> now (for quite some time?) being used for other than management >>>>>> tasks and so should always be present even if INCLUDE_MANAGEMENT >>>>>> is not enabled. Is that sufficient or are there likely to be other >>>>>> changes needed to actually ensure that all works correctly? e.g. >>>>>> any code the service thread executes that is only defined for >>>>>> INCLUDE_MANAGEMENT will need to be compiled out explicitly. >>>>>> >>>>> >>>>> I asked Jie offline to check the minimal build.? I don't think >>>>> there are other INCLUDE_MANAGEMENT actions in the service thread >>>>> and I'm not sure why it was initialized there in the first place. >>>>> The minimal vm would have been broken ie. hashtables would not have >>>>> been cleaned up, etc, but I'm not sure how well that is tested or >>>>> if one would notice. >>>>>> - the service thread and the notification thread are (were?) >>>>>> closely related but now started at completely different times >>>>> >>>>> The notification thread is limited to "services" so it makes sense >>>>> where it is.? The ServiceThread does lots of other things.? Maybe >>>>> it needs renaming in 15. >>>>>> >>>>>> The bug report states the problem as: >>>>>> >>>>>> "The graal crash is because compiled_method_load events are added >>>>>> to the ServiceThread's deferred event queue before the >>>>>> ServiceThread is created so are not walked to keep them from being >>>>>> zombied." >>>>>> >>>>>> so why isn't the solution to ensure the deferred event queue is >>>>>> walked? I'm not clear how starting the service thread relates to >>>>>> walking the queue. >>>>>> >>>>> >>>>> The service thread is responsible for walking the deferred event >>>>> queue.?? See ServiceThread::oops_do/nmethods_do.?? The design could >>>>> be changed to have some global walk somewhere of this queue, but >>>>> essentially this queue is processed by the service thread. >>>> >>>> Sorry I don't follow. I thought "oops_do" and friends are for the GC >>>> threads and/or VMThread to call to process oops when GC updates them. >>> >>> The oops_do and nmethods_do() can be called by a thread walk in >>> handshakes (by the sweeper thread) and by parallel GC thread walks. >>> There isn't a single entry to do the thread-specific closures that we >>> need to do for these deferred event queues.?? I tried a version that >>> walked the queues with a static call but missed some places where it >>> would be needed to make this call (didn't work).? Keeping this >>> associated with the ServiceThread simplifies a lot. >> >> Just to clarify that further, the thread walk requires the thread >> appears in ALL_JAVA_THREADS but that only happens after the >> ServiceThread has been started. So in essence we don't really need the >> ServiceThread to have commenced execution earlier, but we need it to >> have been created. Those two steps are combined in practice. > > Yes.? Then the ServiceThread waits on the Service_lock until notified by > these events: > > ????? while (((sensors_changed = (!UseNotificationThread && > LowMemoryDetector::has_pending_requests())) | > ????????????? (has_jvmti_events = _jvmti_service_queue.has_events()) | > ????????????? (has_gc_notification_event = (!UseNotificationThread && > GCNotifier::has_event())) | > ????????????? (has_dcmd_notification_event = (!UseNotificationThread && > DCmdFactory::has_pending_jmx_notification())) | > ????????????? (stringtable_work = StringTable::has_work()) | > ????????????? (symboltable_work = SymbolTable::has_work()) | > ????????????? (resolved_method_table_work = > ResolvedMethodTable::has_work()) | > ????????????? (thread_id_table_work = ThreadIdTable::has_work()) | > ????????????? (protection_domain_table_work = > SystemDictionary::pd_cache_table()->has_work()) | > ????????????? (oopstorage_work = OopStorage::has_cleanup_work_and_reset()) > ???????????? ) == 0) { > > The first, third and fourth events are from management.cpp events that > were initialized after the ServiceThread was started. > The second event I have changed, to wait until LIVE phase to return true. > The stringtable, symboltable, resolved_method_table, thread_id and pd > table have static _has_work variables initialized to false. > The oopstorage_work has similar, but may update a time-based counter a > bit earlier with the service thread starting earlier.? I think this is > harmless. > > It is possible that after the service thread starts and before the > compiler thread starts, there could be a GC that notifies the > stringtable to clean up.? This seems like a good thing that the GC would > clean up these tables with this order.? I ran the tier4 graal tests and > there were no failures. > > Thanks, > Coleen >> >> Cheers, >> David >> >>> thanks, >>> Coleen >>> >>>> >>>> David >>>> ----- >>>> >>>>> I had an additional change to make the queue non-static but want to >>>>> limit the change at this point. >>>>> >>>>> Thanks, >>>>> Coleen >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> See bug for description of the problems found with the new >>>>>>> Zombie.java test. >>>>>>> >>>>>>> open webrev at >>>>>>> http://cr.openjdk.java.net/~coleenp/2019/8235829.01/webrev >>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8235829 >>>>>>> >>>>>>> Ran tier1 all platforms, and tier2-8 testing, as well as >>>>>>> rerunning original test failure from bug >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8173361. >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>> >>> > From robbin.ehn at oracle.com Wed Dec 18 15:00:10 2019 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 18 Dec 2019 16:00:10 +0100 Subject: [14] RFR 8235829: graal crashes with Zombie.java test In-Reply-To: References: Message-ID: <86019ad1-39fa-cb1a-d34f-6b2b521d9dba@oracle.com> Hi Coleen, I looked at v2: http://cr.openjdk.java.net/~coleenp/2019/8235829.01/webrev Seems good. But we do a lot of work to keep the nmethod alive while in queue. Instead we should try copy the data from nmethod and enqueue this copy. Thus not having to keep the nmethod alive. Not for this change-set, but a potential future simplification. Thanks, Robbin On 12/16/19 12:41 PM, coleen.phillimore at oracle.com wrote: > Summary: Start ServiceThread before compiler threads, and run nmethod barriers > for zgc before adding to the service thread queue, or posting the events on the > java thread queue. > > See bug for description of the problems found with the new Zombie.java test. > > open webrev at http://cr.openjdk.java.net/~coleenp/2019/8235829.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8235829 > > Ran tier1 all platforms, and tier2-8 testing, as well as rerunning original test > failure from bug https://bugs.openjdk.java.net/browse/JDK-8173361. > > Thanks, > Coleen From matthias.baesken at sap.com Wed Dec 18 16:07:28 2019 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 18 Dec 2019 16:07:28 +0000 Subject: building libjvm with -Os for space optimization - was : RE: RFR: 8234525: enable link-time section-gc for linux s390x to remove unused code In-Reply-To: References: Message-ID: * I compiled with clang since I'm on Mac. * Thanks for clarifying, that?s what I thought . (btw I wonder how much effect profile guided optimization would bring in your experiments ) * My opinion is that there are probably more compelling alternatives if reducing binary size is the goal. Even if the tests show that Os/O2 is no different than O3, * who knows if this will be true in the future. * What alternatives do you have in mind ? Best regards, Matthias From: August Nagro Sent: Mittwoch, 18. Dezember 2019 14:42 To: Baesken, Matthias Cc: claes.redestad at oracle.com; Doerr, Martin ; erik.joelsson at oracle.com; build-dev at openjdk.java.net; hotspot-dev at openjdk.java.net Subject: Re: RE: building libjvm with -Os for space optimization - was : RE: RFR: 8234525: enable link-time section-gc for linux s390x to remove unused code I compiled with clang since I'm on Mac. The Renaissance benchmark suite is also a good one that I learned about recently. My opinion is that there are probably more compelling alternatives if reducing binary size is the goal. Even if the tests show that Os/O2 is no different than O3, who knows if this will be true in the future. Regards, - August On Wed, Dec 18, 2019, 1:58 AM Baesken, Matthias > wrote: Hi August , thanks for pointing to your webpage, very interesting ! We did our builds+tests/benchmarks with gcc 7.4.0 , what compiler+version did you use? Probably I should look a bit more into Dacapo (we used that one in the past too sometimes). Best regards, Matthias > > I published some benchmarks of OpenJDK on Mac with Ofast and O3 [1]. > Some microbenchmarks like Netty?s HttpObjectEncoder experienced >100% > speedup with O3, and the more real-world Dacapo suite was ~15% > improvement over O2 (which is exactly the same as Os). I did include a few > other flags, however the speedup was primarily due to optimization level. > > Building with Os is the old wisdom. It used to be the case that many programs > would be faster with the smaller binary size, but this is almost never the case > nowadays. > > - August > > [1]: http://august.nagro.us/optimized-openjdk.html From coleen.phillimore at oracle.com Wed Dec 18 16:36:29 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 18 Dec 2019 11:36:29 -0500 Subject: [14] RFR 8235829: graal crashes with Zombie.java test In-Reply-To: <86019ad1-39fa-cb1a-d34f-6b2b521d9dba@oracle.com> References: <86019ad1-39fa-cb1a-d34f-6b2b521d9dba@oracle.com> Message-ID: Robbin, Thank you for looking at it. On 12/18/19 10:00 AM, Robbin Ehn wrote: > Hi Coleen, > > I looked at v2: > http://cr.openjdk.java.net/~coleenp/2019/8235829.01/webrev > > Seems good. > > But we do a lot of work to keep the nmethod alive while in queue. > Instead we should try copy the data from nmethod and enqueue this copy. > Thus not having to keep the nmethod alive. > Not for this change-set, but a potential future simplification. I agree.? Thank you for thinking of this idea.? This might work a lot better, and I'll try to write this simplification and see if it works for 15. Thanks for the code review. Coleen > > Thanks, Robbin > > On 12/16/19 12:41 PM, coleen.phillimore at oracle.com wrote: >> Summary: Start ServiceThread before compiler threads, and run nmethod >> barriers for zgc before adding to the service thread queue, or >> posting the events on the java thread queue. >> >> See bug for description of the problems found with the new >> Zombie.java test. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2019/8235829.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8235829 >> >> Ran tier1 all platforms, and tier2-8 testing, as well as rerunning >> original test failure from bug >> https://bugs.openjdk.java.net/browse/JDK-8173361. >> >> Thanks, >> Coleen From coleen.phillimore at oracle.com Wed Dec 18 16:42:45 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 18 Dec 2019 11:42:45 -0500 Subject: [14] RFR 8235829: graal crashes with Zombie.java test In-Reply-To: References: <447990f9-d276-56c3-aebe-fa84d0cfcc27@oracle.com> <975b6e46-95c1-8a37-b160-b6a57a9633a8@oracle.com> <06d6e9f4-b919-3062-7d18-1245e66b27b2@oracle.com> <39716b03-dd5e-580e-2c91-004da64bcc19@oracle.com> <53208d1a-b969-d2e7-8657-d95f2d2d22e8@oracle.com> Message-ID: <07b6370a-ff4f-641c-7661-e5b1231a7d8c@oracle.com> On 12/18/19 8:45 AM, David Holmes wrote: > Thanks for the additional info Coleen! > > Just to add a bit more to the initialization history. The > ServiceThread is a generalization of the LowMemoryDetectorThread that > was part of the management API, and so it was initialized in > Management::initialize. When it turned into the ServiceThread - to > process JVMTI deferred events in addition to the low-memory-detector > events - the initialization placement remained the same. Then later > the INCLUDE_MANAGEMENT guards were added (JDK-7189254, October 2012). > Later still we started adding other items of work for the > ServiceThread. The earliest was the AllocationContextService > notification in September 2014 but as that no longer exists I can't > tell if that was the first non-management related use. Then the > StringTable use was added 18 months ago - which definitely was outside > the realm of the management API. So that is when the MinimalVM was > first "broken". So it is good that is fixed. > > With regard to the placement in the initialization order, my remaining > concern was with JVMTI event processing that might happen via events > generated very early in the init sequence. But you have now modified > things so that we will only process events in the LIVE phase, which > only activates after all the class library initialization is complete. > > So overall I'm no longer significantly concerned about the change to > the initialization order as I think you have it all covered. Thanks > for bearing with me and all the off-list discussion. Thank you for having this discussion with me and provoking me to recheck the ServiceThread.?? I think we can do further work to future-proof initialization order but the design needs to be improved. Thanks for reviewing, Coleen > > Cheers, > David > ----- > > On 18/12/2019 1:27 am, coleen.phillimore at oracle.com wrote: >> >> >> On 12/16/19 11:04 PM, David Holmes wrote: >>> Clarification ... >>> >>> On 17/12/2019 12:40 pm, coleen.phillimore at oracle.com wrote: >>>> >>>> Short answer below. >>>> >>>> On 12/16/19 5:51 PM, David Holmes wrote: >>>>> Hi Coleen, >>>>> >>>>> A quick initial response ... >>>>> >>>>> On 16/12/2019 11:26 pm, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> >>>>>> On 12/16/19 8:04 AM, David Holmes wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> On 16/12/2019 9:41 pm, coleen.phillimore at oracle.com wrote: >>>>>>>> Summary: Start ServiceThread before compiler threads, and run >>>>>>>> nmethod barriers for zgc before adding to the service thread >>>>>>>> queue, or posting the events on the java thread queue. >>>>>>> >>>>>>> I can't comment on most of this but the earlier starting of the >>>>>>> service thread has some concerns: >>>>>>> >>>>>>> - there is a lot of JDK level initialization which now will not >>>>>>> have happened before the service thread is started and it is far >>>>>>> from obvious that all possible initialization dependencies will >>>>>>> be satisfied >>>>>> >>>>>> I agree that the order of initialization is very sensitive. From >>>>>> the actions that the service thread does, the one that I found >>>>>> was a problem was that events were posted before the LIVE phase >>>>>> (see comment in has_events()), which could have happened with the >>>>>> existing code, but the window for the race is a lot smaller. ? >>>>>> The other actions can be run if there's a GC before >>>>>> initialization but would be a bug in the initialization code, and >>>>>> I didn't find these bugs in all my testing. There are some >>>>>> ordering dependencies that do have odd side effects (between the >>>>>> compiler thread startup and initialization jsr292 classes) which >>>>>> have comments.? This patch doesn't touch those. >>>>>> >>>>>>> >>>>>>> - current starting of the service thread in >>>>>>> Management::initialize is guarded by "#if INCLUDE_MANAGEMENT", >>>>>>> but now you are starting the service thread unconditionally for >>>>>>> all builds. Hmm just saw your latest comment to the bug report - >>>>>>> so the service thread is now (for quite some time?) being used >>>>>>> for other than management tasks and so should always be present >>>>>>> even if INCLUDE_MANAGEMENT is not enabled. Is that sufficient or >>>>>>> are there likely to be other changes needed to actually ensure >>>>>>> that all works correctly? e.g. any code the service thread >>>>>>> executes that is only defined for INCLUDE_MANAGEMENT will need >>>>>>> to be compiled out explicitly. >>>>>>> >>>>>> >>>>>> I asked Jie offline to check the minimal build.? I don't think >>>>>> there are other INCLUDE_MANAGEMENT actions in the service thread >>>>>> and I'm not sure why it was initialized there in the first place. >>>>>> The minimal vm would have been broken ie. hashtables would not >>>>>> have been cleaned up, etc, but I'm not sure how well that is >>>>>> tested or if one would notice. >>>>>>> - the service thread and the notification thread are (were?) >>>>>>> closely related but now started at completely different times >>>>>> >>>>>> The notification thread is limited to "services" so it makes >>>>>> sense where it is.? The ServiceThread does lots of other things.? >>>>>> Maybe it needs renaming in 15. >>>>>>> >>>>>>> The bug report states the problem as: >>>>>>> >>>>>>> "The graal crash is because compiled_method_load events are >>>>>>> added to the ServiceThread's deferred event queue before the >>>>>>> ServiceThread is created so are not walked to keep them from >>>>>>> being zombied." >>>>>>> >>>>>>> so why isn't the solution to ensure the deferred event queue is >>>>>>> walked? I'm not clear how starting the service thread relates to >>>>>>> walking the queue. >>>>>>> >>>>>> >>>>>> The service thread is responsible for walking the deferred event >>>>>> queue.?? See ServiceThread::oops_do/nmethods_do.?? The design >>>>>> could be changed to have some global walk somewhere of this >>>>>> queue, but essentially this queue is processed by the service >>>>>> thread. >>>>> >>>>> Sorry I don't follow. I thought "oops_do" and friends are for the >>>>> GC threads and/or VMThread to call to process oops when GC updates >>>>> them. >>>> >>>> The oops_do and nmethods_do() can be called by a thread walk in >>>> handshakes (by the sweeper thread) and by parallel GC thread walks. >>>> There isn't a single entry to do the thread-specific closures that >>>> we need to do for these deferred event queues.?? I tried a version >>>> that walked the queues with a static call but missed some places >>>> where it would be needed to make this call (didn't work).? Keeping >>>> this associated with the ServiceThread simplifies a lot. >>> >>> Just to clarify that further, the thread walk requires the thread >>> appears in ALL_JAVA_THREADS but that only happens after the >>> ServiceThread has been started. So in essence we don't really need >>> the ServiceThread to have commenced execution earlier, but we need >>> it to have been created. Those two steps are combined in practice. >> >> Yes.? Then the ServiceThread waits on the Service_lock until notified >> by these events: >> >> ?????? while (((sensors_changed = (!UseNotificationThread && >> LowMemoryDetector::has_pending_requests())) | >> ?????????????? (has_jvmti_events = _jvmti_service_queue.has_events()) | >> ?????????????? (has_gc_notification_event = (!UseNotificationThread >> && GCNotifier::has_event())) | >> ?????????????? (has_dcmd_notification_event = (!UseNotificationThread >> && DCmdFactory::has_pending_jmx_notification())) | >> ?????????????? (stringtable_work = StringTable::has_work()) | >> ?????????????? (symboltable_work = SymbolTable::has_work()) | >> ?????????????? (resolved_method_table_work = >> ResolvedMethodTable::has_work()) | >> ?????????????? (thread_id_table_work = ThreadIdTable::has_work()) | >> ?????????????? (protection_domain_table_work = >> SystemDictionary::pd_cache_table()->has_work()) | >> ?????????????? (oopstorage_work = >> OopStorage::has_cleanup_work_and_reset()) >> ????????????? ) == 0) { >> >> The first, third and fourth events are from management.cpp events >> that were initialized after the ServiceThread was started. >> The second event I have changed, to wait until LIVE phase to return >> true. >> The stringtable, symboltable, resolved_method_table, thread_id and pd >> table have static _has_work variables initialized to false. >> The oopstorage_work has similar, but may update a time-based counter >> a bit earlier with the service thread starting earlier. I think this >> is harmless. >> >> It is possible that after the service thread starts and before the >> compiler thread starts, there could be a GC that notifies the >> stringtable to clean up.? This seems like a good thing that the GC >> would clean up these tables with this order.? I ran the tier4 graal >> tests and there were no failures. >> >> Thanks, >> Coleen >>> >>> Cheers, >>> David >>> >>>> thanks, >>>> Coleen >>>> >>>>> >>>>> David >>>>> ----- >>>>> >>>>>> I had an additional change to make the queue non-static but want >>>>>> to limit the change at this point. >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>>> See bug for description of the problems found with the new >>>>>>>> Zombie.java test. >>>>>>>> >>>>>>>> open webrev at >>>>>>>> http://cr.openjdk.java.net/~coleenp/2019/8235829.01/webrev >>>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8235829 >>>>>>>> >>>>>>>> Ran tier1 all platforms, and tier2-8 testing, as well as >>>>>>>> rerunning original test failure from bug >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8173361. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>>> >>>> >> From daniel.daugherty at oracle.com Wed Dec 18 17:37:29 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 18 Dec 2019 12:37:29 -0500 Subject: RFR: 8235966: Process obsolete flags less aggressively In-Reply-To: References: <674e4f0d-563f-3bb4-dee0-74aaacb9b270@oracle.com> Message-ID: Hi David, On 12/17/19 5:03 PM, David Holmes wrote: > Hi Dan, > > Thanks for taking a look. Updated webrev: > > http://cr.openjdk.java.net/~dholmes/8235966/webrev.v2/ src/hotspot/share/runtime/arguments.cpp ??? I like the updates to header comment for verify_special_jvm_flags(). Thumbs up. > > Discussion below. Replies below. > > On 18/12/2019 1:47 am, Daniel D. Daugherty wrote: >> On 12/16/19 12:16 AM, David Holmes wrote: >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8235966 >>> webrev: http://cr.openjdk.java.net/~dholmes/8235966/webrev/ >> >> src/hotspot/share/runtime/arguments.cpp >> ???? L745: ????? // if flag has become obsolete it should not have a >> "globals" flag defined anymore. >> ???? L746: ????? if (!version_less_than(JDK_Version::current(), >> flag.obsolete_in)) { >> ???? L747: ??????? if (JVMFlag::find_declared_flag(flag.name) != NULL) { >> ???? L748: ????????? // Temporarily disable the warning: 8196739 >> ???? L749: ????????? // warning("Global variable for obsolete special >> flag entry \"%s\" should be removed", flag.name); >> ???? L750: ??????? } >> ???? L751: ????? } >> ???????? It seems like we've been down a similar road before: >> >> ???????? JDK-8196739 Disable obsolete/expired VM flag transitional >> warnings >> ???????? https://bugs.openjdk.java.net/browse/JDK-8196739 >> >> ???????? This one may ring a bell... Fixed by dholmes in jdk11-b01... >> :-) >> >> ???????? And this followup sub-task to re-enable that warning: >> >> ???????? JDK-8196741 Re-enable obsolete/expired VM flag transitional >> warnings >> ???????? https://bugs.openjdk.java.net/browse/JDK-8196741 >> >> ???????? was closed as "Won't fix" on 2019.08.02. >> >> ???????? So the obvious questions: >> >> ???????? - Why is the new warning less problematic to tests that don't >> ?????????? tolerate unexpected output? > > Two different situations. The commented out warning happens > unconditionally when you run the VM and it finds any flag marked > obsolete that hasn't been removed. Hence every single test will > encounter this warning. Ouch on such verbosity. > The situation I am modifying is when a test uses a flag that is marked > for obsoletion. In the majority of cases the flag is already > deprecated and so already issuing a deprecation warning that the test > has to handle. Without my change there would still be an obsoletion > warning, so this test is in for a warning no matter what. Good that your change only comes into play when the flag is used. > Also note that for hotspot at least we have strived to make tests > tolerate unexpected output. The reason JDK-8196741 was closed as > "won't fix" was because other areas wouldn't commit to doing that. Yup. Got it. > >> ???????? - If you move forward with this fix, then I think think code >> ?????????? block needs to be removed or modified or am I missing >> something? > > I've rewritten the comment at the head of verify_special_jvm_flags to > explain why we can't issue a warning, and have deleted the block. Thanks for deleting the stale code. > >> ???????? There's a similar commented out check on L757-L765, but that >> one >> ???????? is for an expired flag... You might want to adjust/delete it >> also? > > Deleted. Thanks. > >> ???? L753: ??????? warning("Special flag entry \"%s\" must be >> explicitly obsoleted before expired.", flag.name); >> ???? L754: ??????? success = false; >> ???????? nit - s/before expired/before being expired/ >> ???????? Update: I now see that "style" is in several places in this >> ???????????? function. I'm not sure what to think here... it grates, >> ?? ? ? ? ? ? but I can live with it. >> >> ???????? nit - L75[34] indented too much by two spaces. > > Fixed. > >> ???? L962: ????????? return real_name; >> ???????? nit - indented too much by two spaces. > > Fixed. > >> >> Trying to understand the modified logic in argument processing is >> making my head spin... > > Mine too. It took a few attempts to put the logic in the right place > and make adjustments so that it all works as expected for a correctly > specified flag and an erroneous one. I keep trying to convince myself that we're improving this flag and options code with each release... :-) > >> - You've added a call to is_obsolete_flag() in >> ?? handle_aliases_and_deprecation() and is_obsolete_flag() >> ?? is where the new warning is output: >> >> ???? warning("Temporarily processing option %s; support is scheduled >> for removal in %s" >> >> ?? handle_aliases_and_deprecation() is called from six different places, >> ?? but the call sites are different based on the argument pattern so I >> ?? have (mostly) convinced myself that there should not be any duplicate >> ?? warning lines. > > Right - handle_aliases_and_deprecation is only called for a > syntactically correct flag based on those patterns. It normally > filters out obsoleted/expired flags and lets them fall through to > later error processing (in process_argument after parse_arg returns > false). That error processing is where the normal obsoletion check is > performed. So I had to not filter the flag in > handle_aliases_and_deprecation in this case, but still produce the > warning for a malformed flag. E.g. > > java -XX:+UseParallelOldGC -version > Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing > option UseParallelOldGC; support is scheduled for removal in 15.0 > java version "15-internal" 2020-09-15 > > java -XX:UseParallelOldGC -version > Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing > option UseParallelOldGC; support is scheduled for removal in 15.0 > Missing +/- setting for VM option 'UseParallelOldGC' > Error: Could not create the Java Virtual Machine. Thanks for the example. That helps a lot. > >> So I now understand the new logic that allows an obsoleted option >> to be specified with a warning as long as the option still exists. >> I'm good with the technical change, but... >> >> I'm worried about tests that don't tolerate the new warning mesg, >> i.e., why wouldn't this become an issue again: >> >> JDK-8196739 Disable obsolete/expired VM flag transitional warnings > > Explained above. Yup and thanks. Dan > > Thanks, > David > >> Dan >> >> >>> >>> When a flag is marked as obsolete in the special-flags table we will >>> ignore it and issue a warning that it is being ignored, as soon as >>> we bump the version of the JDK. That means that any tests still >>> using the obsolete flag may start to fail, leading to a surge of >>> test failures at the start of a release cycle. For example for JDK >>> 15 we have a whole bunch of JFR tests that fail because they still >>> try to work with UseParallelOldGC. In another case >>> runtime/cds/appcds/FieldLayoutFlags.java passes only be accident. >>> >>> When a flag is marked as obsolete for a release, all code involving >>> that flag (including tests that use it) must be updated within that >>> release and the flag itself removed. Whilst this is typically >>> scheduled early in a release cycle it isn't reasonable to expect it >>> to all occur within the first couple of days of the release cycle, >>> nor do we want to have to ProblemList a bunch of tests when they >>> start failing. >>> >>> What I propose is to instead allow an obsolete flag to continue to >>> be processed as long as that code removal has not actually occurred >>> - with an adjusted warning. The change I propose: >>> >>> - only treats an obsolete flag as obsolete if the flag cannot be found >>> - added a new flag verification rule that disallows obsoletion in an >>> undefined version, but expiration in a specific version i.e. we must >>> always explicitly obsolete a flag before we expire it. >>> >>> The only downside here is that if we actually forget to file an >>> issue for the actual obsoletion work we may not notice via testing. >>> Of course whenever a change is made to the flags table to add an >>> entry then the issue to do the obsoletion should be filed at the >>> same time. >>> >>> Thanks, >>> David >>> ----- >>> >> From serguei.spitsyn at oracle.com Wed Dec 18 18:33:08 2019 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 18 Dec 2019 10:33:08 -0800 Subject: [14] RFR 8235829: graal crashes with Zombie.java test In-Reply-To: <07b6370a-ff4f-641c-7661-e5b1231a7d8c@oracle.com> References: <447990f9-d276-56c3-aebe-fa84d0cfcc27@oracle.com> <975b6e46-95c1-8a37-b160-b6a57a9633a8@oracle.com> <06d6e9f4-b919-3062-7d18-1245e66b27b2@oracle.com> <39716b03-dd5e-580e-2c91-004da64bcc19@oracle.com> <53208d1a-b969-d2e7-8657-d95f2d2d22e8@oracle.com> <07b6370a-ff4f-641c-7661-e5b1231a7d8c@oracle.com> Message-ID: <102b841c-6797-dcda-4e33-e7282a2336b0@oracle.com> Hi Coleen, Just wanted to confirm the webrev V2 version looks okay to me. Sorry for replying on the wrong mailing thread. Thanks, Serguei On 12/18/19 08:42, coleen.phillimore at oracle.com wrote: > > > On 12/18/19 8:45 AM, David Holmes wrote: >> Thanks for the additional info Coleen! >> >> Just to add a bit more to the initialization history. The >> ServiceThread is a generalization of the LowMemoryDetectorThread that >> was part of the management API, and so it was initialized in >> Management::initialize. When it turned into the ServiceThread - to >> process JVMTI deferred events in addition to the low-memory-detector >> events - the initialization placement remained the same. Then later >> the INCLUDE_MANAGEMENT guards were added (JDK-7189254, October 2012). >> Later still we started adding other items of work for the >> ServiceThread. The earliest was the AllocationContextService >> notification in September 2014 but as that no longer exists I can't >> tell if that was the first non-management related use. Then the >> StringTable use was added 18 months ago - which definitely was >> outside the realm of the management API. So that is when the >> MinimalVM was first "broken". So it is good that is fixed. >> >> With regard to the placement in the initialization order, my >> remaining concern was with JVMTI event processing that might happen >> via events generated very early in the init sequence. But you have >> now modified things so that we will only process events in the LIVE >> phase, which only activates after all the class library >> initialization is complete. >> >> So overall I'm no longer significantly concerned about the change to >> the initialization order as I think you have it all covered. Thanks >> for bearing with me and all the off-list discussion. > > Thank you for having this discussion with me and provoking me to > recheck the ServiceThread.?? I think we can do further work to > future-proof initialization order but the design needs to be improved. > > Thanks for reviewing, > Coleen >> >> Cheers, >> David >> ----- >> >> On 18/12/2019 1:27 am, coleen.phillimore at oracle.com wrote: >>> >>> >>> On 12/16/19 11:04 PM, David Holmes wrote: >>>> Clarification ... >>>> >>>> On 17/12/2019 12:40 pm, coleen.phillimore at oracle.com wrote: >>>>> >>>>> Short answer below. >>>>> >>>>> On 12/16/19 5:51 PM, David Holmes wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> A quick initial response ... >>>>>> >>>>>> On 16/12/2019 11:26 pm, coleen.phillimore at oracle.com wrote: >>>>>>> >>>>>>> >>>>>>> On 12/16/19 8:04 AM, David Holmes wrote: >>>>>>>> Hi Coleen, >>>>>>>> >>>>>>>> On 16/12/2019 9:41 pm, coleen.phillimore at oracle.com wrote: >>>>>>>>> Summary: Start ServiceThread before compiler threads, and run >>>>>>>>> nmethod barriers for zgc before adding to the service thread >>>>>>>>> queue, or posting the events on the java thread queue. >>>>>>>> >>>>>>>> I can't comment on most of this but the earlier starting of the >>>>>>>> service thread has some concerns: >>>>>>>> >>>>>>>> - there is a lot of JDK level initialization which now will not >>>>>>>> have happened before the service thread is started and it is >>>>>>>> far from obvious that all possible initialization dependencies >>>>>>>> will be satisfied >>>>>>> >>>>>>> I agree that the order of initialization is very sensitive. From >>>>>>> the actions that the service thread does, the one that I found >>>>>>> was a problem was that events were posted before the LIVE phase >>>>>>> (see comment in has_events()), which could have happened with >>>>>>> the existing code, but the window for the race is a lot smaller. >>>>>>> ? The other actions can be run if there's a GC before >>>>>>> initialization but would be a bug in the initialization code, >>>>>>> and I didn't find these bugs in all my testing. There are some >>>>>>> ordering dependencies that do have odd side effects (between the >>>>>>> compiler thread startup and initialization jsr292 classes) which >>>>>>> have comments.? This patch doesn't touch those. >>>>>>> >>>>>>>> >>>>>>>> - current starting of the service thread in >>>>>>>> Management::initialize is guarded by "#if INCLUDE_MANAGEMENT", >>>>>>>> but now you are starting the service thread unconditionally for >>>>>>>> all builds. Hmm just saw your latest comment to the bug report >>>>>>>> - so the service thread is now (for quite some time?) being >>>>>>>> used for other than management tasks and so should always be >>>>>>>> present even if INCLUDE_MANAGEMENT is not enabled. Is that >>>>>>>> sufficient or are there likely to be other changes needed to >>>>>>>> actually ensure that all works correctly? e.g. any code the >>>>>>>> service thread executes that is only defined for >>>>>>>> INCLUDE_MANAGEMENT will need to be compiled out explicitly. >>>>>>>> >>>>>>> >>>>>>> I asked Jie offline to check the minimal build.? I don't think >>>>>>> there are other INCLUDE_MANAGEMENT actions in the service thread >>>>>>> and I'm not sure why it was initialized there in the first >>>>>>> place. The minimal vm would have been broken ie. hashtables >>>>>>> would not have been cleaned up, etc, but I'm not sure how well >>>>>>> that is tested or if one would notice. >>>>>>>> - the service thread and the notification thread are (were?) >>>>>>>> closely related but now started at completely different times >>>>>>> >>>>>>> The notification thread is limited to "services" so it makes >>>>>>> sense where it is.? The ServiceThread does lots of other >>>>>>> things.? Maybe it needs renaming in 15. >>>>>>>> >>>>>>>> The bug report states the problem as: >>>>>>>> >>>>>>>> "The graal crash is because compiled_method_load events are >>>>>>>> added to the ServiceThread's deferred event queue before the >>>>>>>> ServiceThread is created so are not walked to keep them from >>>>>>>> being zombied." >>>>>>>> >>>>>>>> so why isn't the solution to ensure the deferred event queue is >>>>>>>> walked? I'm not clear how starting the service thread relates >>>>>>>> to walking the queue. >>>>>>>> >>>>>>> >>>>>>> The service thread is responsible for walking the deferred event >>>>>>> queue.?? See ServiceThread::oops_do/nmethods_do.?? The design >>>>>>> could be changed to have some global walk somewhere of this >>>>>>> queue, but essentially this queue is processed by the service >>>>>>> thread. >>>>>> >>>>>> Sorry I don't follow. I thought "oops_do" and friends are for the >>>>>> GC threads and/or VMThread to call to process oops when GC >>>>>> updates them. >>>>> >>>>> The oops_do and nmethods_do() can be called by a thread walk in >>>>> handshakes (by the sweeper thread) and by parallel GC thread >>>>> walks. There isn't a single entry to do the thread-specific >>>>> closures that we need to do for these deferred event queues.?? I >>>>> tried a version that walked the queues with a static call but >>>>> missed some places where it would be needed to make this call >>>>> (didn't work).? Keeping this associated with the ServiceThread >>>>> simplifies a lot. >>>> >>>> Just to clarify that further, the thread walk requires the thread >>>> appears in ALL_JAVA_THREADS but that only happens after the >>>> ServiceThread has been started. So in essence we don't really need >>>> the ServiceThread to have commenced execution earlier, but we need >>>> it to have been created. Those two steps are combined in practice. >>> >>> Yes.? Then the ServiceThread waits on the Service_lock until >>> notified by these events: >>> >>> ?????? while (((sensors_changed = (!UseNotificationThread && >>> LowMemoryDetector::has_pending_requests())) | >>> ?????????????? (has_jvmti_events = _jvmti_service_queue.has_events()) | >>> ?????????????? (has_gc_notification_event = (!UseNotificationThread >>> && GCNotifier::has_event())) | >>> ?????????????? (has_dcmd_notification_event = >>> (!UseNotificationThread && >>> DCmdFactory::has_pending_jmx_notification())) | >>> ?????????????? (stringtable_work = StringTable::has_work()) | >>> ?????????????? (symboltable_work = SymbolTable::has_work()) | >>> ?????????????? (resolved_method_table_work = >>> ResolvedMethodTable::has_work()) | >>> ?????????????? (thread_id_table_work = ThreadIdTable::has_work()) | >>> ?????????????? (protection_domain_table_work = >>> SystemDictionary::pd_cache_table()->has_work()) | >>> ?????????????? (oopstorage_work = >>> OopStorage::has_cleanup_work_and_reset()) >>> ????????????? ) == 0) { >>> >>> The first, third and fourth events are from management.cpp events >>> that were initialized after the ServiceThread was started. >>> The second event I have changed, to wait until LIVE phase to return >>> true. >>> The stringtable, symboltable, resolved_method_table, thread_id and >>> pd table have static _has_work variables initialized to false. >>> The oopstorage_work has similar, but may update a time-based counter >>> a bit earlier with the service thread starting earlier. I think this >>> is harmless. >>> >>> It is possible that after the service thread starts and before the >>> compiler thread starts, there could be a GC that notifies the >>> stringtable to clean up.? This seems like a good thing that the GC >>> would clean up these tables with this order.? I ran the tier4 graal >>> tests and there were no failures. >>> >>> Thanks, >>> Coleen >>>> >>>> Cheers, >>>> David >>>> >>>>> thanks, >>>>> Coleen >>>>> >>>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>>> I had an additional change to make the queue non-static but want >>>>>>> to limit the change at this point. >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>>> See bug for description of the problems found with the new >>>>>>>>> Zombie.java test. >>>>>>>>> >>>>>>>>> open webrev at >>>>>>>>> http://cr.openjdk.java.net/~coleenp/2019/8235829.01/webrev >>>>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8235829 >>>>>>>>> >>>>>>>>> Ran tier1 all platforms, and tier2-8 testing, as well as >>>>>>>>> rerunning original test failure from bug >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8173361. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Coleen >>>>>>> >>>>> >>> > From augustnagro at gmail.com Wed Dec 18 18:39:22 2019 From: augustnagro at gmail.com (August Nagro) Date: Wed, 18 Dec 2019 12:39:22 -0600 Subject: building libjvm with -Os for space optimization - was : RE: RFR: 8234525: enable link-time section-gc for linux s390x to remove unused code In-Reply-To: References: Message-ID: > (btw I wonder how much effect profile guided optimization would bring in your experiments ) That's good idea; I'm also curious about PGO. I know some apps get 10-20% performance boost, however setting up PGO is pretty inconvenient. But maybe having it run over the DaCapo / Renaissance benchmarks would make a good profile. I also used some flags that OpenJDK could pickup, like `-fomit-frame-pointer` on non-debug builds. I also haven't had any problems running with the higher optimization, Ofast. > What alternatives do you have in mind? There are many opportunities I think. One way would be to allow jar / jmod files to use a modern compression algorithm (like LZ4). Not only would distributions be smaller but probably faster as well, since the cost of IO >> decompression in many cases. The size of OpenJDK's jar/jmod files is much larger than libjvm.so. And this isn't my idea, I saw it on Aleksey Shiplev's twitter some time ago: https://twitter.com/shipilev/status/1100396679285665794 On Wed, Dec 18, 2019 at 10:07 AM Baesken, Matthias wrote: > > > - I compiled with clang since I'm on Mac. > - > > > > Thanks for clarifying, that?s what I thought . > > > > (btw I wonder how much effect profile guided optimization would bring in > your experiments ) > > > > > > - My opinion is that there are probably more compelling alternatives > if reducing binary size is the goal. Even if the tests show that Os/O2 is > no different than O3, > - who knows if this will be true in the future. > - > > > > What alternatives do you have in mind ? > > > > Best regards, Matthias > > > > > > *From:* August Nagro > *Sent:* Mittwoch, 18. Dezember 2019 14:42 > *To:* Baesken, Matthias > *Cc:* claes.redestad at oracle.com; Doerr, Martin ; > erik.joelsson at oracle.com; build-dev at openjdk.java.net; > hotspot-dev at openjdk.java.net > *Subject:* Re: RE: building libjvm with -Os for space optimization - was > : RE: RFR: 8234525: enable link-time section-gc for linux s390x to remove > unused code > > > > I compiled with clang since I'm on Mac. > > > > The Renaissance benchmark suite is also a good one that I learned about > recently. > > > > My opinion is that there are probably more compelling alternatives if > reducing binary size is the goal. Even if the tests show that Os/O2 is no > different than O3, who knows if this will be true in the future. > > > > Regards, > > > > - August > > > > On Wed, Dec 18, 2019, 1:58 AM Baesken, Matthias > wrote: > > Hi August , thanks for pointing to your webpage, very interesting ! > > We did our builds+tests/benchmarks with gcc 7.4.0 , what > compiler+version did you use? > > Probably I should look a bit more into Dacapo (we used that one in the > past too sometimes). > > Best regards, Matthias > > > > > > I published some benchmarks of OpenJDK on Mac with Ofast and O3 [1]. > > Some microbenchmarks like Netty?s HttpObjectEncoder experienced >100% > > speedup with O3, and the more real-world Dacapo suite was ~15% > > improvement over O2 (which is exactly the same as Os). I did include a > few > > other flags, however the speedup was primarily due to optimization level. > > > > Building with Os is the old wisdom. It used to be the case that many > programs > > would be faster with the smaller binary size, but this is almost never > the case > > nowadays. > > > > - August > > > > [1]: http://august.nagro.us/optimized-openjdk.html > > From augustnagro at gmail.com Wed Dec 18 19:51:54 2019 From: augustnagro at gmail.com (August Nagro) Date: Wed, 18 Dec 2019 13:51:54 -0600 Subject: Bounds Check Elimination with Fast-Range In-Reply-To: <28CF6CF4-3E20-489C-9659-C50E4222EC22@oracle.com> References: <19544187-6670-4A65-AF47-297F38E8D555@gmail.com> <87r22549fg.fsf@oldenburg2.str.redhat.com> <1AB809E9-27B3-423E-AB1A-448D6B4689F6@oracle.com> <877e3x0wji.fsf@oldenburg2.str.redhat.com> <28CF6CF4-3E20-489C-9659-C50E4222EC22@oracle.com> Message-ID: John, Apologies for the late response, school has been busy lately. I do like the idea of growing by a 'size table' or similar, especially if the growth rate decreases as the hashmap size increases. Since the probability of fragmentation increases with the number of resizes, bumping down the resize factor could be a good solution. To remove the bounds-checking in C2, I've taken a look at the subop class. I think I understand the change, but I'm wondering about the best way to handle a negative hash value. In fast range the hash & size must be less than 2^32, so that the result of their multiplication fits in a long. So one way is to do hash * (long) size, or similar. However, there's no unsigned multiply in Java, so if hash is negative it is widened to a negative long. One could use Integer::toUnsignedLong, or just directly `hash & 0xffffffffL`. However, this complicates the code shape. So I wonder if there is a better way. One option might be to make a Math::fastRange(int hash, int size) method that is a hotspot intrinsic and directly uses the unsigned instructions. Would appreciate some guidance on this. - August On Tue, Dec 3, 2019 at 12:06 AM John Rose wrote: > On Nov 18, 2019, at 12:17 PM, Florian Weimer wrote: > > > int bucket = fr(h * M); // M = 0x2357BD or something > > or maybe something fast and sloppy like: > > int bucket = fr(h + (h << 8)); > > > Surely this one works, since fr is the final operation. > The shift/add is just a mixing step to precondition the input. > > Just for the record, I?d like to keep brainstorming a bit more, > though surely this sort of thing is of limited interest. So, > just a little more from me on this. > > If we had BITR I?d want to try something like fr(h - bitr(h)). > > But what I keep wishing for a good one- or two-cycle instruction > that will mix all input bits into all the output bits, so that any > change in one input bit is likely to cause cascading changes in > many output bits, perhaps even 50% on average. A pair of AES > steps is a good example of this. I think AES would be superior to > multiply (for mixing) when used on 128 bit payloads or larger, so > it looks appealing (to me) for vectorizable hashing applications. > Though it is overkill on scalars, I think it points in promising > directions for scalars also. > > > or even: > > int bucket = fr(h) ^ (h & (N-1)); > > Does this really work? I don't think so. > > > Oops, you are right. Something like it might work, though. > > The idea, on paper, is that h & (N-1) is less than N, for any N >=1. > And if N-1 has a high enough pop-count the information content is close to > 50% of h (though maybe 50% of the bits are masked out). The xor of two > quasi-independent values both less than N is, well, less than 2^(ceil lg > N), > not N, which is a bug. Oops. There are ways to quickly combine two values > less than N and reduce the result to less than N: You do a conditional > subtract of N if the sum is >= N. > > So the tactical area I?m trying to explore here is to take two reduced > hashes developed in parallel, which depend on different bits of the > input, and combine them into a single stronger hash (not by ^). > > Maybe (I can?t resist hacking at it some more): > > int h1 = H1(h), h2 = H2(h); > int bucket = CCA(h1 - h2, N); > // where H1 := fr, H2(h) := (h & (N-1)) > // where CCA(x, N) := x + ((x >> 31) & N) // Conditional Compensating Add > > In this framework, a better H2 for favoring the low bits of h might be > H2(h) := ((1< the number of low bits of h that feed into the final bucket selection, > while fr (H1) arguably maximizes the number of influential high bits. > > I think this kind of perturbation is quite expensive. Arm's BITR should > be helpful here. > > > Yes, BITR is a helpful building block. If I understand you correctly, it > needs to be combined with other operations, such as multiply, shift, xor, > etc., > and can overcome biases towards high bits or towards low bits that come > from the simple arithmetic definitions of the other mixing operations. > > The hack with CCA(h1 - h2, N) seems competitive with a BITR-based > mixing step, since H2 can be very simple. > > A scalar variant of two AES steps (with xor of a second register or > constant > parameter at both stages) would be a better building block for strongly > mixing bits. > Or some other shallow linear network with a layer of non-linear S-boxes. > > But even though this operation is commonly needed and > easily implemented in hardware, it's rarely found in CPUs. > > > Yep; the whole cottage industry of building clever mixing functions > out of hand calculator functions could be sidelined if CPUs gave us good > cheap mixing primitives out of the box. The crypto literature is full of > them, and many are designed to be easy to implement in silicon. > > ? John > > P.S. I mention AES because I?m familiar with that bit of crypto tech, and > also because I actually tried it out once on the smhasher quality > benchmark. > No surprise in hindsight; it passes the quality tests with just two rounds. > Given that it is as cheap as multiplication, and handles twice as many > bits at a time, but requires two steps for full mixing, it would seem to > be competitive with multiplication as a mixing step. It has no built-in > biases towards high or low bits, so that?s an advantage over > multiplication. > > Why two rounds? The one-round version has flaws, as a hash function, > which are obvious on inspection of the simple structure of an AES round. > Not every output bit is data-dependent on every input bit of one round, > but two rounds swirls them all together. Are back-to-back AES rounds > expensive? Maybe, although that?s how the instructions are designed to > be used, about 10 of them back to back to do real crypto. > > From coleen.phillimore at oracle.com Wed Dec 18 22:06:08 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 18 Dec 2019 17:06:08 -0500 Subject: [14] RFR 8235829: graal crashes with Zombie.java test In-Reply-To: <102b841c-6797-dcda-4e33-e7282a2336b0@oracle.com> References: <447990f9-d276-56c3-aebe-fa84d0cfcc27@oracle.com> <975b6e46-95c1-8a37-b160-b6a57a9633a8@oracle.com> <06d6e9f4-b919-3062-7d18-1245e66b27b2@oracle.com> <39716b03-dd5e-580e-2c91-004da64bcc19@oracle.com> <53208d1a-b969-d2e7-8657-d95f2d2d22e8@oracle.com> <07b6370a-ff4f-641c-7661-e5b1231a7d8c@oracle.com> <102b841c-6797-dcda-4e33-e7282a2336b0@oracle.com> Message-ID: <7803e51d-42d0-fe15-41fb-70a7d8a630ab@oracle.com> Thanks Serguei! Coleen On 12/18/19 1:33 PM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > Just wanted to confirm the webrev V2 version looks okay to me. > Sorry for replying on the wrong mailing thread. > > Thanks, > Serguei > > > On 12/18/19 08:42, coleen.phillimore at oracle.com wrote: >> >> >> On 12/18/19 8:45 AM, David Holmes wrote: >>> Thanks for the additional info Coleen! >>> >>> Just to add a bit more to the initialization history. The >>> ServiceThread is a generalization of the LowMemoryDetectorThread >>> that was part of the management API, and so it was initialized in >>> Management::initialize. When it turned into the ServiceThread - to >>> process JVMTI deferred events in addition to the low-memory-detector >>> events - the initialization placement remained the same. Then later >>> the INCLUDE_MANAGEMENT guards were added (JDK-7189254, October >>> 2012). Later still we started adding other items of work for the >>> ServiceThread. The earliest was the AllocationContextService >>> notification in September 2014 but as that no longer exists I can't >>> tell if that was the first non-management related use. Then the >>> StringTable use was added 18 months ago - which definitely was >>> outside the realm of the management API. So that is when the >>> MinimalVM was first "broken". So it is good that is fixed. >>> >>> With regard to the placement in the initialization order, my >>> remaining concern was with JVMTI event processing that might happen >>> via events generated very early in the init sequence. But you have >>> now modified things so that we will only process events in the LIVE >>> phase, which only activates after all the class library >>> initialization is complete. >>> >>> So overall I'm no longer significantly concerned about the change to >>> the initialization order as I think you have it all covered. Thanks >>> for bearing with me and all the off-list discussion. >> >> Thank you for having this discussion with me and provoking me to >> recheck the ServiceThread.?? I think we can do further work to >> future-proof initialization order but the design needs to be improved. >> >> Thanks for reviewing, >> Coleen >>> >>> Cheers, >>> David >>> ----- >>> >>> On 18/12/2019 1:27 am, coleen.phillimore at oracle.com wrote: >>>> >>>> >>>> On 12/16/19 11:04 PM, David Holmes wrote: >>>>> Clarification ... >>>>> >>>>> On 17/12/2019 12:40 pm, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> Short answer below. >>>>>> >>>>>> On 12/16/19 5:51 PM, David Holmes wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> A quick initial response ... >>>>>>> >>>>>>> On 16/12/2019 11:26 pm, coleen.phillimore at oracle.com wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 12/16/19 8:04 AM, David Holmes wrote: >>>>>>>>> Hi Coleen, >>>>>>>>> >>>>>>>>> On 16/12/2019 9:41 pm, coleen.phillimore at oracle.com wrote: >>>>>>>>>> Summary: Start ServiceThread before compiler threads, and run >>>>>>>>>> nmethod barriers for zgc before adding to the service thread >>>>>>>>>> queue, or posting the events on the java thread queue. >>>>>>>>> >>>>>>>>> I can't comment on most of this but the earlier starting of >>>>>>>>> the service thread has some concerns: >>>>>>>>> >>>>>>>>> - there is a lot of JDK level initialization which now will >>>>>>>>> not have happened before the service thread is started and it >>>>>>>>> is far from obvious that all possible initialization >>>>>>>>> dependencies will be satisfied >>>>>>>> >>>>>>>> I agree that the order of initialization is very sensitive. >>>>>>>> From the actions that the service thread does, the one that I >>>>>>>> found was a problem was that events were posted before the LIVE >>>>>>>> phase (see comment in has_events()), which could have happened >>>>>>>> with the existing code, but the window for the race is a lot >>>>>>>> smaller. ? The other actions can be run if there's a GC before >>>>>>>> initialization but would be a bug in the initialization code, >>>>>>>> and I didn't find these bugs in all my testing. There are some >>>>>>>> ordering dependencies that do have odd side effects (between >>>>>>>> the compiler thread startup and initialization jsr292 classes) >>>>>>>> which have comments. This patch doesn't touch those. >>>>>>>> >>>>>>>>> >>>>>>>>> - current starting of the service thread in >>>>>>>>> Management::initialize is guarded by "#if INCLUDE_MANAGEMENT", >>>>>>>>> but now you are starting the service thread unconditionally >>>>>>>>> for all builds. Hmm just saw your latest comment to the bug >>>>>>>>> report - so the service thread is now (for quite some time?) >>>>>>>>> being used for other than management tasks and so should >>>>>>>>> always be present even if INCLUDE_MANAGEMENT is not enabled. >>>>>>>>> Is that sufficient or are there likely to be other changes >>>>>>>>> needed to actually ensure that all works correctly? e.g. any >>>>>>>>> code the service thread executes that is only defined for >>>>>>>>> INCLUDE_MANAGEMENT will need to be compiled out explicitly. >>>>>>>>> >>>>>>>> >>>>>>>> I asked Jie offline to check the minimal build.? I don't think >>>>>>>> there are other INCLUDE_MANAGEMENT actions in the service >>>>>>>> thread and I'm not sure why it was initialized there in the >>>>>>>> first place. The minimal vm would have been broken ie. >>>>>>>> hashtables would not have been cleaned up, etc, but I'm not >>>>>>>> sure how well that is tested or if one would notice. >>>>>>>>> - the service thread and the notification thread are (were?) >>>>>>>>> closely related but now started at completely different times >>>>>>>> >>>>>>>> The notification thread is limited to "services" so it makes >>>>>>>> sense where it is.? The ServiceThread does lots of other >>>>>>>> things.? Maybe it needs renaming in 15. >>>>>>>>> >>>>>>>>> The bug report states the problem as: >>>>>>>>> >>>>>>>>> "The graal crash is because compiled_method_load events are >>>>>>>>> added to the ServiceThread's deferred event queue before the >>>>>>>>> ServiceThread is created so are not walked to keep them from >>>>>>>>> being zombied." >>>>>>>>> >>>>>>>>> so why isn't the solution to ensure the deferred event queue >>>>>>>>> is walked? I'm not clear how starting the service thread >>>>>>>>> relates to walking the queue. >>>>>>>>> >>>>>>>> >>>>>>>> The service thread is responsible for walking the deferred >>>>>>>> event queue.?? See ServiceThread::oops_do/nmethods_do.?? The >>>>>>>> design could be changed to have some global walk somewhere of >>>>>>>> this queue, but essentially this queue is processed by the >>>>>>>> service thread. >>>>>>> >>>>>>> Sorry I don't follow. I thought "oops_do" and friends are for >>>>>>> the GC threads and/or VMThread to call to process oops when GC >>>>>>> updates them. >>>>>> >>>>>> The oops_do and nmethods_do() can be called by a thread walk in >>>>>> handshakes (by the sweeper thread) and by parallel GC thread >>>>>> walks. There isn't a single entry to do the thread-specific >>>>>> closures that we need to do for these deferred event queues.?? I >>>>>> tried a version that walked the queues with a static call but >>>>>> missed some places where it would be needed to make this call >>>>>> (didn't work).? Keeping this associated with the ServiceThread >>>>>> simplifies a lot. >>>>> >>>>> Just to clarify that further, the thread walk requires the thread >>>>> appears in ALL_JAVA_THREADS but that only happens after the >>>>> ServiceThread has been started. So in essence we don't really need >>>>> the ServiceThread to have commenced execution earlier, but we need >>>>> it to have been created. Those two steps are combined in practice. >>>> >>>> Yes.? Then the ServiceThread waits on the Service_lock until >>>> notified by these events: >>>> >>>> ?????? while (((sensors_changed = (!UseNotificationThread && >>>> LowMemoryDetector::has_pending_requests())) | >>>> ?????????????? (has_jvmti_events = >>>> _jvmti_service_queue.has_events()) | >>>> ?????????????? (has_gc_notification_event = (!UseNotificationThread >>>> && GCNotifier::has_event())) | >>>> ?????????????? (has_dcmd_notification_event = >>>> (!UseNotificationThread && >>>> DCmdFactory::has_pending_jmx_notification())) | >>>> ?????????????? (stringtable_work = StringTable::has_work()) | >>>> ?????????????? (symboltable_work = SymbolTable::has_work()) | >>>> ?????????????? (resolved_method_table_work = >>>> ResolvedMethodTable::has_work()) | >>>> ?????????????? (thread_id_table_work = ThreadIdTable::has_work()) | >>>> ?????????????? (protection_domain_table_work = >>>> SystemDictionary::pd_cache_table()->has_work()) | >>>> ?????????????? (oopstorage_work = >>>> OopStorage::has_cleanup_work_and_reset()) >>>> ????????????? ) == 0) { >>>> >>>> The first, third and fourth events are from management.cpp events >>>> that were initialized after the ServiceThread was started. >>>> The second event I have changed, to wait until LIVE phase to return >>>> true. >>>> The stringtable, symboltable, resolved_method_table, thread_id and >>>> pd table have static _has_work variables initialized to false. >>>> The oopstorage_work has similar, but may update a time-based >>>> counter a bit earlier with the service thread starting earlier. I >>>> think this is harmless. >>>> >>>> It is possible that after the service thread starts and before the >>>> compiler thread starts, there could be a GC that notifies the >>>> stringtable to clean up.? This seems like a good thing that the GC >>>> would clean up these tables with this order.? I ran the tier4 graal >>>> tests and there were no failures. >>>> >>>> Thanks, >>>> Coleen >>>>> >>>>> Cheers, >>>>> David >>>>> >>>>>> thanks, >>>>>> Coleen >>>>>> >>>>>>> >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> I had an additional change to make the queue non-static but >>>>>>>> want to limit the change at this point. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> >>>>>>>>>> See bug for description of the problems found with the new >>>>>>>>>> Zombie.java test. >>>>>>>>>> >>>>>>>>>> open webrev at >>>>>>>>>> http://cr.openjdk.java.net/~coleenp/2019/8235829.01/webrev >>>>>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8235829 >>>>>>>>>> >>>>>>>>>> Ran tier1 all platforms, and tier2-8 testing, as well as >>>>>>>>>> rerunning original test failure from bug >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8173361. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Coleen >>>>>>>> >>>>>> >>>> >> > From daniel.daugherty at oracle.com Wed Dec 18 23:14:29 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 18 Dec 2019 18:14:29 -0500 Subject: Urgent RFR(T): 8236226: fix merge error in src/hotspot/share/gc/z/zRootsIterator.cpp Message-ID: Greetings, A merge error crept into the latest sync from JDK14 -> JDK15. I consider this a trivial fix. Here's the context diff: $ hg diff diff -r 87266ac324d7 src/hotspot/share/gc/z/zRootsIterator.cpp --- a/src/hotspot/share/gc/z/zRootsIterator.cpp Wed Dec 18 23:46:55 2019 +0100 +++ b/src/hotspot/share/gc/z/zRootsIterator.cpp Wed Dec 18 18:11:08 2019 -0500 @@ -44,7 +44,8 @@ ?#include "memory/universe.hpp" ?#include "prims/jvmtiExport.hpp" ?#include "prims/resolvedMethodTable.hpp" -#include "runtime/atomic.hpp"#include "runtime/safepoint.hpp" +#include "runtime/atomic.hpp" +#include "runtime/safepoint.hpp" ?#include "runtime/synchronizer.hpp" ?#include "runtime/thread.hpp" ?#include "runtime/vmThread.hpp" Thanks, in advance, for any comments, questions or suggestions. Dan From david.holmes at oracle.com Wed Dec 18 23:19:13 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 19 Dec 2019 09:19:13 +1000 Subject: Urgent RFR(T): 8236226: fix merge error in src/hotspot/share/gc/z/zRootsIterator.cpp In-Reply-To: References: Message-ID: <4d987a81-2048-29a2-7fd3-7e548c582a32@oracle.com> Ship it! Thanks, David On 19/12/2019 9:14 am, Daniel D. Daugherty wrote: > Greetings, > > A merge error crept into the latest sync from JDK14 -> JDK15. I consider > this a trivial fix. > > Here's the context diff: > > $ hg diff > diff -r 87266ac324d7 src/hotspot/share/gc/z/zRootsIterator.cpp > --- a/src/hotspot/share/gc/z/zRootsIterator.cpp Wed Dec 18 23:46:55 2019 > +0100 > +++ b/src/hotspot/share/gc/z/zRootsIterator.cpp Wed Dec 18 18:11:08 2019 > -0500 > @@ -44,7 +44,8 @@ > ?#include "memory/universe.hpp" > ?#include "prims/jvmtiExport.hpp" > ?#include "prims/resolvedMethodTable.hpp" > -#include "runtime/atomic.hpp"#include "runtime/safepoint.hpp" > +#include "runtime/atomic.hpp" > +#include "runtime/safepoint.hpp" > ?#include "runtime/synchronizer.hpp" > ?#include "runtime/thread.hpp" > ?#include "runtime/vmThread.hpp" > > > Thanks, in advance, for any comments, questions or suggestions. > > Dan > From kim.barrett at oracle.com Wed Dec 18 23:19:48 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 18 Dec 2019 18:19:48 -0500 Subject: Urgent RFR(T): 8236226: fix merge error in src/hotspot/share/gc/z/zRootsIterator.cpp In-Reply-To: References: Message-ID: <6041E4BD-1702-4E11-9F9D-A4817F48A7B9@oracle.com> > On Dec 18, 2019, at 6:14 PM, Daniel D. Daugherty wrote: > > Greetings, > > A merge error crept into the latest sync from JDK14 -> JDK15. I consider > this a trivial fix. > > Here's the context diff: > > $ hg diff > diff -r 87266ac324d7 src/hotspot/share/gc/z/zRootsIterator.cpp > --- a/src/hotspot/share/gc/z/zRootsIterator.cpp Wed Dec 18 23:46:55 2019 +0100 > +++ b/src/hotspot/share/gc/z/zRootsIterator.cpp Wed Dec 18 18:11:08 2019 -0500 > @@ -44,7 +44,8 @@ > #include "memory/universe.hpp" > #include "prims/jvmtiExport.hpp" > #include "prims/resolvedMethodTable.hpp" > -#include "runtime/atomic.hpp"#include "runtime/safepoint.hpp" > +#include "runtime/atomic.hpp" > +#include "runtime/safepoint.hpp" > #include "runtime/synchronizer.hpp" > #include "runtime/thread.hpp" > #include "runtime/vmThread.hpp" > > > Thanks, in advance, for any comments, questions or suggestions. > > Dan Looks good. From daniel.daugherty at oracle.com Wed Dec 18 23:19:56 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 18 Dec 2019 18:19:56 -0500 Subject: Urgent RFR(T): 8236226: fix merge error in src/hotspot/share/gc/z/zRootsIterator.cpp In-Reply-To: <4d987a81-2048-29a2-7fd3-7e548c582a32@oracle.com> References: <4d987a81-2048-29a2-7fd3-7e548c582a32@oracle.com> Message-ID: <8845b15e-3a86-83a5-2b36-c0934ef17cf1@oracle.com> Thanks! Should I push this fix in hopes that it is the only merge error or should I do a Linux test build? Dan On 12/18/19 6:19 PM, David Holmes wrote: > Ship it! > > Thanks, > David > > On 19/12/2019 9:14 am, Daniel D. Daugherty wrote: >> Greetings, >> >> A merge error crept into the latest sync from JDK14 -> JDK15. I consider >> this a trivial fix. >> >> Here's the context diff: >> >> $ hg diff >> diff -r 87266ac324d7 src/hotspot/share/gc/z/zRootsIterator.cpp >> --- a/src/hotspot/share/gc/z/zRootsIterator.cpp Wed Dec 18 23:46:55 >> 2019 +0100 >> +++ b/src/hotspot/share/gc/z/zRootsIterator.cpp Wed Dec 18 18:11:08 >> 2019 -0500 >> @@ -44,7 +44,8 @@ >> ??#include "memory/universe.hpp" >> ??#include "prims/jvmtiExport.hpp" >> ??#include "prims/resolvedMethodTable.hpp" >> -#include "runtime/atomic.hpp"#include "runtime/safepoint.hpp" >> +#include "runtime/atomic.hpp" >> +#include "runtime/safepoint.hpp" >> ??#include "runtime/synchronizer.hpp" >> ??#include "runtime/thread.hpp" >> ??#include "runtime/vmThread.hpp" >> >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> From david.holmes at oracle.com Wed Dec 18 23:20:57 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 19 Dec 2019 09:20:57 +1000 Subject: Urgent RFR(T): 8236226: fix merge error in src/hotspot/share/gc/z/zRootsIterator.cpp In-Reply-To: <8845b15e-3a86-83a5-2b36-c0934ef17cf1@oracle.com> References: <4d987a81-2048-29a2-7fd3-7e548c582a32@oracle.com> <8845b15e-3a86-83a5-2b36-c0934ef17cf1@oracle.com> Message-ID: Push it now and try to avoid CI failure. :) David On 19/12/2019 9:19 am, Daniel D. Daugherty wrote: > Thanks! Should I push this fix in hopes that it is the only merge > error or should I do a Linux test build? > > Dan > > > On 12/18/19 6:19 PM, David Holmes wrote: >> Ship it! >> >> Thanks, >> David >> >> On 19/12/2019 9:14 am, Daniel D. Daugherty wrote: >>> Greetings, >>> >>> A merge error crept into the latest sync from JDK14 -> JDK15. I consider >>> this a trivial fix. >>> >>> Here's the context diff: >>> >>> $ hg diff >>> diff -r 87266ac324d7 src/hotspot/share/gc/z/zRootsIterator.cpp >>> --- a/src/hotspot/share/gc/z/zRootsIterator.cpp Wed Dec 18 23:46:55 >>> 2019 +0100 >>> +++ b/src/hotspot/share/gc/z/zRootsIterator.cpp Wed Dec 18 18:11:08 >>> 2019 -0500 >>> @@ -44,7 +44,8 @@ >>> ??#include "memory/universe.hpp" >>> ??#include "prims/jvmtiExport.hpp" >>> ??#include "prims/resolvedMethodTable.hpp" >>> -#include "runtime/atomic.hpp"#include "runtime/safepoint.hpp" >>> +#include "runtime/atomic.hpp" >>> +#include "runtime/safepoint.hpp" >>> ??#include "runtime/synchronizer.hpp" >>> ??#include "runtime/thread.hpp" >>> ??#include "runtime/vmThread.hpp" >>> >>> >>> Thanks, in advance, for any comments, questions or suggestions. >>> >>> Dan >>> > From kim.barrett at oracle.com Wed Dec 18 23:21:26 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 18 Dec 2019 18:21:26 -0500 Subject: Urgent RFR(T): 8236226: fix merge error in src/hotspot/share/gc/z/zRootsIterator.cpp In-Reply-To: <8845b15e-3a86-83a5-2b36-c0934ef17cf1@oracle.com> References: <4d987a81-2048-29a2-7fd3-7e548c582a32@oracle.com> <8845b15e-3a86-83a5-2b36-c0934ef17cf1@oracle.com> Message-ID: > On Dec 18, 2019, at 6:19 PM, Daniel D. Daugherty wrote: > > Thanks! Should I push this fix in hopes that it is the only merge > error or should I do a Linux test build? I think go ahead and push. From david.holmes at oracle.com Wed Dec 18 23:21:51 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 19 Dec 2019 09:21:51 +1000 Subject: Urgent RFR(T): 8236226: fix merge error in src/hotspot/share/gc/z/zRootsIterator.cpp In-Reply-To: References: <4d987a81-2048-29a2-7fd3-7e548c582a32@oracle.com> <8845b15e-3a86-83a5-2b36-c0934ef17cf1@oracle.com> Message-ID: Too late :( On 19/12/2019 9:20 am, David Holmes wrote: > Push it now and try to avoid CI failure. :) > > David > > On 19/12/2019 9:19 am, Daniel D. Daugherty wrote: >> Thanks! Should I push this fix in hopes that it is the only merge >> error or should I do a Linux test build? >> >> Dan >> >> >> On 12/18/19 6:19 PM, David Holmes wrote: >>> Ship it! >>> >>> Thanks, >>> David >>> >>> On 19/12/2019 9:14 am, Daniel D. Daugherty wrote: >>>> Greetings, >>>> >>>> A merge error crept into the latest sync from JDK14 -> JDK15. I >>>> consider >>>> this a trivial fix. >>>> >>>> Here's the context diff: >>>> >>>> $ hg diff >>>> diff -r 87266ac324d7 src/hotspot/share/gc/z/zRootsIterator.cpp >>>> --- a/src/hotspot/share/gc/z/zRootsIterator.cpp Wed Dec 18 23:46:55 >>>> 2019 +0100 >>>> +++ b/src/hotspot/share/gc/z/zRootsIterator.cpp Wed Dec 18 18:11:08 >>>> 2019 -0500 >>>> @@ -44,7 +44,8 @@ >>>> ??#include "memory/universe.hpp" >>>> ??#include "prims/jvmtiExport.hpp" >>>> ??#include "prims/resolvedMethodTable.hpp" >>>> -#include "runtime/atomic.hpp"#include "runtime/safepoint.hpp" >>>> +#include "runtime/atomic.hpp" >>>> +#include "runtime/safepoint.hpp" >>>> ??#include "runtime/synchronizer.hpp" >>>> ??#include "runtime/thread.hpp" >>>> ??#include "runtime/vmThread.hpp" >>>> >>>> >>>> Thanks, in advance, for any comments, questions or suggestions. >>>> >>>> Dan >>>> >> From daniel.daugherty at oracle.com Wed Dec 18 23:23:57 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 18 Dec 2019 18:23:57 -0500 Subject: Urgent RFR(T): 8236226: fix merge error in src/hotspot/share/gc/z/zRootsIterator.cpp In-Reply-To: References: <4d987a81-2048-29a2-7fd3-7e548c582a32@oracle.com> <8845b15e-3a86-83a5-2b36-c0934ef17cf1@oracle.com> Message-ID: The CI build failure was how I saw the problem. This fix is pushed. I have a Linux-X64 build going now. Dan On 12/18/19 6:21 PM, David Holmes wrote: > Too late :( > > On 19/12/2019 9:20 am, David Holmes wrote: >> Push it now and try to avoid CI failure. :) >> >> David >> >> On 19/12/2019 9:19 am, Daniel D. Daugherty wrote: >>> Thanks! Should I push this fix in hopes that it is the only merge >>> error or should I do a Linux test build? >>> >>> Dan >>> >>> >>> On 12/18/19 6:19 PM, David Holmes wrote: >>>> Ship it! >>>> >>>> Thanks, >>>> David >>>> >>>> On 19/12/2019 9:14 am, Daniel D. Daugherty wrote: >>>>> Greetings, >>>>> >>>>> A merge error crept into the latest sync from JDK14 -> JDK15. I >>>>> consider >>>>> this a trivial fix. >>>>> >>>>> Here's the context diff: >>>>> >>>>> $ hg diff >>>>> diff -r 87266ac324d7 src/hotspot/share/gc/z/zRootsIterator.cpp >>>>> --- a/src/hotspot/share/gc/z/zRootsIterator.cpp Wed Dec 18 >>>>> 23:46:55 2019 +0100 >>>>> +++ b/src/hotspot/share/gc/z/zRootsIterator.cpp Wed Dec 18 >>>>> 18:11:08 2019 -0500 >>>>> @@ -44,7 +44,8 @@ >>>>> ??#include "memory/universe.hpp" >>>>> ??#include "prims/jvmtiExport.hpp" >>>>> ??#include "prims/resolvedMethodTable.hpp" >>>>> -#include "runtime/atomic.hpp"#include "runtime/safepoint.hpp" >>>>> +#include "runtime/atomic.hpp" >>>>> +#include "runtime/safepoint.hpp" >>>>> ??#include "runtime/synchronizer.hpp" >>>>> ??#include "runtime/thread.hpp" >>>>> ??#include "runtime/vmThread.hpp" >>>>> >>>>> >>>>> Thanks, in advance, for any comments, questions or suggestions. >>>>> >>>>> Dan >>>>> >>> From daniel.daugherty at oracle.com Wed Dec 18 23:36:21 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 18 Dec 2019 18:36:21 -0500 Subject: Urgent RFR(T): 8236226: fix merge error in src/hotspot/share/gc/z/zRootsIterator.cpp In-Reply-To: References: <4d987a81-2048-29a2-7fd3-7e548c582a32@oracle.com> <8845b15e-3a86-83a5-2b36-c0934ef17cf1@oracle.com> Message-ID: <8cc45afb-f56e-d211-d954-5ad8049533fe@oracle.com> My local Linux-X64 'release' bits build passed so odds are good that JDK-8236226 is all we needed. Dan On 12/18/19 6:23 PM, Daniel D. Daugherty wrote: > The CI build failure was how I saw the problem. > > This fix is pushed. I have a Linux-X64 build going now. > > Dan > > > On 12/18/19 6:21 PM, David Holmes wrote: >> Too late :( >> >> On 19/12/2019 9:20 am, David Holmes wrote: >>> Push it now and try to avoid CI failure. :) >>> >>> David >>> >>> On 19/12/2019 9:19 am, Daniel D. Daugherty wrote: >>>> Thanks! Should I push this fix in hopes that it is the only merge >>>> error or should I do a Linux test build? >>>> >>>> Dan >>>> >>>> >>>> On 12/18/19 6:19 PM, David Holmes wrote: >>>>> Ship it! >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> On 19/12/2019 9:14 am, Daniel D. Daugherty wrote: >>>>>> Greetings, >>>>>> >>>>>> A merge error crept into the latest sync from JDK14 -> JDK15. I >>>>>> consider >>>>>> this a trivial fix. >>>>>> >>>>>> Here's the context diff: >>>>>> >>>>>> $ hg diff >>>>>> diff -r 87266ac324d7 src/hotspot/share/gc/z/zRootsIterator.cpp >>>>>> --- a/src/hotspot/share/gc/z/zRootsIterator.cpp Wed Dec 18 >>>>>> 23:46:55 2019 +0100 >>>>>> +++ b/src/hotspot/share/gc/z/zRootsIterator.cpp Wed Dec 18 >>>>>> 18:11:08 2019 -0500 >>>>>> @@ -44,7 +44,8 @@ >>>>>> ??#include "memory/universe.hpp" >>>>>> ??#include "prims/jvmtiExport.hpp" >>>>>> ??#include "prims/resolvedMethodTable.hpp" >>>>>> -#include "runtime/atomic.hpp"#include "runtime/safepoint.hpp" >>>>>> +#include "runtime/atomic.hpp" >>>>>> +#include "runtime/safepoint.hpp" >>>>>> ??#include "runtime/synchronizer.hpp" >>>>>> ??#include "runtime/thread.hpp" >>>>>> ??#include "runtime/vmThread.hpp" >>>>>> >>>>>> >>>>>> Thanks, in advance, for any comments, questions or suggestions. >>>>>> >>>>>> Dan >>>>>> >>>> > > From aph at redhat.com Thu Dec 19 10:31:44 2019 From: aph at redhat.com (Andrew Haley) Date: Thu, 19 Dec 2019 11:31:44 +0100 Subject: Bounds Check Elimination with Fast-Range In-Reply-To: <28CF6CF4-3E20-489C-9659-C50E4222EC22@oracle.com> References: <19544187-6670-4A65-AF47-297F38E8D555@gmail.com> <87r22549fg.fsf@oldenburg2.str.redhat.com> <1AB809E9-27B3-423E-AB1A-448D6B4689F6@oracle.com> <877e3x0wji.fsf@oldenburg2.str.redhat.com> <28CF6CF4-3E20-489C-9659-C50E4222EC22@oracle.com> Message-ID: <2ca24471-a5e8-580a-66bc-37b0eece8898@redhat.com> On 12/3/19 6:06 AM, John Rose wrote: > But what I keep wishing for a good one- or two-cycle instruction > that will mix all input bits into all the output bits, so that any > change in one input bit is likely to cause cascading changes in > many output bits, perhaps even 50% on average. A pair of AES > steps is a good example of this. I've searched for something like this too. However, I experimented with two rounds of AES and I didn't get very good results. From what I remember, it took at least three or four rounds to get decent mixing, let alone full avalanche. Also, the latency of AES instructions tends to be a few cycles and the latency of moving data from integer to vector registers is as many as five cycles. So I gave up. This was a while ago, so probably badly remembered... -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From augustnagro at gmail.com Thu Dec 19 14:38:49 2019 From: augustnagro at gmail.com (August Nagro) Date: Thu, 19 Dec 2019 08:38:49 -0600 Subject: Bounds Check Elimination with Fast-Range In-Reply-To: <2ca24471-a5e8-580a-66bc-37b0eece8898@redhat.com> References: <19544187-6670-4A65-AF47-297F38E8D555@gmail.com> <87r22549fg.fsf@oldenburg2.str.redhat.com> <1AB809E9-27B3-423E-AB1A-448D6B4689F6@oracle.com> <877e3x0wji.fsf@oldenburg2.str.redhat.com> <28CF6CF4-3E20-489C-9659-C50E4222EC22@oracle.com> <2ca24471-a5e8-580a-66bc-37b0eece8898@redhat.com> Message-ID: <2F5736F4-3B57-4E97-9A0D-9B32B2EDD82A@gmail.com> One thing I?ve come to realize is that the bit-mixing step in fast-range does not require a high quality hash function. It just needs to distribute the bits so that the probability of falling into the buckets [0, 2^32), [2^32, 2 * 2^32), [2*2^32, 3*2^32), ?, is even. This is why I?m so enthusiastic about fibonacci hashing for this operation, since it costs only a single multiply. To prove it works, take a look at this sample. The integer gFactor is equal to 2^32 / phi, where phi is the Golden Ratio. Note that hash * gFactor is 32-bit multiplication, and we AND with 0xffffffffL for the unsigned multiply with int size. Set::add returns false if the element is present. import java.util.HashSet; import java.util.Set; class Scratch { public static void main(String[] args) { int gFactor = -1640531527; int size = 1_000; int range = 4_000; int collisions = 0; Set set = new HashSet<>(); for (int hash = 0; hash < range; ++hash) { int reduced = (int) (((hash * gFactor) & 0xffffffffL) * size >>> 32); if (!set.add(reduced)) collisions++; } System.out.println("Optimal collisions: " + (range - size)); System.out.println("Actual collisions: " + collisions); } } > On Dec 19, 2019, at 4:31 AM, Andrew Haley wrote: > > On 12/3/19 6:06 AM, John Rose wrote: >> But what I keep wishing for a good one- or two-cycle instruction >> that will mix all input bits into all the output bits, so that any >> change in one input bit is likely to cause cascading changes in >> many output bits, perhaps even 50% on average. A pair of AES >> steps is a good example of this. > > I've searched for something like this too. However, I > experimented with two rounds of AES and I didn't get very good > results. From what I remember, it took at least three or four > rounds to get decent mixing, let alone full avalanche. > > Also, the latency of AES instructions tends to be a few cycles > and the latency of moving data from integer to vector registers > is as many as five cycles. So I gave up. > > This was a while ago, so probably badly remembered... > > -- > Andrew Haley (he/him) > Java Platform Lead Engineer > Red Hat UK Ltd. > https://keybase.io/andrewhaley > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 > From vladimir.x.ivanov at oracle.com Thu Dec 19 18:15:26 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 19 Dec 2019 21:15:26 +0300 Subject: [15] RFR (L): 7175279: Don't use x87 FPU on x86-64 In-Reply-To: References: <02b93959-9ec3-45c1-9ba7-ebd5682548e4@oracle.com> Message-ID: Thanks for the feedback, Vladimir. > c1_CodeStubs.hpp - I think it should be stronger than assert to catch it > in product too (we can do check in product because it is not performance > critical code). Do you prefer to see guarantee/fatal instead? Frankly speaking, even the assert doesn't look warranted enough. I put it there mainly to validate my own changes. I could have introduced ConversionStubs for x86-64 as well, but decided to simplify the implementation. Considering x86-32 is the only consumer, I'd prefer to have ConversionStub x86_32-specific instead and completely hide it from other platforms, but it requires putting more #ifdefs in shared code which I don't like either. So, if you see a value in having a runtime check in product binaries there, I'll put it there. > c1_LinearScan.cpp - I think IA64 is used for Itanium. For 64-bit x86 we > use AMD64: > > https://hg.openjdk.java.net/jdk/jdk/file/cfaa2457a60a/src/hotspot/share/utilities/macros.hpp#l456 Yes, you are right. Good catch! :-) Best regards, Vladimir Ivanov > On 12/17/19 4:50 AM, Vladimir Ivanov wrote: >> http://cr.openjdk.java.net/~vlivanov/7175279/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-7175279 >> >> There was a major rewrite of math intrinsics which in 9 time frame >> which almost completely eliminated x87 code in x86-64 code base. >> >> Proposed patch removes the rest and makes x86-64 code x87-free. >> >> The main motivation for the patch is to completely eliminate >> non-strictfp behaving code in order to prepare the JVM for JEP 306 [1] >> and related enhancements [2]. >> >> Most of the changes are in C1, but there is one case in template >> interpreter (java_lang_math_abs) which now uses >> StubRoutines::x86::double_sign_mask(). It forces its initialization to >> be moved to StubRoutines::initialize1(). >> >> x87 instructions are made available only on x86-32. >> >> C1 changes involve removing FPU support on x86-64 and effectively make >> x86-specific support in linear scan allocator [2] x86-32-only. >> >> Testing: tier1-6, build on x86-23, linux-aarch64, linux-arm32. >> >> Best regards, >> Vladimir Ivanov >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8175916 >> >> [2] https://bugs.openjdk.java.net/browse/JDK-8136414 >> >> [3] >> http://hg.openjdk.java.net/jdk/jdk/file/ff7cd49f2aef/src/hotspot/cpu/x86/c1_LinearScan_x86.cpp >> From vladimir.kozlov at oracle.com Thu Dec 19 18:34:25 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 19 Dec 2019 10:34:25 -0800 Subject: [15] RFR (L): 7175279: Don't use x87 FPU on x86-64 In-Reply-To: References: <02b93959-9ec3-45c1-9ba7-ebd5682548e4@oracle.com> Message-ID: <536139cd-53a1-f697-9c83-296f2d311536@oracle.com> On 12/19/19 10:15 AM, Vladimir Ivanov wrote: > Thanks for the feedback, Vladimir. > >> c1_CodeStubs.hpp - I think it should be stronger than assert to catch it in product too (we can do check in product >> because it is not performance critical code). > > Do you prefer to see guarantee/fatal instead? > > Frankly speaking, even the assert doesn't look warranted enough. > I put it there mainly to validate my own changes. I could have introduced ConversionStubs for x86-64 as well, but > decided to simplify the implementation. > > Considering x86-32 is the only consumer, I'd prefer to have ConversionStub x86_32-specific instead and completely hide > it from other platforms, but it requires putting more #ifdefs in shared code which I don't like either. > > So, if you see a value in having a runtime check in product binaries there, I'll put it there. Make ConversionStub x86_32-specific only if possible. From what I see it is only LIR_OpConvert in c1_LIR.hpp we have to deal with. I actually can't see how it could be only 32-specific. Hmm? Vladimir K > >> c1_LinearScan.cpp - I think IA64 is used for Itanium. For 64-bit x86 we use AMD64: >> >> https://hg.openjdk.java.net/jdk/jdk/file/cfaa2457a60a/src/hotspot/share/utilities/macros.hpp#l456 > > Yes, you are right. Good catch! :-) > > Best regards, > Vladimir Ivanov > >> On 12/17/19 4:50 AM, Vladimir Ivanov wrote: >>> http://cr.openjdk.java.net/~vlivanov/7175279/webrev.00/ >>> https://bugs.openjdk.java.net/browse/JDK-7175279 >>> >>> There was a major rewrite of math intrinsics which in 9 time frame which almost completely eliminated x87 code in >>> x86-64 code base. >>> >>> Proposed patch removes the rest and makes x86-64 code x87-free. >>> >>> The main motivation for the patch is to completely eliminate non-strictfp behaving code in order to prepare the JVM >>> for JEP 306 [1] and related enhancements [2]. >>> >>> Most of the changes are in C1, but there is one case in template interpreter (java_lang_math_abs) which now uses >>> StubRoutines::x86::double_sign_mask(). It forces its initialization to be moved to StubRoutines::initialize1(). >>> >>> x87 instructions are made available only on x86-32. >>> >>> C1 changes involve removing FPU support on x86-64 and effectively make x86-specific support in linear scan allocator >>> [2] x86-32-only. >>> >>> Testing: tier1-6, build on x86-23, linux-aarch64, linux-arm32. >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8175916 >>> >>> [2] https://bugs.openjdk.java.net/browse/JDK-8136414 >>> >>> [3] http://hg.openjdk.java.net/jdk/jdk/file/ff7cd49f2aef/src/hotspot/cpu/x86/c1_LinearScan_x86.cpp From vladimir.x.ivanov at oracle.com Thu Dec 19 18:40:46 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 19 Dec 2019 21:40:46 +0300 Subject: [15] RFR (L): 7175279: Don't use x87 FPU on x86-64 In-Reply-To: <536139cd-53a1-f697-9c83-296f2d311536@oracle.com> References: <02b93959-9ec3-45c1-9ba7-ebd5682548e4@oracle.com> <536139cd-53a1-f697-9c83-296f2d311536@oracle.com> Message-ID: <14878e70-359e-0716-1fb2-cc54bc77f093@oracle.com> > Make ConversionStub x86_32-specific only if possible. From what I see it > is only LIR_OpConvert in c1_LIR.hpp we have to deal with. I actually > can't see how it could be only 32-specific. Hmm? I experimented with it, but it requires #ifdefs in c1_LIR.cpp/hpp which I don't like. So, I don't consider it as an option right now. Best regards, Vladimir Ivanov >>> c1_LinearScan.cpp - I think IA64 is used for Itanium. For 64-bit x86 >>> we use AMD64: >>> >>> https://hg.openjdk.java.net/jdk/jdk/file/cfaa2457a60a/src/hotspot/share/utilities/macros.hpp#l456 >> >> >> Yes, you are right. Good catch! :-) >> >> Best regards, >> Vladimir Ivanov >> >>> On 12/17/19 4:50 AM, Vladimir Ivanov wrote: >>>> http://cr.openjdk.java.net/~vlivanov/7175279/webrev.00/ >>>> https://bugs.openjdk.java.net/browse/JDK-7175279 >>>> >>>> There was a major rewrite of math intrinsics which in 9 time frame >>>> which almost completely eliminated x87 code in x86-64 code base. >>>> >>>> Proposed patch removes the rest and makes x86-64 code x87-free. >>>> >>>> The main motivation for the patch is to completely eliminate >>>> non-strictfp behaving code in order to prepare the JVM for JEP 306 >>>> [1] and related enhancements [2]. >>>> >>>> Most of the changes are in C1, but there is one case in template >>>> interpreter (java_lang_math_abs) which now uses >>>> StubRoutines::x86::double_sign_mask(). It forces its initialization >>>> to be moved to StubRoutines::initialize1(). >>>> >>>> x87 instructions are made available only on x86-32. >>>> >>>> C1 changes involve removing FPU support on x86-64 and effectively >>>> make x86-specific support in linear scan allocator [2] x86-32-only. >>>> >>>> Testing: tier1-6, build on x86-23, linux-aarch64, linux-arm32. >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8175916 >>>> >>>> [2] https://bugs.openjdk.java.net/browse/JDK-8136414 >>>> >>>> [3] >>>> http://hg.openjdk.java.net/jdk/jdk/file/ff7cd49f2aef/src/hotspot/cpu/x86/c1_LinearScan_x86.cpp >>>> From vladimir.kozlov at oracle.com Thu Dec 19 18:52:53 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 19 Dec 2019 10:52:53 -0800 Subject: [15] RFR (L): 7175279: Don't use x87 FPU on x86-64 In-Reply-To: <14878e70-359e-0716-1fb2-cc54bc77f093@oracle.com> References: <02b93959-9ec3-45c1-9ba7-ebd5682548e4@oracle.com> <536139cd-53a1-f697-9c83-296f2d311536@oracle.com> <14878e70-359e-0716-1fb2-cc54bc77f093@oracle.com> Message-ID: <660d8e71-6fcd-3e71-fac3-7a4caa787872@oracle.com> On 12/19/19 10:40 AM, Vladimir Ivanov wrote: > >> Make ConversionStub x86_32-specific only if possible. From what I see it is only LIR_OpConvert in c1_LIR.hpp we have >> to deal with. I actually can't see how it could be only 32-specific. Hmm? > > I experimented with it, but it requires #ifdefs in c1_LIR.cpp/hpp which I don't like. So, I don't consider it as an > option right now. Okay, NOT_IA32( ShouldNotReachHere() ) with comment in c1_CodeStubs.hpp should be enough for now. Vladimir K > > Best regards, > Vladimir Ivanov > >>>> c1_LinearScan.cpp - I think IA64 is used for Itanium. For 64-bit x86 we use AMD64: >>>> >>>> https://hg.openjdk.java.net/jdk/jdk/file/cfaa2457a60a/src/hotspot/share/utilities/macros.hpp#l456 >>> >>> >>> Yes, you are right. Good catch! :-) >>> >>> Best regards, >>> Vladimir Ivanov >>> >>>> On 12/17/19 4:50 AM, Vladimir Ivanov wrote: >>>>> http://cr.openjdk.java.net/~vlivanov/7175279/webrev.00/ >>>>> https://bugs.openjdk.java.net/browse/JDK-7175279 >>>>> >>>>> There was a major rewrite of math intrinsics which in 9 time frame which almost completely eliminated x87 code in >>>>> x86-64 code base. >>>>> >>>>> Proposed patch removes the rest and makes x86-64 code x87-free. >>>>> >>>>> The main motivation for the patch is to completely eliminate non-strictfp behaving code in order to prepare the JVM >>>>> for JEP 306 [1] and related enhancements [2]. >>>>> >>>>> Most of the changes are in C1, but there is one case in template interpreter (java_lang_math_abs) which now uses >>>>> StubRoutines::x86::double_sign_mask(). It forces its initialization to be moved to StubRoutines::initialize1(). >>>>> >>>>> x87 instructions are made available only on x86-32. >>>>> >>>>> C1 changes involve removing FPU support on x86-64 and effectively make x86-specific support in linear scan >>>>> allocator [2] x86-32-only. >>>>> >>>>> Testing: tier1-6, build on x86-23, linux-aarch64, linux-arm32. >>>>> >>>>> Best regards, >>>>> Vladimir Ivanov >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8175916 >>>>> >>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8136414 >>>>> >>>>> [3] http://hg.openjdk.java.net/jdk/jdk/file/ff7cd49f2aef/src/hotspot/cpu/x86/c1_LinearScan_x86.cpp From vladimir.x.ivanov at oracle.com Thu Dec 19 21:58:19 2019 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 20 Dec 2019 00:58:19 +0300 Subject: [15] RFR (L): 7175279: Don't use x87 FPU on x86-64 In-Reply-To: <660d8e71-6fcd-3e71-fac3-7a4caa787872@oracle.com> References: <02b93959-9ec3-45c1-9ba7-ebd5682548e4@oracle.com> <536139cd-53a1-f697-9c83-296f2d311536@oracle.com> <14878e70-359e-0716-1fb2-cc54bc77f093@oracle.com> <660d8e71-6fcd-3e71-fac3-7a4caa787872@oracle.com> Message-ID: <0b0897e0-1dbc-306d-b2cb-31de13fb8b34@oracle.com> >>> Make ConversionStub x86_32-specific only if possible. From what I see >>> it is only LIR_OpConvert in c1_LIR.hpp we have to deal with. I >>> actually can't see how it could be only 32-specific. Hmm? >> >> I experimented with it, but it requires #ifdefs in c1_LIR.cpp/hpp >> which I don't like. So, I don't consider it as an? > option right now. > > Okay, NOT_IA32( ShouldNotReachHere() ) with comment in c1_CodeStubs.hpp > should be enough for now. Incremental diff: http://cr.openjdk.java.net/~vlivanov/7175279/webrev.01-00/ Best regards, Vladimir Ivanov >>>>> c1_LinearScan.cpp - I think IA64 is used for Itanium. For 64-bit >>>>> x86 we use AMD64: >>>>> >>>>> https://hg.openjdk.java.net/jdk/jdk/file/cfaa2457a60a/src/hotspot/share/utilities/macros.hpp#l456 >>>> >>>> >>>> >>>> Yes, you are right. Good catch! :-) >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>>> On 12/17/19 4:50 AM, Vladimir Ivanov wrote: >>>>>> http://cr.openjdk.java.net/~vlivanov/7175279/webrev.00/ >>>>>> https://bugs.openjdk.java.net/browse/JDK-7175279 >>>>>> >>>>>> There was a major rewrite of math intrinsics which in 9 time frame >>>>>> which almost completely eliminated x87 code in x86-64 code base. >>>>>> >>>>>> Proposed patch removes the rest and makes x86-64 code x87-free. >>>>>> >>>>>> The main motivation for the patch is to completely eliminate >>>>>> non-strictfp behaving code in order to prepare the JVM for JEP 306 >>>>>> [1] and related enhancements [2]. >>>>>> >>>>>> Most of the changes are in C1, but there is one case in template >>>>>> interpreter (java_lang_math_abs) which now uses >>>>>> StubRoutines::x86::double_sign_mask(). It forces its >>>>>> initialization to be moved to StubRoutines::initialize1(). >>>>>> >>>>>> x87 instructions are made available only on x86-32. >>>>>> >>>>>> C1 changes involve removing FPU support on x86-64 and effectively >>>>>> make x86-specific support in linear scan allocator [2] x86-32-only. >>>>>> >>>>>> Testing: tier1-6, build on x86-23, linux-aarch64, linux-arm32. >>>>>> >>>>>> Best regards, >>>>>> Vladimir Ivanov >>>>>> >>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8175916 >>>>>> >>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8136414 >>>>>> >>>>>> [3] >>>>>> http://hg.openjdk.java.net/jdk/jdk/file/ff7cd49f2aef/src/hotspot/cpu/x86/c1_LinearScan_x86.cpp >>>>>> From vladimir.kozlov at oracle.com Thu Dec 19 22:00:02 2019 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 19 Dec 2019 14:00:02 -0800 Subject: [15] RFR (L): 7175279: Don't use x87 FPU on x86-64 In-Reply-To: <0b0897e0-1dbc-306d-b2cb-31de13fb8b34@oracle.com> References: <0b0897e0-1dbc-306d-b2cb-31de13fb8b34@oracle.com> Message-ID: <7063EB29-D415-4A48-BA4F-B16C5A1F52F8@oracle.com> Good Thanks Vladimir > On Dec 19, 2019, at 1:58 PM, Vladimir Ivanov wrote: > > ? >>>> Make ConversionStub x86_32-specific only if possible. From what I see it is only LIR_OpConvert in c1_LIR.hpp we have to deal with. I actually can't see how it could be only 32-specific. Hmm? >>> >>> I experimented with it, but it requires #ifdefs in c1_LIR.cpp/hpp which I don't like. So, I don't consider it as an > option right now. >> Okay, NOT_IA32( ShouldNotReachHere() ) with comment in c1_CodeStubs.hpp should be enough for now. > > Incremental diff: > http://cr.openjdk.java.net/~vlivanov/7175279/webrev.01-00/ > > Best regards, > Vladimir Ivanov > >>>>>> c1_LinearScan.cpp - I think IA64 is used for Itanium. For 64-bit x86 we use AMD64: >>>>>> >>>>>> https://hg.openjdk.java.net/jdk/jdk/file/cfaa2457a60a/src/hotspot/share/utilities/macros.hpp#l456 >>>>> >>>>> >>>>> >>>>> Yes, you are right. Good catch! :-) >>>>> >>>>> Best regards, >>>>> Vladimir Ivanov >>>>> >>>>>> On 12/17/19 4:50 AM, Vladimir Ivanov wrote: >>>>>>> http://cr.openjdk.java.net/~vlivanov/7175279/webrev.00/ >>>>>>> https://bugs.openjdk.java.net/browse/JDK-7175279 >>>>>>> >>>>>>> There was a major rewrite of math intrinsics which in 9 time frame which almost completely eliminated x87 code in x86-64 code base. >>>>>>> >>>>>>> Proposed patch removes the rest and makes x86-64 code x87-free. >>>>>>> >>>>>>> The main motivation for the patch is to completely eliminate non-strictfp behaving code in order to prepare the JVM for JEP 306 [1] and related enhancements [2]. >>>>>>> >>>>>>> Most of the changes are in C1, but there is one case in template interpreter (java_lang_math_abs) which now uses StubRoutines::x86::double_sign_mask(). It forces its initialization to be moved to StubRoutines::initialize1(). >>>>>>> >>>>>>> x87 instructions are made available only on x86-32. >>>>>>> >>>>>>> C1 changes involve removing FPU support on x86-64 and effectively make x86-specific support in linear scan allocator [2] x86-32-only. >>>>>>> >>>>>>> Testing: tier1-6, build on x86-23, linux-aarch64, linux-arm32. >>>>>>> >>>>>>> Best regards, >>>>>>> Vladimir Ivanov >>>>>>> >>>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8175916 >>>>>>> >>>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8136414 >>>>>>> >>>>>>> [3] http://hg.openjdk.java.net/jdk/jdk/file/ff7cd49f2aef/src/hotspot/cpu/x86/c1_LinearScan_x86.cpp From david.holmes at oracle.com Thu Dec 19 22:20:53 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 20 Dec 2019 08:20:53 +1000 Subject: RFR: 8235966: Process obsolete flags less aggressively In-Reply-To: References: <674e4f0d-563f-3bb4-dee0-74aaacb9b270@oracle.com> Message-ID: Thanks Dan. FTR I've updated the bug report with an extension to this proposal, which is to add back the flag table validation checks to use via a gtest that we only enable after a certain build in the release cycle (it always passes before then). That way we avoid the problems I've outlined with the initial version bump but also have a safety net in place to ensure we don't forget to actually obsolete/expire flags. Cheers, David On 19/12/2019 3:37 am, Daniel D. Daugherty wrote: > Hi David, > > On 12/17/19 5:03 PM, David Holmes wrote: >> Hi Dan, >> >> Thanks for taking a look. Updated webrev: >> >> http://cr.openjdk.java.net/~dholmes/8235966/webrev.v2/ > > src/hotspot/share/runtime/arguments.cpp > ??? I like the updates to header comment for verify_special_jvm_flags(). > > Thumbs up. > > >> >> Discussion below. > > Replies below. > > >> >> On 18/12/2019 1:47 am, Daniel D. Daugherty wrote: >>> On 12/16/19 12:16 AM, David Holmes wrote: >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8235966 >>>> webrev: http://cr.openjdk.java.net/~dholmes/8235966/webrev/ >>> >>> src/hotspot/share/runtime/arguments.cpp >>> ???? L745: ????? // if flag has become obsolete it should not have a >>> "globals" flag defined anymore. >>> ???? L746: ????? if (!version_less_than(JDK_Version::current(), >>> flag.obsolete_in)) { >>> ???? L747: ??????? if (JVMFlag::find_declared_flag(flag.name) != NULL) { >>> ???? L748: ????????? // Temporarily disable the warning: 8196739 >>> ???? L749: ????????? // warning("Global variable for obsolete special >>> flag entry \"%s\" should be removed", flag.name); >>> ???? L750: ??????? } >>> ???? L751: ????? } >>> ???????? It seems like we've been down a similar road before: >>> >>> ???????? JDK-8196739 Disable obsolete/expired VM flag transitional >>> warnings >>> ???????? https://bugs.openjdk.java.net/browse/JDK-8196739 >>> >>> ???????? This one may ring a bell... Fixed by dholmes in jdk11-b01... >>> :-) >>> >>> ???????? And this followup sub-task to re-enable that warning: >>> >>> ???????? JDK-8196741 Re-enable obsolete/expired VM flag transitional >>> warnings >>> ???????? https://bugs.openjdk.java.net/browse/JDK-8196741 >>> >>> ???????? was closed as "Won't fix" on 2019.08.02. >>> >>> ???????? So the obvious questions: >>> >>> ???????? - Why is the new warning less problematic to tests that don't >>> ?????????? tolerate unexpected output? >> >> Two different situations. The commented out warning happens >> unconditionally when you run the VM and it finds any flag marked >> obsolete that hasn't been removed. Hence every single test will >> encounter this warning. > > Ouch on such verbosity. > > >> The situation I am modifying is when a test uses a flag that is marked >> for obsoletion. In the majority of cases the flag is already >> deprecated and so already issuing a deprecation warning that the test >> has to handle. Without my change there would still be an obsoletion >> warning, so this test is in for a warning no matter what. > > Good that your change only comes into play when the flag is used. > > >> Also note that for hotspot at least we have strived to make tests >> tolerate unexpected output. The reason JDK-8196741 was closed as >> "won't fix" was because other areas wouldn't commit to doing that. > > Yup. Got it. > > >> >>> ???????? - If you move forward with this fix, then I think think code >>> ?????????? block needs to be removed or modified or am I missing >>> something? >> >> I've rewritten the comment at the head of verify_special_jvm_flags to >> explain why we can't issue a warning, and have deleted the block. > > Thanks for deleting the stale code. > > >> >>> ???????? There's a similar commented out check on L757-L765, but that >>> one >>> ???????? is for an expired flag... You might want to adjust/delete it >>> also? >> >> Deleted. > > Thanks. > > >> >>> ???? L753: ??????? warning("Special flag entry \"%s\" must be >>> explicitly obsoleted before expired.", flag.name); >>> ???? L754: ??????? success = false; >>> ???????? nit - s/before expired/before being expired/ >>> ???????? Update: I now see that "style" is in several places in this >>> ???????????? function. I'm not sure what to think here... it grates, >>> ?? ? ? ? ? ? but I can live with it. >>> >>> ???????? nit - L75[34] indented too much by two spaces. >> >> Fixed. >> >>> ???? L962: ????????? return real_name; >>> ???????? nit - indented too much by two spaces. >> >> Fixed. >> >>> >>> Trying to understand the modified logic in argument processing is >>> making my head spin... >> >> Mine too. It took a few attempts to put the logic in the right place >> and make adjustments so that it all works as expected for a correctly >> specified flag and an erroneous one. > > I keep trying to convince myself that we're improving this flag and > options code with each release... :-) > > >> >>> - You've added a call to is_obsolete_flag() in >>> ?? handle_aliases_and_deprecation() and is_obsolete_flag() >>> ?? is where the new warning is output: >>> >>> ???? warning("Temporarily processing option %s; support is scheduled >>> for removal in %s" >>> >>> ?? handle_aliases_and_deprecation() is called from six different places, >>> ?? but the call sites are different based on the argument pattern so I >>> ?? have (mostly) convinced myself that there should not be any duplicate >>> ?? warning lines. >> >> Right - handle_aliases_and_deprecation is only called for a >> syntactically correct flag based on those patterns. It normally >> filters out obsoleted/expired flags and lets them fall through to >> later error processing (in process_argument after parse_arg returns >> false). That error processing is where the normal obsoletion check is >> performed. So I had to not filter the flag in >> handle_aliases_and_deprecation in this case, but still produce the >> warning for a malformed flag. E.g. >> >> java -XX:+UseParallelOldGC -version >> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing >> option UseParallelOldGC; support is scheduled for removal in 15.0 >> java version "15-internal" 2020-09-15 >> >> java -XX:UseParallelOldGC -version >> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing >> option UseParallelOldGC; support is scheduled for removal in 15.0 >> Missing +/- setting for VM option 'UseParallelOldGC' >> Error: Could not create the Java Virtual Machine. > > Thanks for the example. That helps a lot. > > >> >>> So I now understand the new logic that allows an obsoleted option >>> to be specified with a warning as long as the option still exists. >>> I'm good with the technical change, but... >>> >>> I'm worried about tests that don't tolerate the new warning mesg, >>> i.e., why wouldn't this become an issue again: >>> >>> JDK-8196739 Disable obsolete/expired VM flag transitional warnings >> >> Explained above. > > Yup and thanks. > > Dan > > >> >> Thanks, >> David >> >>> Dan >>> >>> >>>> >>>> When a flag is marked as obsolete in the special-flags table we will >>>> ignore it and issue a warning that it is being ignored, as soon as >>>> we bump the version of the JDK. That means that any tests still >>>> using the obsolete flag may start to fail, leading to a surge of >>>> test failures at the start of a release cycle. For example for JDK >>>> 15 we have a whole bunch of JFR tests that fail because they still >>>> try to work with UseParallelOldGC. In another case >>>> runtime/cds/appcds/FieldLayoutFlags.java passes only be accident. >>>> >>>> When a flag is marked as obsolete for a release, all code involving >>>> that flag (including tests that use it) must be updated within that >>>> release and the flag itself removed. Whilst this is typically >>>> scheduled early in a release cycle it isn't reasonable to expect it >>>> to all occur within the first couple of days of the release cycle, >>>> nor do we want to have to ProblemList a bunch of tests when they >>>> start failing. >>>> >>>> What I propose is to instead allow an obsolete flag to continue to >>>> be processed as long as that code removal has not actually occurred >>>> - with an adjusted warning. The change I propose: >>>> >>>> - only treats an obsolete flag as obsolete if the flag cannot be found >>>> - added a new flag verification rule that disallows obsoletion in an >>>> undefined version, but expiration in a specific version i.e. we must >>>> always explicitly obsolete a flag before we expire it. >>>> >>>> The only downside here is that if we actually forget to file an >>>> issue for the actual obsoletion work we may not notice via testing. >>>> Of course whenever a change is made to the flags table to add an >>>> entry then the issue to do the obsoletion should be filed at the >>>> same time. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>> > From Sergey.Bylokhov at oracle.com Sun Dec 22 20:24:09 2019 From: Sergey.Bylokhov at oracle.com (Sergey Bylokhov) Date: Sun, 22 Dec 2019 23:24:09 +0300 Subject: [15] Review Request: 8235975 Update copyright year to match last edit in jdk repository for 2014/15/16/17/18 Message-ID: <1e7d0395-fc57-4d5b-9cfa-c33e0f6462d5@oracle.com> Hello. Please review the fix for JDK 15. Bug: https://bugs.openjdk.java.net/browse/JDK-8235975 Patch (2 Mb): http://cr.openjdk.java.net/~serb/8235975/webrev.02/open.patch Fix: http://cr.openjdk.java.net/~serb/8235975/webrev.02 I have updated the source code copyrights by the "update_copyright_year.sh" script for 2014/15/16/18/19 years, unfortunately, cannot run it for 2017 because of: "JDK-8187443: Forest Consolidation: Move files to unified layout" which touched all files. -- Best regards, Sergey. From david.holmes at oracle.com Sun Dec 22 22:18:02 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 23 Dec 2019 08:18:02 +1000 Subject: [15] Review Request: 8235975 Update copyright year to match last edit in jdk repository for 2014/15/16/17/18 In-Reply-To: <1e7d0395-fc57-4d5b-9cfa-c33e0f6462d5@oracle.com> References: <1e7d0395-fc57-4d5b-9cfa-c33e0f6462d5@oracle.com> Message-ID: <541ce510-9eb7-d91d-154e-861126fd24c2@oracle.com> Hi Sergey, I've looked at the hotspot files. Changes seem okay, though for the files with dual-copyrights it isn't clear whether the Oracle copyright or other copyright should be updated for any given change. But it seems to me that if anyone wants their specific copyright updated then it is up to them to make that change at the time. Thanks for doing this tedious work. David On 23/12/2019 6:24 am, Sergey Bylokhov wrote: > Hello. > Please review the fix for JDK 15. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8235975 > Patch (2 Mb): http://cr.openjdk.java.net/~serb/8235975/webrev.02/open.patch > Fix: http://cr.openjdk.java.net/~serb/8235975/webrev.02 > > I have updated the source code copyrights by the "update_copyright_year.sh" > script for 2014/15/16/18/19 years, unfortunately, cannot run it for 2017 > because of: "JDK-8187443: Forest Consolidation: Move files to unified > layout" > which touched all files. > > From weijun.wang at oracle.com Mon Dec 23 01:47:07 2019 From: weijun.wang at oracle.com (Weijun Wang) Date: Mon, 23 Dec 2019 09:47:07 +0800 Subject: [15] Review Request: 8235975 Update copyright year to match last edit in jdk repository for 2014/15/16/17/18 In-Reply-To: <1e7d0395-fc57-4d5b-9cfa-c33e0f6462d5@oracle.com> References: <1e7d0395-fc57-4d5b-9cfa-c33e0f6462d5@oracle.com> Message-ID: <73BB85F1-9B0A-46DB-BA9B-AA3FC84977C7@oracle.com> I noticed a problem of myself but do not know if update_copyright_year.sh can deal with it more correctly. For example, --- old/src/java.security.jgss/share/classes/sun/security/krb5/internal/crypto/Aes128CtsHmacSha2EType.java 2019-12-22 19:00:54.000000000 +0300 +++ new/src/java.security.jgss/share/classes/sun/security/krb5/internal/crypto/Aes128CtsHmacSha2EType.java 2019-12-22 19:00:53.000000000 +0300 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2017, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it Here, I prepared the change in 2017 and pushed it on 2018-01-22 and forgot to update the year and the file hasn't been updated since it's first version. So precisely "Copyright (c) 2018" should be the most correct. Thanks, Max > On Dec 23, 2019, at 4:24 AM, Sergey Bylokhov wrote: > > Hello. > Please review the fix for JDK 15. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8235975 > Patch (2 Mb): http://cr.openjdk.java.net/~serb/8235975/webrev.02/open.patch > Fix: http://cr.openjdk.java.net/~serb/8235975/webrev.02 > > I have updated the source code copyrights by the "update_copyright_year.sh" > script for 2014/15/16/18/19 years, unfortunately, cannot run it for 2017 > because of: "JDK-8187443: Forest Consolidation: Move files to unified layout" > which touched all files. > > > -- > Best regards, Sergey. From dean.long at oracle.com Mon Dec 23 07:22:47 2019 From: dean.long at oracle.com (Dean Long) Date: Sun, 22 Dec 2019 23:22:47 -0800 Subject: [15] Review Request: 8235975 Update copyright year to match last edit in jdk repository for 2014/15/16/17/18 In-Reply-To: <1e7d0395-fc57-4d5b-9cfa-c33e0f6462d5@oracle.com> References: <1e7d0395-fc57-4d5b-9cfa-c33e0f6462d5@oracle.com> Message-ID: The changes to the src/jdk.internal.vm.compiler tree is going to complicate our automated sync with upstream Graal.? Our sync script sets the date based on changes in the Graal repo, so a file that was last modified in 2018 (in upstream Graal) but was added to JDK in 2019 would still have 2018 as the date. dl On 12/22/19 12:24 PM, Sergey Bylokhov wrote: > Hello. > Please review the fix for JDK 15. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8235975 > Patch (2 Mb): > http://cr.openjdk.java.net/~serb/8235975/webrev.02/open.patch > Fix: http://cr.openjdk.java.net/~serb/8235975/webrev.02 > > I have updated the source code copyrights by the > "update_copyright_year.sh" > script for 2014/15/16/18/19 years, unfortunately, cannot run it for 2017 > because of: "JDK-8187443: Forest Consolidation: Move files to unified > layout" > which touched all files. > > From igor.ignatyev at oracle.com Mon Dec 23 20:42:15 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 23 Dec 2019 12:42:15 -0800 Subject: RFR(S): 8236111 : narrow allowSmartActionArgs disabling In-Reply-To: References: Message-ID: ping? > On Dec 17, 2019, at 11:30 AM, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev/8236111/webrev.00/ >> 31 lines changed: 20 ins; 11 del; 0 mod; > > Hi all, > > could you please review this small patch which enables allowSmartActionArgs in hotspot and jdk test suites and disables them in a small number of test directories? the patch also removes TEST.properties files which enabled allowSmartActionArgs as they aren't needed anymore. > > from JBS: >> currently, allowSmartActionArgs is disabled for the whole hotspot and jdk test suites and enabled just in few places. this makes it a bit harder for people to use smart action arguments in these test suites as they have to not to forget to enable them. and given in all the other test suites, smart action arguments are enabled, it can be confusing and frustrating. > > > testing: tier1-5 > JBS: https://bugs.openjdk.java.net/browse/JDK-8236111 > webrev: http://cr.openjdk.java.net/~iignatyev/8236111/webrev.00/ > > Thanks, > -- Igor From david.holmes at oracle.com Mon Dec 23 21:33:16 2019 From: david.holmes at oracle.com (David Holmes) Date: Tue, 24 Dec 2019 07:33:16 +1000 Subject: RFR(S): 8236111 : narrow allowSmartActionArgs disabling In-Reply-To: References: Message-ID: <423ea31a-ebf8-4cba-72a4-6fbb934f7789@oracle.com> Hi Igor, Hotspot changes seem fine. Can't comment on jdk tests. Thanks, David On 24/12/2019 6:42 am, Igor Ignatyev wrote: > ping? > >> On Dec 17, 2019, at 11:30 AM, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev/8236111/webrev.00/ >>> 31 lines changed: 20 ins; 11 del; 0 mod; >> >> Hi all, >> >> could you please review this small patch which enables allowSmartActionArgs in hotspot and jdk test suites and disables them in a small number of test directories? the patch also removes TEST.properties files which enabled allowSmartActionArgs as they aren't needed anymore. >> >> from JBS: >>> currently, allowSmartActionArgs is disabled for the whole hotspot and jdk test suites and enabled just in few places. this makes it a bit harder for people to use smart action arguments in these test suites as they have to not to forget to enable them. and given in all the other test suites, smart action arguments are enabled, it can be confusing and frustrating. >> >> >> testing: tier1-5 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8236111 >> webrev: http://cr.openjdk.java.net/~iignatyev/8236111/webrev.00/ >> >> Thanks, >> -- Igor > From igor.ignatyev at oracle.com Tue Dec 24 04:13:18 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 23 Dec 2019 20:13:18 -0800 Subject: RFR(S): 8236111 : narrow allowSmartActionArgs disabling In-Reply-To: <423ea31a-ebf8-4cba-72a4-6fbb934f7789@oracle.com> References: <423ea31a-ebf8-4cba-72a4-6fbb934f7789@oracle.com> Message-ID: <0BA46866-3DEA-44BF-B87C-2B59B84196C9@oracle.com> Thanks David. core-libs folks, could you please review jdk part of this patch? Thanks, -- Igor > On Dec 23, 2019, at 1:33 PM, David Holmes wrote: > > Hi Igor, > > Hotspot changes seem fine. Can't comment on jdk tests. > > Thanks, > David > > On 24/12/2019 6:42 am, Igor Ignatyev wrote: >> ping? >>> On Dec 17, 2019, at 11:30 AM, Igor Ignatyev wrote: >>> >>> http://cr.openjdk.java.net/~iignatyev/8236111/webrev.00/ >>>> 31 lines changed: 20 ins; 11 del; 0 mod; >>> >>> Hi all, >>> >>> could you please review this small patch which enables allowSmartActionArgs in hotspot and jdk test suites and disables them in a small number of test directories? the patch also removes TEST.properties files which enabled allowSmartActionArgs as they aren't needed anymore. >>> >>> from JBS: >>>> currently, allowSmartActionArgs is disabled for the whole hotspot and jdk test suites and enabled just in few places. this makes it a bit harder for people to use smart action arguments in these test suites as they have to not to forget to enable them. and given in all the other test suites, smart action arguments are enabled, it can be confusing and frustrating. >>> >>> >>> testing: tier1-5 >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8236111 >>> webrev: http://cr.openjdk.java.net/~iignatyev/8236111/webrev.00/ >>> >>> Thanks, >>> -- Igor From Sergey.Bylokhov at oracle.com Tue Dec 24 18:22:15 2019 From: Sergey.Bylokhov at oracle.com (Sergey Bylokhov) Date: Tue, 24 Dec 2019 21:22:15 +0300 Subject: [15] Review Request: 8235975 Update copyright year to match last edit in jdk repository for 2014/15/16/17/18 In-Reply-To: <1e7d0395-fc57-4d5b-9cfa-c33e0f6462d5@oracle.com> References: <1e7d0395-fc57-4d5b-9cfa-c33e0f6462d5@oracle.com> Message-ID: <3460a6f6-6178-cc45-5840-0f215eebc53f@oracle.com> Hello. Here is an updated version: Bug: https://bugs.openjdk.java.net/browse/JDK-8235975 Patch (2 Mb): http://cr.openjdk.java.net/~serb/8235975/webrev.03/open.patch Fix: http://cr.openjdk.java.net/~serb/8235975/webrev.03/ - "jdk.internal.vm.compiler" is removed from the patch. - "Aes128CtsHmacSha2EType.java" is updated to "Copyright (c) 2018" On 12/22/19 11:24 pm, Sergey Bylokhov wrote: > Hello. > Please review the fix for JDK 15. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8235975 > Patch (2 Mb): http://cr.openjdk.java.net/~serb/8235975/webrev.02/open.patch > Fix: http://cr.openjdk.java.net/~serb/8235975/webrev.02 > > I have updated the source code copyrights by the "update_copyright_year.sh" > script for 2014/15/16/18/19 years, unfortunately, cannot run it for 2017 > because of: "JDK-8187443: Forest Consolidation: Move files to unified layout" > which touched all files. > > -- Best regards, Sergey.