From calvin.cheung at oracle.com Tue May 1 00:24:22 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Mon, 30 Apr 2018 17:24:22 -0700 Subject: RFR(S): 8200466: Revisit the setting of _transitive_interfaces in InstanceKlass In-Reply-To: <3fa86f67-be5a-50c4-8d04-707ff7ab185c@oracle.com> References: <5AE2AB27.5040805@oracle.com> <9ce88e64-599e-fd6e-ab70-4ce22f253c56@oracle.com> <5AE78393.6060203@oracle.com> <3fa86f67-be5a-50c4-8d04-707ff7ab185c@oracle.com> Message-ID: <5AE7B3B6.6020903@oracle.com> Hi Coleen, I've tried the following simple change: --- a/src/hotspot/share/classfile/classFileParser.cpp +++ b/src/hotspot/share/classfile/classFileParser.cpp @@ -3592,6 +3592,7 @@ this_klass->set_methods(_methods); this_klass->set_inner_classes(_inner_classes); this_klass->set_local_interfaces(_local_interfaces); + this_klass->set_super(const_cast(_super_klass)); this_klass->set_transitive_interfaces(_transitive_interfaces); this_klass->set_annotations(_combined_annotations); but it crashes in vtableEntry::verify() during fastdebug build: if (!vtklass->is_subtype_of(method()->method_holder())) { #ifndef PRODUCT print(); #endif fatal("vtableEntry " PTR_FORMAT ": method is from subclass", p2i(this)); <<<<-- crashes here } Here's the call stack from the hs err log: V [libjvm.so+0xb32bbe] report_fatal(char const*, int, char const*, ...)+0xfe V [libjvm.so+0x1215963] klassVtable::verify(outputStream*, bool)+0x113 V [libjvm.so+0x1219753] klassVtable::initialize_vtable(bool, Thread*)+0x1293 V [libjvm.so+0x1834b6f] Universe::reinitialize_vtable_of(Klass*, Thread*)+0x2f V [libjvm.so+0x1834c2c] Universe::reinitialize_vtable_of(Klass*, Thread*)+0xec V [libjvm.so+0x1834c2c] Universe::reinitialize_vtable_of(Klass*, Thread*)+0xec V [libjvm.so+0x1837b3f] universe_post_init()+0x1af V [libjvm.so+0xe99aad] init_globals()+0xed V [libjvm.so+0x17f8b2c] Threads::create_vm(JavaVMInitArgs*, bool*)+0x2cc V [libjvm.so+0x1018af8] JNI_CreateJavaVM+0xa8 C [libjli.so+0x3e81] JavaMain+0x81 I will send you the hs err and build logs off list. thanks, Calvin On 4/30/18, 2:20 PM, coleen.phillimore at oracle.com wrote: > > > On 4/30/18 4:58 PM, Calvin Cheung wrote: >> Hi Coleen, >> >> Thanks for your review. >> >> On 4/30/18, 10:23 AM, coleen.phillimore at oracle.com wrote: >>> >>> This sharing of metadata has been prone to bugs. Can you have the >>> same bug with the _secondary_super field? >>> InstanceKlass::deallocate_contents() checks: >>> >>> secondary_supers() != transitive_interfaces() && >>> >>> and will free the field possibly from the super class if >>> transitive_interfaces() is still NULL. (If I understand this >>> correctly). >> If it passes the conditions check in the 'if' block, it will >> deallocate the _secondary_supers. >> It maybe fine because the crash was in visit_all_interfaces() where >> it doesn't dereference the _secondary_supers. > > It seems like it would deallocate _secondary_supers if an OOM occurred > after it's set to the super's transitive_interfaces and before > _transitive_interfaces is set. There could be a state that could > cause this to happen. I would have to trace through to see what sort > of crash would happen. > >>> >>> Wouldn't it be easier just to eagerly set the super class, and keep >>> the call to initialize_supers() where it is? >> Do you mean adding the following in >> ClassFileParser::apply_parsed_class_metadata() and reverting the rest >> of the changes? >> >> this_klass->set_super(const_cast(_super_klass)); >> > > Yes, can you try this? This line with a comment would be less > complicated than passing NULL to initialize_supers() for the > arrayKlass versions. > > >> BTW, I think the first arg to initialize_supers() should be casted to >> Klass* like above because the _super_klass is already an InstanceKlass*. >> >> ik->initialize_supers(const_cast(_super_klass) ... >> >> > I think we still need some cast here and above because _super_klass is > defined as const, and this function and set_super are both not > const-safe, which is a bigger change. Or you could make > ClassFileParser::_super_klass not const since const is cast away. > >>> >>> http://cr.openjdk.java.net/~ccheung/8200466/webrev.00/src/hotspot/share/classfile/classFileParser.cpp.udiff.html >>> >>> >>> **** 5884,5893 ***** >>> *--- 5889,5899 ----* >>> *Annotations::free_contents(_loader_data, _fields_annotations);* >>> *Annotations::free_contents(_loader_data, _fields_type_annotations);* >>> *}* >>> ** >>> *clear_class_metadata();* >>> *+ _transitive_interfaces = NULL;* >>> ** >>> >>> clear_class_metadata() already sets the _transitive_interfaces to >>> NULL, so this shouldn't be needed. >> The current change does not set the _trasitive_interfaces to NULL in >> clear_class_metadata(). >> >> From the udiff above: >> >> **** 5832,5842 ***** >> *--- 5838,5847 ----* >> * _cp = NULL;* >> * _fields = NULL;* >> * _methods = NULL;* >> * _inner_classes = NULL;* >> * _local_interfaces = NULL;* >> * _transitive_interfaces = NULL;* >> * _combined_annotations = NULL;* >> * _annotations = _type_annotations = NULL;* >> * _fields_annotations = _fields_type_annotations = NULL;* >> * }* > > Oh I see it now and why you changed it. Can you see if setting the > Klass::_super ahead of setting _transitive_interfaces would work better? > > thanks, > Coleen > >> >> thanks, >> Calvin >>> >>> thanks, >>> Coleen >>> >>> On 4/27/18 12:46 AM, Calvin Cheung wrote: >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8200466 >>>> >>>> webrev: http://cr.openjdk.java.net/~ccheung/8200466/webrev.00/ >>>> >>>> This bug was discovered during the investigation of JDK-8200078 >>>> . >>>> >>>> I've included the following comment in classFileParser.cpp to >>>> summarize the change: >>>> >>>> // Delay the setting of _transitive_interfaces until after >>>> initialize_supers() in >>>> // fill_instance_klass(). It is because the _transitive_interfaces >>>> may be shared with >>>> // its _super. If an OOM occurs while loading the current klass, >>>> its _super field >>>> // may not have been set. When GC tries to free the klass, the >>>> _transitive_interfaces >>>> // may be deallocated mistakenly in >>>> InstanceKlass::deallocate_interfaces(). Subsequent >>>> // dereferences to the deallocated _transitive_interfaces will >>>> result in a crash. >>>> >>>> Testing: (on Oracle platforms) >>>> hs-tier{1,2,3} >>>> closed (soon will be open) PCL (parallel class loading) tests >>>> >>>> thanks, >>>> Calvin >>> > From yasuenag at gmail.com Tue May 1 00:41:28 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Tue, 1 May 2018 09:41:28 +0900 Subject: Event descriptions are truncated in logs Message-ID: <1f4e4a23-1799-977d-b317-ca1216c22352@gmail.com> Hi all, I found event description in hs_error log and VM.info dcmd are trucated as following: ``` Event: 1.977 Thread 0x00007f1720010800 Exception (0x00000000d7a338f0) thrown at [/scratch/opt/mach5/mesos/work_dir/slaves/9190d864-6621-4810-ba08-d8d8c75ba674-S702/framewo ``` I think we can fix it as this webrev. Is it acceptable? If so, I will file it to JBS and send review request. http://cr.openjdk.java.net/~ysuenaga/event-msg-truncated/ This webrev shows event description as following: ``` Event: 3.660 Thread 0x00007fa174018800 Exception (0x00000000d778f878) thrown at [/home/ysuenaga/OpenJDK/jdk/src/hotspot/share/interpreter/linkResolver.cpp, line 755] ``` But I concern this change might not work when HotSpot is built with VS 2013 or earlier. This change uses vsnprintf() to get length of string, but _vscprintf() will be used when older Visual Studio is used to build. Thanks, Yasumasa From kim.barrett at oracle.com Tue May 1 00:42:40 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 30 Apr 2018 20:42:40 -0400 Subject: RFR: JDK-8202319: Fix compilation warnings in Solaris debug builds for DevStudio 12.6 In-Reply-To: <4df85ebd-aca1-46fe-8104-5c6f50517256@oracle.com> References: <5AE1FFD0.1090803@oracle.com> <5ED019CA-50CB-4AAB-A22A-E7BC6FE0157D@oracle.com> <43683acb-7f30-e79a-42dc-0d4693e2ac21@oracle.com> <4df85ebd-aca1-46fe-8104-5c6f50517256@oracle.com> Message-ID: > On Apr 27, 2018, at 7:25 PM, David Holmes wrote: > > We discussed this offline and Gary pointed out that, at least in the VMError case the attempt to SEGV by dereferencing zero is one of a specific number of crash inducing cases, others of which include trying to trigger SEGV at non-zero address and explicit signal raising. So changing the code to raise the signal directly is not really appropriate - and the code in VMError knows the attempt may not result in a crash. > > So I am okay with just disabling the compilation warnings for these two cases. I feel pretty uncomfortable with code that is intentionally invoking undefined behavior and expecting anything at all useful to come from that. I think in the frame::oops_do_internal case, the code used when CrashGCForDumpingJavaThread should use os::signal_raise, as suggested by David. Except there is the problem that David diagnosed in JDK-8139300, that OSX raise sends a signal to the main thread instead of the current thread. I wonder if that is still true with our new minimum supported OSX version? Man pages on my Mac say raise sends the signal to the current thread. (And I'm surprised that bug was fixed that way, rather than using os::signal_raise and making sure that worked properly.) In VMError::controlled_crash, not only is there no guarantee the write will crash, there's also no guarantee the break will do anything either. Since executing the write clearly invokes UB, the subsequent break can be assumed to be unreachable, or demons may fly out your nose. But since both of these are !PRODUCT, I guess I'm okay with suppressing the warning for now, as a way to move forward with the compiler upgrade. However, I'd like the scope of that warning suppression narrowed if possible, such as by moving it down to inside the #ifndef PRODUCT protecting the test code. We don't need the suppression for the entire file. For the frame case, does Solaris support any sort of push/pop diagnostic control? And it looks like there is further cleanup needed in these areas. From yumin.qi at gmail.com Tue May 1 05:48:28 2018 From: yumin.qi at gmail.com (yumin qi) Date: Mon, 30 Apr 2018 22:48:28 -0700 Subject: Fat jar with AppCDS In-Reply-To: <8e0d891a-c00c-8f57-362a-6a4684a6cc21@oracle.com> References: <8e0d891a-c00c-8f57-362a-6a4684a6cc21@oracle.com> Message-ID: I will try your patch and update you --- thanks! Yumin On Mon, Apr 30, 2018 at 3:05 PM, Ioi Lam wrote: > Hi Volker, > > Thank you so much for writing the tool. We have written some internal > tools that work similarly. As you said, the current interface is awkward > (because it's experimental). I have some ideas for how to improve support > for custom loaders in CDS, and I'll send out a separate mail for discussion. > > I have made a small patch your cl4cds tool to support "fat" JAR files. You > can find the patch and a test case from: > > http://cr.openjdk.java.net/~iklam/jdk11/fatjarcds/ > > It basically extract all entries like > > jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/ > validation-api-2.0.1.Final.jar!/ > > into a local file > > ./tmp/BOOT-INF/lib/validation-api-2.0.1.Final.jar > > and read from this location during archive dumping. > > > Yumin, could you try it out and see if it works for you? > > Thanks > - Ioi > > > > On 4/24/18 11:14 PM, yumin qi wrote: > > Hi, Volker > > Thanks! I will try your tool. > > Yumin > > On Tue, Apr 24, 2018 at 5:42 PM, Volker Simonis > wrote: > >> Hi Yumin, >> >> your right with your observation. For applications like Tomcat or >> Eclipse you can currently usually only share about 20%-30% of the >> application classes, even with AppCDS. >> >> To overcome this problem I wrote a small tool called "cl4cds" (i.e. >> class list for CDS) which can be used to generate a classlist out of >> the "Xlog:class+load" output. Unfortunately this is currently an >> extra, off-line step which should obviously be integrated right into >> HotSpot. But for the time being it allows you to take advantage of >> AppCDS for custom classloaders and share a much bigger amount of >> classes even for application server scenarios. >> >> Regards, >> Volker >> >> [1] https://github.com/simonis/cl4cds >> >> On Tue, Apr 24, 2018 at 2:49 AM, yumin qi wrote: >> > Hi,Ioi >> > >> > There are some applications with fat jar, that is, like web jars, are >> > loaded by its own class loaders, are not dumped when run with current >> mode. >> > AppCDS can dump the jars in path and loaded by App class loader, but it >> > does not dump the classes loaded by the class loader "in the jars", >> which >> > are loaded in runtime. >> > >> > Do you have any plan to work on this issue? The classes loaded during >> > runtime are NOT generated classes, they are just designed this way and >> > stored in other jar files(fat). >> > >> > We are working on the issue, so if you have recommendation or you >> already >> > worked on it please let me know. >> > >> > Thanks >> > Yumin >> > > > From adinn at redhat.com Tue May 1 10:32:34 2018 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 1 May 2018 11:32:34 +0100 Subject: RFR(L): 8201572: Improve Metaspace Statistics In-Reply-To: References: Message-ID: On 30/04/18 16:53, Thomas St?fe wrote: > I promise, this is the last addition to the change. Any followups I > will put into separate changes. > > Full http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.03/webrev/ > > Delta to last: http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-02-to-03/webrev/ > > Only very minimal changes to the waste section. > - Added "committed but unused" > - Added "overhead in in-use chunks" > - Printing a total I'm still happy with this. > Still need a second reviewer. I have pinged Zhengyu in case he is able to review it. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From thomas.stuefe at gmail.com Tue May 1 10:45:35 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 01 May 2018 10:45:35 +0000 Subject: RFR(L): 8201572: Improve Metaspace Statistics In-Reply-To: References: Message-ID: On Tue 1. May 2018 at 12:32, Andrew Dinn wrote: > On 30/04/18 16:53, Thomas St?fe wrote: > > I promise, this is the last addition to the change. Any followups I > > will put into separate changes. > > > > Full > http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.03/webrev/ > > > > Delta to last: > http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-02-to-03/webrev/ > > > > Only very minimal changes to the waste section. > > - Added "committed but unused" > > - Added "overhead in in-use chunks" > > - Printing a total > > I'm still happy with this. > > > Still need a second reviewer. > I have pinged Zhengyu in case he is able to review it. > > regards, > > > Andrew Dinn Thanks, Andrew! > > From zgu at redhat.com Tue May 1 13:28:39 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 1 May 2018 09:28:39 -0400 Subject: RFR(L): 8201572: Improve Metaspace Statistics In-Reply-To: References: Message-ID: Hi Thomas, This is a very useful feature, thanks for doing this. A few minor comments: - metaspace.cpp L#178 - 182: indents for case/default stmts. L#184: is this assert still necessary? - metaspace.hpp L#103 and L#108: leftover? - ostream.hpp Not sure if print_human_readable_size()/print_percentage() belong here. Otherwise, looks good to me. Thanks, -Zhengyu On 04/26/2018 02:03 AM, Thomas St?fe wrote: > Hi all, > > I still need an additional reviewer. > > new Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.02/webrev/index.html > incremental to v1: > http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-01-to-02/webrev/ > > new: just build- and test fixes: > - I added final cr() to a number of places where LogStream was used to > pipe output from functions I changed. Otherwise ~LogStream will assert > complaining about unfinished lines. I plan to change this, see > https://bugs.openjdk.java.net/browse/JDK-8202303, however for now, > lets go with the additional cr() > - build fixes for MacOS and Solaris. > > jdk-submit test suite ran clean through. We are currently running the > patch through our test suites, so far all looks good. > > Thanks, Thomas > > > > > On Tue, Apr 24, 2018 at 6:28 PM, Thomas St?fe wrote: >> Hi all, >> >> New webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-01/webrev/ >> Incremental: http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-00-to-01/webrev/ >> >> This one now cleanly applies to jdk-jdk and does not need other patches. >> >> Changes: >> - removed MetaWord occurrences in ostream.cpp, replaces with BytesPerWord >> - added per-spacetype printing to PrintNMTStatistics output to make it >> more closely resemble the output before. Example output: >> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/PrintNMTStatistics.txt >> - Fixed Windows build (missed precompiled.hpp) >> >> Thanks, Thomas >> >> >> On Mon, Apr 23, 2018 at 6:32 PM, Thomas St?fe wrote: >>> Hi all, >>> >>> may I please have opinions and reviews for the following improvement. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8201572 >>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.00/webrev/ >>> >>> (Note: it needs patch for 8202074 [3] to apply cleanly) >>> >>> This is a combined feature-add and cleanup for the metaspace >>> reporting. It should have no effect on metaspace allocation itself. >>> >>> This change does a number of things: >>> >>> - It improves the "VM.metaspace" jcmd, making it hopefully much more versatile. >>> >>> - It improves the way statistics are collected: The number of times we >>> walk the CLDG or the internal metaspace structures is reduced. It also >>> adds another global counter to MetaspaceUtils. >>> >>> - Though this was not primarily intended as a cleanup change, a number >>> of cleanups are part of this change. A number of functions were >>> removed. Where new code was added, I attempted to define and follow a >>> new scheme using namespaces and separate files. See below for details. >>> >>> --- >>> >>> Details: >>> >>> -> Improvements of jcmd: >>> >>> The jcmd "VM.metaspace" command now features a number of sub command >>> which control the details printed: >>> >>> jcmd VM.metaspace [basic] [show-loaders] [by-chunktype] [by-spacetype] >>> [vslist] [vsmap] [scale] >>> >>> All of them except for "basic" can be combined with each other. >>> >>> "basic" >>> shows a basic report; it needs no CLDG walk and no locks, so it >>> should be safe under most conditions. >>> "show-loaders" >>> this shows metaspace usage by loaders (basically what VM.metaspace >>> did before - this was originally added by Zhengyu Gu) >>> "by-chunktype" >>> Numbers are broken down by chunk type (small, medium...) in addition >>> to total numbers >>> "by-spacetype" >>> Numbers are broken down by space type (bootloader, anonymous, >>> reflection...) in addition to total numbers >>> "vslist" >>> The virtual space list is printed in detail >>> "vsmap" >>> Prints the virtual space map (ascii map of chunk placements) >>> >>> When no arguments are given, a summary is displayed. >>> >>> In addition to the above mentioned features we now print >>> - a "Waste" section detailing wastage. >>> - in debug mode, a section with some internal counters. This was just >>> curiosity on my part, I wanted to see how "hot" some of the metaspace >>> allocation paths actually get. >>> - We now count overhead too, i.e. space needed for chunk headers (was >>> counted as "used" before) >>> - We now also show deallocated space (block freelist) >>> >>> >>> Some output examples: >>> >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace.txt >>> >>> "basic" >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_basic.txt >>> >>> "by-chunktype" >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_by-chunktype.txt >>> >>> "by-spacetype" >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_by_spacetype.txt >>> >>> "show-loaders" >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_showloaders.txt >>> >>> "show-loaders" and "by-chunktype combined" >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_showloaders_by-chunktype.txt >>> >>> "vslist" >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_vslist.txt >>> >>> "vsmap" >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_vsmap.txt >>> >>> --- >>> >>> -> The way we collect statistics was streamlined. I tried to get rid >>> of all functions which were walking structures to just sum up a single >>> value; not only was that unnecessary code duplication, it was also >>> inefficient. >>> >>> This means most of the SpaceManager::sum_xxxx() functions are gone, as >>> well as all the different MetaspaceUtils::xxx_xxx_slow() functions. >>> >>> They were replaced by functions named "collect_statistics" or >>> "add_statistics", which accumulate all needed data in one go into a >>> statistics structure. This is more efficient since we usually need >>> more than one value and so it makes sense to collect them in one go. >>> >>> The newly added statistics objects are: >>> >>> - UsedChunksStatistics >>> - SpaceManagerStatistics >>> - ClassLoaderMetaspaceStatistics >>> >>> They live in a new file (memory/metaspace/metaspaceStatistics). >>> >>> I also ripped out the preexisting ChunkManagerStatistics, modified it >>> to follow the same convention, and moved it into this new file. >>> >>> -> We now have a new central reporting function, >>> MetaspaceUtils::print_report() - this was once Zhengyu's >>> print_for_nmt() function, but I expanded it and renamed it to be the >>> central entry point. Reporting options are handed into it via flags. >>> >>> In addition to that, we have a new >>> MetaspaceUtils::print_basic_report(), which prints a short summary >>> under the promise of not walking or locking anything. This is used at >>> error reporting time to print a short summary to the hs-err file. It >>> is also used for VM.info(). >>> >>> -> All new functions now accept the "scale" parameter known from NMT. >>> I expanded it a bit to have a "dynamic" setting. In that setting, the >>> printer chooses the best unit for a given value. I find it works quite >>> well, so I made it the default. I also changed printing to correctly >>> print very-small-but-not-0 numbers or percentages, where before they >>> were rounded to 0. See linked output examples. >>> >>> -> Other things to note: >>> >>> - MetaspaceUtils::dump was used to print some details at Metaspace >>> OOM. This now calls MetaspaceUtils::print_basic_report() directly, and >>> I removed the dump() function. >>> >>> - I moved printing the summary out of the PrintCLDMetaspaceInfoClosure >>> and into the print_report function. PrintCLDMetaspaceInfoClosure still >>> prints the individual loader metaspace usage numbers. >>> >>> - I streamlined naming of values somewhat. Where we refer to in-use >>> chunks, I now use consistently "capacity", "free", "waste", "used" and >>> the new "overhead", and the assumption is that capacity is the sum of >>> all the other values. >>> >>> - About the newly added namespaces: Since the patch is already quite >>> big, I did not want to load it with more cleanup. However, neither did >>> I want to just dump the new classes and functions into metaspace.cpp. >>> So as a compromise I tried to follow a new scheme with newly added >>> code. If you agree on this organization, this may be the way to split >>> metaspace.cpp in some follow up change. >>> >>> The idea was to move all metaspace-internal stuff into >>> memory/metaspace and hide it from the outside. Also, I did put >>> metaspace coding into an own namespace ("metaspace") and >>> metaspace-internal coding into "metaspace::internal". This is just a >>> proposal, we can bike shed. Future changes could move more classes out >>> of metaspace.cpp into metaspace/ and the metaspace::internal namespace >>> to make metaspace.cpp malleable again. >>> >>> Thank you, >>> >>> Kind Regards, Thomas >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> -------------------- >>> [1] https://bugs.openjdk.java.net/browse/JDK-8201572 >>> [2] http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.00/webrev/ >>> [3] http://cr.openjdk.java.net/~stuefe/webrevs/8202074-metaspace-retire-after-humongous-chunks-are-allocated/webrev.00/webrev/ From coleen.phillimore at oracle.com Tue May 1 13:54:13 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 1 May 2018 09:54:13 -0400 Subject: RFR(L): 8201572: Improve Metaspace Statistics In-Reply-To: References: Message-ID: <9c8e564f-86c7-7733-0c7b-0871da853953@oracle.com> Hi, This isn't a full review but I like that you've added a metaspace namespace and directory under memory for it. This will make a good foundation for doing this change: https://bugs.openjdk.java.net/browse/JDK-8176808 Thanks, Coleen On 4/26/18 2:03 AM, Thomas St?fe wrote: > Hi all, > > I still need an additional reviewer. > > new Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.02/webrev/index.html > incremental to v1: > http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-01-to-02/webrev/ > > new: just build- and test fixes: > - I added final cr() to a number of places where LogStream was used to > pipe output from functions I changed. Otherwise ~LogStream will assert > complaining about unfinished lines. I plan to change this, see > https://bugs.openjdk.java.net/browse/JDK-8202303, however for now, > lets go with the additional cr() > - build fixes for MacOS and Solaris. > > jdk-submit test suite ran clean through. We are currently running the > patch through our test suites, so far all looks good. > > Thanks, Thomas > > > > > On Tue, Apr 24, 2018 at 6:28 PM, Thomas St?fe wrote: >> Hi all, >> >> New webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-01/webrev/ >> Incremental: http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-00-to-01/webrev/ >> >> This one now cleanly applies to jdk-jdk and does not need other patches. >> >> Changes: >> - removed MetaWord occurrences in ostream.cpp, replaces with BytesPerWord >> - added per-spacetype printing to PrintNMTStatistics output to make it >> more closely resemble the output before. Example output: >> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/PrintNMTStatistics.txt >> - Fixed Windows build (missed precompiled.hpp) >> >> Thanks, Thomas >> >> >> On Mon, Apr 23, 2018 at 6:32 PM, Thomas St?fe wrote: >>> Hi all, >>> >>> may I please have opinions and reviews for the following improvement. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8201572 >>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.00/webrev/ >>> >>> (Note: it needs patch for 8202074 [3] to apply cleanly) >>> >>> This is a combined feature-add and cleanup for the metaspace >>> reporting. It should have no effect on metaspace allocation itself. >>> >>> This change does a number of things: >>> >>> - It improves the "VM.metaspace" jcmd, making it hopefully much more versatile. >>> >>> - It improves the way statistics are collected: The number of times we >>> walk the CLDG or the internal metaspace structures is reduced. It also >>> adds another global counter to MetaspaceUtils. >>> >>> - Though this was not primarily intended as a cleanup change, a number >>> of cleanups are part of this change. A number of functions were >>> removed. Where new code was added, I attempted to define and follow a >>> new scheme using namespaces and separate files. See below for details. >>> >>> --- >>> >>> Details: >>> >>> -> Improvements of jcmd: >>> >>> The jcmd "VM.metaspace" command now features a number of sub command >>> which control the details printed: >>> >>> jcmd VM.metaspace [basic] [show-loaders] [by-chunktype] [by-spacetype] >>> [vslist] [vsmap] [scale] >>> >>> All of them except for "basic" can be combined with each other. >>> >>> "basic" >>> shows a basic report; it needs no CLDG walk and no locks, so it >>> should be safe under most conditions. >>> "show-loaders" >>> this shows metaspace usage by loaders (basically what VM.metaspace >>> did before - this was originally added by Zhengyu Gu) >>> "by-chunktype" >>> Numbers are broken down by chunk type (small, medium...) in addition >>> to total numbers >>> "by-spacetype" >>> Numbers are broken down by space type (bootloader, anonymous, >>> reflection...) in addition to total numbers >>> "vslist" >>> The virtual space list is printed in detail >>> "vsmap" >>> Prints the virtual space map (ascii map of chunk placements) >>> >>> When no arguments are given, a summary is displayed. >>> >>> In addition to the above mentioned features we now print >>> - a "Waste" section detailing wastage. >>> - in debug mode, a section with some internal counters. This was just >>> curiosity on my part, I wanted to see how "hot" some of the metaspace >>> allocation paths actually get. >>> - We now count overhead too, i.e. space needed for chunk headers (was >>> counted as "used" before) >>> - We now also show deallocated space (block freelist) >>> >>> >>> Some output examples: >>> >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace.txt >>> >>> "basic" >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_basic.txt >>> >>> "by-chunktype" >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_by-chunktype.txt >>> >>> "by-spacetype" >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_by_spacetype.txt >>> >>> "show-loaders" >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_showloaders.txt >>> >>> "show-loaders" and "by-chunktype combined" >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_showloaders_by-chunktype.txt >>> >>> "vslist" >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_vslist.txt >>> >>> "vsmap" >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_vsmap.txt >>> >>> --- >>> >>> -> The way we collect statistics was streamlined. I tried to get rid >>> of all functions which were walking structures to just sum up a single >>> value; not only was that unnecessary code duplication, it was also >>> inefficient. >>> >>> This means most of the SpaceManager::sum_xxxx() functions are gone, as >>> well as all the different MetaspaceUtils::xxx_xxx_slow() functions. >>> >>> They were replaced by functions named "collect_statistics" or >>> "add_statistics", which accumulate all needed data in one go into a >>> statistics structure. This is more efficient since we usually need >>> more than one value and so it makes sense to collect them in one go. >>> >>> The newly added statistics objects are: >>> >>> - UsedChunksStatistics >>> - SpaceManagerStatistics >>> - ClassLoaderMetaspaceStatistics >>> >>> They live in a new file (memory/metaspace/metaspaceStatistics). >>> >>> I also ripped out the preexisting ChunkManagerStatistics, modified it >>> to follow the same convention, and moved it into this new file. >>> >>> -> We now have a new central reporting function, >>> MetaspaceUtils::print_report() - this was once Zhengyu's >>> print_for_nmt() function, but I expanded it and renamed it to be the >>> central entry point. Reporting options are handed into it via flags. >>> >>> In addition to that, we have a new >>> MetaspaceUtils::print_basic_report(), which prints a short summary >>> under the promise of not walking or locking anything. This is used at >>> error reporting time to print a short summary to the hs-err file. It >>> is also used for VM.info(). >>> >>> -> All new functions now accept the "scale" parameter known from NMT. >>> I expanded it a bit to have a "dynamic" setting. In that setting, the >>> printer chooses the best unit for a given value. I find it works quite >>> well, so I made it the default. I also changed printing to correctly >>> print very-small-but-not-0 numbers or percentages, where before they >>> were rounded to 0. See linked output examples. >>> >>> -> Other things to note: >>> >>> - MetaspaceUtils::dump was used to print some details at Metaspace >>> OOM. This now calls MetaspaceUtils::print_basic_report() directly, and >>> I removed the dump() function. >>> >>> - I moved printing the summary out of the PrintCLDMetaspaceInfoClosure >>> and into the print_report function. PrintCLDMetaspaceInfoClosure still >>> prints the individual loader metaspace usage numbers. >>> >>> - I streamlined naming of values somewhat. Where we refer to in-use >>> chunks, I now use consistently "capacity", "free", "waste", "used" and >>> the new "overhead", and the assumption is that capacity is the sum of >>> all the other values. >>> >>> - About the newly added namespaces: Since the patch is already quite >>> big, I did not want to load it with more cleanup. However, neither did >>> I want to just dump the new classes and functions into metaspace.cpp. >>> So as a compromise I tried to follow a new scheme with newly added >>> code. If you agree on this organization, this may be the way to split >>> metaspace.cpp in some follow up change. >>> >>> The idea was to move all metaspace-internal stuff into >>> memory/metaspace and hide it from the outside. Also, I did put >>> metaspace coding into an own namespace ("metaspace") and >>> metaspace-internal coding into "metaspace::internal". This is just a >>> proposal, we can bike shed. Future changes could move more classes out >>> of metaspace.cpp into metaspace/ and the metaspace::internal namespace >>> to make metaspace.cpp malleable again. >>> >>> Thank you, >>> >>> Kind Regards, Thomas >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> -------------------- >>> [1] https://bugs.openjdk.java.net/browse/JDK-8201572 >>> [2] http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.00/webrev/ >>> [3] http://cr.openjdk.java.net/~stuefe/webrevs/8202074-metaspace-retire-after-humongous-chunks-are-allocated/webrev.00/webrev/ From daniel.daugherty at oracle.com Tue May 1 17:00:22 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 1 May 2018 13:00:22 -0400 Subject: RFR(M): 8191798 redo nested ThreadsListHandle to drop Threads_lock Message-ID: <2fa7fff2-7fce-5255-a6a2-2c9066594a5c@oracle.com> Greetings, We have a fix for the following Thread-SMR bug: ??? JDK-8191798 redo nested ThreadsListHandle to drop Threads_lock ??? https://bugs.openjdk.java.net/browse/JDK-8191798 Erik O is the primary author for this fix; I've only made minor tweaks here and there to the comments, the code and the tests. I have the Thread-SMR stress testing setup on my Solaris-X64 server so I'm the one shepherding the fix forward. Summary: Refactor Thread hazard ptrs and nested ThreadsLists into ??? SafeThreadsListPtr. Hazard ptr management logic remains the same, but the nested ThreadsList management logic uses a linked list of SafeThreadsListPtr and safe counters instead of a linked list of NestedThreadsList and the Threads_lock. Webrev URLs: http://cr.openjdk.java.net/~dcubed/8191798-webrev/1_for_jdk_jdk.full_no_code_motion/ ??? This is the webrev to look at if you want to see the entire ??? proposed fix without code motion noise. http://cr.openjdk.java.net/~dcubed/8191798-webrev/1_for_jdk_jdk.code_motion_only/ ??? This webrev is only code motion and is here for completeness. http://cr.openjdk.java.net/~dcubed/8191798-webrev/1_for_jdk_jdk.full_with_code_motion/ ??? This webrev is everything and is here for completeness. Testing: - Mach5 builds-tier1,jdk-tier1,jdk-tier2,jdk-tier3,hs-tier1,hs-tier2,hs-tier3 ? - multiple rounds - no test failures - Solaris X64 Thread-SMR stress testing - 2 x 24+ hour runs - no failures ? related to these changes Thanks, in advance, for any comments, suggestions or questions. Dan and Erik From yumin.qi at gmail.com Tue May 1 17:18:54 2018 From: yumin.qi at gmail.com (yumin qi) Date: Tue, 1 May 2018 10:18:54 -0700 Subject: Fat jar with AppCDS In-Reply-To: References: <8e0d891a-c00c-8f57-362a-6a4684a6cc21@oracle.com> Message-ID: Hi, Ioi I tried your cl4cds.java and it works! Thanks. One way to work this around may be supply a jcmd operation to dump .jsa file? When the application started, usually the server will run for long time. Yumin On Mon, Apr 30, 2018 at 10:48 PM, yumin qi wrote: > I will try your patch and update you --- thanks! > > Yumin > > On Mon, Apr 30, 2018 at 3:05 PM, Ioi Lam wrote: > >> Hi Volker, >> >> Thank you so much for writing the tool. We have written some internal >> tools that work similarly. As you said, the current interface is awkward >> (because it's experimental). I have some ideas for how to improve support >> for custom loaders in CDS, and I'll send out a separate mail for discussion. >> >> I have made a small patch your cl4cds tool to support "fat" JAR files. >> You can find the patch and a test case from: >> >> http://cr.openjdk.java.net/~iklam/jdk11/fatjarcds/ >> >> It basically extract all entries like >> >> jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/valida >> tion-api-2.0.1.Final.jar!/ >> >> into a local file >> >> ./tmp/BOOT-INF/lib/validation-api-2.0.1.Final.jar >> >> and read from this location during archive dumping. >> >> >> Yumin, could you try it out and see if it works for you? >> >> Thanks >> - Ioi >> >> >> >> On 4/24/18 11:14 PM, yumin qi wrote: >> >> Hi, Volker >> >> Thanks! I will try your tool. >> >> Yumin >> >> On Tue, Apr 24, 2018 at 5:42 PM, Volker Simonis > > wrote: >> >>> Hi Yumin, >>> >>> your right with your observation. For applications like Tomcat or >>> Eclipse you can currently usually only share about 20%-30% of the >>> application classes, even with AppCDS. >>> >>> To overcome this problem I wrote a small tool called "cl4cds" (i.e. >>> class list for CDS) which can be used to generate a classlist out of >>> the "Xlog:class+load" output. Unfortunately this is currently an >>> extra, off-line step which should obviously be integrated right into >>> HotSpot. But for the time being it allows you to take advantage of >>> AppCDS for custom classloaders and share a much bigger amount of >>> classes even for application server scenarios. >>> >>> Regards, >>> Volker >>> >>> [1] https://github.com/simonis/cl4cds >>> >>> On Tue, Apr 24, 2018 at 2:49 AM, yumin qi wrote: >>> > Hi,Ioi >>> > >>> > There are some applications with fat jar, that is, like web jars, are >>> > loaded by its own class loaders, are not dumped when run with current >>> mode. >>> > AppCDS can dump the jars in path and loaded by App class loader, but it >>> > does not dump the classes loaded by the class loader "in the jars", >>> which >>> > are loaded in runtime. >>> > >>> > Do you have any plan to work on this issue? The classes loaded during >>> > runtime are NOT generated classes, they are just designed this way and >>> > stored in other jar files(fat). >>> > >>> > We are working on the issue, so if you have recommendation or you >>> already >>> > worked on it please let me know. >>> > >>> > Thanks >>> > Yumin >>> >> >> >> > From coleen.phillimore at oracle.com Tue May 1 17:29:48 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 1 May 2018 13:29:48 -0400 Subject: RFR(S): 8200466: Revisit the setting of _transitive_interfaces in InstanceKlass In-Reply-To: <5AE7B3B6.6020903@oracle.com> References: <5AE2AB27.5040805@oracle.com> <9ce88e64-599e-fd6e-ab70-4ce22f253c56@oracle.com> <5AE78393.6060203@oracle.com> <3fa86f67-be5a-50c4-8d04-707ff7ab185c@oracle.com> <5AE7B3B6.6020903@oracle.com> Message-ID: <9c17e3a0-23b0-7c39-788d-d70f2efd40dc@oracle.com> Hi Calvin, Ok, I had a look at this.? It seems that the relative initialization orders of _super and the primary superclasses is unfathonably complicated. I'm fine with your initial change as is. Thanks, Coleen On 4/30/18 8:24 PM, Calvin Cheung wrote: > Hi Coleen, > > I've tried the following simple change: > > --- a/src/hotspot/share/classfile/classFileParser.cpp > +++ b/src/hotspot/share/classfile/classFileParser.cpp > @@ -3592,6 +3592,7 @@ > ?? this_klass->set_methods(_methods); > ?? this_klass->set_inner_classes(_inner_classes); > ?? this_klass->set_local_interfaces(_local_interfaces); > + this_klass->set_super(const_cast(_super_klass)); > this_klass->set_transitive_interfaces(_transitive_interfaces); > ?? this_klass->set_annotations(_combined_annotations); > > but it crashes in vtableEntry::verify() during fastdebug build: > > ??? if (!vtklass->is_subtype_of(method()->method_holder())) { > #ifndef PRODUCT > ????? print(); > #endif > ????? fatal("vtableEntry " PTR_FORMAT ": method is from subclass", > p2i(this)); <<<<-- crashes here > ??? } > > Here's the call stack from the hs err log: > > V? [libjvm.so+0xb32bbe]? report_fatal(char const*, int, char const*, > ...)+0xfe > V? [libjvm.so+0x1215963]? klassVtable::verify(outputStream*, bool)+0x113 > V? [libjvm.so+0x1219753]? klassVtable::initialize_vtable(bool, > Thread*)+0x1293 > V? [libjvm.so+0x1834b6f]? Universe::reinitialize_vtable_of(Klass*, > Thread*)+0x2f > V? [libjvm.so+0x1834c2c]? Universe::reinitialize_vtable_of(Klass*, > Thread*)+0xec > V? [libjvm.so+0x1834c2c]? Universe::reinitialize_vtable_of(Klass*, > Thread*)+0xec > V? [libjvm.so+0x1837b3f]? universe_post_init()+0x1af > V? [libjvm.so+0xe99aad]? init_globals()+0xed > V? [libjvm.so+0x17f8b2c]? Threads::create_vm(JavaVMInitArgs*, > bool*)+0x2cc > V? [libjvm.so+0x1018af8]? JNI_CreateJavaVM+0xa8 > C? [libjli.so+0x3e81]? JavaMain+0x81 > > I will send you the hs err and build logs off list. > > thanks, > Calvin > > On 4/30/18, 2:20 PM, coleen.phillimore at oracle.com wrote: >> >> >> On 4/30/18 4:58 PM, Calvin Cheung wrote: >>> Hi Coleen, >>> >>> Thanks for your review. >>> >>> On 4/30/18, 10:23 AM, coleen.phillimore at oracle.com wrote: >>>> >>>> This sharing of metadata has been prone to bugs.? Can you have the >>>> same bug with the _secondary_super field? >>>> InstanceKlass::deallocate_contents() checks: >>>> >>>> ????? secondary_supers() != transitive_interfaces() && >>>> >>>> and will free the field possibly from the super class if >>>> transitive_interfaces() is still NULL.? (If I understand this >>>> correctly). >>> If it passes the conditions check in the 'if' block, it will >>> deallocate the _secondary_supers. >>> It maybe fine because the crash was in visit_all_interfaces() where >>> it doesn't dereference the _secondary_supers. >> >> It seems like it would deallocate _secondary_supers if an OOM >> occurred after it's set to the super's transitive_interfaces and >> before _transitive_interfaces is set.? There could be a state that >> could cause this to happen.? I would have to trace through to see >> what sort of crash would happen. >> >>>> >>>> Wouldn't it be easier just to eagerly set the super class, and keep >>>> the call to initialize_supers() where it is? >>> Do you mean adding the following in >>> ClassFileParser::apply_parsed_class_metadata() and reverting the >>> rest of the changes? >>> >>> this_klass->set_super(const_cast(_super_klass)); >>> >> >> Yes, can you try this?? This line with a comment would be less >> complicated than passing NULL to initialize_supers() for the >> arrayKlass versions. >> >> >>> BTW, I think the first arg to initialize_supers() should be casted >>> to Klass* like above because the _super_klass is already an >>> InstanceKlass*. >>> >>> ik->initialize_supers(const_cast(_super_klass) ... >>> >>> >> I think we still need some cast here and above because _super_klass >> is defined as const, and this function and set_super are both not >> const-safe, which is a bigger change. Or you could make >> ClassFileParser::_super_klass not const since const is cast away. >> >>>> >>>> http://cr.openjdk.java.net/~ccheung/8200466/webrev.00/src/hotspot/share/classfile/classFileParser.cpp.udiff.html >>>> >>>> >>>> **** 5884,5893 ***** >>>> *--- 5889,5899 ----* >>>> *Annotations::free_contents(_loader_data, _fields_annotations);* >>>> *Annotations::free_contents(_loader_data, _fields_type_annotations);* >>>> *}* >>>> ** >>>> *clear_class_metadata();* >>>> *+ _transitive_interfaces = NULL;* >>>> ** >>>> >>>> clear_class_metadata() already sets the _transitive_interfaces to >>>> NULL, so this shouldn't be needed. >>> The current change does not set the _trasitive_interfaces to NULL in >>> clear_class_metadata(). >>> >>> From the udiff above: >>> >>> **** 5832,5842 ***** >>> *--- 5838,5847 ----* >>> *???? _cp = NULL;* >>> *???? _fields = NULL;* >>> *???? _methods = NULL;* >>> *???? _inner_classes = NULL;* >>> *???? _local_interfaces = NULL;* >>> *???? _transitive_interfaces = NULL;* >>> *???? _combined_annotations = NULL;* >>> *???? _annotations = _type_annotations = NULL;* >>> *???? _fields_annotations = _fields_type_annotations = NULL;* >>> *?? }* >> >> Oh I see it now and why you changed it.? Can you see if setting the >> Klass::_super ahead of setting _transitive_interfaces would work better? >> >> thanks, >> Coleen >> >>> >>> thanks, >>> Calvin >>>> >>>> thanks, >>>> Coleen >>>> >>>> On 4/27/18 12:46 AM, Calvin Cheung wrote: >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8200466 >>>>> >>>>> webrev: http://cr.openjdk.java.net/~ccheung/8200466/webrev.00/ >>>>> >>>>> This bug was discovered during the investigation of JDK-8200078 >>>>> . >>>>> >>>>> I've included the following comment in classFileParser.cpp to >>>>> summarize the change: >>>>> >>>>> // Delay the setting of _transitive_interfaces until after >>>>> initialize_supers() in >>>>> // fill_instance_klass(). It is because the _transitive_interfaces >>>>> may be shared with >>>>> // its _super. If an OOM occurs while loading the current klass, >>>>> its _super field >>>>> // may not have been set. When GC tries to free the klass, the >>>>> _transitive_interfaces >>>>> // may be deallocated mistakenly in >>>>> InstanceKlass::deallocate_interfaces(). Subsequent >>>>> // dereferences to the deallocated _transitive_interfaces will >>>>> result in a crash. >>>>> >>>>> Testing: (on Oracle platforms) >>>>> ??? hs-tier{1,2,3} >>>>> ??? closed (soon will be open) PCL (parallel class loading) tests >>>>> >>>>> thanks, >>>>> Calvin >>>> >> From calvin.cheung at oracle.com Tue May 1 18:07:06 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Tue, 01 May 2018 11:07:06 -0700 Subject: RFR(S): 8200466: Revisit the setting of _transitive_interfaces in InstanceKlass In-Reply-To: <9c17e3a0-23b0-7c39-788d-d70f2efd40dc@oracle.com> References: <5AE2AB27.5040805@oracle.com> <9ce88e64-599e-fd6e-ab70-4ce22f253c56@oracle.com> <5AE78393.6060203@oracle.com> <3fa86f67-be5a-50c4-8d04-707ff7ab185c@oracle.com> <5AE7B3B6.6020903@oracle.com> <9c17e3a0-23b0-7c39-788d-d70f2efd40dc@oracle.com> Message-ID: <5AE8ACCA.9010506@oracle.com> Hi Coleen, Thanks for taking a closer look at it. I'll sync up my repo, do another round of testing, before pushing the change. thanks, Calvin On 5/1/18, 10:29 AM, coleen.phillimore at oracle.com wrote: > > Hi Calvin, > > Ok, I had a look at this. It seems that the relative initialization > orders of _super and the primary superclasses is unfathonably > complicated. > I'm fine with your initial change as is. > > Thanks, > Coleen > > > On 4/30/18 8:24 PM, Calvin Cheung wrote: >> Hi Coleen, >> >> I've tried the following simple change: >> >> --- a/src/hotspot/share/classfile/classFileParser.cpp >> +++ b/src/hotspot/share/classfile/classFileParser.cpp >> @@ -3592,6 +3592,7 @@ >> this_klass->set_methods(_methods); >> this_klass->set_inner_classes(_inner_classes); >> this_klass->set_local_interfaces(_local_interfaces); >> + this_klass->set_super(const_cast(_super_klass)); >> this_klass->set_transitive_interfaces(_transitive_interfaces); >> this_klass->set_annotations(_combined_annotations); >> >> but it crashes in vtableEntry::verify() during fastdebug build: >> >> if (!vtklass->is_subtype_of(method()->method_holder())) { >> #ifndef PRODUCT >> print(); >> #endif >> fatal("vtableEntry " PTR_FORMAT ": method is from subclass", >> p2i(this)); <<<<-- crashes here >> } >> >> Here's the call stack from the hs err log: >> >> V [libjvm.so+0xb32bbe] report_fatal(char const*, int, char const*, >> ...)+0xfe >> V [libjvm.so+0x1215963] klassVtable::verify(outputStream*, bool)+0x113 >> V [libjvm.so+0x1219753] klassVtable::initialize_vtable(bool, >> Thread*)+0x1293 >> V [libjvm.so+0x1834b6f] Universe::reinitialize_vtable_of(Klass*, >> Thread*)+0x2f >> V [libjvm.so+0x1834c2c] Universe::reinitialize_vtable_of(Klass*, >> Thread*)+0xec >> V [libjvm.so+0x1834c2c] Universe::reinitialize_vtable_of(Klass*, >> Thread*)+0xec >> V [libjvm.so+0x1837b3f] universe_post_init()+0x1af >> V [libjvm.so+0xe99aad] init_globals()+0xed >> V [libjvm.so+0x17f8b2c] Threads::create_vm(JavaVMInitArgs*, >> bool*)+0x2cc >> V [libjvm.so+0x1018af8] JNI_CreateJavaVM+0xa8 >> C [libjli.so+0x3e81] JavaMain+0x81 >> >> I will send you the hs err and build logs off list. >> >> thanks, >> Calvin >> >> On 4/30/18, 2:20 PM, coleen.phillimore at oracle.com wrote: >>> >>> >>> On 4/30/18 4:58 PM, Calvin Cheung wrote: >>>> Hi Coleen, >>>> >>>> Thanks for your review. >>>> >>>> On 4/30/18, 10:23 AM, coleen.phillimore at oracle.com wrote: >>>>> >>>>> This sharing of metadata has been prone to bugs. Can you have the >>>>> same bug with the _secondary_super field? >>>>> InstanceKlass::deallocate_contents() checks: >>>>> >>>>> secondary_supers() != transitive_interfaces() && >>>>> >>>>> and will free the field possibly from the super class if >>>>> transitive_interfaces() is still NULL. (If I understand this >>>>> correctly). >>>> If it passes the conditions check in the 'if' block, it will >>>> deallocate the _secondary_supers. >>>> It maybe fine because the crash was in visit_all_interfaces() where >>>> it doesn't dereference the _secondary_supers. >>> >>> It seems like it would deallocate _secondary_supers if an OOM >>> occurred after it's set to the super's transitive_interfaces and >>> before _transitive_interfaces is set. There could be a state that >>> could cause this to happen. I would have to trace through to see >>> what sort of crash would happen. >>> >>>>> >>>>> Wouldn't it be easier just to eagerly set the super class, and >>>>> keep the call to initialize_supers() where it is? >>>> Do you mean adding the following in >>>> ClassFileParser::apply_parsed_class_metadata() and reverting the >>>> rest of the changes? >>>> >>>> this_klass->set_super(const_cast(_super_klass)); >>>> >>> >>> Yes, can you try this? This line with a comment would be less >>> complicated than passing NULL to initialize_supers() for the >>> arrayKlass versions. >>> >>> >>>> BTW, I think the first arg to initialize_supers() should be casted >>>> to Klass* like above because the _super_klass is already an >>>> InstanceKlass*. >>>> >>>> ik->initialize_supers(const_cast(_super_klass) ... >>>> >>>> >>> I think we still need some cast here and above because _super_klass >>> is defined as const, and this function and set_super are both not >>> const-safe, which is a bigger change. Or you could make >>> ClassFileParser::_super_klass not const since const is cast away. >>> >>>>> >>>>> http://cr.openjdk.java.net/~ccheung/8200466/webrev.00/src/hotspot/share/classfile/classFileParser.cpp.udiff.html >>>>> >>>>> >>>>> **** 5884,5893 ***** >>>>> *--- 5889,5899 ----* >>>>> *Annotations::free_contents(_loader_data, _fields_annotations);* >>>>> *Annotations::free_contents(_loader_data, _fields_type_annotations);* >>>>> *}* >>>>> ** >>>>> *clear_class_metadata();* >>>>> *+ _transitive_interfaces = NULL;* >>>>> ** >>>>> >>>>> clear_class_metadata() already sets the _transitive_interfaces to >>>>> NULL, so this shouldn't be needed. >>>> The current change does not set the _trasitive_interfaces to NULL >>>> in clear_class_metadata(). >>>> >>>> From the udiff above: >>>> >>>> **** 5832,5842 ***** >>>> *--- 5838,5847 ----* >>>> * _cp = NULL;* >>>> * _fields = NULL;* >>>> * _methods = NULL;* >>>> * _inner_classes = NULL;* >>>> * _local_interfaces = NULL;* >>>> * _transitive_interfaces = NULL;* >>>> * _combined_annotations = NULL;* >>>> * _annotations = _type_annotations = NULL;* >>>> * _fields_annotations = _fields_type_annotations = NULL;* >>>> * }* >>> >>> Oh I see it now and why you changed it. Can you see if setting the >>> Klass::_super ahead of setting _transitive_interfaces would work >>> better? >>> >>> thanks, >>> Coleen >>> >>>> >>>> thanks, >>>> Calvin >>>>> >>>>> thanks, >>>>> Coleen >>>>> >>>>> On 4/27/18 12:46 AM, Calvin Cheung wrote: >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8200466 >>>>>> >>>>>> webrev: http://cr.openjdk.java.net/~ccheung/8200466/webrev.00/ >>>>>> >>>>>> This bug was discovered during the investigation of JDK-8200078 >>>>>> . >>>>>> >>>>>> I've included the following comment in classFileParser.cpp to >>>>>> summarize the change: >>>>>> >>>>>> // Delay the setting of _transitive_interfaces until after >>>>>> initialize_supers() in >>>>>> // fill_instance_klass(). It is because the >>>>>> _transitive_interfaces may be shared with >>>>>> // its _super. If an OOM occurs while loading the current klass, >>>>>> its _super field >>>>>> // may not have been set. When GC tries to free the klass, the >>>>>> _transitive_interfaces >>>>>> // may be deallocated mistakenly in >>>>>> InstanceKlass::deallocate_interfaces(). Subsequent >>>>>> // dereferences to the deallocated _transitive_interfaces will >>>>>> result in a crash. >>>>>> >>>>>> Testing: (on Oracle platforms) >>>>>> hs-tier{1,2,3} >>>>>> closed (soon will be open) PCL (parallel class loading) tests >>>>>> >>>>>> thanks, >>>>>> Calvin >>>>> >>> > From ioi.lam at oracle.com Tue May 1 18:32:36 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 1 May 2018 11:32:36 -0700 Subject: Improving AppCDS for Custom Loaders In-Reply-To: References: <8e0d891a-c00c-8f57-362a-6a4684a6cc21@oracle.com> Message-ID: <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> PROBLEM: As discussed with Volker and Yumin in previous e-mails, AppCDS has some experimental support for custom class loaders. However, it's not very easy to use. For example, you can write a classlist like this: ??? java/lang/Object id: 1 ??? CustomLoadee id: 2 super: 1 source: /tmp/foo.jar The CustomLoadee class will be stored in the shared archive with a CRC code. During runtime, if a customed loader wants to load a class of the same name, and its classfile has the same size and CRC as the archived class, the archived version will be loaded. This speeds up class loading by avoiding parsing the class file, and saves space by sharing the mmap'ed class metadata across processes. You can see an example test at: http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java However, the current scheme requires you to specify all the super classes and interfaces. There's no support provided by the -XX:DumpLoadedClassList option.It can be helped somewhat with Volker's tool: https://github.com/simonis/cl4cds POSSIBLE SOLUTIONS: 1. "Dump-as-you-go". As suggested by Yumin, we can provide a jcmd to ask a running JVM process to dump all of its loaded classes, including those loaded by custom loaders, into an archive. An alternative is to dump the archive at JVM exit time (or when you press Ctrl-C, etc. 2. Add information about the custom classes for -XX:DumpLoadedClassList. The trouble is some class loaders don't specify a code source that can be understood by the built-in class loaders. For example, the "Fat Jars" would have a code source like jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/ also, many custom loaders would pre-process the classfile data before defining the class, so we can't simply archive the version of the class on disk. One possible solution for #2 is to include the class file data in the -XX:DumpLoadedClassList output: java/lang/Object id: 1 ??? CustomLoadee id: 2 super: 1 source: base64 yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0 PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA ??? AgAPAAcAAAAKAAEABQADAAYACA== Of the 2 solutions: #1 seems easier to use, but may require more invasive modifications in the VM, especially if you want to be able to continue execution after dumping. #2 would be easier to implement, but the classlist might be huge. Also, #2 would allow post-processing tools to remove unneeded classes, or merge two runs into a single list. The output of #1 is essentially a binary blob that's impossible for off-line analysis/optimizations. Any comments, or suggestions for alternatives? Thanks - Ioi From erik.osterlund at oracle.com Tue May 1 19:43:47 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 1 May 2018 21:43:47 +0200 Subject: RFR(M): 8191798 redo nested ThreadsListHandle to drop Threads_lock In-Reply-To: <2fa7fff2-7fce-5255-a6a2-2c9066594a5c@oracle.com> References: <2fa7fff2-7fce-5255-a6a2-2c9066594a5c@oracle.com> Message-ID: Hi Dan, In case my opinion counts, I think this looks great. Thank you for the thorough testing and various improvements of this patch. Thanks, /Erik On 2018-05-01 19:00, Daniel D. Daugherty wrote: > Greetings, > > We have a fix for the following Thread-SMR bug: > > ??? JDK-8191798 redo nested ThreadsListHandle to drop Threads_lock > ??? https://bugs.openjdk.java.net/browse/JDK-8191798 > > Erik O is the primary author for this fix; I've only made minor tweaks > here and there to the comments, the code and the tests. I have the > Thread-SMR stress testing setup on my Solaris-X64 server so I'm the > one shepherding the fix forward. > > Summary: Refactor Thread hazard ptrs and nested ThreadsLists into > ??? SafeThreadsListPtr. > > Hazard ptr management logic remains the same, but the nested ThreadsList > management logic uses a linked list of SafeThreadsListPtr and safe > counters > instead of a linked list of NestedThreadsList and the Threads_lock. > > > Webrev URLs: > > http://cr.openjdk.java.net/~dcubed/8191798-webrev/1_for_jdk_jdk.full_no_code_motion/ > > ??? This is the webrev to look at if you want to see the entire > ??? proposed fix without code motion noise. > > http://cr.openjdk.java.net/~dcubed/8191798-webrev/1_for_jdk_jdk.code_motion_only/ > > ??? This webrev is only code motion and is here for completeness. > > http://cr.openjdk.java.net/~dcubed/8191798-webrev/1_for_jdk_jdk.full_with_code_motion/ > > ??? This webrev is everything and is here for completeness. > > Testing: > - Mach5 > builds-tier1,jdk-tier1,jdk-tier2,jdk-tier3,hs-tier1,hs-tier2,hs-tier3 > ? - multiple rounds - no test failures > - Solaris X64 Thread-SMR stress testing - 2 x 24+ hour runs - no failures > ? related to these changes > > Thanks, in advance, for any comments, suggestions or questions. > > Dan and Erik From daniel.daugherty at oracle.com Tue May 1 20:07:12 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 1 May 2018 16:07:12 -0400 Subject: RFR(M): 8191798 redo nested ThreadsListHandle to drop Threads_lock In-Reply-To: References: <2fa7fff2-7fce-5255-a6a2-2c9066594a5c@oracle.com> Message-ID: On 5/1/18 3:43 PM, Erik ?sterlund wrote: > Hi Dan, > > In case my opinion counts, I think this looks great. As is usual for Thread-SMR stuff, I plan to list both you and I as contributors and as reviewers. Robbin reviewed internally and I expect he will chime in here. David H often reviews Thread-SMR stuff but he's a bit busy right now... I'm hoping Kim B. chimes in since I think he ran into this issue with some of his recent changes... So thanks for the (re-)review! > Thank you for the thorough testing and various improvements of this > patch. You are quite welcome. Dan > > Thanks, > /Erik > > On 2018-05-01 19:00, Daniel D. Daugherty wrote: >> Greetings, >> >> We have a fix for the following Thread-SMR bug: >> >> ??? JDK-8191798 redo nested ThreadsListHandle to drop Threads_lock >> ??? https://bugs.openjdk.java.net/browse/JDK-8191798 >> >> Erik O is the primary author for this fix; I've only made minor tweaks >> here and there to the comments, the code and the tests. I have the >> Thread-SMR stress testing setup on my Solaris-X64 server so I'm the >> one shepherding the fix forward. >> >> Summary: Refactor Thread hazard ptrs and nested ThreadsLists into >> ??? SafeThreadsListPtr. >> >> Hazard ptr management logic remains the same, but the nested ThreadsList >> management logic uses a linked list of SafeThreadsListPtr and safe >> counters >> instead of a linked list of NestedThreadsList and the Threads_lock. >> >> >> Webrev URLs: >> >> http://cr.openjdk.java.net/~dcubed/8191798-webrev/1_for_jdk_jdk.full_no_code_motion/ >> >> ??? This is the webrev to look at if you want to see the entire >> ??? proposed fix without code motion noise. >> >> http://cr.openjdk.java.net/~dcubed/8191798-webrev/1_for_jdk_jdk.code_motion_only/ >> >> ??? This webrev is only code motion and is here for completeness. >> >> http://cr.openjdk.java.net/~dcubed/8191798-webrev/1_for_jdk_jdk.full_with_code_motion/ >> >> ??? This webrev is everything and is here for completeness. >> >> Testing: >> - Mach5 >> builds-tier1,jdk-tier1,jdk-tier2,jdk-tier3,hs-tier1,hs-tier2,hs-tier3 >> ? - multiple rounds - no test failures >> - Solaris X64 Thread-SMR stress testing - 2 x 24+ hour runs - no >> failures >> ? related to these changes >> >> Thanks, in advance, for any comments, suggestions or questions. >> >> Dan and Erik > From calvin.cheung at oracle.com Tue May 1 21:57:14 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Tue, 01 May 2018 14:57:14 -0700 Subject: RFR(xs): 8202130: [TESTBUG] Some appcds regression test cases fail with "Error: VM option 'PrintSystemDictionaryAtExit' is notproduct and is available only in debug version of VM" Message-ID: <5AE8E2BA.30702@oracle.com> JBS: https://bugs.openjdk.java.net/browse/JDK-8202130 webrev: http://cr.openjdk.java.net/~ccheung/8202130/webrev.00/ The PrintSystemDictionaryAtExit vm option is a debug option and is not available in release builds. It was there for debugging purposes while the tests were being developed. Removing the PrintSystemDictionaryAtExit vm option from the tests. Tested with fastdebug and release builds on Oracle platforms. thanks, Calvin From kim.barrett at oracle.com Tue May 1 22:02:34 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 1 May 2018 18:02:34 -0400 Subject: RFR: JDK-8202319: Fix compilation warnings in Solaris debug builds for DevStudio 12.6 In-Reply-To: References: <5AE1FFD0.1090803@oracle.com> <5ED019CA-50CB-4AAB-A22A-E7BC6FE0157D@oracle.com> <43683acb-7f30-e79a-42dc-0d4693e2ac21@oracle.com> <4df85ebd-aca1-46fe-8104-5c6f50517256@oracle.com> Message-ID: > On Apr 30, 2018, at 8:42 PM, Kim Barrett wrote: > And it looks like there is further cleanup needed in these areas. I?ve created these followups: https://bugs.openjdk.java.net/browse/JDK-8202508 https://bugs.openjdk.java.net/browse/JDK-8202509 From zgu at redhat.com Wed May 2 00:06:58 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 1 May 2018 20:06:58 -0400 Subject: RFR(xs): 8202130: [TESTBUG] Some appcds regression test cases fail with "Error: VM option 'PrintSystemDictionaryAtExit' is notproduct and is available only in debug version of VM" In-Reply-To: <5AE8E2BA.30702@oracle.com> References: <5AE8E2BA.30702@oracle.com> Message-ID: Looks good to me. -Zhengyu On 05/01/2018 05:57 PM, Calvin Cheung wrote: > JBS: https://bugs.openjdk.java.net/browse/JDK-8202130 > > webrev: http://cr.openjdk.java.net/~ccheung/8202130/webrev.00/ > > The PrintSystemDictionaryAtExit vm option is a debug option and is not > available in release builds. It was there for debugging purposes while > the tests were being developed. Removing the PrintSystemDictionaryAtExit > vm option from the tests. > > Tested with fastdebug and release builds on Oracle platforms. > > thanks, > Calvin From calvin.cheung at oracle.com Wed May 2 00:23:31 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Tue, 01 May 2018 17:23:31 -0700 Subject: RFR(xs): 8202130: [TESTBUG] Some appcds regression test cases fail with "Error: VM option 'PrintSystemDictionaryAtExit' is notproduct and is available only in debug version of VM" In-Reply-To: References: <5AE8E2BA.30702@oracle.com> Message-ID: <5AE90503.3070006@oracle.com> Thanks, Zhengyu. Calvin On 5/1/18, 5:06 PM, Zhengyu Gu wrote: > Looks good to me. > > -Zhengyu > > On 05/01/2018 05:57 PM, Calvin Cheung wrote: >> JBS: https://bugs.openjdk.java.net/browse/JDK-8202130 >> >> webrev: http://cr.openjdk.java.net/~ccheung/8202130/webrev.00/ >> >> The PrintSystemDictionaryAtExit vm option is a debug option and is >> not available in release builds. It was there for debugging purposes >> while the tests were being developed. Removing the >> PrintSystemDictionaryAtExit vm option from the tests. >> >> Tested with fastdebug and release builds on Oracle platforms. >> >> thanks, >> Calvin From ioi.lam at oracle.com Wed May 2 05:17:38 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 1 May 2018 22:17:38 -0700 Subject: RFR 8197954 Remove unnecessary intermediary APIs from AppCDS implementation Message-ID: https://bugs.openjdk.java.net/browse/JDK-8197954 http://cr.openjdk.java.net/~iklam/jdk11/8197954-remove-unnecessary-appcds-api.v01/ Summary: Before AppCDS was open sourced, we had a few "Ext" classes in sharedClassUtil.hpp that abstracted away the CDS operations related to the non-boot class loaders. This API made it possible to build the Oracle JDK with AppCDS included, or build the Open JDK with AppCDS removed. With the open sourcing of AppCDS, this abstraction layer is no longer necessary. I have moved the contents of the "Ext" classes into their proper locations and removed the sharedClassUtil.hpp/cpp files. Most of the changes are just moving things around. There shouldn't be behavioral changes. The files classLoaderExt.hpp/cpp still exists -- it encapsulates the classpath management code and is not strictly unnecessary. Some clean up may be needed there, but I'll probably do that in a separate RFE. Testing with hotspot tiers1~4. Thanks - Ioi From thomas.stuefe at gmail.com Wed May 2 05:26:37 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 2 May 2018 07:26:37 +0200 Subject: RFR(xxs): 8202303: LogStream should autoflush on destruction In-Reply-To: References: Message-ID: Ping... this is a really simple one :) Please review. Thanks! On Thu, Apr 26, 2018 at 8:34 AM, Thomas St?fe wrote: > Hi all, > > please review this tiny change: > > https://bugs.openjdk.java.net/browse/JDK-8202303 > http://cr.openjdk.java.net/~stuefe/webrevs/8202303-logstream-autoflush/webrev.00/webrev/ > > Basically, LogStream should not assert upon desctruction when having > buffered unfinished lines; instead, it should autoflush. > > (Note: the changes to the TestLineBufferAllocation test are just to > keep this test in line with the rest of the tests in regards to how > the fake log file is handled). > > More details in my earlier mail to hs-runtime: > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-April/027592.html > > Thanks, Thomas From thomas.stuefe at gmail.com Wed May 2 05:36:54 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 2 May 2018 07:36:54 +0200 Subject: RFR(L): 8201572: Improve Metaspace Statistics In-Reply-To: <9c8e564f-86c7-7733-0c7b-0871da853953@oracle.com> References: <9c8e564f-86c7-7733-0c7b-0871da853953@oracle.com> Message-ID: Hi Coleen, On Tue, May 1, 2018 at 3:54 PM, wrote: > > Hi, This isn't a full review but I like that you've added a metaspace > namespace and directory under memory for it. > This will make a good foundation for doing this change: > > https://bugs.openjdk.java.net/browse/JDK-8176808 > Yes, that is what I thought. When ripping up metaspace.cpp, should we be concerned that we loose history for the ripped out code? ..Thomas > Thanks, > Coleen > > > On 4/26/18 2:03 AM, Thomas St?fe wrote: >> >> Hi all, >> >> I still need an additional reviewer. >> >> new Webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.02/webrev/index.html >> incremental to v1: >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-01-to-02/webrev/ >> >> new: just build- and test fixes: >> - I added final cr() to a number of places where LogStream was used to >> pipe output from functions I changed. Otherwise ~LogStream will assert >> complaining about unfinished lines. I plan to change this, see >> https://bugs.openjdk.java.net/browse/JDK-8202303, however for now, >> lets go with the additional cr() >> - build fixes for MacOS and Solaris. >> >> jdk-submit test suite ran clean through. We are currently running the >> patch through our test suites, so far all looks good. >> >> Thanks, Thomas >> >> >> >> >> On Tue, Apr 24, 2018 at 6:28 PM, Thomas St?fe >> wrote: >>> >>> Hi all, >>> >>> New webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-01/webrev/ >>> Incremental: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-00-to-01/webrev/ >>> >>> This one now cleanly applies to jdk-jdk and does not need other patches. >>> >>> Changes: >>> - removed MetaWord occurrences in ostream.cpp, replaces with BytesPerWord >>> - added per-spacetype printing to PrintNMTStatistics output to make it >>> more closely resemble the output before. Example output: >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/PrintNMTStatistics.txt >>> - Fixed Windows build (missed precompiled.hpp) >>> >>> Thanks, Thomas >>> >>> >>> On Mon, Apr 23, 2018 at 6:32 PM, Thomas St?fe >>> wrote: >>>> >>>> Hi all, >>>> >>>> may I please have opinions and reviews for the following improvement. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8201572 >>>> Webrev: >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.00/webrev/ >>>> >>>> (Note: it needs patch for 8202074 [3] to apply cleanly) >>>> >>>> This is a combined feature-add and cleanup for the metaspace >>>> reporting. It should have no effect on metaspace allocation itself. >>>> >>>> This change does a number of things: >>>> >>>> - It improves the "VM.metaspace" jcmd, making it hopefully much more >>>> versatile. >>>> >>>> - It improves the way statistics are collected: The number of times we >>>> walk the CLDG or the internal metaspace structures is reduced. It also >>>> adds another global counter to MetaspaceUtils. >>>> >>>> - Though this was not primarily intended as a cleanup change, a number >>>> of cleanups are part of this change. A number of functions were >>>> removed. Where new code was added, I attempted to define and follow a >>>> new scheme using namespaces and separate files. See below for details. >>>> >>>> --- >>>> >>>> Details: >>>> >>>> -> Improvements of jcmd: >>>> >>>> The jcmd "VM.metaspace" command now features a number of sub command >>>> which control the details printed: >>>> >>>> jcmd VM.metaspace [basic] [show-loaders] [by-chunktype] [by-spacetype] >>>> [vslist] [vsmap] [scale] >>>> >>>> All of them except for "basic" can be combined with each other. >>>> >>>> "basic" >>>> shows a basic report; it needs no CLDG walk and no locks, so it >>>> should be safe under most conditions. >>>> "show-loaders" >>>> this shows metaspace usage by loaders (basically what VM.metaspace >>>> did before - this was originally added by Zhengyu Gu) >>>> "by-chunktype" >>>> Numbers are broken down by chunk type (small, medium...) in addition >>>> to total numbers >>>> "by-spacetype" >>>> Numbers are broken down by space type (bootloader, anonymous, >>>> reflection...) in addition to total numbers >>>> "vslist" >>>> The virtual space list is printed in detail >>>> "vsmap" >>>> Prints the virtual space map (ascii map of chunk placements) >>>> >>>> When no arguments are given, a summary is displayed. >>>> >>>> In addition to the above mentioned features we now print >>>> - a "Waste" section detailing wastage. >>>> - in debug mode, a section with some internal counters. This was just >>>> curiosity on my part, I wanted to see how "hot" some of the metaspace >>>> allocation paths actually get. >>>> - We now count overhead too, i.e. space needed for chunk headers (was >>>> counted as "used" before) >>>> - We now also show deallocated space (block freelist) >>>> >>>> >>>> Some output examples: >>>> >>>> >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace.txt >>>> >>>> "basic" >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_basic.txt >>>> >>>> "by-chunktype" >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_by-chunktype.txt >>>> >>>> "by-spacetype" >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_by_spacetype.txt >>>> >>>> "show-loaders" >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_showloaders.txt >>>> >>>> "show-loaders" and "by-chunktype combined" >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_showloaders_by-chunktype.txt >>>> >>>> "vslist" >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_vslist.txt >>>> >>>> "vsmap" >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_vsmap.txt >>>> >>>> --- >>>> >>>> -> The way we collect statistics was streamlined. I tried to get rid >>>> of all functions which were walking structures to just sum up a single >>>> value; not only was that unnecessary code duplication, it was also >>>> inefficient. >>>> >>>> This means most of the SpaceManager::sum_xxxx() functions are gone, as >>>> well as all the different MetaspaceUtils::xxx_xxx_slow() functions. >>>> >>>> They were replaced by functions named "collect_statistics" or >>>> "add_statistics", which accumulate all needed data in one go into a >>>> statistics structure. This is more efficient since we usually need >>>> more than one value and so it makes sense to collect them in one go. >>>> >>>> The newly added statistics objects are: >>>> >>>> - UsedChunksStatistics >>>> - SpaceManagerStatistics >>>> - ClassLoaderMetaspaceStatistics >>>> >>>> They live in a new file (memory/metaspace/metaspaceStatistics). >>>> >>>> I also ripped out the preexisting ChunkManagerStatistics, modified it >>>> to follow the same convention, and moved it into this new file. >>>> >>>> -> We now have a new central reporting function, >>>> MetaspaceUtils::print_report() - this was once Zhengyu's >>>> print_for_nmt() function, but I expanded it and renamed it to be the >>>> central entry point. Reporting options are handed into it via flags. >>>> >>>> In addition to that, we have a new >>>> MetaspaceUtils::print_basic_report(), which prints a short summary >>>> under the promise of not walking or locking anything. This is used at >>>> error reporting time to print a short summary to the hs-err file. It >>>> is also used for VM.info(). >>>> >>>> -> All new functions now accept the "scale" parameter known from NMT. >>>> I expanded it a bit to have a "dynamic" setting. In that setting, the >>>> printer chooses the best unit for a given value. I find it works quite >>>> well, so I made it the default. I also changed printing to correctly >>>> print very-small-but-not-0 numbers or percentages, where before they >>>> were rounded to 0. See linked output examples. >>>> >>>> -> Other things to note: >>>> >>>> - MetaspaceUtils::dump was used to print some details at Metaspace >>>> OOM. This now calls MetaspaceUtils::print_basic_report() directly, and >>>> I removed the dump() function. >>>> >>>> - I moved printing the summary out of the PrintCLDMetaspaceInfoClosure >>>> and into the print_report function. PrintCLDMetaspaceInfoClosure still >>>> prints the individual loader metaspace usage numbers. >>>> >>>> - I streamlined naming of values somewhat. Where we refer to in-use >>>> chunks, I now use consistently "capacity", "free", "waste", "used" and >>>> the new "overhead", and the assumption is that capacity is the sum of >>>> all the other values. >>>> >>>> - About the newly added namespaces: Since the patch is already quite >>>> big, I did not want to load it with more cleanup. However, neither did >>>> I want to just dump the new classes and functions into metaspace.cpp. >>>> So as a compromise I tried to follow a new scheme with newly added >>>> code. If you agree on this organization, this may be the way to split >>>> metaspace.cpp in some follow up change. >>>> >>>> The idea was to move all metaspace-internal stuff into >>>> memory/metaspace and hide it from the outside. Also, I did put >>>> metaspace coding into an own namespace ("metaspace") and >>>> metaspace-internal coding into "metaspace::internal". This is just a >>>> proposal, we can bike shed. Future changes could move more classes out >>>> of metaspace.cpp into metaspace/ and the metaspace::internal namespace >>>> to make metaspace.cpp malleable again. >>>> >>>> Thank you, >>>> >>>> Kind Regards, Thomas >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> -------------------- >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8201572 >>>> [2] >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.00/webrev/ >>>> [3] >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8202074-metaspace-retire-after-humongous-chunks-are-allocated/webrev.00/webrev/ > > From thomas.stuefe at gmail.com Wed May 2 06:06:01 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 2 May 2018 08:06:01 +0200 Subject: RFR(L): 8201572: Improve Metaspace Statistics In-Reply-To: References: Message-ID: Hi Zhengyu, New webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.04/webrev/ Delta to last: http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-03-to-04/webrev/ Comments below. On Tue, May 1, 2018 at 3:28 PM, Zhengyu Gu wrote: > Hi Thomas, > > This is a very useful feature, thanks for doing this. > > A few minor comments: > > - metaspace.cpp > L#178 - 182: indents for case/default stmts. Fixed. > L#184: is this assert still necessary? No, you are right. Removed. Note that it does not show up in the delta webrev, just the full. I made an mq mistake somewhere. > > - metaspace.hpp > L#103 and L#108: leftover? Yes. Fixed. > > - ostream.hpp > Not sure if print_human_readable_size()/print_percentage() belong > here. I put them there because I thought them to be useful general purpose functions, but I can generalize them in a separate change. I moved them to metaspaceCommon.hpp/cpp. > > Otherwise, looks good to me. > Great, thanks for the review! ..Thomas > Thanks, > > -Zhengyu > > > > > On 04/26/2018 02:03 AM, Thomas St?fe wrote: >> >> Hi all, >> >> I still need an additional reviewer. >> >> new Webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.02/webrev/index.html >> incremental to v1: >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-01-to-02/webrev/ >> >> new: just build- and test fixes: >> - I added final cr() to a number of places where LogStream was used to >> pipe output from functions I changed. Otherwise ~LogStream will assert >> complaining about unfinished lines. I plan to change this, see >> https://bugs.openjdk.java.net/browse/JDK-8202303, however for now, >> lets go with the additional cr() >> - build fixes for MacOS and Solaris. >> >> jdk-submit test suite ran clean through. We are currently running the >> patch through our test suites, so far all looks good. >> >> Thanks, Thomas >> >> >> >> >> On Tue, Apr 24, 2018 at 6:28 PM, Thomas St?fe >> wrote: >>> >>> Hi all, >>> >>> New webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-01/webrev/ >>> Incremental: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-00-to-01/webrev/ >>> >>> This one now cleanly applies to jdk-jdk and does not need other patches. >>> >>> Changes: >>> - removed MetaWord occurrences in ostream.cpp, replaces with BytesPerWord >>> - added per-spacetype printing to PrintNMTStatistics output to make it >>> more closely resemble the output before. Example output: >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/PrintNMTStatistics.txt >>> - Fixed Windows build (missed precompiled.hpp) >>> >>> Thanks, Thomas >>> >>> >>> On Mon, Apr 23, 2018 at 6:32 PM, Thomas St?fe >>> wrote: >>>> >>>> Hi all, >>>> >>>> may I please have opinions and reviews for the following improvement. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8201572 >>>> Webrev: >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.00/webrev/ >>>> >>>> (Note: it needs patch for 8202074 [3] to apply cleanly) >>>> >>>> This is a combined feature-add and cleanup for the metaspace >>>> reporting. It should have no effect on metaspace allocation itself. >>>> >>>> This change does a number of things: >>>> >>>> - It improves the "VM.metaspace" jcmd, making it hopefully much more >>>> versatile. >>>> >>>> - It improves the way statistics are collected: The number of times we >>>> walk the CLDG or the internal metaspace structures is reduced. It also >>>> adds another global counter to MetaspaceUtils. >>>> >>>> - Though this was not primarily intended as a cleanup change, a number >>>> of cleanups are part of this change. A number of functions were >>>> removed. Where new code was added, I attempted to define and follow a >>>> new scheme using namespaces and separate files. See below for details. >>>> >>>> --- >>>> >>>> Details: >>>> >>>> -> Improvements of jcmd: >>>> >>>> The jcmd "VM.metaspace" command now features a number of sub command >>>> which control the details printed: >>>> >>>> jcmd VM.metaspace [basic] [show-loaders] [by-chunktype] [by-spacetype] >>>> [vslist] [vsmap] [scale] >>>> >>>> All of them except for "basic" can be combined with each other. >>>> >>>> "basic" >>>> shows a basic report; it needs no CLDG walk and no locks, so it >>>> should be safe under most conditions. >>>> "show-loaders" >>>> this shows metaspace usage by loaders (basically what VM.metaspace >>>> did before - this was originally added by Zhengyu Gu) >>>> "by-chunktype" >>>> Numbers are broken down by chunk type (small, medium...) in addition >>>> to total numbers >>>> "by-spacetype" >>>> Numbers are broken down by space type (bootloader, anonymous, >>>> reflection...) in addition to total numbers >>>> "vslist" >>>> The virtual space list is printed in detail >>>> "vsmap" >>>> Prints the virtual space map (ascii map of chunk placements) >>>> >>>> When no arguments are given, a summary is displayed. >>>> >>>> In addition to the above mentioned features we now print >>>> - a "Waste" section detailing wastage. >>>> - in debug mode, a section with some internal counters. This was just >>>> curiosity on my part, I wanted to see how "hot" some of the metaspace >>>> allocation paths actually get. >>>> - We now count overhead too, i.e. space needed for chunk headers (was >>>> counted as "used" before) >>>> - We now also show deallocated space (block freelist) >>>> >>>> >>>> Some output examples: >>>> >>>> >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace.txt >>>> >>>> "basic" >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_basic.txt >>>> >>>> "by-chunktype" >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_by-chunktype.txt >>>> >>>> "by-spacetype" >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_by_spacetype.txt >>>> >>>> "show-loaders" >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_showloaders.txt >>>> >>>> "show-loaders" and "by-chunktype combined" >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_showloaders_by-chunktype.txt >>>> >>>> "vslist" >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_vslist.txt >>>> >>>> "vsmap" >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_vsmap.txt >>>> >>>> --- >>>> >>>> -> The way we collect statistics was streamlined. I tried to get rid >>>> of all functions which were walking structures to just sum up a single >>>> value; not only was that unnecessary code duplication, it was also >>>> inefficient. >>>> >>>> This means most of the SpaceManager::sum_xxxx() functions are gone, as >>>> well as all the different MetaspaceUtils::xxx_xxx_slow() functions. >>>> >>>> They were replaced by functions named "collect_statistics" or >>>> "add_statistics", which accumulate all needed data in one go into a >>>> statistics structure. This is more efficient since we usually need >>>> more than one value and so it makes sense to collect them in one go. >>>> >>>> The newly added statistics objects are: >>>> >>>> - UsedChunksStatistics >>>> - SpaceManagerStatistics >>>> - ClassLoaderMetaspaceStatistics >>>> >>>> They live in a new file (memory/metaspace/metaspaceStatistics). >>>> >>>> I also ripped out the preexisting ChunkManagerStatistics, modified it >>>> to follow the same convention, and moved it into this new file. >>>> >>>> -> We now have a new central reporting function, >>>> MetaspaceUtils::print_report() - this was once Zhengyu's >>>> print_for_nmt() function, but I expanded it and renamed it to be the >>>> central entry point. Reporting options are handed into it via flags. >>>> >>>> In addition to that, we have a new >>>> MetaspaceUtils::print_basic_report(), which prints a short summary >>>> under the promise of not walking or locking anything. This is used at >>>> error reporting time to print a short summary to the hs-err file. It >>>> is also used for VM.info(). >>>> >>>> -> All new functions now accept the "scale" parameter known from NMT. >>>> I expanded it a bit to have a "dynamic" setting. In that setting, the >>>> printer chooses the best unit for a given value. I find it works quite >>>> well, so I made it the default. I also changed printing to correctly >>>> print very-small-but-not-0 numbers or percentages, where before they >>>> were rounded to 0. See linked output examples. >>>> >>>> -> Other things to note: >>>> >>>> - MetaspaceUtils::dump was used to print some details at Metaspace >>>> OOM. This now calls MetaspaceUtils::print_basic_report() directly, and >>>> I removed the dump() function. >>>> >>>> - I moved printing the summary out of the PrintCLDMetaspaceInfoClosure >>>> and into the print_report function. PrintCLDMetaspaceInfoClosure still >>>> prints the individual loader metaspace usage numbers. >>>> >>>> - I streamlined naming of values somewhat. Where we refer to in-use >>>> chunks, I now use consistently "capacity", "free", "waste", "used" and >>>> the new "overhead", and the assumption is that capacity is the sum of >>>> all the other values. >>>> >>>> - About the newly added namespaces: Since the patch is already quite >>>> big, I did not want to load it with more cleanup. However, neither did >>>> I want to just dump the new classes and functions into metaspace.cpp. >>>> So as a compromise I tried to follow a new scheme with newly added >>>> code. If you agree on this organization, this may be the way to split >>>> metaspace.cpp in some follow up change. >>>> >>>> The idea was to move all metaspace-internal stuff into >>>> memory/metaspace and hide it from the outside. Also, I did put >>>> metaspace coding into an own namespace ("metaspace") and >>>> metaspace-internal coding into "metaspace::internal". This is just a >>>> proposal, we can bike shed. Future changes could move more classes out >>>> of metaspace.cpp into metaspace/ and the metaspace::internal namespace >>>> to make metaspace.cpp malleable again. >>>> >>>> Thank you, >>>> >>>> Kind Regards, Thomas >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> -------------------- >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8201572 >>>> [2] >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.00/webrev/ >>>> [3] >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8202074-metaspace-retire-after-humongous-chunks-are-allocated/webrev.00/webrev/ From david.holmes at oracle.com Wed May 2 07:19:27 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 2 May 2018 17:19:27 +1000 Subject: RFR(xxs): 8202303: LogStream should autoflush on destruction In-Reply-To: References: Message-ID: <255436cb-d990-8985-1e98-07e08ccf22c8@oracle.com> Hi Thomas, On 2/05/2018 3:26 PM, Thomas St?fe wrote: > Ping... > > this is a really simple one :) Please review. Thanks! Was hoping our logging experts would chime in here. This looks okay to me. My main concern would be that everything is torn down in the correct order for this to work reliably. Thanks, David > On Thu, Apr 26, 2018 at 8:34 AM, Thomas St?fe wrote: >> Hi all, >> >> please review this tiny change: >> >> https://bugs.openjdk.java.net/browse/JDK-8202303 >> http://cr.openjdk.java.net/~stuefe/webrevs/8202303-logstream-autoflush/webrev.00/webrev/ >> >> Basically, LogStream should not assert upon desctruction when having >> buffered unfinished lines; instead, it should autoflush. >> >> (Note: the changes to the TestLineBufferAllocation test are just to >> keep this test in line with the rest of the tests in regards to how >> the fake log file is handled). >> >> More details in my earlier mail to hs-runtime: >> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-April/027592.html >> >> Thanks, Thomas From robbin.ehn at oracle.com Wed May 2 09:41:49 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 2 May 2018 11:41:49 +0200 Subject: RFR(xxs): 8202303: LogStream should autoflush on destruction In-Reply-To: <255436cb-d990-8985-1e98-07e08ccf22c8@oracle.com> References: <255436cb-d990-8985-1e98-07e08ccf22c8@oracle.com> Message-ID: On 2018-05-02 09:19, David Holmes wrote: > Hi Thomas, > > On 2/05/2018 3:26 PM, Thomas St?fe wrote: >> Ping... >> >> this is a really simple one :) Please review. Thanks! > > Was hoping our logging experts would chime in here. > > This looks okay to me. My main concern would be that everything is torn down in > the correct order for this to work reliably. +1 I think it's okey since we always copy the the string to the internal buffer. Thanks, Robbin > > Thanks, > David > >> On Thu, Apr 26, 2018 at 8:34 AM, Thomas St?fe wrote: >>> Hi all, >>> >>> please review this tiny change: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8202303 >>> http://cr.openjdk.java.net/~stuefe/webrevs/8202303-logstream-autoflush/webrev.00/webrev/ >>> >>> >>> Basically, LogStream should not assert upon desctruction when having >>> buffered unfinished lines; instead, it should autoflush. >>> >>> (Note: the changes to the TestLineBufferAllocation test are just to >>> keep this test in line with the rest of the tests in regards to how >>> the fake log file is handled). >>> >>> More details in my earlier mail to hs-runtime: >>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-April/027592.html >>> >>> >>> Thanks, Thomas From robbin.ehn at oracle.com Wed May 2 10:23:15 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 2 May 2018 12:23:15 +0200 Subject: RFR(M): 8191798 redo nested ThreadsListHandle to drop Threads_lock In-Reply-To: References: <2fa7fff2-7fce-5255-a6a2-2c9066594a5c@oracle.com> Message-ID: <1870c0f9-e200-0425-58c9-802d7dfac1b6@oracle.com> On 2018-05-01 22:07, Daniel D. Daugherty wrote: > On 5/1/18 3:43 PM, Erik ?sterlund wrote: >> Hi Dan, >> >> In case my opinion counts, I think this looks great. +1 > > As is usual for Thread-SMR stuff, I plan to list both you and I as > contributors and as reviewers. Robbin reviewed internally and I > expect he will chime in here. David H often reviews Thread-SMR stuff > but he's a bit busy right now... I'm hoping Kim B. chimes in since > I think he ran into this issue with some of his recent changes... > > So thanks for the (re-)review! > > >> Thank you for the thorough testing and various improvements of this patch. Yes, thanks Dan! /Robbin > > You are quite welcome. > > Dan > > >> >> Thanks, >> /Erik >> >> On 2018-05-01 19:00, Daniel D. Daugherty wrote: >>> Greetings, >>> >>> We have a fix for the following Thread-SMR bug: >>> >>> ??? JDK-8191798 redo nested ThreadsListHandle to drop Threads_lock >>> ??? https://bugs.openjdk.java.net/browse/JDK-8191798 >>> >>> Erik O is the primary author for this fix; I've only made minor tweaks >>> here and there to the comments, the code and the tests. I have the >>> Thread-SMR stress testing setup on my Solaris-X64 server so I'm the >>> one shepherding the fix forward. >>> >>> Summary: Refactor Thread hazard ptrs and nested ThreadsLists into >>> ??? SafeThreadsListPtr. >>> >>> Hazard ptr management logic remains the same, but the nested ThreadsList >>> management logic uses a linked list of SafeThreadsListPtr and safe counters >>> instead of a linked list of NestedThreadsList and the Threads_lock. >>> >>> >>> Webrev URLs: >>> >>> http://cr.openjdk.java.net/~dcubed/8191798-webrev/1_for_jdk_jdk.full_no_code_motion/ >>> >>> ??? This is the webrev to look at if you want to see the entire >>> ??? proposed fix without code motion noise. >>> >>> http://cr.openjdk.java.net/~dcubed/8191798-webrev/1_for_jdk_jdk.code_motion_only/ >>> >>> ??? This webrev is only code motion and is here for completeness. >>> >>> http://cr.openjdk.java.net/~dcubed/8191798-webrev/1_for_jdk_jdk.full_with_code_motion/ >>> >>> ??? This webrev is everything and is here for completeness. >>> >>> Testing: >>> - Mach5 builds-tier1,jdk-tier1,jdk-tier2,jdk-tier3,hs-tier1,hs-tier2,hs-tier3 >>> ? - multiple rounds - no test failures >>> - Solaris X64 Thread-SMR stress testing - 2 x 24+ hour runs - no failures >>> ? related to these changes >>> >>> Thanks, in advance, for any comments, suggestions or questions. >>> >>> Dan and Erik >> > From zgu at redhat.com Wed May 2 10:26:29 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 2 May 2018 06:26:29 -0400 Subject: RFR(xxs): 8202303: LogStream should autoflush on destruction In-Reply-To: References: Message-ID: <1e878959-0840-9954-d8af-149e25d153d7@redhat.com> Looks good to me. -Zhengyu On 05/02/2018 01:26 AM, Thomas St?fe wrote: > Ping... > > this is a really simple one :) Please review. Thanks! > > On Thu, Apr 26, 2018 at 8:34 AM, Thomas St?fe wrote: >> Hi all, >> >> please review this tiny change: >> >> https://bugs.openjdk.java.net/browse/JDK-8202303 >> http://cr.openjdk.java.net/~stuefe/webrevs/8202303-logstream-autoflush/webrev.00/webrev/ >> >> Basically, LogStream should not assert upon desctruction when having >> buffered unfinished lines; instead, it should autoflush. >> >> (Note: the changes to the TestLineBufferAllocation test are just to >> keep this test in line with the rest of the tests in regards to how >> the fake log file is handled). >> >> More details in my earlier mail to hs-runtime: >> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-April/027592.html >> >> Thanks, Thomas From zgu at redhat.com Wed May 2 10:33:30 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 2 May 2018 06:33:30 -0400 Subject: RFR(L): 8201572: Improve Metaspace Statistics In-Reply-To: References: Message-ID: <1237b34e-3ee6-1582-eb35-81b65f155898@redhat.com> Looks good to me. Thanks, -Zhengyu On 05/02/2018 02:06 AM, Thomas St?fe wrote: > Hi Zhengyu, > > > New webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.04/webrev/ > Delta to last: http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-03-to-04/webrev/ > > Comments below. > > On Tue, May 1, 2018 at 3:28 PM, Zhengyu Gu wrote: >> Hi Thomas, >> >> This is a very useful feature, thanks for doing this. >> >> A few minor comments: >> >> - metaspace.cpp >> L#178 - 182: indents for case/default stmts. > > Fixed. > >> L#184: is this assert still necessary? > > No, you are right. Removed. Note that it does not show up in the delta > webrev, just the full. I made an mq mistake somewhere. > >> >> - metaspace.hpp >> L#103 and L#108: leftover? > > Yes. Fixed. > >> >> - ostream.hpp >> Not sure if print_human_readable_size()/print_percentage() belong >> here. > > I put them there because I thought them to be useful general purpose > functions, but I can generalize them in a separate change. I moved > them to metaspaceCommon.hpp/cpp. > >> >> Otherwise, looks good to me. >> > > Great, thanks for the review! > > ..Thomas > >> Thanks, >> >> -Zhengyu >> >> >> >> >> On 04/26/2018 02:03 AM, Thomas St?fe wrote: >>> >>> Hi all, >>> >>> I still need an additional reviewer. >>> >>> new Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.02/webrev/index.html >>> incremental to v1: >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-01-to-02/webrev/ >>> >>> new: just build- and test fixes: >>> - I added final cr() to a number of places where LogStream was used to >>> pipe output from functions I changed. Otherwise ~LogStream will assert >>> complaining about unfinished lines. I plan to change this, see >>> https://bugs.openjdk.java.net/browse/JDK-8202303, however for now, >>> lets go with the additional cr() >>> - build fixes for MacOS and Solaris. >>> >>> jdk-submit test suite ran clean through. We are currently running the >>> patch through our test suites, so far all looks good. >>> >>> Thanks, Thomas >>> >>> >>> >>> >>> On Tue, Apr 24, 2018 at 6:28 PM, Thomas St?fe >>> wrote: >>>> >>>> Hi all, >>>> >>>> New webrev: >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-01/webrev/ >>>> Incremental: >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-00-to-01/webrev/ >>>> >>>> This one now cleanly applies to jdk-jdk and does not need other patches. >>>> >>>> Changes: >>>> - removed MetaWord occurrences in ostream.cpp, replaces with BytesPerWord >>>> - added per-spacetype printing to PrintNMTStatistics output to make it >>>> more closely resemble the output before. Example output: >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/PrintNMTStatistics.txt >>>> - Fixed Windows build (missed precompiled.hpp) >>>> >>>> Thanks, Thomas >>>> >>>> >>>> On Mon, Apr 23, 2018 at 6:32 PM, Thomas St?fe >>>> wrote: >>>>> >>>>> Hi all, >>>>> >>>>> may I please have opinions and reviews for the following improvement. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8201572 >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.00/webrev/ >>>>> >>>>> (Note: it needs patch for 8202074 [3] to apply cleanly) >>>>> >>>>> This is a combined feature-add and cleanup for the metaspace >>>>> reporting. It should have no effect on metaspace allocation itself. >>>>> >>>>> This change does a number of things: >>>>> >>>>> - It improves the "VM.metaspace" jcmd, making it hopefully much more >>>>> versatile. >>>>> >>>>> - It improves the way statistics are collected: The number of times we >>>>> walk the CLDG or the internal metaspace structures is reduced. It also >>>>> adds another global counter to MetaspaceUtils. >>>>> >>>>> - Though this was not primarily intended as a cleanup change, a number >>>>> of cleanups are part of this change. A number of functions were >>>>> removed. Where new code was added, I attempted to define and follow a >>>>> new scheme using namespaces and separate files. See below for details. >>>>> >>>>> --- >>>>> >>>>> Details: >>>>> >>>>> -> Improvements of jcmd: >>>>> >>>>> The jcmd "VM.metaspace" command now features a number of sub command >>>>> which control the details printed: >>>>> >>>>> jcmd VM.metaspace [basic] [show-loaders] [by-chunktype] [by-spacetype] >>>>> [vslist] [vsmap] [scale] >>>>> >>>>> All of them except for "basic" can be combined with each other. >>>>> >>>>> "basic" >>>>> shows a basic report; it needs no CLDG walk and no locks, so it >>>>> should be safe under most conditions. >>>>> "show-loaders" >>>>> this shows metaspace usage by loaders (basically what VM.metaspace >>>>> did before - this was originally added by Zhengyu Gu) >>>>> "by-chunktype" >>>>> Numbers are broken down by chunk type (small, medium...) in addition >>>>> to total numbers >>>>> "by-spacetype" >>>>> Numbers are broken down by space type (bootloader, anonymous, >>>>> reflection...) in addition to total numbers >>>>> "vslist" >>>>> The virtual space list is printed in detail >>>>> "vsmap" >>>>> Prints the virtual space map (ascii map of chunk placements) >>>>> >>>>> When no arguments are given, a summary is displayed. >>>>> >>>>> In addition to the above mentioned features we now print >>>>> - a "Waste" section detailing wastage. >>>>> - in debug mode, a section with some internal counters. This was just >>>>> curiosity on my part, I wanted to see how "hot" some of the metaspace >>>>> allocation paths actually get. >>>>> - We now count overhead too, i.e. space needed for chunk headers (was >>>>> counted as "used" before) >>>>> - We now also show deallocated space (block freelist) >>>>> >>>>> >>>>> Some output examples: >>>>> >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace.txt >>>>> >>>>> "basic" >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_basic.txt >>>>> >>>>> "by-chunktype" >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_by-chunktype.txt >>>>> >>>>> "by-spacetype" >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_by_spacetype.txt >>>>> >>>>> "show-loaders" >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_showloaders.txt >>>>> >>>>> "show-loaders" and "by-chunktype combined" >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_showloaders_by-chunktype.txt >>>>> >>>>> "vslist" >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_vslist.txt >>>>> >>>>> "vsmap" >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_vsmap.txt >>>>> >>>>> --- >>>>> >>>>> -> The way we collect statistics was streamlined. I tried to get rid >>>>> of all functions which were walking structures to just sum up a single >>>>> value; not only was that unnecessary code duplication, it was also >>>>> inefficient. >>>>> >>>>> This means most of the SpaceManager::sum_xxxx() functions are gone, as >>>>> well as all the different MetaspaceUtils::xxx_xxx_slow() functions. >>>>> >>>>> They were replaced by functions named "collect_statistics" or >>>>> "add_statistics", which accumulate all needed data in one go into a >>>>> statistics structure. This is more efficient since we usually need >>>>> more than one value and so it makes sense to collect them in one go. >>>>> >>>>> The newly added statistics objects are: >>>>> >>>>> - UsedChunksStatistics >>>>> - SpaceManagerStatistics >>>>> - ClassLoaderMetaspaceStatistics >>>>> >>>>> They live in a new file (memory/metaspace/metaspaceStatistics). >>>>> >>>>> I also ripped out the preexisting ChunkManagerStatistics, modified it >>>>> to follow the same convention, and moved it into this new file. >>>>> >>>>> -> We now have a new central reporting function, >>>>> MetaspaceUtils::print_report() - this was once Zhengyu's >>>>> print_for_nmt() function, but I expanded it and renamed it to be the >>>>> central entry point. Reporting options are handed into it via flags. >>>>> >>>>> In addition to that, we have a new >>>>> MetaspaceUtils::print_basic_report(), which prints a short summary >>>>> under the promise of not walking or locking anything. This is used at >>>>> error reporting time to print a short summary to the hs-err file. It >>>>> is also used for VM.info(). >>>>> >>>>> -> All new functions now accept the "scale" parameter known from NMT. >>>>> I expanded it a bit to have a "dynamic" setting. In that setting, the >>>>> printer chooses the best unit for a given value. I find it works quite >>>>> well, so I made it the default. I also changed printing to correctly >>>>> print very-small-but-not-0 numbers or percentages, where before they >>>>> were rounded to 0. See linked output examples. >>>>> >>>>> -> Other things to note: >>>>> >>>>> - MetaspaceUtils::dump was used to print some details at Metaspace >>>>> OOM. This now calls MetaspaceUtils::print_basic_report() directly, and >>>>> I removed the dump() function. >>>>> >>>>> - I moved printing the summary out of the PrintCLDMetaspaceInfoClosure >>>>> and into the print_report function. PrintCLDMetaspaceInfoClosure still >>>>> prints the individual loader metaspace usage numbers. >>>>> >>>>> - I streamlined naming of values somewhat. Where we refer to in-use >>>>> chunks, I now use consistently "capacity", "free", "waste", "used" and >>>>> the new "overhead", and the assumption is that capacity is the sum of >>>>> all the other values. >>>>> >>>>> - About the newly added namespaces: Since the patch is already quite >>>>> big, I did not want to load it with more cleanup. However, neither did >>>>> I want to just dump the new classes and functions into metaspace.cpp. >>>>> So as a compromise I tried to follow a new scheme with newly added >>>>> code. If you agree on this organization, this may be the way to split >>>>> metaspace.cpp in some follow up change. >>>>> >>>>> The idea was to move all metaspace-internal stuff into >>>>> memory/metaspace and hide it from the outside. Also, I did put >>>>> metaspace coding into an own namespace ("metaspace") and >>>>> metaspace-internal coding into "metaspace::internal". This is just a >>>>> proposal, we can bike shed. Future changes could move more classes out >>>>> of metaspace.cpp into metaspace/ and the metaspace::internal namespace >>>>> to make metaspace.cpp malleable again. >>>>> >>>>> Thank you, >>>>> >>>>> Kind Regards, Thomas >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -------------------- >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8201572 >>>>> [2] >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.00/webrev/ >>>>> [3] >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8202074-metaspace-retire-after-humongous-chunks-are-allocated/webrev.00/webrev/ From goetz.lindenmaier at sap.com Wed May 2 11:21:07 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 2 May 2018 11:21:07 +0000 Subject: [ping] RE: RFR(M): 8199852: Print more information about class loaders in LinkageErrors. In-Reply-To: References: <0e13b51363ef4b6f9aece78e3a332afb@sap.com> Message-ID: <5b99bafb1ceb440a8892676dcc1a3ec6@sap.com> Hi David, > Sorry really strapped for time these days ... This is ok, but I will keep pinging ... ?? A great, general thanks to you, as you really spend a lot of effort on reviewing changes by SAP. That helps us a lot and we appreciate your efforts! > src/hotspot/share/classfile/javaClasses.cpp > src/hotspot/share/classfile/javaClasses.hpp > The description of describe_external in the hpp doesn't match the > implementation in the cpp: parent: x versus child of x I updated the comment: - // Prints "" (instance of , parent: "" ) - // or (parent: "" ). + // Prints "" (instance of , child of "" ). + // If a classloader has no name, it prints instead. The output + // for well known loaders (system/platform) is abbreviated. > src/hotspot/share/classfile/systemDictionary.cpp > > 2208 throwException = true; > 2209 ss.print("loader constraint violation: loader %s", > 2210 > java_lang_ClassLoader::describe_external(class_loader())); > 2211 ss.print(" wants to load %s %s.", > 2212 k->external_kind(), k->external_name()); > 2213 Klass *existing_klass = > constraints()->find_constrained_klass(name, class_loader); > 2214 if (existing_klass->class_loader() != class_loader()) { > 2215 ss.print(" A different %s with the same name was > previously loaded by %s.", > 2216 existing_klass->external_kind(), > 2217 > java_lang_ClassLoader::describe_external(existing_klass->class_loader())); > 2218 } > > I still find this way too verbose. I won't oppose it but I don't like > it. I know you're trying to solve your remote support problem, but I > worry about the newbie trying to learn and experiment and being totally > bamboozled by the exception message. (I thought the old ones were bad > enough - but necessary in their level of preciseness.) I understand. Thanks for not opposing. > 3125 // Caller needs ResourceMark. > > Nit: Why change this line but not line 3131? In general single-line > comments don't need to be written as grammatically correct sentences > with full punctuation. Only if it is a multi-sentence comment should you > need to do that. Reverted, leftover from my other edits in that code that were removed. > Only comment I'll make on the tests is that being too specific about the > exact message makes the tests more fragile. Yes, but there are so many details in the message I would like to assure (like the proper name of classes with '.') that I think this is necessary. I'll push the change through the jdk-submit forest. (It's through all our tests, anyways). After pushing, I will also update the bug to list the final exception messages. Can I consider it reviewed by you, then? Best regards, Goetz. > > Thanks, > David > ----- > > > On 26/04/2018 1:30 AM, Lindenmaier, Goetz wrote: > > Hi Lois, David, George or others, > > > > I would like to finish this one up, could I please get some > > (hopefully final) reviews? > > > > Thanks, > > Goetz. > > > >> -----Original Message----- > >> From: Lindenmaier, Goetz > >> Sent: Mittwoch, 18. April 2018 09:55 > >> To: 'David Holmes' ; 'hotspot-runtime- > >> dev at openjdk.java.net' > >> Subject: RE: RFR(M): 8199852: Print more information about class loaders > in > >> LinkageErrors. > >> > >> Hi, > >> > >> I rebased the webrev to the jdk repo: > >> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03-jdk/ > >> Could I please get the final go? > >> > >> Thanks and best regards, > >> Goetz. > >> > >> > >>> -----Original Message----- > >>> From: Lindenmaier, Goetz > >>> Sent: Freitag, 13. April 2018 10:10 > >>> To: 'David Holmes' ; hotspot-runtime- > >>> dev at openjdk.java.net > >>> Subject: RE: RFR(M): 8199852: Print more information about class loaders > in > >>> LinkageErrors. > >>> > >>> Hi David, > >>> > >>> I fixed what you mentioned here. I also found an older mail > >>> of yours with some comments I implemented now. I copied > >>> that mail's content here to have only one mail ... find references > >>> to some incremental webrevs in my replies below. > >>> > >>> Further I moved the test classes of test differentLE/ into a package. > >>> > >>> Comprehensive new webrev: > >>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03 > >>> > >>> > >>>> -----Original Message----- > >>>> From: David Holmes > >>>> Sent: Thursday, April 12, 2018 8:50 AM > >>>> To: Lindenmaier, Goetz ; hotspot- > >> runtime- > >>>> dev at openjdk.java.net > >>> ... > >>>> This looks reasonable to me. Two minor comments: > >>>> - updated tests need updated copyright notice > >>>> - I think you can avoid duplicating test1() to test2() by passing in the > >>>> loader name and the expected message as parameters > >>> Fixed both. Incremental webrev: > >>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- > >>> incremental2/ > >>> > >>> > >>>> -----Original Message----- > >>>> From: David Holmes > >>>> Sent: Sunday, March 25, 2018 11:49 PM > >>>> To: Lindenmaier, Goetz ; hotspot- > >> runtime- > >>>> dev at openjdk.java.net > >>> ... > >>>>> Maybe I should suppress printing verbose information about these > >>>>> well known loaders. (There is is_builtin_class_loader_data() to > >>>>> identify such.) > >>>> > >>>> Yes it might be good to simplify things for the predefined loaders. > >>>> Though I'd use: > >>>> SystemDictionary::is_platform_class_loader > >>>> SystemDictionary::is_system_class_loader > >>>> for that check. > >>> I implemented omitting the parent for these loaders. > >>> > >>>> Maybe it makes sense to have a set pattern for this descriptive text, > >>>> and use "unnamed" if the loader has no name: > >>>> > >>>> loader , instanceof , child of loader ... > >>> I implemented printing "" for empty names. I would like > >>> To keep the format with brackets, because the enclosing text might use > >>> commas, too, and then the sentence structure might get confusing. > >>> > >>> See this incremental webrev for these two changes: > >>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- > >>> incremental1a/ > >>> > >>>> Add this bug to the @bug in the test(s). > >>> Done. > >>> > >>>>> I'm not clear where you distinguish type and class. > >>>> Using "class" is inaccurate as it can be a class, interface or array type. > >>>> > >>>> In the above the entity is a "type" as we are referring to a >>>> classloader> pair. But you can also just read it as "class or interface > >>>> or array type" > >>> So why not print what it really is? I had added external_kind() for this in > >>> My previous exception message change. > >>> Unfortunately only in SystemDictionary::check_constraints() this is easily > >>> possible, and it improves the message I think. > >>> > >>> Incremental webrev for these two changes: > >>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- > >>> incremental1b/ > >>> > >>> I also implemented a test for the call to external_kind(), but I missed to > put > >>> that into > >>> the incremental webrev. > >>> > >>> Best regards, > >>> Goetz. > > From coleen.phillimore at oracle.com Wed May 2 11:58:13 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 2 May 2018 07:58:13 -0400 Subject: RFR(L): 8201572: Improve Metaspace Statistics In-Reply-To: References: <9c8e564f-86c7-7733-0c7b-0871da853953@oracle.com> Message-ID: <62cf88d3-e2c4-8f2e-82e4-1a3c9840a628@oracle.com> On 5/2/18 1:36 AM, Thomas St?fe wrote: > Hi Coleen, > > On Tue, May 1, 2018 at 3:54 PM, wrote: >> Hi, This isn't a full review but I like that you've added a metaspace >> namespace and directory under memory for it. >> This will make a good foundation for doing this change: >> >> https://bugs.openjdk.java.net/browse/JDK-8176808 >> > Yes, that is what I thought. > > When ripping up metaspace.cpp, should we be concerned that we loose > history for the ripped out code? The history will be there if one looks for it, or in older repositories. thanks, Coleen > > ..Thomas > >> Thanks, >> Coleen >> >> >> On 4/26/18 2:03 AM, Thomas St?fe wrote: >>> Hi all, >>> >>> I still need an additional reviewer. >>> >>> new Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.02/webrev/index.html >>> incremental to v1: >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-01-to-02/webrev/ >>> >>> new: just build- and test fixes: >>> - I added final cr() to a number of places where LogStream was used to >>> pipe output from functions I changed. Otherwise ~LogStream will assert >>> complaining about unfinished lines. I plan to change this, see >>> https://bugs.openjdk.java.net/browse/JDK-8202303, however for now, >>> lets go with the additional cr() >>> - build fixes for MacOS and Solaris. >>> >>> jdk-submit test suite ran clean through. We are currently running the >>> patch through our test suites, so far all looks good. >>> >>> Thanks, Thomas >>> >>> >>> >>> >>> On Tue, Apr 24, 2018 at 6:28 PM, Thomas St?fe >>> wrote: >>>> Hi all, >>>> >>>> New webrev: >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-01/webrev/ >>>> Incremental: >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev-00-to-01/webrev/ >>>> >>>> This one now cleanly applies to jdk-jdk and does not need other patches. >>>> >>>> Changes: >>>> - removed MetaWord occurrences in ostream.cpp, replaces with BytesPerWord >>>> - added per-spacetype printing to PrintNMTStatistics output to make it >>>> more closely resemble the output before. Example output: >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/PrintNMTStatistics.txt >>>> - Fixed Windows build (missed precompiled.hpp) >>>> >>>> Thanks, Thomas >>>> >>>> >>>> On Mon, Apr 23, 2018 at 6:32 PM, Thomas St?fe >>>> wrote: >>>>> Hi all, >>>>> >>>>> may I please have opinions and reviews for the following improvement. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8201572 >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.00/webrev/ >>>>> >>>>> (Note: it needs patch for 8202074 [3] to apply cleanly) >>>>> >>>>> This is a combined feature-add and cleanup for the metaspace >>>>> reporting. It should have no effect on metaspace allocation itself. >>>>> >>>>> This change does a number of things: >>>>> >>>>> - It improves the "VM.metaspace" jcmd, making it hopefully much more >>>>> versatile. >>>>> >>>>> - It improves the way statistics are collected: The number of times we >>>>> walk the CLDG or the internal metaspace structures is reduced. It also >>>>> adds another global counter to MetaspaceUtils. >>>>> >>>>> - Though this was not primarily intended as a cleanup change, a number >>>>> of cleanups are part of this change. A number of functions were >>>>> removed. Where new code was added, I attempted to define and follow a >>>>> new scheme using namespaces and separate files. See below for details. >>>>> >>>>> --- >>>>> >>>>> Details: >>>>> >>>>> -> Improvements of jcmd: >>>>> >>>>> The jcmd "VM.metaspace" command now features a number of sub command >>>>> which control the details printed: >>>>> >>>>> jcmd VM.metaspace [basic] [show-loaders] [by-chunktype] [by-spacetype] >>>>> [vslist] [vsmap] [scale] >>>>> >>>>> All of them except for "basic" can be combined with each other. >>>>> >>>>> "basic" >>>>> shows a basic report; it needs no CLDG walk and no locks, so it >>>>> should be safe under most conditions. >>>>> "show-loaders" >>>>> this shows metaspace usage by loaders (basically what VM.metaspace >>>>> did before - this was originally added by Zhengyu Gu) >>>>> "by-chunktype" >>>>> Numbers are broken down by chunk type (small, medium...) in addition >>>>> to total numbers >>>>> "by-spacetype" >>>>> Numbers are broken down by space type (bootloader, anonymous, >>>>> reflection...) in addition to total numbers >>>>> "vslist" >>>>> The virtual space list is printed in detail >>>>> "vsmap" >>>>> Prints the virtual space map (ascii map of chunk placements) >>>>> >>>>> When no arguments are given, a summary is displayed. >>>>> >>>>> In addition to the above mentioned features we now print >>>>> - a "Waste" section detailing wastage. >>>>> - in debug mode, a section with some internal counters. This was just >>>>> curiosity on my part, I wanted to see how "hot" some of the metaspace >>>>> allocation paths actually get. >>>>> - We now count overhead too, i.e. space needed for chunk headers (was >>>>> counted as "used" before) >>>>> - We now also show deallocated space (block freelist) >>>>> >>>>> >>>>> Some output examples: >>>>> >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace.txt >>>>> >>>>> "basic" >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_basic.txt >>>>> >>>>> "by-chunktype" >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_by-chunktype.txt >>>>> >>>>> "by-spacetype" >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_by_spacetype.txt >>>>> >>>>> "show-loaders" >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_showloaders.txt >>>>> >>>>> "show-loaders" and "by-chunktype combined" >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_showloaders_by-chunktype.txt >>>>> >>>>> "vslist" >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_vslist.txt >>>>> >>>>> "vsmap" >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/examples/VM.metaspace_vsmap.txt >>>>> >>>>> --- >>>>> >>>>> -> The way we collect statistics was streamlined. I tried to get rid >>>>> of all functions which were walking structures to just sum up a single >>>>> value; not only was that unnecessary code duplication, it was also >>>>> inefficient. >>>>> >>>>> This means most of the SpaceManager::sum_xxxx() functions are gone, as >>>>> well as all the different MetaspaceUtils::xxx_xxx_slow() functions. >>>>> >>>>> They were replaced by functions named "collect_statistics" or >>>>> "add_statistics", which accumulate all needed data in one go into a >>>>> statistics structure. This is more efficient since we usually need >>>>> more than one value and so it makes sense to collect them in one go. >>>>> >>>>> The newly added statistics objects are: >>>>> >>>>> - UsedChunksStatistics >>>>> - SpaceManagerStatistics >>>>> - ClassLoaderMetaspaceStatistics >>>>> >>>>> They live in a new file (memory/metaspace/metaspaceStatistics). >>>>> >>>>> I also ripped out the preexisting ChunkManagerStatistics, modified it >>>>> to follow the same convention, and moved it into this new file. >>>>> >>>>> -> We now have a new central reporting function, >>>>> MetaspaceUtils::print_report() - this was once Zhengyu's >>>>> print_for_nmt() function, but I expanded it and renamed it to be the >>>>> central entry point. Reporting options are handed into it via flags. >>>>> >>>>> In addition to that, we have a new >>>>> MetaspaceUtils::print_basic_report(), which prints a short summary >>>>> under the promise of not walking or locking anything. This is used at >>>>> error reporting time to print a short summary to the hs-err file. It >>>>> is also used for VM.info(). >>>>> >>>>> -> All new functions now accept the "scale" parameter known from NMT. >>>>> I expanded it a bit to have a "dynamic" setting. In that setting, the >>>>> printer chooses the best unit for a given value. I find it works quite >>>>> well, so I made it the default. I also changed printing to correctly >>>>> print very-small-but-not-0 numbers or percentages, where before they >>>>> were rounded to 0. See linked output examples. >>>>> >>>>> -> Other things to note: >>>>> >>>>> - MetaspaceUtils::dump was used to print some details at Metaspace >>>>> OOM. This now calls MetaspaceUtils::print_basic_report() directly, and >>>>> I removed the dump() function. >>>>> >>>>> - I moved printing the summary out of the PrintCLDMetaspaceInfoClosure >>>>> and into the print_report function. PrintCLDMetaspaceInfoClosure still >>>>> prints the individual loader metaspace usage numbers. >>>>> >>>>> - I streamlined naming of values somewhat. Where we refer to in-use >>>>> chunks, I now use consistently "capacity", "free", "waste", "used" and >>>>> the new "overhead", and the assumption is that capacity is the sum of >>>>> all the other values. >>>>> >>>>> - About the newly added namespaces: Since the patch is already quite >>>>> big, I did not want to load it with more cleanup. However, neither did >>>>> I want to just dump the new classes and functions into metaspace.cpp. >>>>> So as a compromise I tried to follow a new scheme with newly added >>>>> code. If you agree on this organization, this may be the way to split >>>>> metaspace.cpp in some follow up change. >>>>> >>>>> The idea was to move all metaspace-internal stuff into >>>>> memory/metaspace and hide it from the outside. Also, I did put >>>>> metaspace coding into an own namespace ("metaspace") and >>>>> metaspace-internal coding into "metaspace::internal". This is just a >>>>> proposal, we can bike shed. Future changes could move more classes out >>>>> of metaspace.cpp into metaspace/ and the metaspace::internal namespace >>>>> to make metaspace.cpp malleable again. >>>>> >>>>> Thank you, >>>>> >>>>> Kind Regards, Thomas >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -------------------- >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8201572 >>>>> [2] >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8201572-improve-metaspace-reporting/webrev.00/webrev/ >>>>> [3] >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8202074-metaspace-retire-after-humongous-chunks-are-allocated/webrev.00/webrev/ >> From david.holmes at oracle.com Wed May 2 12:31:41 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 2 May 2018 22:31:41 +1000 Subject: [ping] RE: RFR(M): 8199852: Print more information about class loaders in LinkageErrors. In-Reply-To: <5b99bafb1ceb440a8892676dcc1a3ec6@sap.com> References: <0e13b51363ef4b6f9aece78e3a332afb@sap.com> <5b99bafb1ceb440a8892676dcc1a3ec6@sap.com> Message-ID: <792d97cc-0e32-4d81-4838-5cc958b7242c@oracle.com> Hi Goetz, Yes you can consider this Reviewed. Thanks, David On 2/05/2018 9:21 PM, Lindenmaier, Goetz wrote: > Hi David, > >> Sorry really strapped for time these days ... > This is ok, but I will keep pinging ... ?? > A great, general thanks to you, as you really spend a lot of effort on > reviewing changes by SAP. That helps us a lot and we appreciate > your efforts! > >> src/hotspot/share/classfile/javaClasses.cpp >> src/hotspot/share/classfile/javaClasses.hpp >> The description of describe_external in the hpp doesn't match the >> implementation in the cpp: parent: x versus child of x > I updated the comment: > - // Prints "" (instance of , parent: "" ) > - // or (parent: "" ). > + // Prints "" (instance of , child of "" ). > + // If a classloader has no name, it prints instead. The output > + // for well known loaders (system/platform) is abbreviated. > >> src/hotspot/share/classfile/systemDictionary.cpp >> >> 2208 throwException = true; >> 2209 ss.print("loader constraint violation: loader %s", >> 2210 >> java_lang_ClassLoader::describe_external(class_loader())); >> 2211 ss.print(" wants to load %s %s.", >> 2212 k->external_kind(), k->external_name()); >> 2213 Klass *existing_klass = >> constraints()->find_constrained_klass(name, class_loader); >> 2214 if (existing_klass->class_loader() != class_loader()) { >> 2215 ss.print(" A different %s with the same name was >> previously loaded by %s.", >> 2216 existing_klass->external_kind(), >> 2217 >> java_lang_ClassLoader::describe_external(existing_klass->class_loader())); >> 2218 } >> >> I still find this way too verbose. I won't oppose it but I don't like >> it. I know you're trying to solve your remote support problem, but I >> worry about the newbie trying to learn and experiment and being totally >> bamboozled by the exception message. (I thought the old ones were bad >> enough - but necessary in their level of preciseness.) > I understand. Thanks for not opposing. > > >> 3125 // Caller needs ResourceMark. >> >> Nit: Why change this line but not line 3131? In general single-line >> comments don't need to be written as grammatically correct sentences >> with full punctuation. Only if it is a multi-sentence comment should you >> need to do that. > Reverted, leftover from my other edits in that code that were removed. > >> Only comment I'll make on the tests is that being too specific about the >> exact message makes the tests more fragile. > Yes, but there are so many details in the message I would like to > assure (like the proper name of classes with '.') that I think this is > necessary. > > I'll push the change through the jdk-submit forest. (It's through all > our tests, anyways). After pushing, I will also update the bug to > list the final exception messages. > Can I consider it reviewed by you, then? > > Best regards, > Goetz. > > >> >> Thanks, >> David >> ----- >> >> >> On 26/04/2018 1:30 AM, Lindenmaier, Goetz wrote: >>> Hi Lois, David, George or others, >>> >>> I would like to finish this one up, could I please get some >>> (hopefully final) reviews? >>> >>> Thanks, >>> Goetz. >>> >>>> -----Original Message----- >>>> From: Lindenmaier, Goetz >>>> Sent: Mittwoch, 18. April 2018 09:55 >>>> To: 'David Holmes' ; 'hotspot-runtime- >>>> dev at openjdk.java.net' >>>> Subject: RE: RFR(M): 8199852: Print more information about class loaders >> in >>>> LinkageErrors. >>>> >>>> Hi, >>>> >>>> I rebased the webrev to the jdk repo: >>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03-jdk/ >>>> Could I please get the final go? >>>> >>>> Thanks and best regards, >>>> Goetz. >>>> >>>> >>>>> -----Original Message----- >>>>> From: Lindenmaier, Goetz >>>>> Sent: Freitag, 13. April 2018 10:10 >>>>> To: 'David Holmes' ; hotspot-runtime- >>>>> dev at openjdk.java.net >>>>> Subject: RE: RFR(M): 8199852: Print more information about class loaders >> in >>>>> LinkageErrors. >>>>> >>>>> Hi David, >>>>> >>>>> I fixed what you mentioned here. I also found an older mail >>>>> of yours with some comments I implemented now. I copied >>>>> that mail's content here to have only one mail ... find references >>>>> to some incremental webrevs in my replies below. >>>>> >>>>> Further I moved the test classes of test differentLE/ into a package. >>>>> >>>>> Comprehensive new webrev: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03 >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: David Holmes >>>>>> Sent: Thursday, April 12, 2018 8:50 AM >>>>>> To: Lindenmaier, Goetz ; hotspot- >>>> runtime- >>>>>> dev at openjdk.java.net >>>>> ... >>>>>> This looks reasonable to me. Two minor comments: >>>>>> - updated tests need updated copyright notice >>>>>> - I think you can avoid duplicating test1() to test2() by passing in the >>>>>> loader name and the expected message as parameters >>>>> Fixed both. Incremental webrev: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- >>>>> incremental2/ >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: David Holmes >>>>>> Sent: Sunday, March 25, 2018 11:49 PM >>>>>> To: Lindenmaier, Goetz ; hotspot- >>>> runtime- >>>>>> dev at openjdk.java.net >>>>> ... >>>>>>> Maybe I should suppress printing verbose information about these >>>>>>> well known loaders. (There is is_builtin_class_loader_data() to >>>>>>> identify such.) >>>>>> >>>>>> Yes it might be good to simplify things for the predefined loaders. >>>>>> Though I'd use: >>>>>> SystemDictionary::is_platform_class_loader >>>>>> SystemDictionary::is_system_class_loader >>>>>> for that check. >>>>> I implemented omitting the parent for these loaders. >>>>> >>>>>> Maybe it makes sense to have a set pattern for this descriptive text, >>>>>> and use "unnamed" if the loader has no name: >>>>>> >>>>>> loader , instanceof , child of loader ... >>>>> I implemented printing "" for empty names. I would like >>>>> To keep the format with brackets, because the enclosing text might use >>>>> commas, too, and then the sentence structure might get confusing. >>>>> >>>>> See this incremental webrev for these two changes: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- >>>>> incremental1a/ >>>>> >>>>>> Add this bug to the @bug in the test(s). >>>>> Done. >>>>> >>>>>>> I'm not clear where you distinguish type and class. >>>>>> Using "class" is inaccurate as it can be a class, interface or array type. >>>>>> >>>>>> In the above the entity is a "type" as we are referring to a >>>>> classloader> pair. But you can also just read it as "class or interface >>>>>> or array type" >>>>> So why not print what it really is? I had added external_kind() for this in >>>>> My previous exception message change. >>>>> Unfortunately only in SystemDictionary::check_constraints() this is easily >>>>> possible, and it improves the message I think. >>>>> >>>>> Incremental webrev for these two changes: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- >>>>> incremental1b/ >>>>> >>>>> I also implemented a test for the call to external_kind(), but I missed to >> put >>>>> that into >>>>> the incremental webrev. >>>>> >>>>> Best regards, >>>>> Goetz. >>> From daniel.daugherty at oracle.com Wed May 2 12:43:04 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 2 May 2018 08:43:04 -0400 Subject: RFR(M): 8191798 redo nested ThreadsListHandle to drop Threads_lock In-Reply-To: <1870c0f9-e200-0425-58c9-802d7dfac1b6@oracle.com> References: <2fa7fff2-7fce-5255-a6a2-2c9066594a5c@oracle.com> <1870c0f9-e200-0425-58c9-802d7dfac1b6@oracle.com> Message-ID: On 5/2/18 6:23 AM, Robbin Ehn wrote: > On 2018-05-01 22:07, Daniel D. Daugherty wrote: >> On 5/1/18 3:43 PM, Erik ?sterlund wrote: >>> Hi Dan, >>> >>> In case my opinion counts, I think this looks great. > > +1 Thanks for the re-review! > >> >> As is usual for Thread-SMR stuff, I plan to list both you and I as >> contributors and as reviewers. Robbin reviewed internally and I >> expect he will chime in here. David H often reviews Thread-SMR stuff >> but he's a bit busy right now... I'm hoping Kim B. chimes in since >> I think he ran into this issue with some of his recent changes... >> >> So thanks for the (re-)review! >> >> >>> Thank you for the thorough testing and various improvements of this >>> patch. > > Yes, thanks Dan! You are also quite welcome. Dan > > /Robbin > >> >> You are quite welcome. >> >> Dan >> >> >>> >>> Thanks, >>> /Erik >>> >>> On 2018-05-01 19:00, Daniel D. Daugherty wrote: >>>> Greetings, >>>> >>>> We have a fix for the following Thread-SMR bug: >>>> >>>> ??? JDK-8191798 redo nested ThreadsListHandle to drop Threads_lock >>>> ??? https://bugs.openjdk.java.net/browse/JDK-8191798 >>>> >>>> Erik O is the primary author for this fix; I've only made minor tweaks >>>> here and there to the comments, the code and the tests. I have the >>>> Thread-SMR stress testing setup on my Solaris-X64 server so I'm the >>>> one shepherding the fix forward. >>>> >>>> Summary: Refactor Thread hazard ptrs and nested ThreadsLists into >>>> ??? SafeThreadsListPtr. >>>> >>>> Hazard ptr management logic remains the same, but the nested >>>> ThreadsList >>>> management logic uses a linked list of SafeThreadsListPtr and safe >>>> counters >>>> instead of a linked list of NestedThreadsList and the Threads_lock. >>>> >>>> >>>> Webrev URLs: >>>> >>>> http://cr.openjdk.java.net/~dcubed/8191798-webrev/1_for_jdk_jdk.full_no_code_motion/ >>>> >>>> ??? This is the webrev to look at if you want to see the entire >>>> ??? proposed fix without code motion noise. >>>> >>>> http://cr.openjdk.java.net/~dcubed/8191798-webrev/1_for_jdk_jdk.code_motion_only/ >>>> >>>> ??? This webrev is only code motion and is here for completeness. >>>> >>>> http://cr.openjdk.java.net/~dcubed/8191798-webrev/1_for_jdk_jdk.full_with_code_motion/ >>>> >>>> ??? This webrev is everything and is here for completeness. >>>> >>>> Testing: >>>> - Mach5 >>>> builds-tier1,jdk-tier1,jdk-tier2,jdk-tier3,hs-tier1,hs-tier2,hs-tier3 >>>> ? - multiple rounds - no test failures >>>> - Solaris X64 Thread-SMR stress testing - 2 x 24+ hour runs - no >>>> failures >>>> ? related to these changes >>>> >>>> Thanks, in advance, for any comments, suggestions or questions. >>>> >>>> Dan and Erik >>> >> From david.holmes at oracle.com Wed May 2 12:56:43 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 2 May 2018 22:56:43 +1000 Subject: [ping] RE: RFR(M): 8199852: Print more information about class loaders in LinkageErrors. In-Reply-To: <5b99bafb1ceb440a8892676dcc1a3ec6@sap.com> References: <0e13b51363ef4b6f9aece78e3a332afb@sap.com> <5b99bafb1ceb440a8892676dcc1a3ec6@sap.com> Message-ID: <541d524f-1d40-ade5-1751-86a8df3d9f43@oracle.com> Ps. On 2/05/2018 9:21 PM, Lindenmaier, Goetz wrote: > Hi David, > >> Sorry really strapped for time these days ... > This is ok, but I will keep pinging ... ?? > A great, general thanks to you, as you really spend a lot of effort on > reviewing changes by SAP. That helps us a lot and we appreciate > your efforts! Thanks for that. Cheers, David ----- >> src/hotspot/share/classfile/javaClasses.cpp >> src/hotspot/share/classfile/javaClasses.hpp >> The description of describe_external in the hpp doesn't match the >> implementation in the cpp: parent: x versus child of x > I updated the comment: > - // Prints "" (instance of , parent: "" ) > - // or (parent: "" ). > + // Prints "" (instance of , child of "" ). > + // If a classloader has no name, it prints instead. The output > + // for well known loaders (system/platform) is abbreviated. > >> src/hotspot/share/classfile/systemDictionary.cpp >> >> 2208 throwException = true; >> 2209 ss.print("loader constraint violation: loader %s", >> 2210 >> java_lang_ClassLoader::describe_external(class_loader())); >> 2211 ss.print(" wants to load %s %s.", >> 2212 k->external_kind(), k->external_name()); >> 2213 Klass *existing_klass = >> constraints()->find_constrained_klass(name, class_loader); >> 2214 if (existing_klass->class_loader() != class_loader()) { >> 2215 ss.print(" A different %s with the same name was >> previously loaded by %s.", >> 2216 existing_klass->external_kind(), >> 2217 >> java_lang_ClassLoader::describe_external(existing_klass->class_loader())); >> 2218 } >> >> I still find this way too verbose. I won't oppose it but I don't like >> it. I know you're trying to solve your remote support problem, but I >> worry about the newbie trying to learn and experiment and being totally >> bamboozled by the exception message. (I thought the old ones were bad >> enough - but necessary in their level of preciseness.) > I understand. Thanks for not opposing. > > >> 3125 // Caller needs ResourceMark. >> >> Nit: Why change this line but not line 3131? In general single-line >> comments don't need to be written as grammatically correct sentences >> with full punctuation. Only if it is a multi-sentence comment should you >> need to do that. > Reverted, leftover from my other edits in that code that were removed. > >> Only comment I'll make on the tests is that being too specific about the >> exact message makes the tests more fragile. > Yes, but there are so many details in the message I would like to > assure (like the proper name of classes with '.') that I think this is > necessary. > > I'll push the change through the jdk-submit forest. (It's through all > our tests, anyways). After pushing, I will also update the bug to > list the final exception messages. > Can I consider it reviewed by you, then? > > Best regards, > Goetz. > > >> >> Thanks, >> David >> ----- >> >> >> On 26/04/2018 1:30 AM, Lindenmaier, Goetz wrote: >>> Hi Lois, David, George or others, >>> >>> I would like to finish this one up, could I please get some >>> (hopefully final) reviews? >>> >>> Thanks, >>> Goetz. >>> >>>> -----Original Message----- >>>> From: Lindenmaier, Goetz >>>> Sent: Mittwoch, 18. April 2018 09:55 >>>> To: 'David Holmes' ; 'hotspot-runtime- >>>> dev at openjdk.java.net' >>>> Subject: RE: RFR(M): 8199852: Print more information about class loaders >> in >>>> LinkageErrors. >>>> >>>> Hi, >>>> >>>> I rebased the webrev to the jdk repo: >>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03-jdk/ >>>> Could I please get the final go? >>>> >>>> Thanks and best regards, >>>> Goetz. >>>> >>>> >>>>> -----Original Message----- >>>>> From: Lindenmaier, Goetz >>>>> Sent: Freitag, 13. April 2018 10:10 >>>>> To: 'David Holmes' ; hotspot-runtime- >>>>> dev at openjdk.java.net >>>>> Subject: RE: RFR(M): 8199852: Print more information about class loaders >> in >>>>> LinkageErrors. >>>>> >>>>> Hi David, >>>>> >>>>> I fixed what you mentioned here. I also found an older mail >>>>> of yours with some comments I implemented now. I copied >>>>> that mail's content here to have only one mail ... find references >>>>> to some incremental webrevs in my replies below. >>>>> >>>>> Further I moved the test classes of test differentLE/ into a package. >>>>> >>>>> Comprehensive new webrev: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03 >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: David Holmes >>>>>> Sent: Thursday, April 12, 2018 8:50 AM >>>>>> To: Lindenmaier, Goetz ; hotspot- >>>> runtime- >>>>>> dev at openjdk.java.net >>>>> ... >>>>>> This looks reasonable to me. Two minor comments: >>>>>> - updated tests need updated copyright notice >>>>>> - I think you can avoid duplicating test1() to test2() by passing in the >>>>>> loader name and the expected message as parameters >>>>> Fixed both. Incremental webrev: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- >>>>> incremental2/ >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: David Holmes >>>>>> Sent: Sunday, March 25, 2018 11:49 PM >>>>>> To: Lindenmaier, Goetz ; hotspot- >>>> runtime- >>>>>> dev at openjdk.java.net >>>>> ... >>>>>>> Maybe I should suppress printing verbose information about these >>>>>>> well known loaders. (There is is_builtin_class_loader_data() to >>>>>>> identify such.) >>>>>> >>>>>> Yes it might be good to simplify things for the predefined loaders. >>>>>> Though I'd use: >>>>>> SystemDictionary::is_platform_class_loader >>>>>> SystemDictionary::is_system_class_loader >>>>>> for that check. >>>>> I implemented omitting the parent for these loaders. >>>>> >>>>>> Maybe it makes sense to have a set pattern for this descriptive text, >>>>>> and use "unnamed" if the loader has no name: >>>>>> >>>>>> loader , instanceof , child of loader ... >>>>> I implemented printing "" for empty names. I would like >>>>> To keep the format with brackets, because the enclosing text might use >>>>> commas, too, and then the sentence structure might get confusing. >>>>> >>>>> See this incremental webrev for these two changes: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- >>>>> incremental1a/ >>>>> >>>>>> Add this bug to the @bug in the test(s). >>>>> Done. >>>>> >>>>>>> I'm not clear where you distinguish type and class. >>>>>> Using "class" is inaccurate as it can be a class, interface or array type. >>>>>> >>>>>> In the above the entity is a "type" as we are referring to a >>>>> classloader> pair. But you can also just read it as "class or interface >>>>>> or array type" >>>>> So why not print what it really is? I had added external_kind() for this in >>>>> My previous exception message change. >>>>> Unfortunately only in SystemDictionary::check_constraints() this is easily >>>>> possible, and it improves the message I think. >>>>> >>>>> Incremental webrev for these two changes: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- >>>>> incremental1b/ >>>>> >>>>> I also implemented a test for the call to external_kind(), but I missed to >> put >>>>> that into >>>>> the incremental webrev. >>>>> >>>>> Best regards, >>>>> Goetz. >>> From goetz.lindenmaier at sap.com Wed May 2 13:51:20 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 2 May 2018 13:51:20 +0000 Subject: [ping] RE: RFR(M): 8199852: Print more information about class loaders in LinkageErrors. In-Reply-To: <792d97cc-0e32-4d81-4838-5cc958b7242c@oracle.com> References: <0e13b51363ef4b6f9aece78e3a332afb@sap.com> <5b99bafb1ceb440a8892676dcc1a3ec6@sap.com> <792d97cc-0e32-4d81-4838-5cc958b7242c@oracle.com> Message-ID: Thanks a lot! Best regards, Goetz. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Mittwoch, 2. Mai 2018 14:32 > To: Lindenmaier, Goetz ; hotspot-runtime- > dev at openjdk.java.net > Subject: Re: [ping] RE: RFR(M): 8199852: Print more information about class > loaders in LinkageErrors. > > Hi Goetz, > > Yes you can consider this Reviewed. > > Thanks, > David > > On 2/05/2018 9:21 PM, Lindenmaier, Goetz wrote: > > Hi David, > > > >> Sorry really strapped for time these days ... > > This is ok, but I will keep pinging ... ?? > > A great, general thanks to you, as you really spend a lot of effort on > > reviewing changes by SAP. That helps us a lot and we appreciate > > your efforts! > > > >> src/hotspot/share/classfile/javaClasses.cpp > >> src/hotspot/share/classfile/javaClasses.hpp > >> The description of describe_external in the hpp doesn't match the > >> implementation in the cpp: parent: x versus child of x > > I updated the comment: > > - // Prints "" (instance of , parent: "" > ) > > - // or (parent: "" ). > > + // Prints "" (instance of , child of "" > ). > > + // If a classloader has no name, it prints instead. The output > > + // for well known loaders (system/platform) is abbreviated. > > > >> src/hotspot/share/classfile/systemDictionary.cpp > >> > >> 2208 throwException = true; > >> 2209 ss.print("loader constraint violation: loader %s", > >> 2210 > >> java_lang_ClassLoader::describe_external(class_loader())); > >> 2211 ss.print(" wants to load %s %s.", > >> 2212 k->external_kind(), k->external_name()); > >> 2213 Klass *existing_klass = > >> constraints()->find_constrained_klass(name, class_loader); > >> 2214 if (existing_klass->class_loader() != class_loader()) { > >> 2215 ss.print(" A different %s with the same name was > >> previously loaded by %s.", > >> 2216 existing_klass->external_kind(), > >> 2217 > >> java_lang_ClassLoader::describe_external(existing_klass- > >class_loader())); > >> 2218 } > >> > >> I still find this way too verbose. I won't oppose it but I don't like > >> it. I know you're trying to solve your remote support problem, but I > >> worry about the newbie trying to learn and experiment and being totally > >> bamboozled by the exception message. (I thought the old ones were bad > >> enough - but necessary in their level of preciseness.) > > I understand. Thanks for not opposing. > > > > > >> 3125 // Caller needs ResourceMark. > >> > >> Nit: Why change this line but not line 3131? In general single-line > >> comments don't need to be written as grammatically correct sentences > >> with full punctuation. Only if it is a multi-sentence comment should you > >> need to do that. > > Reverted, leftover from my other edits in that code that were removed. > > > >> Only comment I'll make on the tests is that being too specific about the > >> exact message makes the tests more fragile. > > Yes, but there are so many details in the message I would like to > > assure (like the proper name of classes with '.') that I think this is > > necessary. > > > > I'll push the change through the jdk-submit forest. (It's through all > > our tests, anyways). After pushing, I will also update the bug to > > list the final exception messages. > > Can I consider it reviewed by you, then? > > > > Best regards, > > Goetz. > > > > > >> > >> Thanks, > >> David > >> ----- > >> > >> > >> On 26/04/2018 1:30 AM, Lindenmaier, Goetz wrote: > >>> Hi Lois, David, George or others, > >>> > >>> I would like to finish this one up, could I please get some > >>> (hopefully final) reviews? > >>> > >>> Thanks, > >>> Goetz. > >>> > >>>> -----Original Message----- > >>>> From: Lindenmaier, Goetz > >>>> Sent: Mittwoch, 18. April 2018 09:55 > >>>> To: 'David Holmes' ; 'hotspot-runtime- > >>>> dev at openjdk.java.net' > >>>> Subject: RE: RFR(M): 8199852: Print more information about class > loaders > >> in > >>>> LinkageErrors. > >>>> > >>>> Hi, > >>>> > >>>> I rebased the webrev to the jdk repo: > >>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- > jdk/ > >>>> Could I please get the final go? > >>>> > >>>> Thanks and best regards, > >>>> Goetz. > >>>> > >>>> > >>>>> -----Original Message----- > >>>>> From: Lindenmaier, Goetz > >>>>> Sent: Freitag, 13. April 2018 10:10 > >>>>> To: 'David Holmes' ; hotspot-runtime- > >>>>> dev at openjdk.java.net > >>>>> Subject: RE: RFR(M): 8199852: Print more information about class > loaders > >> in > >>>>> LinkageErrors. > >>>>> > >>>>> Hi David, > >>>>> > >>>>> I fixed what you mentioned here. I also found an older mail > >>>>> of yours with some comments I implemented now. I copied > >>>>> that mail's content here to have only one mail ... find references > >>>>> to some incremental webrevs in my replies below. > >>>>> > >>>>> Further I moved the test classes of test differentLE/ into a package. > >>>>> > >>>>> Comprehensive new webrev: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03 > >>>>> > >>>>> > >>>>>> -----Original Message----- > >>>>>> From: David Holmes > >>>>>> Sent: Thursday, April 12, 2018 8:50 AM > >>>>>> To: Lindenmaier, Goetz ; hotspot- > >>>> runtime- > >>>>>> dev at openjdk.java.net > >>>>> ... > >>>>>> This looks reasonable to me. Two minor comments: > >>>>>> - updated tests need updated copyright notice > >>>>>> - I think you can avoid duplicating test1() to test2() by passing in the > >>>>>> loader name and the expected message as parameters > >>>>> Fixed both. Incremental webrev: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- > >>>>> incremental2/ > >>>>> > >>>>> > >>>>>> -----Original Message----- > >>>>>> From: David Holmes > >>>>>> Sent: Sunday, March 25, 2018 11:49 PM > >>>>>> To: Lindenmaier, Goetz ; hotspot- > >>>> runtime- > >>>>>> dev at openjdk.java.net > >>>>> ... > >>>>>>> Maybe I should suppress printing verbose information about these > >>>>>>> well known loaders. (There is is_builtin_class_loader_data() to > >>>>>>> identify such.) > >>>>>> > >>>>>> Yes it might be good to simplify things for the predefined loaders. > >>>>>> Though I'd use: > >>>>>> SystemDictionary::is_platform_class_loader > >>>>>> SystemDictionary::is_system_class_loader > >>>>>> for that check. > >>>>> I implemented omitting the parent for these loaders. > >>>>> > >>>>>> Maybe it makes sense to have a set pattern for this descriptive text, > >>>>>> and use "unnamed" if the loader has no name: > >>>>>> > >>>>>> loader , instanceof , child of loader ... > >>>>> I implemented printing "" for empty names. I would like > >>>>> To keep the format with brackets, because the enclosing text might > use > >>>>> commas, too, and then the sentence structure might get confusing. > >>>>> > >>>>> See this incremental webrev for these two changes: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- > >>>>> incremental1a/ > >>>>> > >>>>>> Add this bug to the @bug in the test(s). > >>>>> Done. > >>>>> > >>>>>>> I'm not clear where you distinguish type and class. > >>>>>> Using "class" is inaccurate as it can be a class, interface or array type. > >>>>>> > >>>>>> In the above the entity is a "type" as we are referring to a name, > >>>>>> classloader> pair. But you can also just read it as "class or interface > >>>>>> or array type" > >>>>> So why not print what it really is? I had added external_kind() for this > in > >>>>> My previous exception message change. > >>>>> Unfortunately only in SystemDictionary::check_constraints() this is > easily > >>>>> possible, and it improves the message I think. > >>>>> > >>>>> Incremental webrev for these two changes: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- > >>>>> incremental1b/ > >>>>> > >>>>> I also implemented a test for the call to external_kind(), but I missed > to > >> put > >>>>> that into > >>>>> the incremental webrev. > >>>>> > >>>>> Best regards, > >>>>> Goetz. > >>> From lutz.schmidt at sap.com Wed May 2 14:20:09 2018 From: lutz.schmidt at sap.com (Schmidt, Lutz) Date: Wed, 2 May 2018 14:20:09 +0000 Subject: [CAUTION] RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics Message-ID: <42CEFFC8-7FD1-469C-A870-4DD5EC2C2E7E@sap.com> Hi Martin, your changes look good. For the additional synchronization on s390 in particular, I do not expect a serious performance impact. The checkpoint synchronization part has always been the expensive, but rarely required, part. PPC may be another story. We knoow that a full-blown sync really hurts. We will see... Thanks, Lutz ?On 26.04.18, 16:23, "hotspot-runtime-dev on behalf of Doerr, Martin" wrote: Hi, I have renamed "cmpxchg_memory_order" to "atomic_memory_order" and added support to all Read-Modify-Write atomics with support for all C++11 semantics except seq_cst. The latter has issues on PPC64 so we're currently using our own memory_order_conservative instead, which is the default. Please review my new webrev: http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.01/ We'll test Windows, MacOS, linux x86+ppc64+s390, AIX and Solaris SPARC. It'd be great if somebody could check arm/aarch64 and zero. This change may enable optimizations for arm/aarch64 (not yet included). Best regards, Martin -----Original Message----- From: Doerr, Martin Sent: Donnerstag, 26. April 2018 11:20 To: 'David Holmes' ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) Subject: RE: RFR(M): 8202080: Introduce ordering semantics for Atomic::add Hi David, sounds like we are on the same page, now. I think we should have some kind of summary of what was analyzed. But that belongs to the other thread (JDK- 8154736). > The bugs in this kind of code are very subtle and rarely exposed through normal levels of testing. Right. We have experienced this often enough. So it is definitely in our interest to have correct memory barriers. Mistakes will hit us (PPC64). So I appreciate that you insist on careful analysis. I'll prepare a new webrev with conservative semantics for all read-modify-write atomics by default. Best regards, Martin From lois.foltan at oracle.com Wed May 2 14:58:42 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Wed, 2 May 2018 10:58:42 -0400 Subject: [ping] RE: RFR(M): 8199852: Print more information about class loaders in LinkageErrors. In-Reply-To: <5b99bafb1ceb440a8892676dcc1a3ec6@sap.com> References: <0e13b51363ef4b6f9aece78e3a332afb@sap.com> <5b99bafb1ceb440a8892676dcc1a3ec6@sap.com> Message-ID: <4567ab7a-7f88-519e-c7df-c7c21928c396@oracle.com> Hi Goetz, I'm ok with going forward with this change, you have my review. However, I would like to record some concerns I do have about this change which I hope RFEs will be created for to clean up. 1. the change introduces another duplicate way to obtain the class loader's name via java_lang_ClassLoader::describe_external, where Klass::class_loader_and_module_name() exists today and could be modified 2. the method describe_external() obtains the class loader's name in a different manner than SystemDictionary::loader_name().? How to obtain the class loader's name should be standardized within the VM. 3. previously I expressed concerns about adding the parent loader's information into the LinkageError message since I think this makes the message wordy and to me this seems like information that is more appropriate for logging. Thanks, Lois On 5/2/2018 7:21 AM, Lindenmaier, Goetz wrote: > Hi David, > >> Sorry really strapped for time these days ... > This is ok, but I will keep pinging ... ?? > A great, general thanks to you, as you really spend a lot of effort on > reviewing changes by SAP. That helps us a lot and we appreciate > your efforts! > >> src/hotspot/share/classfile/javaClasses.cpp >> src/hotspot/share/classfile/javaClasses.hpp >> The description of describe_external in the hpp doesn't match the >> implementation in the cpp: parent: x versus child of x > I updated the comment: > - // Prints "" (instance of , parent: "" ) > - // or (parent: "" ). > + // Prints "" (instance of , child of "" ). > + // If a classloader has no name, it prints instead. The output > + // for well known loaders (system/platform) is abbreviated. > >> src/hotspot/share/classfile/systemDictionary.cpp >> >> 2208 throwException = true; >> 2209 ss.print("loader constraint violation: loader %s", >> 2210 >> java_lang_ClassLoader::describe_external(class_loader())); >> 2211 ss.print(" wants to load %s %s.", >> 2212 k->external_kind(), k->external_name()); >> 2213 Klass *existing_klass = >> constraints()->find_constrained_klass(name, class_loader); >> 2214 if (existing_klass->class_loader() != class_loader()) { >> 2215 ss.print(" A different %s with the same name was >> previously loaded by %s.", >> 2216 existing_klass->external_kind(), >> 2217 >> java_lang_ClassLoader::describe_external(existing_klass->class_loader())); >> 2218 } >> >> I still find this way too verbose. I won't oppose it but I don't like >> it. I know you're trying to solve your remote support problem, but I >> worry about the newbie trying to learn and experiment and being totally >> bamboozled by the exception message. (I thought the old ones were bad >> enough - but necessary in their level of preciseness.) > I understand. Thanks for not opposing. > > >> 3125 // Caller needs ResourceMark. >> >> Nit: Why change this line but not line 3131? In general single-line >> comments don't need to be written as grammatically correct sentences >> with full punctuation. Only if it is a multi-sentence comment should you >> need to do that. > Reverted, leftover from my other edits in that code that were removed. > >> Only comment I'll make on the tests is that being too specific about the >> exact message makes the tests more fragile. > Yes, but there are so many details in the message I would like to > assure (like the proper name of classes with '.') that I think this is > necessary. > > I'll push the change through the jdk-submit forest. (It's through all > our tests, anyways). After pushing, I will also update the bug to > list the final exception messages. > Can I consider it reviewed by you, then? > > Best regards, > Goetz. > > >> Thanks, >> David >> ----- >> >> >> On 26/04/2018 1:30 AM, Lindenmaier, Goetz wrote: >>> Hi Lois, David, George or others, >>> >>> I would like to finish this one up, could I please get some >>> (hopefully final) reviews? >>> >>> Thanks, >>> Goetz. >>> >>>> -----Original Message----- >>>> From: Lindenmaier, Goetz >>>> Sent: Mittwoch, 18. April 2018 09:55 >>>> To: 'David Holmes' ; 'hotspot-runtime- >>>> dev at openjdk.java.net' >>>> Subject: RE: RFR(M): 8199852: Print more information about class loaders >> in >>>> LinkageErrors. >>>> >>>> Hi, >>>> >>>> I rebased the webrev to the jdk repo: >>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03-jdk/ >>>> Could I please get the final go? >>>> >>>> Thanks and best regards, >>>> Goetz. >>>> >>>> >>>>> -----Original Message----- >>>>> From: Lindenmaier, Goetz >>>>> Sent: Freitag, 13. April 2018 10:10 >>>>> To: 'David Holmes' ; hotspot-runtime- >>>>> dev at openjdk.java.net >>>>> Subject: RE: RFR(M): 8199852: Print more information about class loaders >> in >>>>> LinkageErrors. >>>>> >>>>> Hi David, >>>>> >>>>> I fixed what you mentioned here. I also found an older mail >>>>> of yours with some comments I implemented now. I copied >>>>> that mail's content here to have only one mail ... find references >>>>> to some incremental webrevs in my replies below. >>>>> >>>>> Further I moved the test classes of test differentLE/ into a package. >>>>> >>>>> Comprehensive new webrev: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03 >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: David Holmes >>>>>> Sent: Thursday, April 12, 2018 8:50 AM >>>>>> To: Lindenmaier, Goetz ; hotspot- >>>> runtime- >>>>>> dev at openjdk.java.net >>>>> ... >>>>>> This looks reasonable to me. Two minor comments: >>>>>> - updated tests need updated copyright notice >>>>>> - I think you can avoid duplicating test1() to test2() by passing in the >>>>>> loader name and the expected message as parameters >>>>> Fixed both. Incremental webrev: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- >>>>> incremental2/ >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: David Holmes >>>>>> Sent: Sunday, March 25, 2018 11:49 PM >>>>>> To: Lindenmaier, Goetz ; hotspot- >>>> runtime- >>>>>> dev at openjdk.java.net >>>>> ... >>>>>>> Maybe I should suppress printing verbose information about these >>>>>>> well known loaders. (There is is_builtin_class_loader_data() to >>>>>>> identify such.) >>>>>> Yes it might be good to simplify things for the predefined loaders. >>>>>> Though I'd use: >>>>>> SystemDictionary::is_platform_class_loader >>>>>> SystemDictionary::is_system_class_loader >>>>>> for that check. >>>>> I implemented omitting the parent for these loaders. >>>>> >>>>>> Maybe it makes sense to have a set pattern for this descriptive text, >>>>>> and use "unnamed" if the loader has no name: >>>>>> >>>>>> loader , instanceof , child of loader ... >>>>> I implemented printing "" for empty names. I would like >>>>> To keep the format with brackets, because the enclosing text might use >>>>> commas, too, and then the sentence structure might get confusing. >>>>> >>>>> See this incremental webrev for these two changes: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- >>>>> incremental1a/ >>>>> >>>>>> Add this bug to the @bug in the test(s). >>>>> Done. >>>>> >>>>>>> I'm not clear where you distinguish type and class. >>>>>> Using "class" is inaccurate as it can be a class, interface or array type. >>>>>> >>>>>> In the above the entity is a "type" as we are referring to a >>>>> classloader> pair. But you can also just read it as "class or interface >>>>>> or array type" >>>>> So why not print what it really is? I had added external_kind() for this in >>>>> My previous exception message change. >>>>> Unfortunately only in SystemDictionary::check_constraints() this is easily >>>>> possible, and it improves the message I think. >>>>> >>>>> Incremental webrev for these two changes: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- >>>>> incremental1b/ >>>>> >>>>> I also implemented a test for the call to external_kind(), but I missed to >> put >>>>> that into >>>>> the incremental webrev. >>>>> >>>>> Best regards, >>>>> Goetz. From matthias.baesken at sap.com Wed May 2 10:59:49 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 2 May 2018 10:59:49 +0000 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: References: Message-ID: Hi Martin, thanks for your input . I add hotspot-runtime-dev . > > I wonder if we really need the sizes in kB in addition to MB. Maybe other > reviewers would like to comment on this, too. > We could also just print the MB value , let's see what other think. Another option might be to have a flexible output (kB for smaller memory values , MB (or GB) for larger ) ? Best regards, Matthias > -----Original Message----- > From: Doerr, Martin > Sent: Mittwoch, 2. Mai 2018 12:53 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows > > Hi Matthias, > > looks like a nice enhancement. We can get substantially more information. > > I wonder if we really need the sizes in kB in addition to MB. Maybe other > reviewers would like to comment on this, too. > > We should have line breaks. > > Best regards, > Martin > > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > Behalf Of Baesken, Matthias > Sent: Montag, 30. April 2018 16:53 > To: 'hotspot-dev at openjdk.java.net' > Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows > > On Windows, > the os::print_memory_info misses a few memory-related infos that might > be interesting : > - current and peak WorkingSet size (= physical memory assigned to the > process) > - current and peak commit charge (also known as "private bytes" in Windows > tools) > - on 32bit Windows : > user-mode portion/free user mode portion of virtual address-space > (Total/AvailVirtual) because it shows how close do we get to the 2-4 GB per > process border. > > > - the current naming of "swap/free-swap" memory is a bit misleading; > in the Windows world swap is a special page file used for UWP apps. > (see Windows Internals : "The swap file" (chapter 5 Memory management) > Windows 8 added another page file called a swap file. It is ... exclusively used > for UWP (Universal Windows Platform) apps. > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the UWP > related values, it is about page file sizes > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just virtual > memory might be more appropriate). > > > https://msdn.microsoft.com/de- > de/library/windows/desktop/aa366770(v=vs.85).aspx > > documents it in the following way: > ullTotalPageFile: > The current committed memory limit for the system or the current process, > whichever is smaller,... > > ullAvailPageFile : > The maximum amount of memory the current process can commit, in bytes. > This value is equal to or smaller than the system-wide available commit value > > > > Aditionally I suggest having output of the various memory-values in M > (megabyte) as well , the k (kilobyte) output sometimes gives very high and > unreadable numbers). > > > Could you please review my change ? > > > Bug : > > https://bugs.openjdk.java.net/browse/JDK-8202427 > > > Change : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > > > Thanks, Matthias > From goetz.lindenmaier at sap.com Wed May 2 15:19:41 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 2 May 2018 15:19:41 +0000 Subject: [ping] RE: RFR(M): 8199852: Print more information about class loaders in LinkageErrors. In-Reply-To: <4567ab7a-7f88-519e-c7df-c7c21928c396@oracle.com> References: <0e13b51363ef4b6f9aece78e3a332afb@sap.com> <5b99bafb1ceb440a8892676dcc1a3ec6@sap.com> <4567ab7a-7f88-519e-c7df-c7c21928c396@oracle.com> Message-ID: <8ff92b7d5df3431aa00ad24b8d52223e@sap.com> Hi Lois, Thanks for your approval of my change! > Hi Goetz, > > I'm ok with going forward with this change, you have my review. However, > I would like to record some concerns I do have about this change which I > hope RFEs will be created for to clean up. > > 1. the change introduces another duplicate way to obtain the class > loader's name via java_lang_ClassLoader::describe_external, where > Klass::class_loader_and_module_name() exists today and could be modified As stated before, I think it would be quite complicated to design one method that delivers a name/string as complex as that of these methods to fit all purposes. I appreciate though to find a common way to put the messages, common wording etc.. I think class_loader_and_module_name() could use describe_external() for the class_loader name. > 2. the method describe_external() obtains the class loader's name in a > different manner than SystemDictionary::loader_name().? How to obtain > the class loader's name should be standardized within the VM. In some intermediate design, I had called describe_external within loader_name. As I understand, loader_name does not report the name of the loader at all, only the class name. This really should be improved. > 3. previously I expressed concerns about adding the parent loader's > information into the LinkageError message since I think this makes the > message wordy and to me this seems like information that is more > appropriate for logging. I think error messages should contain as much information as possible to track down the error. Thanks for approving my change anyways. Best regards, Goetz. > > Thanks, > Lois > > On 5/2/2018 7:21 AM, Lindenmaier, Goetz wrote: > > Hi David, > > > >> Sorry really strapped for time these days ... > > This is ok, but I will keep pinging ... ?? > > A great, general thanks to you, as you really spend a lot of effort on > > reviewing changes by SAP. That helps us a lot and we appreciate > > your efforts! > > > >> src/hotspot/share/classfile/javaClasses.cpp > >> src/hotspot/share/classfile/javaClasses.hpp > >> The description of describe_external in the hpp doesn't match the > >> implementation in the cpp: parent: x versus child of x > > I updated the comment: > > - // Prints "" (instance of , parent: "" > ) > > - // or (parent: "" ). > > + // Prints "" (instance of , child of "" > ). > > + // If a classloader has no name, it prints instead. The output > > + // for well known loaders (system/platform) is abbreviated. > > > >> src/hotspot/share/classfile/systemDictionary.cpp > >> > >> 2208 throwException = true; > >> 2209 ss.print("loader constraint violation: loader %s", > >> 2210 > >> java_lang_ClassLoader::describe_external(class_loader())); > >> 2211 ss.print(" wants to load %s %s.", > >> 2212 k->external_kind(), k->external_name()); > >> 2213 Klass *existing_klass = > >> constraints()->find_constrained_klass(name, class_loader); > >> 2214 if (existing_klass->class_loader() != class_loader()) { > >> 2215 ss.print(" A different %s with the same name was > >> previously loaded by %s.", > >> 2216 existing_klass->external_kind(), > >> 2217 > >> java_lang_ClassLoader::describe_external(existing_klass- > >class_loader())); > >> 2218 } > >> > >> I still find this way too verbose. I won't oppose it but I don't like > >> it. I know you're trying to solve your remote support problem, but I > >> worry about the newbie trying to learn and experiment and being totally > >> bamboozled by the exception message. (I thought the old ones were bad > >> enough - but necessary in their level of preciseness.) > > I understand. Thanks for not opposing. > > > > > >> 3125 // Caller needs ResourceMark. > >> > >> Nit: Why change this line but not line 3131? In general single-line > >> comments don't need to be written as grammatically correct sentences > >> with full punctuation. Only if it is a multi-sentence comment should you > >> need to do that. > > Reverted, leftover from my other edits in that code that were removed. > > > >> Only comment I'll make on the tests is that being too specific about the > >> exact message makes the tests more fragile. > > Yes, but there are so many details in the message I would like to > > assure (like the proper name of classes with '.') that I think this is > > necessary. > > > > I'll push the change through the jdk-submit forest. (It's through all > > our tests, anyways). After pushing, I will also update the bug to > > list the final exception messages. > > Can I consider it reviewed by you, then? > > > > Best regards, > > Goetz. > > > > > >> Thanks, > >> David > >> ----- > >> > >> > >> On 26/04/2018 1:30 AM, Lindenmaier, Goetz wrote: > >>> Hi Lois, David, George or others, > >>> > >>> I would like to finish this one up, could I please get some > >>> (hopefully final) reviews? > >>> > >>> Thanks, > >>> Goetz. > >>> > >>>> -----Original Message----- > >>>> From: Lindenmaier, Goetz > >>>> Sent: Mittwoch, 18. April 2018 09:55 > >>>> To: 'David Holmes' ; 'hotspot-runtime- > >>>> dev at openjdk.java.net' > >>>> Subject: RE: RFR(M): 8199852: Print more information about class > loaders > >> in > >>>> LinkageErrors. > >>>> > >>>> Hi, > >>>> > >>>> I rebased the webrev to the jdk repo: > >>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- > jdk/ > >>>> Could I please get the final go? > >>>> > >>>> Thanks and best regards, > >>>> Goetz. > >>>> > >>>> > >>>>> -----Original Message----- > >>>>> From: Lindenmaier, Goetz > >>>>> Sent: Freitag, 13. April 2018 10:10 > >>>>> To: 'David Holmes' ; hotspot-runtime- > >>>>> dev at openjdk.java.net > >>>>> Subject: RE: RFR(M): 8199852: Print more information about class > loaders > >> in > >>>>> LinkageErrors. > >>>>> > >>>>> Hi David, > >>>>> > >>>>> I fixed what you mentioned here. I also found an older mail > >>>>> of yours with some comments I implemented now. I copied > >>>>> that mail's content here to have only one mail ... find references > >>>>> to some incremental webrevs in my replies below. > >>>>> > >>>>> Further I moved the test classes of test differentLE/ into a package. > >>>>> > >>>>> Comprehensive new webrev: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03 > >>>>> > >>>>> > >>>>>> -----Original Message----- > >>>>>> From: David Holmes > >>>>>> Sent: Thursday, April 12, 2018 8:50 AM > >>>>>> To: Lindenmaier, Goetz ; hotspot- > >>>> runtime- > >>>>>> dev at openjdk.java.net > >>>>> ... > >>>>>> This looks reasonable to me. Two minor comments: > >>>>>> - updated tests need updated copyright notice > >>>>>> - I think you can avoid duplicating test1() to test2() by passing in the > >>>>>> loader name and the expected message as parameters > >>>>> Fixed both. Incremental webrev: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- > >>>>> incremental2/ > >>>>> > >>>>> > >>>>>> -----Original Message----- > >>>>>> From: David Holmes > >>>>>> Sent: Sunday, March 25, 2018 11:49 PM > >>>>>> To: Lindenmaier, Goetz ; hotspot- > >>>> runtime- > >>>>>> dev at openjdk.java.net > >>>>> ... > >>>>>>> Maybe I should suppress printing verbose information about these > >>>>>>> well known loaders. (There is is_builtin_class_loader_data() to > >>>>>>> identify such.) > >>>>>> Yes it might be good to simplify things for the predefined loaders. > >>>>>> Though I'd use: > >>>>>> SystemDictionary::is_platform_class_loader > >>>>>> SystemDictionary::is_system_class_loader > >>>>>> for that check. > >>>>> I implemented omitting the parent for these loaders. > >>>>> > >>>>>> Maybe it makes sense to have a set pattern for this descriptive text, > >>>>>> and use "unnamed" if the loader has no name: > >>>>>> > >>>>>> loader , instanceof , child of loader ... > >>>>> I implemented printing "" for empty names. I would like > >>>>> To keep the format with brackets, because the enclosing text might > use > >>>>> commas, too, and then the sentence structure might get confusing. > >>>>> > >>>>> See this incremental webrev for these two changes: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- > >>>>> incremental1a/ > >>>>> > >>>>>> Add this bug to the @bug in the test(s). > >>>>> Done. > >>>>> > >>>>>>> I'm not clear where you distinguish type and class. > >>>>>> Using "class" is inaccurate as it can be a class, interface or array type. > >>>>>> > >>>>>> In the above the entity is a "type" as we are referring to a name, > >>>>>> classloader> pair. But you can also just read it as "class or interface > >>>>>> or array type" > >>>>> So why not print what it really is? I had added external_kind() for this > in > >>>>> My previous exception message change. > >>>>> Unfortunately only in SystemDictionary::check_constraints() this is > easily > >>>>> possible, and it improves the message I think. > >>>>> > >>>>> Incremental webrev for these two changes: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- > >>>>> incremental1b/ > >>>>> > >>>>> I also implemented a test for the call to external_kind(), but I missed > to > >> put > >>>>> that into > >>>>> the incremental webrev. > >>>>> > >>>>> Best regards, > >>>>> Goetz. From goetz.lindenmaier at sap.com Wed May 2 15:56:36 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 2 May 2018 15:56:36 +0000 Subject: RFR(M): 8201593: Print array length in ArrayIndexOutOfBoundsException. In-Reply-To: References: <504bdd8469bb45cc8c5cf790d1f4b3fc@sap.com> <12e8ad2367484b0b88bcee4bcb77ac47@sap.com> <033adcb3-1ab0-094e-78cf-4062a8868d3e@redhat.com> <7767af56c6744e8aa5479a5dd9626936@sap.com> Message-ID: <9d2753f6975c4477912420a2fe459ef3@sap.com> Hi, I needed to move the edit from c1_LIRGenerator_.cpp to the shared file after "8201543: Modularize C1 GC barriers" New webrev: http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06/ @Stuart Thanks for testing! > so as to accommodate the array pointer you are pushing onto the stack? Yes, what you are pointing out seems to be wrong, I changed it to '2'. Best regards, Goetz. > -----Original Message----- > From: Stuart Monteith [mailto:stuart.monteith at linaro.org] > Sent: Freitag, 27. April 2018 16:37 > To: Lindenmaier, Goetz > Cc: hotspot-compiler-dev at openjdk.java.net; aarch64-port- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; aarch32- > port-dev at openjdk.java.net > Subject: Re: RFR(M): 8201593: Print array length in > ArrayIndexOutOfBoundsException. > > Hi, > JTregs hasn't flagged any issues, so it should be ok. > > Regarding the 32-bit arm code, in "void > RangeCheckStub::emit_code(LIR_Assembler* ce)" should: > ce->verify_reserved_argument_area_size(1); > be > ce->verify_reserved_argument_area_size(2); > > so as to accommodate the array pointer you are pushing onto the stack? > > I've not tested 32-bit arm. > > > BR, > Stuart > > On 26 April 2018 at 15:31, Stuart Monteith > wrote: > > Thanks, I'm happy with that. > > > > The registers have a clean path to call_RT - r22 and r23 aren't used > > inbetween. They are an arbitrary choice - c_rarg0 and c_rarg1 were > > always going to cause problems. If _array->as_pointer_register() > > and/or _index->as_register() or _index->as_jint() were the registers > > we were using as parameters there would be trouble. However, with > > pd_last_allocatable_cpu_reg = 16, that shouldn't happen with r22/23, > > or indeed anything else in the range r17 to r28. > > > > I'm going to run all of JTRegs and seem what that produces now. > > > > BR, > > Stuart > > > > > > > > > > On 26 April 2018 at 15:14, Lindenmaier, Goetz > wrote: > >> Hi Stuart, > >> > >> thanks for fixing this! Webrev with your changes: > >> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/05/ > >> > >>> There is the possibility of overwriting live values though, aren't > >>> there? The registers are saved by call_RT. Should I be concerned about > >>> deopt and debugging going wrong? Furthermore, won't there be issues > in > >>> exception handlers? > >> As I understand, this just has to survive the far_call. > >> The call_RT in c1_Runtime then moves it into the > >> proper argument registers. This is just the handling of an > >> exception, and in these few instructions no java code is > >> executed, no safepoint is passed, so this should be fine. > >> > >> incremental diff: > >> iff -r 874f2b999ff6 src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp > >> --- a/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Mon Apr 16 > 15:17:20 2018 +0200 > >> +++ b/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Thu Apr 26 > 15:55:18 2018 +0200 > >> @@ -75,16 +75,16 @@ > >> } > >> > >> if (_index->is_cpu_register()) { > >> - __ mov(rscratch1, _index->as_register()); > >> + __ mov(r22, _index->as_register()); > >> } else { > >> - __ mov(rscratch1, _index->as_jint()); > >> + __ mov(r22, _index->as_jint()); > >> } > >> Runtime1::StubID stub_id; > >> if (_throw_index_out_of_bounds_exception) { > >> stub_id = Runtime1::throw_index_exception_id; > >> } else { > >> assert(_array != NULL, "sanity"); > >> - __ mov(rscratch2, _array->as_pointer_register()); > >> + __ mov(r23, _array->as_pointer_register()); > >> stub_id = Runtime1::throw_range_check_failed_id; > >> } > >> __ far_call(RuntimeAddress(Runtime1::entry_for(stub_id)), NULL, > rscratch2); > >> diff -r 874f2b999ff6 src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp > >> --- a/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Mon Apr 16 > 15:17:20 2018 +0200 > >> +++ b/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Thu Apr 26 > 15:55:18 2018 +0200 > >> @@ -327,7 +327,7 @@ > >> > >> > >> // target: the entry point of the method that creates and posts the > exception oop > >> -// has_argument: true if the exception needs an argument (passed in > rscratch1) > >> +// has_argument: true if the exception needs arguments (passed in r22 > and r23) > >> > >> OopMapSet* Runtime1::generate_exception_throw(StubAssembler* > sasm, address target, bool has_argument) { > >> // make a frame and preserve the caller's caller-save registers > >> @@ -336,7 +336,7 @@ > >> if (!has_argument) { > >> call_offset = __ call_RT(noreg, noreg, target); > >> } else { > >> - call_offset = __ call_RT(noreg, noreg, target, rscratch1, rscratch2); > >> + call_offset = __ call_RT(noreg, noreg, target, r22, r23); > >> } > >> OopMapSet* oop_maps = new OopMapSet(); > >> oop_maps->add_gc_map(call_offset, oop_map); > >> > >> Best regards, > >> Goetz. > >> > >> > >>> -----Original Message----- > >>> From: Stuart Monteith [mailto:stuart.monteith at linaro.org] > >>> Sent: Donnerstag, 26. April 2018 12:52 > >>> To: Andrew Haley > >>> Cc: Lindenmaier, Goetz ; hotspot- > compiler- > >>> dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; hotspot- > >>> runtime-dev at openjdk.java.net; aarch32-port-dev at openjdk.java.net > >>> Subject: Re: RFR(M): 8201593: Print array length in > >>> ArrayIndexOutOfBoundsException. > >>> > >>> Hi, > >>> Using c_rarg1 and c_rarg2 instead of rscratch1 and overwriting > >>> rscratch2 causes a SIGSEGV. > >>> Using r22 and r23 instead, the test ran successfully. > >>> > >>> In c1_CodeStubs_aarch64.cpp > >>> : > >>> 77 if (_index->is_cpu_register()) { > >>> 78 __ mov(r22, _index->as_register()); > >>> 79 } else { > >>> 80 __ mov(r22, _index->as_jint()); > >>> 81 } > >>> 82 Runtime1::StubID stub_id; > >>> 83 if (_throw_index_out_of_bounds_exception) { > >>> 84 stub_id = Runtime1::throw_index_exception_id; > >>> 85 } else { > >>> 86 assert(_array != NULL, "sanity"); > >>> 87 __ mov(r23, _array->as_pointer_register()); > >>> 88 stub_id = Runtime1::throw_range_check_failed_id; > >>> 89 } > >>> > >>> in c1_Runtime_aarch64.cpp: > >>> > >>> 336 if (!has_argument) { > >>> 337 call_offset = __ call_RT(noreg, noreg, target); > >>> 338 } else { > >>> 339 call_offset = __ call_RT(noreg, noreg, target, r22, r23); > >>> 340 } > >>> > >>> There is the possibility of overwriting live values though, aren't > >>> there? The registers are saved by call_RT. Should I be concerned about > >>> deopt and debugging going wrong? Furthermore, won't there be issues > in > >>> exception handlers? > >>> > >>> BR, > >>> Stuart > >>> > >>> > >>> On 25 April 2018 at 16:49, Stuart Monteith > >>> wrote: > >>> > Indeed - and that is what I am seeing. Usually no parameters are being > >>> > called with this pattern, or rscratch1, with the temporary variable > >>> > being changed to use rscratch2 in such circumstances. > >>> > I'll try c_rarg1 and c_rarg2 - they should pass straight through,if I > >>> > interpret the code correcting. > >>> > > >>> > On 25 April 2018 at 16:26, Andrew Haley wrote: > >>> >> On 04/25/2018 04:00 PM, Stuart Monteith wrote: > >>> >>> I'm not quite sure to solve this yet - we'll need to use the stack in > >>> >>> some safe way. > >>> >> > >>> >> It's not a great idea to pass arguments in rscratch1 or rscratch2. These > >>> >> registers are for use in macros and should be treated as volatile. > Given > >>> >> that you're throwing an exception, registers will be clobbered > anyway. > >>> >> > >>> >> -- > >>> >> Andrew Haley > >>> >> Java Platform Lead Engineer > >>> >> Red Hat UK Ltd. > >>> >> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From volker.simonis at gmail.com Wed May 2 17:00:52 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 2 May 2018 19:00:52 +0200 Subject: Improving AppCDS for Custom Loaders In-Reply-To: <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> References: <8e0d891a-c00c-8f57-362a-6a4684a6cc21@oracle.com> <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> Message-ID: On Tue, May 1, 2018 at 8:32 PM, Ioi Lam wrote: > PROBLEM: > > As discussed with Volker and Yumin in previous e-mails, AppCDS has some > experimental support for custom class loaders. However, it's not very easy > to use. > > For example, you can write a classlist like this: > > java/lang/Object id: 1 > CustomLoadee id: 2 super: 1 source: /tmp/foo.jar > > The CustomLoadee class will be stored in the shared archive with a CRC code. > During runtime, if a customed loader wants to load a class of the same name, > and its classfile has the same size and CRC as the archived class, the > archived version will be loaded. This speeds up class loading by avoiding > parsing the class file, and saves space by sharing the mmap'ed class > metadata across processes. > > You can see an example test at: > > http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java > > However, the current scheme requires you to specify all the super classes > and interfaces. There's no support provided by the -XX:DumpLoadedClassList > option. It can be helped somewhat with Volker's tool: > https://github.com/simonis/cl4cds > > > POSSIBLE SOLUTIONS: > > 1. "Dump-as-you-go". As suggested by Yumin, we can provide a jcmd to ask a > running JVM process to dump all of its loaded classes, including those > loaded by custom loaders, into an archive. An alternative is to dump the > archive at JVM exit time (or when you press Ctrl-C, etc. > > 2. Add information about the custom classes for -XX:DumpLoadedClassList. The > trouble is some class loaders don't specify a code source that can be > understood by the built-in class loaders. For example, the "Fat Jars" would > have a code source like > > > jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/ > > also, many custom loaders would pre-process the classfile data before > defining the class, so we can't simply archive the version of the class on > disk. > > One possible solution for #2 is to include the class file data in the > -XX:DumpLoadedClassList output: > > > java/lang/Object id: 1 > CustomLoadee id: 2 super: 1 source: base64 > > yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0 > > PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA > > CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k > > SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA > > AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA > AgAPAAcAAAAKAAEABQADAAYACA== > > > Of the 2 solutions: > > #1 seems easier to use, but may require more invasive modifications in the > VM, especially if you want to be able to continue execution after dumping. > Not sure what #1 really proposes: dumping the complete .jsa archive at runtime or dumping just the loaded classes. If it's just about dumping the loaded class without generating the .jsa archive there's the problem that by default the VM doesn't store the exact bytes of a class after the class was loaded (except when class transformers are registered). So the class files would have to be re-assembled from the internal VM structures (in the same way this is done for class redefinition) and the resulting class-file may be different from the original bytes (i.e. some attributes may be missing). If #1 is about creating the whole .jsa archive at runtime (or at VM exit) I think that would be the most attractive solution from a usability point of view although I understand that #2 will be easier to implement in the short term. Regarding the argument that #1 will produce a "binary blob" that's true, but that's already true now when we use "Xshare:dump". I think it should be not to hard to implement a tool based an SA which could introspect a .jsa archive. > #2 would be easier to implement, but the classlist might be huge. > > Also, #2 would allow post-processing tools to remove unneeded classes, or > merge two runs into a single list. The output of #1 is essentially a binary > blob that's impossible for off-line analysis/optimizations. > > > Any comments, or suggestions for alternatives? > > Thanks > - Ioi > > From lois.foltan at oracle.com Wed May 2 20:21:46 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Wed, 2 May 2018 16:21:46 -0400 Subject: (11) RFR (M) JDK-8189916: Dynamic Constant support for Sparc Message-ID: <8fb7457a-a734-71e9-0363-4d1eaa232963@oracle.com> Please review this change to the template interpreter to support dynamic constant on Sparc. open webrev at http://cr.openjdk.java.net/~lfoltan/bug_jdk8189916.0/webrev/ bug link https://bugs.openjdk.java.net/browse/JDK-8189916 All current jtreg dynamic constant tests pass when run on Sparc and have had (os.arch != "sparcv9") removed from their test description. Testing: Sparc specific testing (hs-tier1-5, jdk-tier1-3) complete. ?????????????? All tier 1 platforms (hs-tier1-2) in progress. Thanks, Lois From coleen.phillimore at oracle.com Wed May 2 21:47:35 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 2 May 2018 17:47:35 -0400 Subject: (11) RFR (M) JDK-8189916: Dynamic Constant support for Sparc In-Reply-To: <8fb7457a-a734-71e9-0363-4d1eaa232963@oracle.com> References: <8fb7457a-a734-71e9-0363-4d1eaa232963@oracle.com> Message-ID: <5eb8a3db-535f-4991-2688-c675894e2168@oracle.com> http://cr.openjdk.java.net/~lfoltan/bug_jdk8189916.0/webrev/src/hotspot/cpu/sparc/templateTable_sparc.cpp.udiff.html + { // Check for the null sentinel. + // If we just called the VM, that already did the mapping for us, + // but it's harmless to retry. Can you change 'that' to 'it' in the comments in all the platforms? + // Safe to call w ith 0 result typo. Looks good! Coleen On 5/2/18 4:21 PM, Lois Foltan wrote: > Please review this change to the template interpreter to support > dynamic constant on Sparc. > > open webrev at > http://cr.openjdk.java.net/~lfoltan/bug_jdk8189916.0/webrev/ > bug link https://bugs.openjdk.java.net/browse/JDK-8189916 > > All current jtreg dynamic constant tests pass when run on Sparc and > have had (os.arch != "sparcv9") removed from their test description. > > Testing: Sparc specific testing (hs-tier1-5, jdk-tier1-3) complete. > ?????????????? All tier 1 platforms (hs-tier1-2) in progress. > > Thanks, > Lois > From calvin.cheung at oracle.com Wed May 2 22:38:32 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 02 May 2018 15:38:32 -0700 Subject: RFR 8197954 Remove unnecessary intermediary APIs from AppCDS implementation In-Reply-To: References: Message-ID: <5AEA3DE8.3010901@oracle.com> Hi Ioi, The changes look good. I have the following minor comments: 1) classLoaderExt.hpp 32 class ClassListParser; The above can remain as CDS_ONLY. It is needed for 113 static Klass* load_one_class(ClassListParser* parser, TRAPS); The above function should be CDS_ONLY as it is called via the MetaspaceShared::preload_and_dump() code path. 2) sharedPathsMiscInfo.hpp 124 enum { 125 BOOT = 1, 126 NON_EXIST = 2, 127 APP = 5, 128 MODULE = 6 129 }; How about set APP to 3 and MODULE to 4? thanks, Calvin On 5/1/18, 10:17 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8197954 > http://cr.openjdk.java.net/~iklam/jdk11/8197954-remove-unnecessary-appcds-api.v01/ > > > Summary: > > Before AppCDS was open sourced, we had a few "Ext" classes in > sharedClassUtil.hpp > that abstracted away the CDS operations related to the non-boot class > loaders. > This API made it possible to build the Oracle JDK with AppCDS > included, or build > the Open JDK with AppCDS removed. > > With the open sourcing of AppCDS, this abstraction layer is no longer > necessary. I > have moved the contents of the "Ext" classes into their proper > locations and removed > the sharedClassUtil.hpp/cpp files. > > Most of the changes are just moving things around. There shouldn't be > behavioral > changes. > > The files classLoaderExt.hpp/cpp still exists -- it encapsulates the > classpath > management code and is not strictly unnecessary. Some clean up may be > needed there, but > I'll probably do that in a separate RFE. > > Testing with hotspot tiers1~4. > > Thanks > - Ioi > > From ioi.lam at oracle.com Thu May 3 04:46:03 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 2 May 2018 21:46:03 -0700 Subject: RFR 8197954 Remove unnecessary intermediary APIs from AppCDS implementation In-Reply-To: <5AEA3DE8.3010901@oracle.com> References: <5AEA3DE8.3010901@oracle.com> Message-ID: On 5/2/18 3:38 PM, Calvin Cheung wrote: > Hi Ioi, > > The changes look good. I have the following minor comments: > > 1) classLoaderExt.hpp > > 32 class ClassListParser; > I removed the CDS_ONLY() macro to make the code cleaner. The class declaration doesn't have any significant cost in compilation and no cost in the binary, so I think it's better to avoid unnecessary CDS_ONLY macros. > The above can remain as CDS_ONLY. > It is needed for > 113?? static Klass* load_one_class(ClassListParser* parser, TRAPS); > > The above function should be CDS_ONLY as it is called via the > MetaspaceShared::preload_and_dump() code path. > > 2) sharedPathsMiscInfo.hpp > > ?124?? enum { > ?125???? BOOT????? = 1, > ?126???? NON_EXIST = 2, > ?127???? APP?????? = 5, > ?128???? MODULE??? = 6 > ?129?? }; > > How about set APP to 3 and MODULE to 4? > Yes, I will do that. Thanks - Ioi > thanks, > Calvin > > On 5/1/18, 10:17 PM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8197954 >> http://cr.openjdk.java.net/~iklam/jdk11/8197954-remove-unnecessary-appcds-api.v01/ >> >> >> Summary: >> >> Before AppCDS was open sourced, we had a few "Ext" classes in >> sharedClassUtil.hpp >> that abstracted away the CDS operations related to the non-boot class >> loaders. >> This API made it possible to build the Oracle JDK with AppCDS >> included, or build >> the Open JDK with AppCDS removed. >> >> With the open sourcing of AppCDS, this abstraction layer is no longer >> necessary. I >> have moved the contents of the "Ext" classes into their proper >> locations and removed >> the sharedClassUtil.hpp/cpp files. >> >> Most of the changes are just moving things around. There shouldn't be >> behavioral >> changes. >> >> The files classLoaderExt.hpp/cpp still exists -- it encapsulates the >> classpath >> management code and is not strictly unnecessary. Some clean up may be >> needed there, but >> I'll probably do that in a separate RFE. >> >> Testing with hotspot tiers1~4. >> >> Thanks >> - Ioi >> >> From ioi.lam at oracle.com Thu May 3 04:52:22 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 2 May 2018 21:52:22 -0700 Subject: Improving AppCDS for Custom Loaders In-Reply-To: References: <8e0d891a-c00c-8f57-362a-6a4684a6cc21@oracle.com> <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> Message-ID: On 5/2/18 10:00 AM, Volker Simonis wrote: > On Tue, May 1, 2018 at 8:32 PM, Ioi Lam wrote: >> PROBLEM: >> >> As discussed with Volker and Yumin in previous e-mails, AppCDS has some >> experimental support for custom class loaders. However, it's not very easy >> to use. >> >> For example, you can write a classlist like this: >> >> java/lang/Object id: 1 >> CustomLoadee id: 2 super: 1 source: /tmp/foo.jar >> >> The CustomLoadee class will be stored in the shared archive with a CRC code. >> During runtime, if a customed loader wants to load a class of the same name, >> and its classfile has the same size and CRC as the archived class, the >> archived version will be loaded. This speeds up class loading by avoiding >> parsing the class file, and saves space by sharing the mmap'ed class >> metadata across processes. >> >> You can see an example test at: >> >> http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java >> >> However, the current scheme requires you to specify all the super classes >> and interfaces. There's no support provided by the -XX:DumpLoadedClassList >> option. It can be helped somewhat with Volker's tool: >> https://github.com/simonis/cl4cds >> >> >> POSSIBLE SOLUTIONS: >> >> 1. "Dump-as-you-go". As suggested by Yumin, we can provide a jcmd to ask a >> running JVM process to dump all of its loaded classes, including those >> loaded by custom loaders, into an archive. An alternative is to dump the >> archive at JVM exit time (or when you press Ctrl-C, etc. >> >> 2. Add information about the custom classes for -XX:DumpLoadedClassList. The >> trouble is some class loaders don't specify a code source that can be >> understood by the built-in class loaders. For example, the "Fat Jars" would >> have a code source like >> >> >> jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/ >> >> also, many custom loaders would pre-process the classfile data before >> defining the class, so we can't simply archive the version of the class on >> disk. >> >> One possible solution for #2 is to include the class file data in the >> -XX:DumpLoadedClassList output: >> >> >> java/lang/Object id: 1 >> CustomLoadee id: 2 super: 1 source: base64 >> >> yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0 >> >> PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA >> >> CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k >> >> SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA >> >> AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA >> AgAPAAcAAAAKAAEABQADAAYACA== >> >> >> Of the 2 solutions: >> >> #1 seems easier to use, but may require more invasive modifications in the >> VM, especially if you want to be able to continue execution after dumping. >> > Not sure what #1 really proposes: dumping the complete .jsa archive at > runtime or dumping just the loaded classes. > > If it's just about dumping the loaded class without generating the > .jsa archive there's the problem that by default the VM doesn't store > the exact bytes of a class after the class was loaded (except when > class transformers are registered). So the class files would have to > be re-assembled from the internal VM structures (in the same way this > is done for class redefinition) and the resulting class-file may be > different from the original bytes (i.e. some attributes may be > missing). #1 is for creating the JSA file, not just dumping the class files. > If #1 is about creating the whole .jsa archive at runtime (or at VM > exit) I think that would be the most attractive solution from a > usability point of view although I understand that #2 will be easier > to implement in the short term. Regarding the argument that #1 will > produce a "binary blob" that's true, but that's already true now when > we use "Xshare:dump". I think it should be not to hard to implement a > tool based an SA which could introspect a .jsa archive. The argument about the binary blob is to compare it against the text file produced by -XX:DumpLoadedClassList. One use case to consider is when you have a JAR file that contains several apps that each load a unique set of classes. Today, (assuming that custom class loaders are not used), you can run each app once with -XX:DumpLoadedClassList, and then do an ??? cat *.classlist | sort | uniq > combined.classlist and then create an archive that would work for all these apps. With the binary blob, there's no easy way of doing this. It will be very difficult to write a tool to decipher each blob and then somehow combine them into a single one. Thanks - Ioi >> #2 would be easier to implement, but the classlist might be huge. >> >> Also, #2 would allow post-processing tools to remove unneeded classes, or >> merge two runs into a single list. The output of #1 is essentially a binary >> blob that's impossible for off-line analysis/optimizations. >> >> >> Any comments, or suggestions for alternatives? >> >> Thanks >> - Ioi >> >> From thomas.stuefe at gmail.com Thu May 3 05:11:36 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 3 May 2018 07:11:36 +0200 Subject: RFR(xxs): 8202303: LogStream should autoflush on destruction In-Reply-To: References: <255436cb-d990-8985-1e98-07e08ccf22c8@oracle.com> Message-ID: Thanks Robin! On Wed, May 2, 2018 at 11:41 AM, Robbin Ehn wrote: > On 2018-05-02 09:19, David Holmes wrote: >> >> Hi Thomas, >> >> On 2/05/2018 3:26 PM, Thomas St?fe wrote: >>> >>> Ping... >>> >>> this is a really simple one :) Please review. Thanks! >> >> >> Was hoping our logging experts would chime in here. >> >> This looks okay to me. My main concern would be that everything is torn >> down in the correct order for this to work reliably. > > > +1 > > I think it's okey since we always copy the the string to the internal > buffer. > > Thanks, Robbin > > >> >> Thanks, >> David >> >>> On Thu, Apr 26, 2018 at 8:34 AM, Thomas St?fe >>> wrote: >>>> >>>> Hi all, >>>> >>>> please review this tiny change: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8202303 >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8202303-logstream-autoflush/webrev.00/webrev/ >>>> >>>> Basically, LogStream should not assert upon desctruction when having >>>> buffered unfinished lines; instead, it should autoflush. >>>> >>>> (Note: the changes to the TestLineBufferAllocation test are just to >>>> keep this test in line with the rest of the tests in regards to how >>>> the fake log file is handled). >>>> >>>> More details in my earlier mail to hs-runtime: >>>> >>>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-April/027592.html >>>> >>>> Thanks, Thomas From thomas.stuefe at gmail.com Thu May 3 05:11:50 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 3 May 2018 07:11:50 +0200 Subject: RFR(xxs): 8202303: LogStream should autoflush on destruction In-Reply-To: <1e878959-0840-9954-d8af-149e25d153d7@redhat.com> References: <1e878959-0840-9954-d8af-149e25d153d7@redhat.com> Message-ID: Thanks Zhengyu! On Wed, May 2, 2018 at 12:26 PM, Zhengyu Gu wrote: > Looks good to me. > > -Zhengyu > > > On 05/02/2018 01:26 AM, Thomas St?fe wrote: >> >> Ping... >> >> this is a really simple one :) Please review. Thanks! >> >> On Thu, Apr 26, 2018 at 8:34 AM, Thomas St?fe >> wrote: >>> >>> Hi all, >>> >>> please review this tiny change: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8202303 >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8202303-logstream-autoflush/webrev.00/webrev/ >>> >>> Basically, LogStream should not assert upon desctruction when having >>> buffered unfinished lines; instead, it should autoflush. >>> >>> (Note: the changes to the TestLineBufferAllocation test are just to >>> keep this test in line with the rest of the tests in regards to how >>> the fake log file is handled). >>> >>> More details in my earlier mail to hs-runtime: >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-April/027592.html >>> >>> Thanks, Thomas From volker.simonis at gmail.com Thu May 3 06:48:09 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 3 May 2018 08:48:09 +0200 Subject: Improving AppCDS for Custom Loaders In-Reply-To: References: <8e0d891a-c00c-8f57-362a-6a4684a6cc21@oracle.com> <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> Message-ID: On Thu, May 3, 2018 at 6:52 AM, Ioi Lam wrote: > > > On 5/2/18 10:00 AM, Volker Simonis wrote: >> >> On Tue, May 1, 2018 at 8:32 PM, Ioi Lam wrote: >>> >>> PROBLEM: >>> >>> As discussed with Volker and Yumin in previous e-mails, AppCDS has some >>> experimental support for custom class loaders. However, it's not very >>> easy >>> to use. >>> >>> For example, you can write a classlist like this: >>> >>> java/lang/Object id: 1 >>> CustomLoadee id: 2 super: 1 source: /tmp/foo.jar >>> >>> The CustomLoadee class will be stored in the shared archive with a CRC >>> code. >>> During runtime, if a customed loader wants to load a class of the same >>> name, >>> and its classfile has the same size and CRC as the archived class, the >>> archived version will be loaded. This speeds up class loading by avoiding >>> parsing the class file, and saves space by sharing the mmap'ed class >>> metadata across processes. >>> >>> You can see an example test at: >>> >>> >>> http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java >>> >>> However, the current scheme requires you to specify all the super classes >>> and interfaces. There's no support provided by the >>> -XX:DumpLoadedClassList >>> option. It can be helped somewhat with Volker's tool: >>> https://github.com/simonis/cl4cds >>> >>> >>> POSSIBLE SOLUTIONS: >>> >>> 1. "Dump-as-you-go". As suggested by Yumin, we can provide a jcmd to ask >>> a >>> running JVM process to dump all of its loaded classes, including those >>> loaded by custom loaders, into an archive. An alternative is to dump the >>> archive at JVM exit time (or when you press Ctrl-C, etc. >>> >>> 2. Add information about the custom classes for -XX:DumpLoadedClassList. >>> The >>> trouble is some class loaders don't specify a code source that can be >>> understood by the built-in class loaders. For example, the "Fat Jars" >>> would >>> have a code source like >>> >>> >>> >>> jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/ >>> >>> also, many custom loaders would pre-process the classfile data before >>> defining the class, so we can't simply archive the version of the class >>> on >>> disk. >>> >>> One possible solution for #2 is to include the class file data in the >>> -XX:DumpLoadedClassList output: >>> >>> >>> java/lang/Object id: 1 >>> CustomLoadee id: 2 super: 1 source: base64 >>> >>> >>> yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0 >>> >>> >>> PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA >>> >>> >>> CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k >>> >>> >>> SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA >>> >>> >>> AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA >>> AgAPAAcAAAAKAAEABQADAAYACA== >>> >>> >>> Of the 2 solutions: >>> >>> #1 seems easier to use, but may require more invasive modifications in >>> the >>> VM, especially if you want to be able to continue execution after >>> dumping. >>> >> Not sure what #1 really proposes: dumping the complete .jsa archive at >> runtime or dumping just the loaded classes. >> >> If it's just about dumping the loaded class without generating the >> .jsa archive there's the problem that by default the VM doesn't store >> the exact bytes of a class after the class was loaded (except when >> class transformers are registered). So the class files would have to >> be re-assembled from the internal VM structures (in the same way this >> is done for class redefinition) and the resulting class-file may be >> different from the original bytes (i.e. some attributes may be >> missing). > > #1 is for creating the JSA file, not just dumping the class files. >> >> If #1 is about creating the whole .jsa archive at runtime (or at VM >> exit) I think that would be the most attractive solution from a >> usability point of view although I understand that #2 will be easier >> to implement in the short term. Regarding the argument that #1 will >> produce a "binary blob" that's true, but that's already true now when >> we use "Xshare:dump". I think it should be not to hard to implement a >> tool based an SA which could introspect a .jsa archive. > > The argument about the binary blob is to compare it against the text file > produced by -XX:DumpLoadedClassList. > > One use case to consider is when you have a JAR file that contains several > apps that each load a unique set of classes. Today, (assuming that custom > class loaders are not used), you can run each app once with > -XX:DumpLoadedClassList, and then do an > > cat *.classlist | sort | uniq > combined.classlist > > and then create an archive that would work for all these apps. > But is this really y relevant use case? Why would I like to create ONE archive for several apps? This would actually increase the footprint of a single instance which uses this archive. If I have several apps I would expect that users create a specific archive for each app to get the best out of CDS. > With the binary blob, there's no easy way of doing this. It will be very > difficult to write a tool to decipher each blob and then somehow combine > them into a single one. > But if users really wants such a "fat" archive, there's a much easier way: just dump ALL the classes from the .jar file into the archive. A class list for this could easily be assembled either with an external tool like cl4cds (or even a simple shell scripts which converts the output of `unzip -l ` into the correct format). Or, even simpler, by adding a new option to the VM similar to -XX:DumpLoadedClassList which dumps all the classes it can find on the class path (and potentially other, configurable locations). > Thanks > - Ioi > > >>> #2 would be easier to implement, but the classlist might be huge. >>> >>> Also, #2 would allow post-processing tools to remove unneeded classes, or >>> merge two runs into a single list. The output of #1 is essentially a >>> binary >>> blob that's impossible for off-line analysis/optimizations. >>> >>> >>> Any comments, or suggestions for alternatives? >>> >>> Thanks >>> - Ioi >>> >>> > From david.holmes at oracle.com Thu May 3 06:55:56 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 3 May 2018 16:55:56 +1000 Subject: Improving AppCDS for Custom Loaders In-Reply-To: References: <8e0d891a-c00c-8f57-362a-6a4684a6cc21@oracle.com> <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> Message-ID: <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> Just lurking here but ... > But is this really y relevant use case? Why would I like to create ONE > archive for several apps? This would actually increase the footprint > of a single instance which uses this archive. If I have several apps I > would expect that users create a specific archive for each app to get > the best out of CDS. One app instance may get increased footprint but you presumably use CDS because you have multiple apps running (whether the same or not). These apps all share the core JDK classes from the archive so the overall footprint per instance is less. David ----- On 3/05/2018 4:48 PM, Volker Simonis wrote: > On Thu, May 3, 2018 at 6:52 AM, Ioi Lam wrote: >> >> >> On 5/2/18 10:00 AM, Volker Simonis wrote: >>> >>> On Tue, May 1, 2018 at 8:32 PM, Ioi Lam wrote: >>>> >>>> PROBLEM: >>>> >>>> As discussed with Volker and Yumin in previous e-mails, AppCDS has some >>>> experimental support for custom class loaders. However, it's not very >>>> easy >>>> to use. >>>> >>>> For example, you can write a classlist like this: >>>> >>>> java/lang/Object id: 1 >>>> CustomLoadee id: 2 super: 1 source: /tmp/foo.jar >>>> >>>> The CustomLoadee class will be stored in the shared archive with a CRC >>>> code. >>>> During runtime, if a customed loader wants to load a class of the same >>>> name, >>>> and its classfile has the same size and CRC as the archived class, the >>>> archived version will be loaded. This speeds up class loading by avoiding >>>> parsing the class file, and saves space by sharing the mmap'ed class >>>> metadata across processes. >>>> >>>> You can see an example test at: >>>> >>>> >>>> http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java >>>> >>>> However, the current scheme requires you to specify all the super classes >>>> and interfaces. There's no support provided by the >>>> -XX:DumpLoadedClassList >>>> option. It can be helped somewhat with Volker's tool: >>>> https://github.com/simonis/cl4cds >>>> >>>> >>>> POSSIBLE SOLUTIONS: >>>> >>>> 1. "Dump-as-you-go". As suggested by Yumin, we can provide a jcmd to ask >>>> a >>>> running JVM process to dump all of its loaded classes, including those >>>> loaded by custom loaders, into an archive. An alternative is to dump the >>>> archive at JVM exit time (or when you press Ctrl-C, etc. >>>> >>>> 2. Add information about the custom classes for -XX:DumpLoadedClassList. >>>> The >>>> trouble is some class loaders don't specify a code source that can be >>>> understood by the built-in class loaders. For example, the "Fat Jars" >>>> would >>>> have a code source like >>>> >>>> >>>> >>>> jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/ >>>> >>>> also, many custom loaders would pre-process the classfile data before >>>> defining the class, so we can't simply archive the version of the class >>>> on >>>> disk. >>>> >>>> One possible solution for #2 is to include the class file data in the >>>> -XX:DumpLoadedClassList output: >>>> >>>> >>>> java/lang/Object id: 1 >>>> CustomLoadee id: 2 super: 1 source: base64 >>>> >>>> >>>> yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0 >>>> >>>> >>>> PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA >>>> >>>> >>>> CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k >>>> >>>> >>>> SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA >>>> >>>> >>>> AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA >>>> AgAPAAcAAAAKAAEABQADAAYACA== >>>> >>>> >>>> Of the 2 solutions: >>>> >>>> #1 seems easier to use, but may require more invasive modifications in >>>> the >>>> VM, especially if you want to be able to continue execution after >>>> dumping. >>>> >>> Not sure what #1 really proposes: dumping the complete .jsa archive at >>> runtime or dumping just the loaded classes. >>> >>> If it's just about dumping the loaded class without generating the >>> .jsa archive there's the problem that by default the VM doesn't store >>> the exact bytes of a class after the class was loaded (except when >>> class transformers are registered). So the class files would have to >>> be re-assembled from the internal VM structures (in the same way this >>> is done for class redefinition) and the resulting class-file may be >>> different from the original bytes (i.e. some attributes may be >>> missing). >> >> #1 is for creating the JSA file, not just dumping the class files. >>> >>> If #1 is about creating the whole .jsa archive at runtime (or at VM >>> exit) I think that would be the most attractive solution from a >>> usability point of view although I understand that #2 will be easier >>> to implement in the short term. Regarding the argument that #1 will >>> produce a "binary blob" that's true, but that's already true now when >>> we use "Xshare:dump". I think it should be not to hard to implement a >>> tool based an SA which could introspect a .jsa archive. >> >> The argument about the binary blob is to compare it against the text file >> produced by -XX:DumpLoadedClassList. >> >> One use case to consider is when you have a JAR file that contains several >> apps that each load a unique set of classes. Today, (assuming that custom >> class loaders are not used), you can run each app once with >> -XX:DumpLoadedClassList, and then do an >> >> cat *.classlist | sort | uniq > combined.classlist >> >> and then create an archive that would work for all these apps. >> > > But is this really y relevant use case? Why would I like to create ONE > archive for several apps? This would actually increase the footprint > of a single instance which uses this archive. If I have several apps I > would expect that users create a specific archive for each app to get > the best out of CDS. > >> With the binary blob, there's no easy way of doing this. It will be very >> difficult to write a tool to decipher each blob and then somehow combine >> them into a single one. >> > > But if users really wants such a "fat" archive, there's a much easier > way: just dump ALL the classes from the .jar file into the archive. A > class list for this could easily be assembled either with an external > tool like cl4cds (or even a simple shell scripts which converts the > output of `unzip -l ` into the correct format). Or, even > simpler, by adding a new option to the VM similar to > -XX:DumpLoadedClassList which dumps all the classes it can find on the > class path (and potentially other, configurable locations). > >> Thanks >> - Ioi >> >> >>>> #2 would be easier to implement, but the classlist might be huge. >>>> >>>> Also, #2 would allow post-processing tools to remove unneeded classes, or >>>> merge two runs into a single list. The output of #1 is essentially a >>>> binary >>>> blob that's impossible for off-line analysis/optimizations. >>>> >>>> >>>> Any comments, or suggestions for alternatives? >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >> From volker.simonis at gmail.com Thu May 3 07:16:36 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 3 May 2018 09:16:36 +0200 Subject: Improving AppCDS for Custom Loaders In-Reply-To: <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> References: <8e0d891a-c00c-8f57-362a-6a4684a6cc21@oracle.com> <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> Message-ID: On Thu, May 3, 2018 at 8:55 AM, David Holmes wrote: > Just lurking here but ... > >> But is this really y relevant use case? Why would I like to create ONE >> archive for several apps? This would actually increase the footprint >> of a single instance which uses this archive. If I have several apps I >> would expect that users create a specific archive for each app to get >> the best out of CDS. > > > One app instance may get increased footprint but you presumably use CDS > because you have multiple apps running (whether the same or not). These apps > all share the core JDK classes from the archive so the overall footprint per > instance is less. > If we just want to share the core JDK classes that's easy. For that we could mostly use the default class list (or a slightly extended one) which is generated at JDK build time (at JAVA_HOME/lib/classlist). If we want to use ONE archive for several applications and we can accept to have a bigger footprint if running a single (or just a few) applications in parallel I suppose the overhead of simply dumping all the classes from the classpathes of the various applications compared to an accurate solution where we only dump the actually used classes of all applications would be not that big. > David > ----- > > > On 3/05/2018 4:48 PM, Volker Simonis wrote: >> >> On Thu, May 3, 2018 at 6:52 AM, Ioi Lam wrote: >>> >>> >>> >>> On 5/2/18 10:00 AM, Volker Simonis wrote: >>>> >>>> >>>> On Tue, May 1, 2018 at 8:32 PM, Ioi Lam wrote: >>>>> >>>>> >>>>> PROBLEM: >>>>> >>>>> As discussed with Volker and Yumin in previous e-mails, AppCDS has some >>>>> experimental support for custom class loaders. However, it's not very >>>>> easy >>>>> to use. >>>>> >>>>> For example, you can write a classlist like this: >>>>> >>>>> java/lang/Object id: 1 >>>>> CustomLoadee id: 2 super: 1 source: /tmp/foo.jar >>>>> >>>>> The CustomLoadee class will be stored in the shared archive with a CRC >>>>> code. >>>>> During runtime, if a customed loader wants to load a class of the same >>>>> name, >>>>> and its classfile has the same size and CRC as the archived class, the >>>>> archived version will be loaded. This speeds up class loading by >>>>> avoiding >>>>> parsing the class file, and saves space by sharing the mmap'ed class >>>>> metadata across processes. >>>>> >>>>> You can see an example test at: >>>>> >>>>> >>>>> >>>>> http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java >>>>> >>>>> However, the current scheme requires you to specify all the super >>>>> classes >>>>> and interfaces. There's no support provided by the >>>>> -XX:DumpLoadedClassList >>>>> option. It can be helped somewhat with Volker's tool: >>>>> https://github.com/simonis/cl4cds >>>>> >>>>> >>>>> POSSIBLE SOLUTIONS: >>>>> >>>>> 1. "Dump-as-you-go". As suggested by Yumin, we can provide a jcmd to >>>>> ask >>>>> a >>>>> running JVM process to dump all of its loaded classes, including those >>>>> loaded by custom loaders, into an archive. An alternative is to dump >>>>> the >>>>> archive at JVM exit time (or when you press Ctrl-C, etc. >>>>> >>>>> 2. Add information about the custom classes for >>>>> -XX:DumpLoadedClassList. >>>>> The >>>>> trouble is some class loaders don't specify a code source that can be >>>>> understood by the built-in class loaders. For example, the "Fat Jars" >>>>> would >>>>> have a code source like >>>>> >>>>> >>>>> >>>>> >>>>> jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/ >>>>> >>>>> also, many custom loaders would pre-process the classfile data before >>>>> defining the class, so we can't simply archive the version of the class >>>>> on >>>>> disk. >>>>> >>>>> One possible solution for #2 is to include the class file data in the >>>>> -XX:DumpLoadedClassList output: >>>>> >>>>> >>>>> java/lang/Object id: 1 >>>>> CustomLoadee id: 2 super: 1 source: base64 >>>>> >>>>> >>>>> >>>>> yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0 >>>>> >>>>> >>>>> >>>>> PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA >>>>> >>>>> >>>>> >>>>> CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k >>>>> >>>>> >>>>> >>>>> SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA >>>>> >>>>> >>>>> >>>>> AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA >>>>> AgAPAAcAAAAKAAEABQADAAYACA== >>>>> >>>>> >>>>> Of the 2 solutions: >>>>> >>>>> #1 seems easier to use, but may require more invasive modifications in >>>>> the >>>>> VM, especially if you want to be able to continue execution after >>>>> dumping. >>>>> >>>> Not sure what #1 really proposes: dumping the complete .jsa archive at >>>> runtime or dumping just the loaded classes. >>>> >>>> If it's just about dumping the loaded class without generating the >>>> .jsa archive there's the problem that by default the VM doesn't store >>>> the exact bytes of a class after the class was loaded (except when >>>> class transformers are registered). So the class files would have to >>>> be re-assembled from the internal VM structures (in the same way this >>>> is done for class redefinition) and the resulting class-file may be >>>> different from the original bytes (i.e. some attributes may be >>>> missing). >>> >>> >>> #1 is for creating the JSA file, not just dumping the class files. >>>> >>>> >>>> If #1 is about creating the whole .jsa archive at runtime (or at VM >>>> exit) I think that would be the most attractive solution from a >>>> usability point of view although I understand that #2 will be easier >>>> to implement in the short term. Regarding the argument that #1 will >>>> produce a "binary blob" that's true, but that's already true now when >>>> we use "Xshare:dump". I think it should be not to hard to implement a >>>> tool based an SA which could introspect a .jsa archive. >>> >>> >>> The argument about the binary blob is to compare it against the text file >>> produced by -XX:DumpLoadedClassList. >>> >>> One use case to consider is when you have a JAR file that contains >>> several >>> apps that each load a unique set of classes. Today, (assuming that custom >>> class loaders are not used), you can run each app once with >>> -XX:DumpLoadedClassList, and then do an >>> >>> cat *.classlist | sort | uniq > combined.classlist >>> >>> and then create an archive that would work for all these apps. >>> >> >> But is this really y relevant use case? Why would I like to create ONE >> archive for several apps? This would actually increase the footprint >> of a single instance which uses this archive. If I have several apps I >> would expect that users create a specific archive for each app to get >> the best out of CDS. >> >>> With the binary blob, there's no easy way of doing this. It will be very >>> difficult to write a tool to decipher each blob and then somehow combine >>> them into a single one. >>> >> >> But if users really wants such a "fat" archive, there's a much easier >> way: just dump ALL the classes from the .jar file into the archive. A >> class list for this could easily be assembled either with an external >> tool like cl4cds (or even a simple shell scripts which converts the >> output of `unzip -l ` into the correct format). Or, even >> simpler, by adding a new option to the VM similar to >> -XX:DumpLoadedClassList which dumps all the classes it can find on the >> class path (and potentially other, configurable locations). >> >>> Thanks >>> - Ioi >>> >>> >>>>> #2 would be easier to implement, but the classlist might be huge. >>>>> >>>>> Also, #2 would allow post-processing tools to remove unneeded classes, >>>>> or >>>>> merge two runs into a single list. The output of #1 is essentially a >>>>> binary >>>>> blob that's impossible for off-line analysis/optimizations. >>>>> >>>>> >>>>> Any comments, or suggestions for alternatives? >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>> > From david.holmes at oracle.com Thu May 3 09:01:31 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 3 May 2018 19:01:31 +1000 Subject: Improving AppCDS for Custom Loaders In-Reply-To: References: <8e0d891a-c00c-8f57-362a-6a4684a6cc21@oracle.com> <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> Message-ID: <732ce957-b07c-bff2-277b-ca335c348753@oracle.com> On 3/05/2018 5:16 PM, Volker Simonis wrote: > On Thu, May 3, 2018 at 8:55 AM, David Holmes wrote: >> Just lurking here but ... >> >>> But is this really y relevant use case? Why would I like to create ONE >>> archive for several apps? This would actually increase the footprint >>> of a single instance which uses this archive. If I have several apps I >>> would expect that users create a specific archive for each app to get >>> the best out of CDS. >> >> >> One app instance may get increased footprint but you presumably use CDS >> because you have multiple apps running (whether the same or not). These apps >> all share the core JDK classes from the archive so the overall footprint per >> instance is less. >> > > If we just want to share the core JDK classes that's easy. For that we > could mostly use the default class list (or a slightly extended one) > which is generated at JDK build time (at JAVA_HOME/lib/classlist). The point is that you are presumably running multiple instances of multiple apps, hence you want to share one set of core classes across all, and share the app classes across each app instance. > If we want to use ONE archive for several applications and we can > accept to have a bigger footprint if running a single (or just a few) > applications in parallel I suppose the overhead of simply dumping all > the classes from the classpathes of the various applications compared > to an accurate solution where we only dump the actually used classes > of all applications would be not that big. But those "accurate" solutions duplicate the core classes and that's a waste of footprint. David ----- >> David >> ----- >> >> >> On 3/05/2018 4:48 PM, Volker Simonis wrote: >>> >>> On Thu, May 3, 2018 at 6:52 AM, Ioi Lam wrote: >>>> >>>> >>>> >>>> On 5/2/18 10:00 AM, Volker Simonis wrote: >>>>> >>>>> >>>>> On Tue, May 1, 2018 at 8:32 PM, Ioi Lam wrote: >>>>>> >>>>>> >>>>>> PROBLEM: >>>>>> >>>>>> As discussed with Volker and Yumin in previous e-mails, AppCDS has some >>>>>> experimental support for custom class loaders. However, it's not very >>>>>> easy >>>>>> to use. >>>>>> >>>>>> For example, you can write a classlist like this: >>>>>> >>>>>> java/lang/Object id: 1 >>>>>> CustomLoadee id: 2 super: 1 source: /tmp/foo.jar >>>>>> >>>>>> The CustomLoadee class will be stored in the shared archive with a CRC >>>>>> code. >>>>>> During runtime, if a customed loader wants to load a class of the same >>>>>> name, >>>>>> and its classfile has the same size and CRC as the archived class, the >>>>>> archived version will be loaded. This speeds up class loading by >>>>>> avoiding >>>>>> parsing the class file, and saves space by sharing the mmap'ed class >>>>>> metadata across processes. >>>>>> >>>>>> You can see an example test at: >>>>>> >>>>>> >>>>>> >>>>>> http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java >>>>>> >>>>>> However, the current scheme requires you to specify all the super >>>>>> classes >>>>>> and interfaces. There's no support provided by the >>>>>> -XX:DumpLoadedClassList >>>>>> option. It can be helped somewhat with Volker's tool: >>>>>> https://github.com/simonis/cl4cds >>>>>> >>>>>> >>>>>> POSSIBLE SOLUTIONS: >>>>>> >>>>>> 1. "Dump-as-you-go". As suggested by Yumin, we can provide a jcmd to >>>>>> ask >>>>>> a >>>>>> running JVM process to dump all of its loaded classes, including those >>>>>> loaded by custom loaders, into an archive. An alternative is to dump >>>>>> the >>>>>> archive at JVM exit time (or when you press Ctrl-C, etc. >>>>>> >>>>>> 2. Add information about the custom classes for >>>>>> -XX:DumpLoadedClassList. >>>>>> The >>>>>> trouble is some class loaders don't specify a code source that can be >>>>>> understood by the built-in class loaders. For example, the "Fat Jars" >>>>>> would >>>>>> have a code source like >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/ >>>>>> >>>>>> also, many custom loaders would pre-process the classfile data before >>>>>> defining the class, so we can't simply archive the version of the class >>>>>> on >>>>>> disk. >>>>>> >>>>>> One possible solution for #2 is to include the class file data in the >>>>>> -XX:DumpLoadedClassList output: >>>>>> >>>>>> >>>>>> java/lang/Object id: 1 >>>>>> CustomLoadee id: 2 super: 1 source: base64 >>>>>> >>>>>> >>>>>> >>>>>> yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0 >>>>>> >>>>>> >>>>>> >>>>>> PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA >>>>>> >>>>>> >>>>>> >>>>>> CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k >>>>>> >>>>>> >>>>>> >>>>>> SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA >>>>>> >>>>>> >>>>>> >>>>>> AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA >>>>>> AgAPAAcAAAAKAAEABQADAAYACA== >>>>>> >>>>>> >>>>>> Of the 2 solutions: >>>>>> >>>>>> #1 seems easier to use, but may require more invasive modifications in >>>>>> the >>>>>> VM, especially if you want to be able to continue execution after >>>>>> dumping. >>>>>> >>>>> Not sure what #1 really proposes: dumping the complete .jsa archive at >>>>> runtime or dumping just the loaded classes. >>>>> >>>>> If it's just about dumping the loaded class without generating the >>>>> .jsa archive there's the problem that by default the VM doesn't store >>>>> the exact bytes of a class after the class was loaded (except when >>>>> class transformers are registered). So the class files would have to >>>>> be re-assembled from the internal VM structures (in the same way this >>>>> is done for class redefinition) and the resulting class-file may be >>>>> different from the original bytes (i.e. some attributes may be >>>>> missing). >>>> >>>> >>>> #1 is for creating the JSA file, not just dumping the class files. >>>>> >>>>> >>>>> If #1 is about creating the whole .jsa archive at runtime (or at VM >>>>> exit) I think that would be the most attractive solution from a >>>>> usability point of view although I understand that #2 will be easier >>>>> to implement in the short term. Regarding the argument that #1 will >>>>> produce a "binary blob" that's true, but that's already true now when >>>>> we use "Xshare:dump". I think it should be not to hard to implement a >>>>> tool based an SA which could introspect a .jsa archive. >>>> >>>> >>>> The argument about the binary blob is to compare it against the text file >>>> produced by -XX:DumpLoadedClassList. >>>> >>>> One use case to consider is when you have a JAR file that contains >>>> several >>>> apps that each load a unique set of classes. Today, (assuming that custom >>>> class loaders are not used), you can run each app once with >>>> -XX:DumpLoadedClassList, and then do an >>>> >>>> cat *.classlist | sort | uniq > combined.classlist >>>> >>>> and then create an archive that would work for all these apps. >>>> >>> >>> But is this really y relevant use case? Why would I like to create ONE >>> archive for several apps? This would actually increase the footprint >>> of a single instance which uses this archive. If I have several apps I >>> would expect that users create a specific archive for each app to get >>> the best out of CDS. >>> >>>> With the binary blob, there's no easy way of doing this. It will be very >>>> difficult to write a tool to decipher each blob and then somehow combine >>>> them into a single one. >>>> >>> >>> But if users really wants such a "fat" archive, there's a much easier >>> way: just dump ALL the classes from the .jar file into the archive. A >>> class list for this could easily be assembled either with an external >>> tool like cl4cds (or even a simple shell scripts which converts the >>> output of `unzip -l ` into the correct format). Or, even >>> simpler, by adding a new option to the VM similar to >>> -XX:DumpLoadedClassList which dumps all the classes it can find on the >>> class path (and potentially other, configurable locations). >>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>>>> #2 would be easier to implement, but the classlist might be huge. >>>>>> >>>>>> Also, #2 would allow post-processing tools to remove unneeded classes, >>>>>> or >>>>>> merge two runs into a single list. The output of #1 is essentially a >>>>>> binary >>>>>> blob that's impossible for off-line analysis/optimizations. >>>>>> >>>>>> >>>>>> Any comments, or suggestions for alternatives? >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>> >> From volker.simonis at gmail.com Thu May 3 09:14:09 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 3 May 2018 11:14:09 +0200 Subject: Improving AppCDS for Custom Loaders In-Reply-To: <732ce957-b07c-bff2-277b-ca335c348753@oracle.com> References: <8e0d891a-c00c-8f57-362a-6a4684a6cc21@oracle.com> <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> <732ce957-b07c-bff2-277b-ca335c348753@oracle.com> Message-ID: On Thu, May 3, 2018 at 11:01 AM, David Holmes wrote: > On 3/05/2018 5:16 PM, Volker Simonis wrote: >> >> On Thu, May 3, 2018 at 8:55 AM, David Holmes >> wrote: >>> >>> Just lurking here but ... >>> >>>> But is this really y relevant use case? Why would I like to create ONE >>>> archive for several apps? This would actually increase the footprint >>>> of a single instance which uses this archive. If I have several apps I >>>> would expect that users create a specific archive for each app to get >>>> the best out of CDS. >>> >>> >>> >>> One app instance may get increased footprint but you presumably use CDS >>> because you have multiple apps running (whether the same or not). These >>> apps >>> all share the core JDK classes from the archive so the overall footprint >>> per >>> instance is less. >>> >> >> If we just want to share the core JDK classes that's easy. For that we >> could mostly use the default class list (or a slightly extended one) >> which is generated at JDK build time (at JAVA_HOME/lib/classlist). > > > The point is that you are presumably running multiple instances of multiple > apps, hence you want to share one set of core classes across all, and share > the app classes across each app instance. > But that would require two archives: a general one with the core classes and an application specific one for each application. Combining the core classes and the application of various applications will not be optimal because the application classes will be all mixed in the same archive. The archive is being mapped page-wise into the java process so you'll probably end up mapping the whole archive into each process although you'll only use a fraction of the classes in the archive. >> If we want to use ONE archive for several applications and we can >> accept to have a bigger footprint if running a single (or just a few) >> applications in parallel I suppose the overhead of simply dumping all >> the classes from the classpathes of the various applications compared >> to an accurate solution where we only dump the actually used classes >> of all applications would be not that big. > > > But those "accurate" solutions duplicate the core classes and that's a waste > of footprint. > By "accurate" I meant one "fat" archive which contains all the classes USED by several applications plus the core classes. My argument was that such an "accurate" "fat" archive won't be much smaller compared to a "fat" archive which simply contains all the core classes plus all the application classes (i.e. from the application class pathes, no matter if they are ever used or not). But the latter would be much simpler to implement. > David > ----- > > >>> David >>> ----- >>> >>> >>> On 3/05/2018 4:48 PM, Volker Simonis wrote: >>>> >>>> >>>> On Thu, May 3, 2018 at 6:52 AM, Ioi Lam wrote: >>>>> >>>>> >>>>> >>>>> >>>>> On 5/2/18 10:00 AM, Volker Simonis wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Tue, May 1, 2018 at 8:32 PM, Ioi Lam wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> PROBLEM: >>>>>>> >>>>>>> As discussed with Volker and Yumin in previous e-mails, AppCDS has >>>>>>> some >>>>>>> experimental support for custom class loaders. However, it's not very >>>>>>> easy >>>>>>> to use. >>>>>>> >>>>>>> For example, you can write a classlist like this: >>>>>>> >>>>>>> java/lang/Object id: 1 >>>>>>> CustomLoadee id: 2 super: 1 source: /tmp/foo.jar >>>>>>> >>>>>>> The CustomLoadee class will be stored in the shared archive with a >>>>>>> CRC >>>>>>> code. >>>>>>> During runtime, if a customed loader wants to load a class of the >>>>>>> same >>>>>>> name, >>>>>>> and its classfile has the same size and CRC as the archived class, >>>>>>> the >>>>>>> archived version will be loaded. This speeds up class loading by >>>>>>> avoiding >>>>>>> parsing the class file, and saves space by sharing the mmap'ed class >>>>>>> metadata across processes. >>>>>>> >>>>>>> You can see an example test at: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java >>>>>>> >>>>>>> However, the current scheme requires you to specify all the super >>>>>>> classes >>>>>>> and interfaces. There's no support provided by the >>>>>>> -XX:DumpLoadedClassList >>>>>>> option. It can be helped somewhat with Volker's tool: >>>>>>> https://github.com/simonis/cl4cds >>>>>>> >>>>>>> >>>>>>> POSSIBLE SOLUTIONS: >>>>>>> >>>>>>> 1. "Dump-as-you-go". As suggested by Yumin, we can provide a jcmd to >>>>>>> ask >>>>>>> a >>>>>>> running JVM process to dump all of its loaded classes, including >>>>>>> those >>>>>>> loaded by custom loaders, into an archive. An alternative is to dump >>>>>>> the >>>>>>> archive at JVM exit time (or when you press Ctrl-C, etc. >>>>>>> >>>>>>> 2. Add information about the custom classes for >>>>>>> -XX:DumpLoadedClassList. >>>>>>> The >>>>>>> trouble is some class loaders don't specify a code source that can be >>>>>>> understood by the built-in class loaders. For example, the "Fat Jars" >>>>>>> would >>>>>>> have a code source like >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/ >>>>>>> >>>>>>> also, many custom loaders would pre-process the classfile data before >>>>>>> defining the class, so we can't simply archive the version of the >>>>>>> class >>>>>>> on >>>>>>> disk. >>>>>>> >>>>>>> One possible solution for #2 is to include the class file data in the >>>>>>> -XX:DumpLoadedClassList output: >>>>>>> >>>>>>> >>>>>>> java/lang/Object id: 1 >>>>>>> CustomLoadee id: 2 super: 1 source: base64 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA >>>>>>> AgAPAAcAAAAKAAEABQADAAYACA== >>>>>>> >>>>>>> >>>>>>> Of the 2 solutions: >>>>>>> >>>>>>> #1 seems easier to use, but may require more invasive modifications >>>>>>> in >>>>>>> the >>>>>>> VM, especially if you want to be able to continue execution after >>>>>>> dumping. >>>>>>> >>>>>> Not sure what #1 really proposes: dumping the complete .jsa archive at >>>>>> runtime or dumping just the loaded classes. >>>>>> >>>>>> If it's just about dumping the loaded class without generating the >>>>>> .jsa archive there's the problem that by default the VM doesn't store >>>>>> the exact bytes of a class after the class was loaded (except when >>>>>> class transformers are registered). So the class files would have to >>>>>> be re-assembled from the internal VM structures (in the same way this >>>>>> is done for class redefinition) and the resulting class-file may be >>>>>> different from the original bytes (i.e. some attributes may be >>>>>> missing). >>>>> >>>>> >>>>> >>>>> #1 is for creating the JSA file, not just dumping the class files. >>>>>> >>>>>> >>>>>> >>>>>> If #1 is about creating the whole .jsa archive at runtime (or at VM >>>>>> exit) I think that would be the most attractive solution from a >>>>>> usability point of view although I understand that #2 will be easier >>>>>> to implement in the short term. Regarding the argument that #1 will >>>>>> produce a "binary blob" that's true, but that's already true now when >>>>>> we use "Xshare:dump". I think it should be not to hard to implement a >>>>>> tool based an SA which could introspect a .jsa archive. >>>>> >>>>> >>>>> >>>>> The argument about the binary blob is to compare it against the text >>>>> file >>>>> produced by -XX:DumpLoadedClassList. >>>>> >>>>> One use case to consider is when you have a JAR file that contains >>>>> several >>>>> apps that each load a unique set of classes. Today, (assuming that >>>>> custom >>>>> class loaders are not used), you can run each app once with >>>>> -XX:DumpLoadedClassList, and then do an >>>>> >>>>> cat *.classlist | sort | uniq > combined.classlist >>>>> >>>>> and then create an archive that would work for all these apps. >>>>> >>>> >>>> But is this really y relevant use case? Why would I like to create ONE >>>> archive for several apps? This would actually increase the footprint >>>> of a single instance which uses this archive. If I have several apps I >>>> would expect that users create a specific archive for each app to get >>>> the best out of CDS. >>>> >>>>> With the binary blob, there's no easy way of doing this. It will be >>>>> very >>>>> difficult to write a tool to decipher each blob and then somehow >>>>> combine >>>>> them into a single one. >>>>> >>>> >>>> But if users really wants such a "fat" archive, there's a much easier >>>> way: just dump ALL the classes from the .jar file into the archive. A >>>> class list for this could easily be assembled either with an external >>>> tool like cl4cds (or even a simple shell scripts which converts the >>>> output of `unzip -l ` into the correct format). Or, even >>>> simpler, by adding a new option to the VM similar to >>>> -XX:DumpLoadedClassList which dumps all the classes it can find on the >>>> class path (and potentially other, configurable locations). >>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>>>> #2 would be easier to implement, but the classlist might be huge. >>>>>>> >>>>>>> Also, #2 would allow post-processing tools to remove unneeded >>>>>>> classes, >>>>>>> or >>>>>>> merge two runs into a single list. The output of #1 is essentially a >>>>>>> binary >>>>>>> blob that's impossible for off-line analysis/optimizations. >>>>>>> >>>>>>> >>>>>>> Any comments, or suggestions for alternatives? >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>> >>> > From martin.doerr at sap.com Thu May 3 10:50:38 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 3 May 2018 10:50:38 +0000 Subject: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics Message-ID: <4a3b4d59c9f54234b0c0d325a3238cbc@sap.com> Hi Lutz, thanks for reviewing. Can I get more reviews, please? Best regards, Martin -----Original Message----- From: Schmidt, Lutz Sent: Mittwoch, 2. Mai 2018 16:20 To: Doerr, Martin ; David Holmes ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley (aph at redhat.com) ; John Paul Adrian Glaubitz Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics Hi Martin, your changes look good. For the additional synchronization on s390 in particular, I do not expect a serious performance impact. The checkpoint synchronization part has always been the expensive, but rarely required, part. PPC may be another story. We knoow that a full-blown sync really hurts. We will see... Thanks, Lutz ?On 26.04.18, 16:23, "hotspot-runtime-dev on behalf of Doerr, Martin" wrote: Hi, I have renamed "cmpxchg_memory_order" to "atomic_memory_order" and added support to all Read-Modify-Write atomics with support for all C++11 semantics except seq_cst. The latter has issues on PPC64 so we're currently using our own memory_order_conservative instead, which is the default. Please review my new webrev: http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.01/ We'll test Windows, MacOS, linux x86+ppc64+s390, AIX and Solaris SPARC. It'd be great if somebody could check arm/aarch64 and zero. This change may enable optimizations for arm/aarch64 (not yet included). Best regards, Martin -----Original Message----- From: Doerr, Martin Sent: Donnerstag, 26. April 2018 11:20 To: 'David Holmes' ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) Subject: RE: RFR(M): 8202080: Introduce ordering semantics for Atomic::add Hi David, sounds like we are on the same page, now. I think we should have some kind of summary of what was analyzed. But that belongs to the other thread (JDK- 8154736). > The bugs in this kind of code are very subtle and rarely exposed through normal levels of testing. Right. We have experienced this often enough. So it is definitely in our interest to have correct memory barriers. Mistakes will hit us (PPC64). So I appreciate that you insist on careful analysis. I'll prepare a new webrev with conservative semantics for all read-modify-write atomics by default. Best regards, Martin From lois.foltan at oracle.com Thu May 3 11:47:16 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 3 May 2018 07:47:16 -0400 Subject: (11) RFR (M) JDK-8189916: Dynamic Constant support for Sparc In-Reply-To: <5eb8a3db-535f-4991-2688-c675894e2168@oracle.com> References: <8fb7457a-a734-71e9-0363-4d1eaa232963@oracle.com> <5eb8a3db-535f-4991-2688-c675894e2168@oracle.com> Message-ID: <758566c3-8c74-625f-f9e1-b42abe392c55@oracle.com> Thanks for the review Coleen!? New webrev that addresses both the changes to comments you mention below. http://cr.openjdk.java.net/~lfoltan/bug_jdk8189916.1/webrev/ Thanks, Lois On 5/2/2018 5:47 PM, coleen.phillimore at oracle.com wrote: > > http://cr.openjdk.java.net/~lfoltan/bug_jdk8189916.0/webrev/src/hotspot/cpu/sparc/templateTable_sparc.cpp.udiff.html > > > + { // Check for the null sentinel. > + // If we just called the VM, that already did the mapping for us, > + // but it's harmless to retry. > > Can you change 'that' to 'it' in the comments in all the platforms? > > + // Safe to call w ith 0 result > > > typo. > > Looks good! > Coleen > > On 5/2/18 4:21 PM, Lois Foltan wrote: >> Please review this change to the template interpreter to support >> dynamic constant on Sparc. >> >> open webrev at >> http://cr.openjdk.java.net/~lfoltan/bug_jdk8189916.0/webrev/ >> bug link https://bugs.openjdk.java.net/browse/JDK-8189916 >> >> All current jtreg dynamic constant tests pass when run on Sparc and >> have had (os.arch != "sparcv9") removed from their test description. >> >> Testing: Sparc specific testing (hs-tier1-5, jdk-tier1-3) complete. >> ?????????????? All tier 1 platforms (hs-tier1-2) in progress. >> >> Thanks, >> Lois >> > From robbin.ehn at oracle.com Thu May 3 12:34:02 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 3 May 2018 14:34:02 +0200 Subject: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics In-Reply-To: <4a3b4d59c9f54234b0c0d325a3238cbc@sap.com> References: <4a3b4d59c9f54234b0c0d325a3238cbc@sap.com> Message-ID: Hi Martin, I tried to reply to all but: " Is being held until the list moderator can review it for approval. The reason it is being held: Too many recipients to the message " :) I think this looks good, thanks for fixing! /Robbin On 2018-05-03 12:50, Doerr, Martin wrote: > Hi Lutz, > > thanks for reviewing. > > Can I get more reviews, please? > > Best regards, > Martin > > > -----Original Message----- > From: Schmidt, Lutz > Sent: Mittwoch, 2. Mai 2018 16:20 > To: Doerr, Martin ; David Holmes ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley (aph at redhat.com) ; John Paul Adrian Glaubitz > Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics > > Hi Martin, > > your changes look good. > > For the additional synchronization on s390 in particular, I do not expect a serious performance impact. The checkpoint synchronization part has always been the expensive, but rarely required, part. PPC may be another story. We knoow that a full-blown sync really hurts. We will see... > > Thanks, > Lutz > > ?On 26.04.18, 16:23, "hotspot-runtime-dev on behalf of Doerr, Martin" wrote: > > Hi, > > I have renamed "cmpxchg_memory_order" to "atomic_memory_order" and added support to all Read-Modify-Write atomics with support for all C++11 semantics except seq_cst. The latter has issues on PPC64 so we're currently using our own memory_order_conservative instead, which is the default. > > Please review my new webrev: > http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.01/ > > We'll test Windows, MacOS, linux x86+ppc64+s390, AIX and Solaris SPARC. > It'd be great if somebody could check arm/aarch64 and zero. > > This change may enable optimizations for arm/aarch64 (not yet included). > > Best regards, > Martin > > > -----Original Message----- > From: Doerr, Martin > Sent: Donnerstag, 26. April 2018 11:20 > To: 'David Holmes' ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) > Subject: RE: RFR(M): 8202080: Introduce ordering semantics for Atomic::add > > Hi David, > > sounds like we are on the same page, now. I think we should have some kind of summary of what was analyzed. But that belongs to the other thread (JDK- 8154736). > > > The bugs in this kind of code are very subtle and rarely exposed through normal levels of testing. > Right. We have experienced this often enough. So it is definitely in our interest to have correct memory barriers. Mistakes will hit us (PPC64). So I appreciate that you insist on careful analysis. > > I'll prepare a new webrev with conservative semantics for all read-modify-write atomics by default. > > Best regards, > Martin > > > From martin.doerr at sap.com Thu May 3 12:39:28 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 3 May 2018 12:39:28 +0000 Subject: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics In-Reply-To: References: <4a3b4d59c9f54234b0c0d325a3238cbc@sap.com> Message-ID: Hi Robbin, ah, thanks for letting me know. And for reviewing, of course! I've pushed it to jdk/submit, but did't get a response, yet. Maybe I just need to wait. Best regards, Martin -----Original Message----- From: Robbin Ehn [mailto:robbin.ehn at oracle.com] Sent: Donnerstag, 3. Mai 2018 14:34 To: Doerr, Martin ; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics Hi Martin, I tried to reply to all but: " Is being held until the list moderator can review it for approval. The reason it is being held: Too many recipients to the message " :) I think this looks good, thanks for fixing! /Robbin On 2018-05-03 12:50, Doerr, Martin wrote: > Hi Lutz, > > thanks for reviewing. > > Can I get more reviews, please? > > Best regards, > Martin > > > -----Original Message----- > From: Schmidt, Lutz > Sent: Mittwoch, 2. Mai 2018 16:20 > To: Doerr, Martin ; David Holmes ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley (aph at redhat.com) ; John Paul Adrian Glaubitz > Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics > > Hi Martin, > > your changes look good. > > For the additional synchronization on s390 in particular, I do not expect a serious performance impact. The checkpoint synchronization part has always been the expensive, but rarely required, part. PPC may be another story. We knoow that a full-blown sync really hurts. We will see... > > Thanks, > Lutz > > ?On 26.04.18, 16:23, "hotspot-runtime-dev on behalf of Doerr, Martin" wrote: > > Hi, > > I have renamed "cmpxchg_memory_order" to "atomic_memory_order" and added support to all Read-Modify-Write atomics with support for all C++11 semantics except seq_cst. The latter has issues on PPC64 so we're currently using our own memory_order_conservative instead, which is the default. > > Please review my new webrev: > http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.01/ > > We'll test Windows, MacOS, linux x86+ppc64+s390, AIX and Solaris SPARC. > It'd be great if somebody could check arm/aarch64 and zero. > > This change may enable optimizations for arm/aarch64 (not yet included). > > Best regards, > Martin > > > -----Original Message----- > From: Doerr, Martin > Sent: Donnerstag, 26. April 2018 11:20 > To: 'David Holmes' ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) > Subject: RE: RFR(M): 8202080: Introduce ordering semantics for Atomic::add > > Hi David, > > sounds like we are on the same page, now. I think we should have some kind of summary of what was analyzed. But that belongs to the other thread (JDK- 8154736). > > > The bugs in this kind of code are very subtle and rarely exposed through normal levels of testing. > Right. We have experienced this often enough. So it is definitely in our interest to have correct memory barriers. Mistakes will hit us (PPC64). So I appreciate that you insist on careful analysis. > > I'll prepare a new webrev with conservative semantics for all read-modify-write atomics by default. > > Best regards, > Martin > > > From thomas.stuefe at gmail.com Thu May 3 13:02:39 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 3 May 2018 15:02:39 +0200 Subject: RFR(xxs): 8202424: Metaspace: on chunk retirement, use correct lower limit on chunksize when adding blocks to free blocks list. In-Reply-To: References: Message-ID: Hi all, jdk-submit tests all went thru on this one. I still need reviewers. Thanks, Thomas On Mon, Apr 30, 2018 at 4:07 PM, Thomas St?fe wrote: > Hi all, > > please review this small fix, which fixes a small metaspace leak when > retiring chunks. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202424 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8202424-Metaspace-on-chunk-retirement-use-correct-lower-limit-on-chunksize-when-adding-blocks-to-free-blocks-list/webrev.00/webrev/ > > Some time ago, JDK-8164921 got rid of dark matter by lowering the > smallest block size which can be managed by the free block list to > sizeof(MetaBlock). > > When retiring chunks, the leftover space in the old chunk is added to > the free block list via SpaceManager::deallocate(). Here, we still > unnecessarily assume a lower size limit of TreeChunk FreeList >::min_size(), which is unnecessary high and > causes leaks which show up as waste in the retired chunks. > > By correctly using SmallBlocks::small_block_min_size() when > deallocating the leftover chunk space on chunk retirement, we can > reduce that kind of waste to almost nil. > > Thanks, Thomas From martin.doerr at sap.com Thu May 3 13:03:35 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 3 May 2018 13:03:35 +0000 Subject: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics In-Reply-To: <372edaad-f92c-7071-036b-703c86ad25c1@oracle.com> References: <4a3b4d59c9f54234b0c0d325a3238cbc@sap.com> <372edaad-f92c-7071-036b-703c86ad25c1@oracle.com> Message-ID: <3c0af6430a8e43cbbd4178edc88a0512@sap.com> Hi Robbin, thank you for reviewing. Submission forest tests have passed in addition to SAP's nightly builds. I have 2 reviews, now, but I'll wait a little bit before pushing. Best regards, Martin -----Original Message----- From: Robbin Ehn [mailto:robbin.ehn at oracle.com] Sent: Donnerstag, 3. Mai 2018 14:29 To: Doerr, Martin ; Schmidt, Lutz ; David Holmes ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley (aph at redhat.com) ; John Paul Adrian Glaubitz Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics On 2018-05-03 12:50, Doerr, Martin wrote: > Hi Lutz, > > thanks for reviewing. > > Can I get more reviews, please? I think this looks good, thanks for fixing! /Robbin > > Best regards, > Martin > > > -----Original Message----- > From: Schmidt, Lutz > Sent: Mittwoch, 2. Mai 2018 16:20 > To: Doerr, Martin ; David Holmes ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley (aph at redhat.com) ; John Paul Adrian Glaubitz > Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics > > Hi Martin, > > your changes look good. > > For the additional synchronization on s390 in particular, I do not expect a serious performance impact. The checkpoint synchronization part has always been the expensive, but rarely required, part. PPC may be another story. We knoow that a full-blown sync really hurts. We will see... > > Thanks, > Lutz > > ?On 26.04.18, 16:23, "hotspot-runtime-dev on behalf of Doerr, Martin" wrote: > > Hi, > > I have renamed "cmpxchg_memory_order" to "atomic_memory_order" and added support to all Read-Modify-Write atomics with support for all C++11 semantics except seq_cst. The latter has issues on PPC64 so we're currently using our own memory_order_conservative instead, which is the default. > > Please review my new webrev: > http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.01/ > > We'll test Windows, MacOS, linux x86+ppc64+s390, AIX and Solaris SPARC. > It'd be great if somebody could check arm/aarch64 and zero. > > This change may enable optimizations for arm/aarch64 (not yet included). > > Best regards, > Martin > > > -----Original Message----- > From: Doerr, Martin > Sent: Donnerstag, 26. April 2018 11:20 > To: 'David Holmes' ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) > Subject: RE: RFR(M): 8202080: Introduce ordering semantics for Atomic::add > > Hi David, > > sounds like we are on the same page, now. I think we should have some kind of summary of what was analyzed. But that belongs to the other thread (JDK- 8154736). > > > The bugs in this kind of code are very subtle and rarely exposed through normal levels of testing. > Right. We have experienced this often enough. So it is definitely in our interest to have correct memory barriers. Mistakes will hit us (PPC64). So I appreciate that you insist on careful analysis. > > I'll prepare a new webrev with conservative semantics for all read-modify-write atomics by default. > > Best regards, > Martin > > > From robbin.ehn at oracle.com Thu May 3 12:29:12 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 3 May 2018 14:29:12 +0200 Subject: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics In-Reply-To: <4a3b4d59c9f54234b0c0d325a3238cbc@sap.com> References: <4a3b4d59c9f54234b0c0d325a3238cbc@sap.com> Message-ID: <372edaad-f92c-7071-036b-703c86ad25c1@oracle.com> On 2018-05-03 12:50, Doerr, Martin wrote: > Hi Lutz, > > thanks for reviewing. > > Can I get more reviews, please? I think this looks good, thanks for fixing! /Robbin > > Best regards, > Martin > > > -----Original Message----- > From: Schmidt, Lutz > Sent: Mittwoch, 2. Mai 2018 16:20 > To: Doerr, Martin ; David Holmes ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley (aph at redhat.com) ; John Paul Adrian Glaubitz > Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics > > Hi Martin, > > your changes look good. > > For the additional synchronization on s390 in particular, I do not expect a serious performance impact. The checkpoint synchronization part has always been the expensive, but rarely required, part. PPC may be another story. We knoow that a full-blown sync really hurts. We will see... > > Thanks, > Lutz > > ?On 26.04.18, 16:23, "hotspot-runtime-dev on behalf of Doerr, Martin" wrote: > > Hi, > > I have renamed "cmpxchg_memory_order" to "atomic_memory_order" and added support to all Read-Modify-Write atomics with support for all C++11 semantics except seq_cst. The latter has issues on PPC64 so we're currently using our own memory_order_conservative instead, which is the default. > > Please review my new webrev: > http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.01/ > > We'll test Windows, MacOS, linux x86+ppc64+s390, AIX and Solaris SPARC. > It'd be great if somebody could check arm/aarch64 and zero. > > This change may enable optimizations for arm/aarch64 (not yet included). > > Best regards, > Martin > > > -----Original Message----- > From: Doerr, Martin > Sent: Donnerstag, 26. April 2018 11:20 > To: 'David Holmes' ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) > Subject: RE: RFR(M): 8202080: Introduce ordering semantics for Atomic::add > > Hi David, > > sounds like we are on the same page, now. I think we should have some kind of summary of what was analyzed. But that belongs to the other thread (JDK- 8154736). > > > The bugs in this kind of code are very subtle and rarely exposed through normal levels of testing. > Right. We have experienced this often enough. So it is definitely in our interest to have correct memory barriers. Mistakes will hit us (PPC64). So I appreciate that you insist on careful analysis. > > I'll prepare a new webrev with conservative semantics for all read-modify-write atomics by default. > > Best regards, > Martin > > > From zgu at redhat.com Thu May 3 13:24:10 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 3 May 2018 09:24:10 -0400 Subject: RFR(xxs): 8202424: Metaspace: on chunk retirement, use correct lower limit on chunksize when adding blocks to free blocks list. In-Reply-To: References: Message-ID: Looks good to me. -Zhengyu On 05/03/2018 09:02 AM, Thomas St?fe wrote: > Hi all, > > jdk-submit tests all went thru on this one. > > I still need reviewers. > > Thanks, Thomas > > > On Mon, Apr 30, 2018 at 4:07 PM, Thomas St?fe wrote: >> Hi all, >> >> please review this small fix, which fixes a small metaspace leak when >> retiring chunks. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8202424 >> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8202424-Metaspace-on-chunk-retirement-use-correct-lower-limit-on-chunksize-when-adding-blocks-to-free-blocks-list/webrev.00/webrev/ >> >> Some time ago, JDK-8164921 got rid of dark matter by lowering the >> smallest block size which can be managed by the free block list to >> sizeof(MetaBlock). >> >> When retiring chunks, the leftover space in the old chunk is added to >> the free block list via SpaceManager::deallocate(). Here, we still >> unnecessarily assume a lower size limit of TreeChunk> FreeList >::min_size(), which is unnecessary high and >> causes leaks which show up as waste in the retired chunks. >> >> By correctly using SmallBlocks::small_block_min_size() when >> deallocating the leftover chunk space on chunk retirement, we can >> reduce that kind of waste to almost nil. >> >> Thanks, Thomas From thomas.stuefe at gmail.com Thu May 3 13:28:45 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 3 May 2018 15:28:45 +0200 Subject: RFR(xxs): 8202424: Metaspace: on chunk retirement, use correct lower limit on chunksize when adding blocks to free blocks list. In-Reply-To: References: Message-ID: Thanks Zhengyu! On Thu, May 3, 2018 at 3:24 PM, Zhengyu Gu wrote: > Looks good to me. > > -Zhengyu > > > On 05/03/2018 09:02 AM, Thomas St?fe wrote: >> >> Hi all, >> >> jdk-submit tests all went thru on this one. >> >> I still need reviewers. >> >> Thanks, Thomas >> >> >> On Mon, Apr 30, 2018 at 4:07 PM, Thomas St?fe >> wrote: >>> >>> Hi all, >>> >>> please review this small fix, which fixes a small metaspace leak when >>> retiring chunks. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202424 >>> Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8202424-Metaspace-on-chunk-retirement-use-correct-lower-limit-on-chunksize-when-adding-blocks-to-free-blocks-list/webrev.00/webrev/ >>> >>> Some time ago, JDK-8164921 got rid of dark matter by lowering the >>> smallest block size which can be managed by the free block list to >>> sizeof(MetaBlock). >>> >>> When retiring chunks, the leftover space in the old chunk is added to >>> the free block list via SpaceManager::deallocate(). Here, we still >>> unnecessarily assume a lower size limit of TreeChunk>> FreeList >::min_size(), which is unnecessary high and >>> causes leaks which show up as waste in the retired chunks. >>> >>> By correctly using SmallBlocks::small_block_min_size() when >>> deallocating the leftover chunk space on chunk retirement, we can >>> reduce that kind of waste to almost nil. >>> >>> Thanks, Thomas From coleen.phillimore at oracle.com Thu May 3 13:55:23 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 3 May 2018 09:55:23 -0400 Subject: RFR(xxs): 8202424: Metaspace: on chunk retirement, use correct lower limit on chunksize when adding blocks to free blocks list. In-Reply-To: References: Message-ID: <8e0d0fe7-6cb4-b1dc-4cac-ec460bccf6d9@oracle.com> This looks good - thank you for fixing this! Coleen On 4/30/18 10:07 AM, Thomas St?fe wrote: > Hi all, > > please review this small fix, which fixes a small metaspace leak when > retiring chunks. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202424 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8202424-Metaspace-on-chunk-retirement-use-correct-lower-limit-on-chunksize-when-adding-blocks-to-free-blocks-list/webrev.00/webrev/ > > Some time ago, JDK-8164921 got rid of dark matter by lowering the > smallest block size which can be managed by the free block list to > sizeof(MetaBlock). > > When retiring chunks, the leftover space in the old chunk is added to > the free block list via SpaceManager::deallocate(). Here, we still > unnecessarily assume a lower size limit of TreeChunk FreeList >::min_size(), which is unnecessary high and > causes leaks which show up as waste in the retired chunks. > > By correctly using SmallBlocks::small_block_min_size() when > deallocating the leftover chunk space on chunk retirement, we can > reduce that kind of waste to almost nil. > > Thanks, Thomas From thomas.stuefe at gmail.com Thu May 3 14:25:04 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 3 May 2018 16:25:04 +0200 Subject: RFR(xxs): 8202424: Metaspace: on chunk retirement, use correct lower limit on chunksize when adding blocks to free blocks list. In-Reply-To: <8e0d0fe7-6cb4-b1dc-4cac-ec460bccf6d9@oracle.com> References: <8e0d0fe7-6cb4-b1dc-4cac-ec460bccf6d9@oracle.com> Message-ID: Thanks, Coleen! On Thu, May 3, 2018 at 3:55 PM, wrote: > > This looks good - thank you for fixing this! > Coleen > > > On 4/30/18 10:07 AM, Thomas St?fe wrote: >> >> Hi all, >> >> please review this small fix, which fixes a small metaspace leak when >> retiring chunks. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8202424 >> Webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/8202424-Metaspace-on-chunk-retirement-use-correct-lower-limit-on-chunksize-when-adding-blocks-to-free-blocks-list/webrev.00/webrev/ >> >> Some time ago, JDK-8164921 got rid of dark matter by lowering the >> smallest block size which can be managed by the free block list to >> sizeof(MetaBlock). >> >> When retiring chunks, the leftover space in the old chunk is added to >> the free block list via SpaceManager::deallocate(). Here, we still >> unnecessarily assume a lower size limit of TreeChunk> FreeList >::min_size(), which is unnecessary high and >> causes leaks which show up as waste in the retired chunks. >> >> By correctly using SmallBlocks::small_block_min_size() when >> deallocating the leftover chunk space on chunk retirement, we can >> reduce that kind of waste to almost nil. >> >> Thanks, Thomas > > From thomas.stuefe at gmail.com Thu May 3 15:28:03 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 3 May 2018 17:28:03 +0200 Subject: RFR: 8202435: [aix] print program break as part of memory info into hs-err file Message-ID: Hi all, may I please have reviews for this tiny, AIX-specific enhancement. bug: https://bugs.openjdk.java.net/browse/JDK-8202435 webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8202435-aix-print-program-break/webrev.00/webrev/ It prints out program break information as part of os::print_memory_info(). From martin.doerr at sap.com Thu May 3 15:50:39 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 3 May 2018 15:50:39 +0000 Subject: 8202435: [aix] print program break as part of memory info into hs-err file In-Reply-To: References: Message-ID: Hi Thomas, looks good. Just a minor thing: For "kilo" usually lower case "k" is used (like for "The thread stack size specified is invalid: "). But I don't need to see a new webrev for that. Thanks for contributing! Martin -----Original Message----- From: ppc-aix-port-dev [mailto:ppc-aix-port-dev-bounces at openjdk.java.net] On Behalf Of Thomas St?fe Sent: Donnerstag, 3. Mai 2018 17:28 To: Hotspot dev runtime ; ppc-aix-port-dev at openjdk.java.net Subject: RFR: 8202435: [aix] print program break as part of memory info into hs-err file Hi all, may I please have reviews for this tiny, AIX-specific enhancement. bug: https://bugs.openjdk.java.net/browse/JDK-8202435 webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8202435-aix-print-program-break/webrev.00/webrev/ It prints out program break information as part of os::print_memory_info(). From jiangli.zhou at oracle.com Thu May 3 17:34:02 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Thu, 3 May 2018 10:34:02 -0700 Subject: Improving AppCDS for Custom Loaders In-Reply-To: References: <8e0d891a-c00c-8f57-362a-6a4684a6cc21@oracle.com> <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> <732ce957-b07c-bff2-277b-ca335c348753@oracle.com> Message-ID: > On May 3, 2018, at 2:14 AM, Volker Simonis wrote: > > On Thu, May 3, 2018 at 11:01 AM, David Holmes wrote: >> On 3/05/2018 5:16 PM, Volker Simonis wrote: >>> >>> On Thu, May 3, 2018 at 8:55 AM, David Holmes >>> wrote: >>>> >>>> Just lurking here but ... >>>> >>>>> But is this really y relevant use case? Why would I like to create ONE >>>>> archive for several apps? This would actually increase the footprint >>>>> of a single instance which uses this archive. If I have several apps I >>>>> would expect that users create a specific archive for each app to get >>>>> the best out of CDS. >>>> >>>> >>>> >>>> One app instance may get increased footprint but you presumably use CDS >>>> because you have multiple apps running (whether the same or not). These >>>> apps >>>> all share the core JDK classes from the archive so the overall footprint >>>> per >>>> instance is less. >>>> >>> >>> If we just want to share the core JDK classes that's easy. For that we >>> could mostly use the default class list (or a slightly extended one) >>> which is generated at JDK build time (at JAVA_HOME/lib/classlist). >> >> >> The point is that you are presumably running multiple instances of multiple >> apps, hence you want to share one set of core classes across all, and share >> the app classes across each app instance. >> > > But that would require two archives: a general one with the core > classes and an application specific one for each application. > Combining the core classes and the application of various applications > will not be optimal because the application classes will be all mixed > in the same archive. The archive is being mapped page-wise into the > java process so you'll probably end up mapping the whole archive into > each process although you'll only use a fraction of the classes in the > archive. > >>> If we want to use ONE archive for several applications and we can >>> accept to have a bigger footprint if running a single (or just a few) >>> applications in parallel I suppose the overhead of simply dumping all >>> the classes from the classpathes of the various applications compared >>> to an accurate solution where we only dump the actually used classes >>> of all applications would be not that big. >> >> >> But those "accurate" solutions duplicate the core classes and that's a waste >> of footprint. >> > > By "accurate" I meant one "fat" archive which contains all the classes > USED by several applications plus the core classes. My argument was > that such an "accurate" "fat" archive won't be much smaller compared > to a "fat" archive which simply contains all the core classes plus all > the application classes (i.e. from the application class pathes, no > matter if they are ever used or not). But the latter would be much > simpler to implement. The above discussion and an internal proposal for hybrid archiving seem to converge on a few points. If there is no objection to the hybrid archiving proposal internally, maybe we can shared the details of the proposal on openjdk soon. Thanks, Jiangli > >> David >> ----- >> >> >>>> David >>>> ----- >>>> >>>> >>>> On 3/05/2018 4:48 PM, Volker Simonis wrote: >>>>> >>>>> >>>>> On Thu, May 3, 2018 at 6:52 AM, Ioi Lam wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On 5/2/18 10:00 AM, Volker Simonis wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, May 1, 2018 at 8:32 PM, Ioi Lam wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> PROBLEM: >>>>>>>> >>>>>>>> As discussed with Volker and Yumin in previous e-mails, AppCDS has >>>>>>>> some >>>>>>>> experimental support for custom class loaders. However, it's not very >>>>>>>> easy >>>>>>>> to use. >>>>>>>> >>>>>>>> For example, you can write a classlist like this: >>>>>>>> >>>>>>>> java/lang/Object id: 1 >>>>>>>> CustomLoadee id: 2 super: 1 source: /tmp/foo.jar >>>>>>>> >>>>>>>> The CustomLoadee class will be stored in the shared archive with a >>>>>>>> CRC >>>>>>>> code. >>>>>>>> During runtime, if a customed loader wants to load a class of the >>>>>>>> same >>>>>>>> name, >>>>>>>> and its classfile has the same size and CRC as the archived class, >>>>>>>> the >>>>>>>> archived version will be loaded. This speeds up class loading by >>>>>>>> avoiding >>>>>>>> parsing the class file, and saves space by sharing the mmap'ed class >>>>>>>> metadata across processes. >>>>>>>> >>>>>>>> You can see an example test at: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java >>>>>>>> >>>>>>>> However, the current scheme requires you to specify all the super >>>>>>>> classes >>>>>>>> and interfaces. There's no support provided by the >>>>>>>> -XX:DumpLoadedClassList >>>>>>>> option. It can be helped somewhat with Volker's tool: >>>>>>>> https://github.com/simonis/cl4cds >>>>>>>> >>>>>>>> >>>>>>>> POSSIBLE SOLUTIONS: >>>>>>>> >>>>>>>> 1. "Dump-as-you-go". As suggested by Yumin, we can provide a jcmd to >>>>>>>> ask >>>>>>>> a >>>>>>>> running JVM process to dump all of its loaded classes, including >>>>>>>> those >>>>>>>> loaded by custom loaders, into an archive. An alternative is to dump >>>>>>>> the >>>>>>>> archive at JVM exit time (or when you press Ctrl-C, etc. >>>>>>>> >>>>>>>> 2. Add information about the custom classes for >>>>>>>> -XX:DumpLoadedClassList. >>>>>>>> The >>>>>>>> trouble is some class loaders don't specify a code source that can be >>>>>>>> understood by the built-in class loaders. For example, the "Fat Jars" >>>>>>>> would >>>>>>>> have a code source like >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/ >>>>>>>> >>>>>>>> also, many custom loaders would pre-process the classfile data before >>>>>>>> defining the class, so we can't simply archive the version of the >>>>>>>> class >>>>>>>> on >>>>>>>> disk. >>>>>>>> >>>>>>>> One possible solution for #2 is to include the class file data in the >>>>>>>> -XX:DumpLoadedClassList output: >>>>>>>> >>>>>>>> >>>>>>>> java/lang/Object id: 1 >>>>>>>> CustomLoadee id: 2 super: 1 source: base64 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA >>>>>>>> AgAPAAcAAAAKAAEABQADAAYACA== >>>>>>>> >>>>>>>> >>>>>>>> Of the 2 solutions: >>>>>>>> >>>>>>>> #1 seems easier to use, but may require more invasive modifications >>>>>>>> in >>>>>>>> the >>>>>>>> VM, especially if you want to be able to continue execution after >>>>>>>> dumping. >>>>>>>> >>>>>>> Not sure what #1 really proposes: dumping the complete .jsa archive at >>>>>>> runtime or dumping just the loaded classes. >>>>>>> >>>>>>> If it's just about dumping the loaded class without generating the >>>>>>> .jsa archive there's the problem that by default the VM doesn't store >>>>>>> the exact bytes of a class after the class was loaded (except when >>>>>>> class transformers are registered). So the class files would have to >>>>>>> be re-assembled from the internal VM structures (in the same way this >>>>>>> is done for class redefinition) and the resulting class-file may be >>>>>>> different from the original bytes (i.e. some attributes may be >>>>>>> missing). >>>>>> >>>>>> >>>>>> >>>>>> #1 is for creating the JSA file, not just dumping the class files. >>>>>>> >>>>>>> >>>>>>> >>>>>>> If #1 is about creating the whole .jsa archive at runtime (or at VM >>>>>>> exit) I think that would be the most attractive solution from a >>>>>>> usability point of view although I understand that #2 will be easier >>>>>>> to implement in the short term. Regarding the argument that #1 will >>>>>>> produce a "binary blob" that's true, but that's already true now when >>>>>>> we use "Xshare:dump". I think it should be not to hard to implement a >>>>>>> tool based an SA which could introspect a .jsa archive. >>>>>> >>>>>> >>>>>> >>>>>> The argument about the binary blob is to compare it against the text >>>>>> file >>>>>> produced by -XX:DumpLoadedClassList. >>>>>> >>>>>> One use case to consider is when you have a JAR file that contains >>>>>> several >>>>>> apps that each load a unique set of classes. Today, (assuming that >>>>>> custom >>>>>> class loaders are not used), you can run each app once with >>>>>> -XX:DumpLoadedClassList, and then do an >>>>>> >>>>>> cat *.classlist | sort | uniq > combined.classlist >>>>>> >>>>>> and then create an archive that would work for all these apps. >>>>>> >>>>> >>>>> But is this really y relevant use case? Why would I like to create ONE >>>>> archive for several apps? This would actually increase the footprint >>>>> of a single instance which uses this archive. If I have several apps I >>>>> would expect that users create a specific archive for each app to get >>>>> the best out of CDS. >>>>> >>>>>> With the binary blob, there's no easy way of doing this. It will be >>>>>> very >>>>>> difficult to write a tool to decipher each blob and then somehow >>>>>> combine >>>>>> them into a single one. >>>>>> >>>>> >>>>> But if users really wants such a "fat" archive, there's a much easier >>>>> way: just dump ALL the classes from the .jar file into the archive. A >>>>> class list for this could easily be assembled either with an external >>>>> tool like cl4cds (or even a simple shell scripts which converts the >>>>> output of `unzip -l ` into the correct format). Or, even >>>>> simpler, by adding a new option to the VM similar to >>>>> -XX:DumpLoadedClassList which dumps all the classes it can find on the >>>>> class path (and potentially other, configurable locations). >>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>>>>>> #2 would be easier to implement, but the classlist might be huge. >>>>>>>> >>>>>>>> Also, #2 would allow post-processing tools to remove unneeded >>>>>>>> classes, >>>>>>>> or >>>>>>>> merge two runs into a single list. The output of #1 is essentially a >>>>>>>> binary >>>>>>>> blob that's impossible for off-line analysis/optimizations. >>>>>>>> >>>>>>>> >>>>>>>> Any comments, or suggestions for alternatives? >>>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>>> >>>>>>>> >>>>>> >>>> >> From paul.sandoz at oracle.com Thu May 3 18:16:16 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 3 May 2018 11:16:16 -0700 Subject: (11) RFR (M) JDK-8189916: Dynamic Constant support for Sparc In-Reply-To: <8fb7457a-a734-71e9-0363-4d1eaa232963@oracle.com> References: <8fb7457a-a734-71e9-0363-4d1eaa232963@oracle.com> Message-ID: <86396FF7-EE5F-4616-AB28-E492439ABF37@oracle.com> Looks good, i can sort of follow the SPARC specific template code as it?s similar to the shape of the x86 code. Paul. > On May 2, 2018, at 1:21 PM, Lois Foltan wrote: > > Please review this change to the template interpreter to support dynamic constant on Sparc. > > open webrev at http://cr.openjdk.java.net/~lfoltan/bug_jdk8189916.0/webrev/ > bug link https://bugs.openjdk.java.net/browse/JDK-8189916 > > All current jtreg dynamic constant tests pass when run on Sparc and have had (os.arch != "sparcv9") removed from their test description. > > Testing: Sparc specific testing (hs-tier1-5, jdk-tier1-3) complete. > All tier 1 platforms (hs-tier1-2) in progress. > > Thanks, > Lois > From lois.foltan at oracle.com Thu May 3 18:24:38 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 3 May 2018 14:24:38 -0400 Subject: [ping] RE: RFR(M): 8199852: Print more information about class loaders in LinkageErrors. In-Reply-To: <8ff92b7d5df3431aa00ad24b8d52223e@sap.com> References: <0e13b51363ef4b6f9aece78e3a332afb@sap.com> <5b99bafb1ceb440a8892676dcc1a3ec6@sap.com> <4567ab7a-7f88-519e-c7df-c7c21928c396@oracle.com> <8ff92b7d5df3431aa00ad24b8d52223e@sap.com> Message-ID: On 5/2/2018 11:19 AM, Lindenmaier, Goetz wrote: > Hi Lois, > > Thanks for your approval of my change! > >> Hi Goetz, >> >> I'm ok with going forward with this change, you have my review. However, >> I would like to record some concerns I do have about this change which I >> hope RFEs will be created for to clean up. >> >> 1. the change introduces another duplicate way to obtain the class >> loader's name via java_lang_ClassLoader::describe_external, where >> Klass::class_loader_and_module_name() exists today and could be modified > As stated before, I think it would be quite complicated to design one method > that delivers a name/string as complex as that of these methods to fit all purposes. > I appreciate though to find a common way to put the messages, common > wording etc.. I think class_loader_and_module_name() could use > describe_external() for the class_loader name. > >> 2. the method describe_external() obtains the class loader's name in a >> different manner than SystemDictionary::loader_name().? How to obtain >> the class loader's name should be standardized within the VM. > In some intermediate design, I had called describe_external within loader_name. > As I understand, loader_name does not report the name of the loader at all, > only the class name. This really should be improved. Hi Goetz, I've entered an RFE to standardize on the use of ClassLoaderData::loader_name() as the preferred way within the VM to obtain a class loader's name.? See https://bugs.openjdk.java.net/browse/JDK-8202605.? The class loader's name is set within ClassLoaderData::initialize_name_and_klass().??? The aim is to remove SystemDictionary::loader_name() and all independent calls to java_lang_ClassLoader::name() in favor of this method. Thanks, Lois > >> 3. previously I expressed concerns about adding the parent loader's >> information into the LinkageError message since I think this makes the >> message wordy and to me this seems like information that is more >> appropriate for logging. > I think error messages should contain as much information as possible > to track down the error. > Thanks for approving my change anyways. > > Best regards, > Goetz. > >> Thanks, >> Lois >> >> On 5/2/2018 7:21 AM, Lindenmaier, Goetz wrote: >>> Hi David, >>> >>>> Sorry really strapped for time these days ... >>> This is ok, but I will keep pinging ... ?? >>> A great, general thanks to you, as you really spend a lot of effort on >>> reviewing changes by SAP. That helps us a lot and we appreciate >>> your efforts! >>> >>>> src/hotspot/share/classfile/javaClasses.cpp >>>> src/hotspot/share/classfile/javaClasses.hpp >>>> The description of describe_external in the hpp doesn't match the >>>> implementation in the cpp: parent: x versus child of x >>> I updated the comment: >>> - // Prints "" (instance of , parent: "" >> ) >>> - // or (parent: "" ). >>> + // Prints "" (instance of , child of "" >> ). >>> + // If a classloader has no name, it prints instead. The output >>> + // for well known loaders (system/platform) is abbreviated. >>> >>>> src/hotspot/share/classfile/systemDictionary.cpp >>>> >>>> 2208 throwException = true; >>>> 2209 ss.print("loader constraint violation: loader %s", >>>> 2210 >>>> java_lang_ClassLoader::describe_external(class_loader())); >>>> 2211 ss.print(" wants to load %s %s.", >>>> 2212 k->external_kind(), k->external_name()); >>>> 2213 Klass *existing_klass = >>>> constraints()->find_constrained_klass(name, class_loader); >>>> 2214 if (existing_klass->class_loader() != class_loader()) { >>>> 2215 ss.print(" A different %s with the same name was >>>> previously loaded by %s.", >>>> 2216 existing_klass->external_kind(), >>>> 2217 >>>> java_lang_ClassLoader::describe_external(existing_klass- >>> class_loader())); >>>> 2218 } >>>> >>>> I still find this way too verbose. I won't oppose it but I don't like >>>> it. I know you're trying to solve your remote support problem, but I >>>> worry about the newbie trying to learn and experiment and being totally >>>> bamboozled by the exception message. (I thought the old ones were bad >>>> enough - but necessary in their level of preciseness.) >>> I understand. Thanks for not opposing. >>> >>> >>>> 3125 // Caller needs ResourceMark. >>>> >>>> Nit: Why change this line but not line 3131? In general single-line >>>> comments don't need to be written as grammatically correct sentences >>>> with full punctuation. Only if it is a multi-sentence comment should you >>>> need to do that. >>> Reverted, leftover from my other edits in that code that were removed. >>> >>>> Only comment I'll make on the tests is that being too specific about the >>>> exact message makes the tests more fragile. >>> Yes, but there are so many details in the message I would like to >>> assure (like the proper name of classes with '.') that I think this is >>> necessary. >>> >>> I'll push the change through the jdk-submit forest. (It's through all >>> our tests, anyways). After pushing, I will also update the bug to >>> list the final exception messages. >>> Can I consider it reviewed by you, then? >>> >>> Best regards, >>> Goetz. >>> >>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>> >>>> On 26/04/2018 1:30 AM, Lindenmaier, Goetz wrote: >>>>> Hi Lois, David, George or others, >>>>> >>>>> I would like to finish this one up, could I please get some >>>>> (hopefully final) reviews? >>>>> >>>>> Thanks, >>>>> Goetz. >>>>> >>>>>> -----Original Message----- >>>>>> From: Lindenmaier, Goetz >>>>>> Sent: Mittwoch, 18. April 2018 09:55 >>>>>> To: 'David Holmes' ; 'hotspot-runtime- >>>>>> dev at openjdk.java.net' >>>>>> Subject: RE: RFR(M): 8199852: Print more information about class >> loaders >>>> in >>>>>> LinkageErrors. >>>>>> >>>>>> Hi, >>>>>> >>>>>> I rebased the webrev to the jdk repo: >>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- >> jdk/ >>>>>> Could I please get the final go? >>>>>> >>>>>> Thanks and best regards, >>>>>> Goetz. >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Lindenmaier, Goetz >>>>>>> Sent: Freitag, 13. April 2018 10:10 >>>>>>> To: 'David Holmes' ; hotspot-runtime- >>>>>>> dev at openjdk.java.net >>>>>>> Subject: RE: RFR(M): 8199852: Print more information about class >> loaders >>>> in >>>>>>> LinkageErrors. >>>>>>> >>>>>>> Hi David, >>>>>>> >>>>>>> I fixed what you mentioned here. I also found an older mail >>>>>>> of yours with some comments I implemented now. I copied >>>>>>> that mail's content here to have only one mail ... find references >>>>>>> to some incremental webrevs in my replies below. >>>>>>> >>>>>>> Further I moved the test classes of test differentLE/ into a package. >>>>>>> >>>>>>> Comprehensive new webrev: >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03 >>>>>>> >>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: David Holmes >>>>>>>> Sent: Thursday, April 12, 2018 8:50 AM >>>>>>>> To: Lindenmaier, Goetz ; hotspot- >>>>>> runtime- >>>>>>>> dev at openjdk.java.net >>>>>>> ... >>>>>>>> This looks reasonable to me. Two minor comments: >>>>>>>> - updated tests need updated copyright notice >>>>>>>> - I think you can avoid duplicating test1() to test2() by passing in the >>>>>>>> loader name and the expected message as parameters >>>>>>> Fixed both. Incremental webrev: >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- >>>>>>> incremental2/ >>>>>>> >>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: David Holmes >>>>>>>> Sent: Sunday, March 25, 2018 11:49 PM >>>>>>>> To: Lindenmaier, Goetz ; hotspot- >>>>>> runtime- >>>>>>>> dev at openjdk.java.net >>>>>>> ... >>>>>>>>> Maybe I should suppress printing verbose information about these >>>>>>>>> well known loaders. (There is is_builtin_class_loader_data() to >>>>>>>>> identify such.) >>>>>>>> Yes it might be good to simplify things for the predefined loaders. >>>>>>>> Though I'd use: >>>>>>>> SystemDictionary::is_platform_class_loader >>>>>>>> SystemDictionary::is_system_class_loader >>>>>>>> for that check. >>>>>>> I implemented omitting the parent for these loaders. >>>>>>> >>>>>>>> Maybe it makes sense to have a set pattern for this descriptive text, >>>>>>>> and use "unnamed" if the loader has no name: >>>>>>>> >>>>>>>> loader , instanceof , child of loader ... >>>>>>> I implemented printing "" for empty names. I would like >>>>>>> To keep the format with brackets, because the enclosing text might >> use >>>>>>> commas, too, and then the sentence structure might get confusing. >>>>>>> >>>>>>> See this incremental webrev for these two changes: >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- >>>>>>> incremental1a/ >>>>>>> >>>>>>>> Add this bug to the @bug in the test(s). >>>>>>> Done. >>>>>>> >>>>>>>>> I'm not clear where you distinguish type and class. >>>>>>>> Using "class" is inaccurate as it can be a class, interface or array type. >>>>>>>> >>>>>>>> In the above the entity is a "type" as we are referring to a > name, >>>>>>>> classloader> pair. But you can also just read it as "class or interface >>>>>>>> or array type" >>>>>>> So why not print what it really is? I had added external_kind() for this >> in >>>>>>> My previous exception message change. >>>>>>> Unfortunately only in SystemDictionary::check_constraints() this is >> easily >>>>>>> possible, and it improves the message I think. >>>>>>> >>>>>>> Incremental webrev for these two changes: >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852-exMsg_Linkage/03- >>>>>>> incremental1b/ >>>>>>> >>>>>>> I also implemented a test for the call to external_kind(), but I missed >> to >>>> put >>>>>>> that into >>>>>>> the incremental webrev. >>>>>>> >>>>>>> Best regards, >>>>>>> Goetz. From lois.foltan at oracle.com Thu May 3 18:54:30 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 3 May 2018 14:54:30 -0400 Subject: (11) RFR (M) JDK-8189916: Dynamic Constant support for Sparc In-Reply-To: <86396FF7-EE5F-4616-AB28-E492439ABF37@oracle.com> References: <8fb7457a-a734-71e9-0363-4d1eaa232963@oracle.com> <86396FF7-EE5F-4616-AB28-E492439ABF37@oracle.com> Message-ID: <83e36479-6f6a-85b0-e552-2a43f2937d06@oracle.com> Thanks for the review Paul! Lois On 5/3/2018 2:16 PM, Paul Sandoz wrote: > Looks good, i can sort of follow the SPARC specific template code as it?s similar to the shape of the x86 code. > > Paul. > >> On May 2, 2018, at 1:21 PM, Lois Foltan wrote: >> >> Please review this change to the template interpreter to support dynamic constant on Sparc. >> >> open webrev at http://cr.openjdk.java.net/~lfoltan/bug_jdk8189916.0/webrev/ >> bug link https://bugs.openjdk.java.net/browse/JDK-8189916 >> >> All current jtreg dynamic constant tests pass when run on Sparc and have had (os.arch != "sparcv9") removed from their test description. >> >> Testing: Sparc specific testing (hs-tier1-5, jdk-tier1-3) complete. >> All tier 1 platforms (hs-tier1-2) in progress. >> >> Thanks, >> Lois >> From jiangli.zhou at Oracle.COM Thu May 3 19:43:15 2018 From: jiangli.zhou at Oracle.COM (Jiangli Zhou) Date: Thu, 3 May 2018 12:43:15 -0700 Subject: RFR 8197954 Remove unnecessary intermediary APIs from AppCDS implementation In-Reply-To: References: Message-ID: <508E3B72-3D09-4971-AFBD-E979E3ADA646@oracle.com> Hi Ioi, Thank you for removing the _ext files and merging the code together. It?s very helpful. - sharedPathsMiscInfo.cpp 191 // FIXME: why is there no MODULE check? The path check for archived module classes is handled differently than the classes from the -cp or -Xbootclasspath/a because we can quickly determine if an archived module class? origin matches with the runtime module source path. That?s why we don?t do string comparison of the dump time module path and the runtime module path in SharedPathsMiscInfo::check(), which is too restrictive and limited. Could you please replace the FIXME comment? - sharedPathsMiscInfo.hpp 131 const char* type_name(int type) { 132 switch (type) { 133 case BOOT: return "BOOT"; 134 case NON_EXIST: return "NON_EXIST"; <> 135 case APP: return "APP"; 136 case MODULE: return "MODULE"; The types are not well-defined. That?s not introduced by your change. The ?APP? type became inaccurate when we added the ?MODULE? type. It's noticeable now when types are merged together. To avoid overlapping, ?APP' can be changed to ?CLASS_PATH? or ?CP?. How about renaming the types to following: BOOT_PATH CLASS_PATH MODULE_PATH NON_EXIST As your current change does not involve larger scale restructuring, it?s okay if we follow up with a separate RFE for the type renaming. - systemDictionaryShared.cpp 490 // [c] At this point, if the named class was loaded by the 491 // AppClassLoader during archive dump time, we know that it must be 492 // loaded by the AppClassLoader during run time, and will not be loaded 493 // by a delegated class loader. The above only applies to the application classes loaded from -cp, but not modules. An archived module class may not be used at runtime if the module has a different source path at runtime. Those archived module classes are filtered out by the runtime shared class ?visibility? check. 493 // by a delegated class loader. This is true because we have checked the 494 // CLASSPATH and module path to ensure compatibility between dump time and 495 // run time. Please remove ?module path? from above comment. We don?t do string comparison of the runtime module path and dump time module path as CLASSPATH. The comment [C] probably should be split into two part, one for application classes from -cp and the other part for application module classes. 508 // An alternative is to modify the Java code of BuiltinClassLoader.loadClassOrNull(). Should above be removed? Otherwise, more details are needed. Thanks, Jiangli > On May 1, 2018, at 10:17 PM, Ioi Lam wrote: > > https://bugs.openjdk.java.net/browse/JDK-8197954 > http://cr.openjdk.java.net/~iklam/jdk11/8197954-remove-unnecessary-appcds-api.v01/ > > Summary: > > Before AppCDS was open sourced, we had a few "Ext" classes in sharedClassUtil.hpp > that abstracted away the CDS operations related to the non-boot class loaders. > This API made it possible to build the Oracle JDK with AppCDS included, or build > the Open JDK with AppCDS removed. > > With the open sourcing of AppCDS, this abstraction layer is no longer necessary. I > have moved the contents of the "Ext" classes into their proper locations and removed > the sharedClassUtil.hpp/cpp files. > > Most of the changes are just moving things around. There shouldn't be behavioral > changes. > > The files classLoaderExt.hpp/cpp still exists -- it encapsulates the classpath > management code and is not strictly unnecessary. Some clean up may be needed there, but > I'll probably do that in a separate RFE. > > Testing with hotspot tiers1~4. > > Thanks > - Ioi > > From gary.adams at oracle.com Thu May 3 22:01:37 2018 From: gary.adams at oracle.com (gary.adams at oracle.com) Date: Thu, 3 May 2018 18:01:37 -0400 Subject: Fwd: RFR: JDK-8202319: Fix compilation warnings in Solaris debug builds for DevStudio 12.6 In-Reply-To: References: <5AE1FFD0.1090803@oracle.com> Message-ID: <95043b95-1427-3576-4ad9-30aa18d016c7@oracle.com> Attached an updated patch for 8202319. Kim or David could you please sponsor the push of the patch. On 4/26/18 6:49 PM, gary.adams at oracle.com wrote: > > Adding build-dev and hotspot-runtime-dev aliases. > > -------- Forwarded Message -------- > Subject: RFR: JDK-8202319: Fix compilation warnings in Solaris debug > builds for DevStudio 12.6 > Date: Thu, 26 Apr 2018 12:35:28 -0400 > From: Gary Adams > Reply-To: gary.adams at oracle.com > To: OpenJDK Serviceability > > > > Getting the sources ready for the next Solaris developer studio toolchain. > Some additional warnings were found in the debug build. > > ?Issue:https://bugs.openjdk.java.net/browse/JDK-8202319 > Webrev:http://cr.openjdk.java.net/~gadams/8202319/webrev.00/ > > This update conditionally disables some new error checks, if the > new toolchain is used. -------------- next part -------------- # HG changeset patch # User gadams # Date 1525176684 14400 # Tue May 01 08:11:24 2018 -0400 # Node ID c2219a364d79a09d5de1480ee1fc280e6fcb0f16 # Parent 2ace90aec48858c5d82fd64b546c29b395ea5524 8202319: Fix compilation warnings in Solaris debug builds for DevStudio 12.6 Reviewed-by: kbarrett, dholmes diff --git a/src/hotspot/share/runtime/frame.cpp b/src/hotspot/share/runtime/frame.cpp --- a/src/hotspot/share/runtime/frame.cpp +++ b/src/hotspot/share/runtime/frame.cpp @@ -1103,6 +1103,9 @@ void frame::oops_do_internal(OopClosure* f, CodeBlobClosure* cf, RegisterMap* map, bool use_interpreter_oop_map_cache) { #ifndef PRODUCT +#if defined(__SUNPRO_CC) && __SUNPRO_CC >= 0x5140 +#pragma error_messages(off, SEC_NULL_PTR_DEREF) +#endif // simulate GC crash here to dump java thread in error report if (CrashGCForDumpingJavaThread) { char *t = NULL; diff --git a/src/hotspot/share/utilities/vmError.cpp b/src/hotspot/share/utilities/vmError.cpp --- a/src/hotspot/share/utilities/vmError.cpp +++ b/src/hotspot/share/utilities/vmError.cpp @@ -1606,6 +1606,9 @@ } #ifndef PRODUCT +#if defined(__SUNPRO_CC) && __SUNPRO_CC >= 0x5140 +#pragma error_messages(off, SEC_NULL_PTR_DEREF) +#endif typedef void (*voidfun_t)(); // Crash with an authentic sigfpe static void crash_with_sigfpe() { From david.holmes at oracle.com Thu May 3 22:15:39 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 4 May 2018 08:15:39 +1000 Subject: Fwd: RFR: JDK-8202319: Fix compilation warnings in Solaris debug builds for DevStudio 12.6 In-Reply-To: <95043b95-1427-3576-4ad9-30aa18d016c7@oracle.com> References: <5AE1FFD0.1090803@oracle.com> <95043b95-1427-3576-4ad9-30aa18d016c7@oracle.com> Message-ID: <5a4dabc1-852b-aff9-d47b-e0b42667b70b@oracle.com> Looks okay. Thanks, David On 4/05/2018 8:01 AM, gary.adams at oracle.com wrote: > Attached an updated patch for 8202319. > Kim or David could you please sponsor the push of the patch. > > On 4/26/18 6:49 PM, gary.adams at oracle.com wrote: >> >> Adding build-dev and hotspot-runtime-dev aliases. >> >> -------- Forwarded Message -------- >> Subject:???? RFR: JDK-8202319: Fix compilation warnings in Solaris >> debug builds for DevStudio 12.6 >> Date:???? Thu, 26 Apr 2018 12:35:28 -0400 >> From:???? Gary Adams >> Reply-To:???? gary.adams at oracle.com >> To:???? OpenJDK Serviceability >> >> >> >> Getting the sources ready for the next Solaris developer studio >> toolchain. >> Some additional warnings were found in the debug build. >> >> ?? ?Issue:https://bugs.openjdk.java.net/browse/JDK-8202319 >> ??? Webrev:http://cr.openjdk.java.net/~gadams/8202319/webrev.00/ >> >> This update conditionally disables some new error checks, if the >> new toolchain is used. > > From calvin.cheung at oracle.com Thu May 3 22:24:04 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Thu, 03 May 2018 15:24:04 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time Message-ID: <5AEB8C04.4040501@oracle.com> bug: https://bugs.openjdk.java.net/browse/JDK-8202289 webrev: http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ In ClassLoaderExt::process_module_table(), it adds all entries from the ModuleEntryTable with the "file:" protocol to the _module_path_entries and eventually to the _shared_path_table. The FileMapInfo::check_nonempty_dir_in_shared_path_table() will check for non-empty directory in the module path. Ran all CDS and AppCDS tests locally on linux-x64. Will run hs-tier{1,2,3} tests. thanks, Calvin From jiangli.zhou at Oracle.COM Thu May 3 22:42:43 2018 From: jiangli.zhou at Oracle.COM (Jiangli Zhou) Date: Thu, 3 May 2018 15:42:43 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time In-Reply-To: <5AEB8C04.4040501@oracle.com> References: <5AEB8C04.4040501@oracle.com> Message-ID: <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> Hi Calvin, I think we don?t need to check the module path entries if only boot classes are archived. The ?end? is set to app_class_paths_start_index when there are only boot classes are archived. If there are app module classes loaded from jars/directories at dump time, has_platform_or_app_classes() should return true and ?end? is set to the end of the shared path table, which includes all module path entries. The following loop is not needed. The existing loop in the code covers all cases with different ?end? value based on the type of classes loaded in the archive. 366 // may need to check module path entries. 367 if ((end == ClassLoaderExt::app_class_paths_start_index()) && (ClassLoader::num_module_path_entries() > 0)) { 368 for (int i = ClassLoaderExt::app_module_paths_start_index(); i < _shared_path_table_size; i++) { 369 SharedClassPathEntry* e = shared_path(i); 370 has_nonempty_dir = check_nonempty_dir(e); 371 } 372 } We also need a unit test for non-empty directory in module path. Please add. Thanks, Jiangli > On May 3, 2018, at 3:24 PM, Calvin Cheung wrote: > > bug: https://bugs.openjdk.java.net/browse/JDK-8202289 > webrev: http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ > > In ClassLoaderExt::process_module_table(), it adds all entries from the ModuleEntryTable with the "file:" protocol to the _module_path_entries and eventually to the _shared_path_table. > The FileMapInfo::check_nonempty_dir_in_shared_path_table() will check for non-empty directory in the module path. > > Ran all CDS and AppCDS tests locally on linux-x64. > Will run hs-tier{1,2,3} tests. > > thanks, > Calvin From calvin.cheung at oracle.com Thu May 3 23:18:15 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Thu, 03 May 2018 16:18:15 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time In-Reply-To: <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> References: <5AEB8C04.4040501@oracle.com> <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> Message-ID: <5AEB98B7.9030209@oracle.com> On 5/3/18, 3:42 PM, Jiangli Zhou wrote: > Hi Calvin, > > I think we don?t need to check the module path entries if only boot > classes are archived. The ?end? is set to app_class_paths_start_index > when there are only boot classes are archived. If there are app module > classes loaded from jars/directories at dump time, > has_platform_or_app_classes() should return true and ?end? is set to > the end of the shared path table, which includes all module path > entries. The following loop is not needed. The existing loop in the > code covers all cases with different ?end? value based on the type of > classes loaded in the archive. > 366 // may need to check module path entries. > 367 if ((end == ClassLoaderExt::app_class_paths_start_index())&& (ClassLoader::num_module_path_entries()> 0)) { > 368 for (int i = ClassLoaderExt::app_module_paths_start_index(); i< _shared_path_table_size; i++) { > 369 SharedClassPathEntry* e = shared_path(i); > 370 has_nonempty_dir = check_nonempty_dir(e); > 371 } > 372 } The has_platform_or_app_classes() will return false if ClassLoaderExt::record_result() hasn't been called while a class is being loaded during dumping. This could happen before this proposed change if one specifies a --module-path with an non-empty directory containing a module. While the class will be loaded during dumping but its classpath_index will still be -1. Therefore, the has_platform_or_app_classes() will return false and the module path entries won't be checked without the above block of code. > We also need a unit test for non-empty directory in module path. > Please add. It is already in my webrev: http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/MainModuleOnly.java.sdiff.html thanks, Calvin > > Thanks, > Jiangli > >> On May 3, 2018, at 3:24 PM, Calvin Cheung > > wrote: >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8202289 >> webrev: http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ >> >> >> In ClassLoaderExt::process_module_table(), it adds all entries from >> the ModuleEntryTable with the "file:" protocol to the >> _module_path_entries and eventually to the _shared_path_table. >> The FileMapInfo::check_nonempty_dir_in_shared_path_table() will check >> for non-empty directory in the module path. >> >> Ran all CDS and AppCDS tests locally on linux-x64. >> Will run hs-tier{1,2,3} tests. >> >> thanks, >> Calvin > From jiangli.zhou at Oracle.COM Fri May 4 00:33:07 2018 From: jiangli.zhou at Oracle.COM (Jiangli Zhou) Date: Thu, 3 May 2018 17:33:07 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time In-Reply-To: <5AEB98B7.9030209@oracle.com> References: <5AEB8C04.4040501@oracle.com> <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> <5AEB98B7.9030209@oracle.com> Message-ID: <56AA6E39-131E-4D5B-BBEF-9FA8C4A60F5A@oracle.com> Hi Calvin, > On May 3, 2018, at 4:18 PM, Calvin Cheung wrote: > > > > On 5/3/18, 3:42 PM, Jiangli Zhou wrote: >> Hi Calvin, >> >> I think we don?t need to check the module path entries if only boot classes are archived. The ?end? is set to app_class_paths_start_index when there are only boot classes are archived. If there are app module classes loaded from jars/directories at dump time, has_platform_or_app_classes() should return true and ?end? is set to the end of the shared path table, which includes all module path entries. The following loop is not needed. The existing loop in the code covers all cases with different ?end? value based on the type of classes loaded in the archive. >> 366 // may need to check module path entries. >> 367 if ((end == ClassLoaderExt::app_class_paths_start_index()) && (ClassLoader::num_module_path_entries() > 0)) { >> 368 for (int i = ClassLoaderExt::app_module_paths_start_index(); i < _shared_path_table_size; i++) { >> 369 SharedClassPathEntry* e = shared_path(i); >> 370 has_nonempty_dir = check_nonempty_dir(e); >> >> 371 } >> 372 } >> > The has_platform_or_app_classes() will return false if ClassLoaderExt::record_result() hasn't been called while a class is being loaded during dumping. > This could happen before this proposed change if one specifies a --module-path with an non-empty directory containing a module. While the class will be loaded during dumping but its classpath_index will still be -1. Therefore, the has_platform_or_app_classes() will return false and the module path entries won't be checked without the above block of code. With the module directory path being added to the path table now, it should record the correct path table index instead of -1 for module classes loaded from directory at dump time. I applied your patch and stepped through ClassLoader::record_result() with your test case. The following path comparison fails due the the extra ?/? in ?path?. That why it fails to find the correct entry in the path table, which causes the class type cannot be correctly identified. if (strcmp(canonical_path, os::native_path((char*)path)) == 0) { (gdb) p canonical_path $74 = 0x7ffff0aea0f0 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple" (gdb) p os::native_path((char*)path) $75 = 0x7ffff7fd4a65 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple/" The trailing ?/? should be excluded for the path comparison. JDK-8202519 is a related issue in the same area. It might be better to fix it together with this one. Once the above issue is fixed, the class type should be recorded correctly for a module classes loaded from directory at dump time. Then ClassLoaderExt::record_result() can set ?has_app_classes? flag properly. And, no change should be needed in FileMapInfo::check_nonempty_dir_in_shared_path_table(). >> We also need a unit test for non-empty directory in module path. Please add. > It is already in my webrev: > http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/MainModuleOnly.java.sdiff.html Ok. I was expecting a separate unit test. I?m fine with an added test case in MainModuleOnly.java. Thanks, Jiangli > > thanks, > Calvin >> >> Thanks, >> Jiangli >> >>> On May 3, 2018, at 3:24 PM, Calvin Cheung wrote: >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8202289 >>> webrev: http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ >>> >>> In ClassLoaderExt::process_module_table(), it adds all entries from the ModuleEntryTable with the "file:" protocol to the _module_path_entries and eventually to the _shared_path_table. >>> The FileMapInfo::check_nonempty_dir_in_shared_path_table() will check for non-empty directory in the module path. >>> >>> Ran all CDS and AppCDS tests locally on linux-x64. >>> Will run hs-tier{1,2,3} tests. >>> >>> thanks, >>> Calvin >> > From jiangli.zhou at Oracle.COM Fri May 4 01:42:42 2018 From: jiangli.zhou at Oracle.COM (Jiangli Zhou) Date: Thu, 3 May 2018 18:42:42 -0700 Subject: Improving AppCDS for Custom Loaders In-Reply-To: References: <8e0d891a-c00c-8f57-362a-6a4684a6cc21@oracle.com> <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> <732ce957-b07c-bff2-277b-ca335c348753@oracle.com> Message-ID: <660A1644-3A08-455D-931D-D95069C39853@oracle.com> Hi Volker, Here are some details about the hybrid archiving. The goal is to harvest the benefit of archiving by default and improve its usability. The hybrid approach combines a two-layer archiving (proposed by Ioi internally) and static & dynamic archiving techniques: - Statically archive system library classes from a provided classlist using the existing method. The archiving includes class metadata, interned string objects, constant pool resolved_references arrays, class mirror objects, etc. Static archiving can be done at the JDK image build time and shipped together with JDK binary. Following need to be addressed: *Relaxing the runtime CDS/AppCDS boot path check, so the packaged archive can be used after the JDK binary is installed on the target device. JDK-8199807 was created to address this issue and is targeted for JDK 11. *Add the static archiving generation in JDK build steps and package the generated archive with JDK image. The archive can only be generated for the same target (both OS can CPU architecture) as the build platform. I will create a RFE. - Dynamic archiving can done for application classes at the first execution of a specific application * The archive is created on top of the default system archive shipped with the JDK image. A separate top-layer archive file is generated for each different application. * Archiving is done at the end of the application execution before VM exists by relocating the class metadata to the archive spaces. Cleanup also needs to be done for copied class meta data to remove any runtime information. Most of the required functionality already exists today. For example, class metadata relocation was implemented by Ioi in JDK 10. * Only archive class metadata for application in the top layer initially. Archiving java heap objects in the top-layer requires more investigations. Benefits of the hybrid archiving: * The system archive can be shared by different applications and provides memory saving. * Archive for application is created and used transparently. No more profiling step and class list are required! * Separating the system archiving from application archiving reduces the cost of archiving at application execution time. The overhead added to the first execution time is reduced. Thanks, Jiangli > On May 3, 2018, at 10:34 AM, Jiangli Zhou wrote: > > >> On May 3, 2018, at 2:14 AM, Volker Simonis wrote: >> >> On Thu, May 3, 2018 at 11:01 AM, David Holmes wrote: >>> On 3/05/2018 5:16 PM, Volker Simonis wrote: >>>> >>>> On Thu, May 3, 2018 at 8:55 AM, David Holmes >>>> wrote: >>>>> >>>>> Just lurking here but ... >>>>> >>>>>> But is this really y relevant use case? Why would I like to create ONE >>>>>> archive for several apps? This would actually increase the footprint >>>>>> of a single instance which uses this archive. If I have several apps I >>>>>> would expect that users create a specific archive for each app to get >>>>>> the best out of CDS. >>>>> >>>>> >>>>> >>>>> One app instance may get increased footprint but you presumably use CDS >>>>> because you have multiple apps running (whether the same or not). These >>>>> apps >>>>> all share the core JDK classes from the archive so the overall footprint >>>>> per >>>>> instance is less. >>>>> >>>> >>>> If we just want to share the core JDK classes that's easy. For that we >>>> could mostly use the default class list (or a slightly extended one) >>>> which is generated at JDK build time (at JAVA_HOME/lib/classlist). >>> >>> >>> The point is that you are presumably running multiple instances of multiple >>> apps, hence you want to share one set of core classes across all, and share >>> the app classes across each app instance. >>> >> >> But that would require two archives: a general one with the core >> classes and an application specific one for each application. >> Combining the core classes and the application of various applications >> will not be optimal because the application classes will be all mixed >> in the same archive. The archive is being mapped page-wise into the >> java process so you'll probably end up mapping the whole archive into >> each process although you'll only use a fraction of the classes in the >> archive. >> >>>> If we want to use ONE archive for several applications and we can >>>> accept to have a bigger footprint if running a single (or just a few) >>>> applications in parallel I suppose the overhead of simply dumping all >>>> the classes from the classpathes of the various applications compared >>>> to an accurate solution where we only dump the actually used classes >>>> of all applications would be not that big. >>> >>> >>> But those "accurate" solutions duplicate the core classes and that's a waste >>> of footprint. >>> >> >> By "accurate" I meant one "fat" archive which contains all the classes >> USED by several applications plus the core classes. My argument was >> that such an "accurate" "fat" archive won't be much smaller compared >> to a "fat" archive which simply contains all the core classes plus all >> the application classes (i.e. from the application class pathes, no >> matter if they are ever used or not). But the latter would be much >> simpler to implement. > > The above discussion and an internal proposal for hybrid archiving seem to converge on a few points. If there is no objection to the hybrid archiving proposal internally, maybe we can shared the details of the proposal on openjdk soon. > > Thanks, > > Jiangli > > >> >>> David >>> ----- >>> >>> >>>>> David >>>>> ----- >>>>> >>>>> >>>>> On 3/05/2018 4:48 PM, Volker Simonis wrote: >>>>>> >>>>>> >>>>>> On Thu, May 3, 2018 at 6:52 AM, Ioi Lam wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 5/2/18 10:00 AM, Volker Simonis wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, May 1, 2018 at 8:32 PM, Ioi Lam wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> PROBLEM: >>>>>>>>> >>>>>>>>> As discussed with Volker and Yumin in previous e-mails, AppCDS has >>>>>>>>> some >>>>>>>>> experimental support for custom class loaders. However, it's not very >>>>>>>>> easy >>>>>>>>> to use. >>>>>>>>> >>>>>>>>> For example, you can write a classlist like this: >>>>>>>>> >>>>>>>>> java/lang/Object id: 1 >>>>>>>>> CustomLoadee id: 2 super: 1 source: /tmp/foo.jar >>>>>>>>> >>>>>>>>> The CustomLoadee class will be stored in the shared archive with a >>>>>>>>> CRC >>>>>>>>> code. >>>>>>>>> During runtime, if a customed loader wants to load a class of the >>>>>>>>> same >>>>>>>>> name, >>>>>>>>> and its classfile has the same size and CRC as the archived class, >>>>>>>>> the >>>>>>>>> archived version will be loaded. This speeds up class loading by >>>>>>>>> avoiding >>>>>>>>> parsing the class file, and saves space by sharing the mmap'ed class >>>>>>>>> metadata across processes. >>>>>>>>> >>>>>>>>> You can see an example test at: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java >>>>>>>>> >>>>>>>>> However, the current scheme requires you to specify all the super >>>>>>>>> classes >>>>>>>>> and interfaces. There's no support provided by the >>>>>>>>> -XX:DumpLoadedClassList >>>>>>>>> option. It can be helped somewhat with Volker's tool: >>>>>>>>> https://github.com/simonis/cl4cds >>>>>>>>> >>>>>>>>> >>>>>>>>> POSSIBLE SOLUTIONS: >>>>>>>>> >>>>>>>>> 1. "Dump-as-you-go". As suggested by Yumin, we can provide a jcmd to >>>>>>>>> ask >>>>>>>>> a >>>>>>>>> running JVM process to dump all of its loaded classes, including >>>>>>>>> those >>>>>>>>> loaded by custom loaders, into an archive. An alternative is to dump >>>>>>>>> the >>>>>>>>> archive at JVM exit time (or when you press Ctrl-C, etc. >>>>>>>>> >>>>>>>>> 2. Add information about the custom classes for >>>>>>>>> -XX:DumpLoadedClassList. >>>>>>>>> The >>>>>>>>> trouble is some class loaders don't specify a code source that can be >>>>>>>>> understood by the built-in class loaders. For example, the "Fat Jars" >>>>>>>>> would >>>>>>>>> have a code source like >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/ >>>>>>>>> >>>>>>>>> also, many custom loaders would pre-process the classfile data before >>>>>>>>> defining the class, so we can't simply archive the version of the >>>>>>>>> class >>>>>>>>> on >>>>>>>>> disk. >>>>>>>>> >>>>>>>>> One possible solution for #2 is to include the class file data in the >>>>>>>>> -XX:DumpLoadedClassList output: >>>>>>>>> >>>>>>>>> >>>>>>>>> java/lang/Object id: 1 >>>>>>>>> CustomLoadee id: 2 super: 1 source: base64 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA >>>>>>>>> AgAPAAcAAAAKAAEABQADAAYACA== >>>>>>>>> >>>>>>>>> >>>>>>>>> Of the 2 solutions: >>>>>>>>> >>>>>>>>> #1 seems easier to use, but may require more invasive modifications >>>>>>>>> in >>>>>>>>> the >>>>>>>>> VM, especially if you want to be able to continue execution after >>>>>>>>> dumping. >>>>>>>>> >>>>>>>> Not sure what #1 really proposes: dumping the complete .jsa archive at >>>>>>>> runtime or dumping just the loaded classes. >>>>>>>> >>>>>>>> If it's just about dumping the loaded class without generating the >>>>>>>> .jsa archive there's the problem that by default the VM doesn't store >>>>>>>> the exact bytes of a class after the class was loaded (except when >>>>>>>> class transformers are registered). So the class files would have to >>>>>>>> be re-assembled from the internal VM structures (in the same way this >>>>>>>> is done for class redefinition) and the resulting class-file may be >>>>>>>> different from the original bytes (i.e. some attributes may be >>>>>>>> missing). >>>>>>> >>>>>>> >>>>>>> >>>>>>> #1 is for creating the JSA file, not just dumping the class files. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> If #1 is about creating the whole .jsa archive at runtime (or at VM >>>>>>>> exit) I think that would be the most attractive solution from a >>>>>>>> usability point of view although I understand that #2 will be easier >>>>>>>> to implement in the short term. Regarding the argument that #1 will >>>>>>>> produce a "binary blob" that's true, but that's already true now when >>>>>>>> we use "Xshare:dump". I think it should be not to hard to implement a >>>>>>>> tool based an SA which could introspect a .jsa archive. >>>>>>> >>>>>>> >>>>>>> >>>>>>> The argument about the binary blob is to compare it against the text >>>>>>> file >>>>>>> produced by -XX:DumpLoadedClassList. >>>>>>> >>>>>>> One use case to consider is when you have a JAR file that contains >>>>>>> several >>>>>>> apps that each load a unique set of classes. Today, (assuming that >>>>>>> custom >>>>>>> class loaders are not used), you can run each app once with >>>>>>> -XX:DumpLoadedClassList, and then do an >>>>>>> >>>>>>> cat *.classlist | sort | uniq > combined.classlist >>>>>>> >>>>>>> and then create an archive that would work for all these apps. >>>>>>> >>>>>> >>>>>> But is this really y relevant use case? Why would I like to create ONE >>>>>> archive for several apps? This would actually increase the footprint >>>>>> of a single instance which uses this archive. If I have several apps I >>>>>> would expect that users create a specific archive for each app to get >>>>>> the best out of CDS. >>>>>> >>>>>>> With the binary blob, there's no easy way of doing this. It will be >>>>>>> very >>>>>>> difficult to write a tool to decipher each blob and then somehow >>>>>>> combine >>>>>>> them into a single one. >>>>>>> >>>>>> >>>>>> But if users really wants such a "fat" archive, there's a much easier >>>>>> way: just dump ALL the classes from the .jar file into the archive. A >>>>>> class list for this could easily be assembled either with an external >>>>>> tool like cl4cds (or even a simple shell scripts which converts the >>>>>> output of `unzip -l ` into the correct format). Or, even >>>>>> simpler, by adding a new option to the VM similar to >>>>>> -XX:DumpLoadedClassList which dumps all the classes it can find on the >>>>>> class path (and potentially other, configurable locations). >>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>>>> #2 would be easier to implement, but the classlist might be huge. >>>>>>>>> >>>>>>>>> Also, #2 would allow post-processing tools to remove unneeded >>>>>>>>> classes, >>>>>>>>> or >>>>>>>>> merge two runs into a single list. The output of #1 is essentially a >>>>>>>>> binary >>>>>>>>> blob that's impossible for off-line analysis/optimizations. >>>>>>>>> >>>>>>>>> >>>>>>>>> Any comments, or suggestions for alternatives? >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> - Ioi >>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>> > From david.holmes at oracle.com Fri May 4 06:08:40 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 4 May 2018 16:08:40 +1000 Subject: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics In-Reply-To: <3c0af6430a8e43cbbd4178edc88a0512@sap.com> References: <4a3b4d59c9f54234b0c0d325a3238cbc@sap.com> <372edaad-f92c-7071-036b-703c86ad25c1@oracle.com> <3c0af6430a8e43cbbd4178edc88a0512@sap.com> Message-ID: Hi Martin, On 3/05/2018 11:03 PM, Doerr, Martin wrote: > Hi Robbin, > > thank you for reviewing. > > Submission forest tests have passed in addition to SAP's nightly builds. I have 2 reviews, now, but I'll wait a little bit before pushing. Thanks for waiting. I think it important there is a general buy-in to this effort, not just the official "get a review and push". Apologies that it's taken me a while to get to this. memory_order_consume has been deprecated in C++17: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0371r1.html I would suggest we not add it to the VM. 40 enum atomic_memory_order { 41 // We support most semantics like in C++11. I would say something like: // The modes that align with C++11 are intended to // follow the same semantics. 47 // We need to be more conservative than seq_cst on PPC64. Not sure the PPC64 comment is relevant here. memory_order_conservative represents the pre-existing strong fencing requirements. It's not intended to map to anything else, nor intended to be different across platforms. Though that raises the question why you haven't added memory_order_seq_cst? Looking at all the platform code that adds the unused "order" parameter ... I would not be surprised if the forthcoming compiler upgrades result in some additional "unused" warnings. But I guess we'll just deal with them as they are discovered. Otherwise all seems okay. Thanks, David > Best regards, > Martin > > > -----Original Message----- > From: Robbin Ehn [mailto:robbin.ehn at oracle.com] > Sent: Donnerstag, 3. Mai 2018 14:29 > To: Doerr, Martin ; Schmidt, Lutz ; David Holmes ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley (aph at redhat.com) ; John Paul Adrian Glaubitz > Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics > > On 2018-05-03 12:50, Doerr, Martin wrote: >> Hi Lutz, >> >> thanks for reviewing. >> >> Can I get more reviews, please? > > I think this looks good, thanks for fixing! > > /Robbin > >> >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: Schmidt, Lutz >> Sent: Mittwoch, 2. Mai 2018 16:20 >> To: Doerr, Martin ; David Holmes ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley (aph at redhat.com) ; John Paul Adrian Glaubitz >> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics >> >> Hi Martin, >> >> your changes look good. >> >> For the additional synchronization on s390 in particular, I do not expect a serious performance impact. The checkpoint synchronization part has always been the expensive, but rarely required, part. PPC may be another story. We knoow that a full-blown sync really hurts. We will see... >> >> Thanks, >> Lutz >> >> ?On 26.04.18, 16:23, "hotspot-runtime-dev on behalf of Doerr, Martin" wrote: >> >> Hi, >> >> I have renamed "cmpxchg_memory_order" to "atomic_memory_order" and added support to all Read-Modify-Write atomics with support for all C++11 semantics except seq_cst. The latter has issues on PPC64 so we're currently using our own memory_order_conservative instead, which is the default. >> >> Please review my new webrev: >> http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.01/ >> >> We'll test Windows, MacOS, linux x86+ppc64+s390, AIX and Solaris SPARC. >> It'd be great if somebody could check arm/aarch64 and zero. >> >> This change may enable optimizations for arm/aarch64 (not yet included). >> >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: Doerr, Martin >> Sent: Donnerstag, 26. April 2018 11:20 >> To: 'David Holmes' ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) >> Subject: RE: RFR(M): 8202080: Introduce ordering semantics for Atomic::add >> >> Hi David, >> >> sounds like we are on the same page, now. I think we should have some kind of summary of what was analyzed. But that belongs to the other thread (JDK- 8154736). >> >> > The bugs in this kind of code are very subtle and rarely exposed through normal levels of testing. >> Right. We have experienced this often enough. So it is definitely in our interest to have correct memory barriers. Mistakes will hit us (PPC64). So I appreciate that you insist on careful analysis. >> >> I'll prepare a new webrev with conservative semantics for all read-modify-write atomics by default. >> >> Best regards, >> Martin >> >> >> From david.holmes at oracle.com Fri May 4 07:40:35 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 4 May 2018 17:40:35 +1000 Subject: RFR: 8202435: [aix] print program break as part of memory info into hs-err file In-Reply-To: References: Message-ID: Hi Thomas, Looks fine to me. Cheers, David On 4/05/2018 1:28 AM, Thomas St?fe wrote: > Hi all, > > may I please have reviews for this tiny, AIX-specific enhancement. > > bug: https://bugs.openjdk.java.net/browse/JDK-8202435 > webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8202435-aix-print-program-break/webrev.00/webrev/ > > It prints out program break information as part of os::print_memory_info(). > From thomas.stuefe at gmail.com Fri May 4 08:50:07 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 04 May 2018 08:50:07 +0000 Subject: RFR: 8202435: [aix] print program break as part of memory info into hs-err file In-Reply-To: References: Message-ID: Thanks David! On Fri, May 4, 2018, 09:40 David Holmes wrote: > Hi Thomas, > > Looks fine to me. > > Cheers, > David > > On 4/05/2018 1:28 AM, Thomas St?fe wrote: > > Hi all, > > > > may I please have reviews for this tiny, AIX-specific enhancement. > > > > bug: https://bugs.openjdk.java.net/browse/JDK-8202435 > > webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8202435-aix-print-program-break/webrev.00/webrev/ > > > > It prints out program break information as part of > os::print_memory_info(). > > > From thomas.stuefe at gmail.com Fri May 4 08:50:49 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 04 May 2018 08:50:49 +0000 Subject: 8202435: [aix] print program break as part of memory info into hs-err file In-Reply-To: References: Message-ID: Thank you Martin! On Thu, May 3, 2018, 17:50 Doerr, Martin wrote: > Hi Thomas, > > looks good. Just a minor thing: For "kilo" usually lower case "k" is used > (like for "The thread stack size specified is invalid: "). But I don't need > to see a new webrev for that. > > Thanks for contributing! > Martin > > > -----Original Message----- > From: ppc-aix-port-dev [mailto:ppc-aix-port-dev-bounces at openjdk.java.net] > On Behalf Of Thomas St?fe > Sent: Donnerstag, 3. Mai 2018 17:28 > To: Hotspot dev runtime ; > ppc-aix-port-dev at openjdk.java.net > Subject: RFR: 8202435: [aix] print program break as part of memory info > into hs-err file > > Hi all, > > may I please have reviews for this tiny, AIX-specific enhancement. > > bug: https://bugs.openjdk.java.net/browse/JDK-8202435 > webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8202435-aix-print-program-break/webrev.00/webrev/ > > It prints out program break information as part of os::print_memory_info(). > From goetz.lindenmaier at sap.com Fri May 4 09:22:23 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 4 May 2018 09:22:23 +0000 Subject: RFR(M): 8201593: Print array length in ArrayIndexOutOfBoundsException. In-Reply-To: <9d2753f6975c4477912420a2fe459ef3@sap.com> References: <504bdd8469bb45cc8c5cf790d1f4b3fc@sap.com> <12e8ad2367484b0b88bcee4bcb77ac47@sap.com> <033adcb3-1ab0-094e-78cf-4062a8868d3e@redhat.com> <7767af56c6744e8aa5479a5dd9626936@sap.com> <9d2753f6975c4477912420a2fe459ef3@sap.com> Message-ID: <14107212c2664aee9965e9f9da8006ad@sap.com> Hi, thanks to Aleksey and Boris this is now also tested on arm. This final webrev contains some fixes needed in the arm files: http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06-arm/ David, can I consider this as finally reviewed? Best regards, Goetz > -----Original Message----- > From: aarch64-port-dev [mailto:aarch64-port-dev- > bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz > Sent: Mittwoch, 2. Mai 2018 17:57 > To: Stuart Monteith > Cc: hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; aarch32-port- > dev at openjdk.java.net > Subject: [CAUTION] Re: [aarch64-port-dev ] RFR(M): 8201593: Print array > length in ArrayIndexOutOfBoundsException. > > Hi, > > I needed to move the edit from c1_LIRGenerator_.cpp to > the shared file after "8201543: Modularize C1 GC barriers" > New webrev: > http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06/ > > @Stuart > Thanks for testing! > > so as to accommodate the array pointer you are pushing onto the stack? > Yes, what you are pointing out seems to be wrong, I changed it to '2'. > > Best regards, > Goetz. > > > > -----Original Message----- > > From: Stuart Monteith [mailto:stuart.monteith at linaro.org] > > Sent: Freitag, 27. April 2018 16:37 > > To: Lindenmaier, Goetz > > Cc: hotspot-compiler-dev at openjdk.java.net; aarch64-port- > > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; aarch32- > > port-dev at openjdk.java.net > > Subject: Re: RFR(M): 8201593: Print array length in > > ArrayIndexOutOfBoundsException. > > > > Hi, > > JTregs hasn't flagged any issues, so it should be ok. > > > > Regarding the 32-bit arm code, in "void > > RangeCheckStub::emit_code(LIR_Assembler* ce)" should: > > ce->verify_reserved_argument_area_size(1); > > be > > ce->verify_reserved_argument_area_size(2); > > > > so as to accommodate the array pointer you are pushing onto the stack? > > > > I've not tested 32-bit arm. > > > > > > BR, > > Stuart > > > > On 26 April 2018 at 15:31, Stuart Monteith > > wrote: > > > Thanks, I'm happy with that. > > > > > > The registers have a clean path to call_RT - r22 and r23 aren't used > > > inbetween. They are an arbitrary choice - c_rarg0 and c_rarg1 were > > > always going to cause problems. If _array->as_pointer_register() > > > and/or _index->as_register() or _index->as_jint() were the registers > > > we were using as parameters there would be trouble. However, with > > > pd_last_allocatable_cpu_reg = 16, that shouldn't happen with r22/23, > > > or indeed anything else in the range r17 to r28. > > > > > > I'm going to run all of JTRegs and seem what that produces now. > > > > > > BR, > > > Stuart > > > > > > > > > > > > > > > On 26 April 2018 at 15:14, Lindenmaier, Goetz > > wrote: > > >> Hi Stuart, > > >> > > >> thanks for fixing this! Webrev with your changes: > > >> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/05/ > > >> > > >>> There is the possibility of overwriting live values though, aren't > > >>> there? The registers are saved by call_RT. Should I be concerned about > > >>> deopt and debugging going wrong? Furthermore, won't there be > issues > > in > > >>> exception handlers? > > >> As I understand, this just has to survive the far_call. > > >> The call_RT in c1_Runtime then moves it into the > > >> proper argument registers. This is just the handling of an > > >> exception, and in these few instructions no java code is > > >> executed, no safepoint is passed, so this should be fine. > > >> > > >> incremental diff: > > >> iff -r 874f2b999ff6 src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp > > >> --- a/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Mon Apr 16 > > 15:17:20 2018 +0200 > > >> +++ b/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Thu Apr 26 > > 15:55:18 2018 +0200 > > >> @@ -75,16 +75,16 @@ > > >> } > > >> > > >> if (_index->is_cpu_register()) { > > >> - __ mov(rscratch1, _index->as_register()); > > >> + __ mov(r22, _index->as_register()); > > >> } else { > > >> - __ mov(rscratch1, _index->as_jint()); > > >> + __ mov(r22, _index->as_jint()); > > >> } > > >> Runtime1::StubID stub_id; > > >> if (_throw_index_out_of_bounds_exception) { > > >> stub_id = Runtime1::throw_index_exception_id; > > >> } else { > > >> assert(_array != NULL, "sanity"); > > >> - __ mov(rscratch2, _array->as_pointer_register()); > > >> + __ mov(r23, _array->as_pointer_register()); > > >> stub_id = Runtime1::throw_range_check_failed_id; > > >> } > > >> __ far_call(RuntimeAddress(Runtime1::entry_for(stub_id)), NULL, > > rscratch2); > > >> diff -r 874f2b999ff6 src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp > > >> --- a/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Mon Apr 16 > > 15:17:20 2018 +0200 > > >> +++ b/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Thu Apr 26 > > 15:55:18 2018 +0200 > > >> @@ -327,7 +327,7 @@ > > >> > > >> > > >> // target: the entry point of the method that creates and posts the > > exception oop > > >> -// has_argument: true if the exception needs an argument (passed in > > rscratch1) > > >> +// has_argument: true if the exception needs arguments (passed in > r22 > > and r23) > > >> > > >> OopMapSet* Runtime1::generate_exception_throw(StubAssembler* > > sasm, address target, bool has_argument) { > > >> // make a frame and preserve the caller's caller-save registers > > >> @@ -336,7 +336,7 @@ > > >> if (!has_argument) { > > >> call_offset = __ call_RT(noreg, noreg, target); > > >> } else { > > >> - call_offset = __ call_RT(noreg, noreg, target, rscratch1, rscratch2); > > >> + call_offset = __ call_RT(noreg, noreg, target, r22, r23); > > >> } > > >> OopMapSet* oop_maps = new OopMapSet(); > > >> oop_maps->add_gc_map(call_offset, oop_map); > > >> > > >> Best regards, > > >> Goetz. > > >> > > >> > > >>> -----Original Message----- > > >>> From: Stuart Monteith [mailto:stuart.monteith at linaro.org] > > >>> Sent: Donnerstag, 26. April 2018 12:52 > > >>> To: Andrew Haley > > >>> Cc: Lindenmaier, Goetz ; hotspot- > > compiler- > > >>> dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; hotspot- > > >>> runtime-dev at openjdk.java.net; aarch32-port-dev at openjdk.java.net > > >>> Subject: Re: RFR(M): 8201593: Print array length in > > >>> ArrayIndexOutOfBoundsException. > > >>> > > >>> Hi, > > >>> Using c_rarg1 and c_rarg2 instead of rscratch1 and overwriting > > >>> rscratch2 causes a SIGSEGV. > > >>> Using r22 and r23 instead, the test ran successfully. > > >>> > > >>> In c1_CodeStubs_aarch64.cpp > > >>> : > > >>> 77 if (_index->is_cpu_register()) { > > >>> 78 __ mov(r22, _index->as_register()); > > >>> 79 } else { > > >>> 80 __ mov(r22, _index->as_jint()); > > >>> 81 } > > >>> 82 Runtime1::StubID stub_id; > > >>> 83 if (_throw_index_out_of_bounds_exception) { > > >>> 84 stub_id = Runtime1::throw_index_exception_id; > > >>> 85 } else { > > >>> 86 assert(_array != NULL, "sanity"); > > >>> 87 __ mov(r23, _array->as_pointer_register()); > > >>> 88 stub_id = Runtime1::throw_range_check_failed_id; > > >>> 89 } > > >>> > > >>> in c1_Runtime_aarch64.cpp: > > >>> > > >>> 336 if (!has_argument) { > > >>> 337 call_offset = __ call_RT(noreg, noreg, target); > > >>> 338 } else { > > >>> 339 call_offset = __ call_RT(noreg, noreg, target, r22, r23); > > >>> 340 } > > >>> > > >>> There is the possibility of overwriting live values though, aren't > > >>> there? The registers are saved by call_RT. Should I be concerned about > > >>> deopt and debugging going wrong? Furthermore, won't there be > issues > > in > > >>> exception handlers? > > >>> > > >>> BR, > > >>> Stuart > > >>> > > >>> > > >>> On 25 April 2018 at 16:49, Stuart Monteith > > > >>> wrote: > > >>> > Indeed - and that is what I am seeing. Usually no parameters are > being > > >>> > called with this pattern, or rscratch1, with the temporary variable > > >>> > being changed to use rscratch2 in such circumstances. > > >>> > I'll try c_rarg1 and c_rarg2 - they should pass straight through,if I > > >>> > interpret the code correcting. > > >>> > > > >>> > On 25 April 2018 at 16:26, Andrew Haley wrote: > > >>> >> On 04/25/2018 04:00 PM, Stuart Monteith wrote: > > >>> >>> I'm not quite sure to solve this yet - we'll need to use the stack in > > >>> >>> some safe way. > > >>> >> > > >>> >> It's not a great idea to pass arguments in rscratch1 or rscratch2. > These > > >>> >> registers are for use in macros and should be treated as volatile. > > Given > > >>> >> that you're throwing an exception, registers will be clobbered > > anyway. > > >>> >> > > >>> >> -- > > >>> >> Andrew Haley > > >>> >> Java Platform Lead Engineer > > >>> >> Red Hat UK Ltd. > > >>> >> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From volker.simonis at gmail.com Fri May 4 10:01:59 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 4 May 2018 12:01:59 +0200 Subject: Improving AppCDS for Custom Loaders In-Reply-To: <660A1644-3A08-455D-931D-D95069C39853@oracle.com> References: <8e0d891a-c00c-8f57-362a-6a4684a6cc21@oracle.com> <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> <732ce957-b07c-bff2-277b-ca335c348753@oracle.com> <660A1644-3A08-455D-931D-D95069C39853@oracle.com> Message-ID: Hi Jiangli, thanks for sharing the hybrid archiving proposal. I think it is very interesting and useful! One issue I see is that the "system library classes" which should go into the "static" archive are inherently application specific (i.e. if my application uses Swing, it will use most of the AWT/Swing classes). So how will you cope with this problem. Either you put ALL the system classes into the static archive which will be a waste for most applications. Or you just put a small commonly used subset of classes into the static archive which may miss many classes for specific applications. If we would add the possibility to create a dynamic archive at runtime / program end (which I think would be great from a usability perspective) I don't see a big need for two different archives. Two archives will further complicate (and slow down) things like Symbol lookup (we already have two SymbolTable now and we'd probably need a third one if we would have two archives). I don't think that running a few different Java applications on one host is the most important use case for CDS. In such a scenario the current, build time generated archive is probably the best we can do anyway. Instead, I think the most important use case is if we have many instances of the same Java application running on the same host. And this is becoming more common with the raise of containerization. For the latter use case, a dynamically generated, application specific archive would be the optimal solution. Thank you and best regards, Volker On Fri, May 4, 2018 at 3:42 AM, Jiangli Zhou wrote: > Hi Volker, > > Here are some details about the hybrid archiving. The goal is to harvest the benefit of archiving by default and improve its usability. The hybrid approach combines a two-layer archiving (proposed by Ioi internally) and static & dynamic archiving techniques: > > - Statically archive system library classes from a provided classlist using the existing method. The archiving includes class metadata, interned string objects, constant pool resolved_references arrays, class mirror objects, etc. Static archiving can be done at the JDK image build time and shipped together with JDK binary. Following need to be addressed: > *Relaxing the runtime CDS/AppCDS boot path check, so the packaged archive can be used after the JDK binary is installed on the target device. JDK-8199807 was created to address this issue and is targeted for JDK 11. > *Add the static archiving generation in JDK build steps and package the generated archive with JDK image. The archive can only be generated for the same target (both OS can CPU architecture) as the build platform. I will create a RFE. > > - Dynamic archiving can done for application classes at the first execution of a specific application > * The archive is created on top of the default system archive shipped with the JDK image. A separate top-layer archive file is generated for each different application. > * Archiving is done at the end of the application execution before VM exists by relocating the class metadata to the archive spaces. Cleanup also needs to be done for copied class meta data to remove any runtime information. Most of the required functionality already exists today. For example, class metadata relocation was implemented by Ioi in JDK 10. > * Only archive class metadata for application in the top layer initially. Archiving java heap objects in the top-layer requires more investigations. > > Benefits of the hybrid archiving: > * The system archive can be shared by different applications and provides memory saving. > * Archive for application is created and used transparently. No more profiling step and class list are required! > * Separating the system archiving from application archiving reduces the cost of archiving at application execution time. The overhead added to the first execution time is reduced. > > Thanks, > > Jiangli > > >> On May 3, 2018, at 10:34 AM, Jiangli Zhou wrote: >> >> >>> On May 3, 2018, at 2:14 AM, Volker Simonis wrote: >>> >>> On Thu, May 3, 2018 at 11:01 AM, David Holmes wrote: >>>> On 3/05/2018 5:16 PM, Volker Simonis wrote: >>>>> >>>>> On Thu, May 3, 2018 at 8:55 AM, David Holmes >>>>> wrote: >>>>>> >>>>>> Just lurking here but ... >>>>>> >>>>>>> But is this really y relevant use case? Why would I like to create ONE >>>>>>> archive for several apps? This would actually increase the footprint >>>>>>> of a single instance which uses this archive. If I have several apps I >>>>>>> would expect that users create a specific archive for each app to get >>>>>>> the best out of CDS. >>>>>> >>>>>> >>>>>> >>>>>> One app instance may get increased footprint but you presumably use CDS >>>>>> because you have multiple apps running (whether the same or not). These >>>>>> apps >>>>>> all share the core JDK classes from the archive so the overall footprint >>>>>> per >>>>>> instance is less. >>>>>> >>>>> >>>>> If we just want to share the core JDK classes that's easy. For that we >>>>> could mostly use the default class list (or a slightly extended one) >>>>> which is generated at JDK build time (at JAVA_HOME/lib/classlist). >>>> >>>> >>>> The point is that you are presumably running multiple instances of multiple >>>> apps, hence you want to share one set of core classes across all, and share >>>> the app classes across each app instance. >>>> >>> >>> But that would require two archives: a general one with the core >>> classes and an application specific one for each application. >>> Combining the core classes and the application of various applications >>> will not be optimal because the application classes will be all mixed >>> in the same archive. The archive is being mapped page-wise into the >>> java process so you'll probably end up mapping the whole archive into >>> each process although you'll only use a fraction of the classes in the >>> archive. >>> >>>>> If we want to use ONE archive for several applications and we can >>>>> accept to have a bigger footprint if running a single (or just a few) >>>>> applications in parallel I suppose the overhead of simply dumping all >>>>> the classes from the classpathes of the various applications compared >>>>> to an accurate solution where we only dump the actually used classes >>>>> of all applications would be not that big. >>>> >>>> >>>> But those "accurate" solutions duplicate the core classes and that's a waste >>>> of footprint. >>>> >>> >>> By "accurate" I meant one "fat" archive which contains all the classes >>> USED by several applications plus the core classes. My argument was >>> that such an "accurate" "fat" archive won't be much smaller compared >>> to a "fat" archive which simply contains all the core classes plus all >>> the application classes (i.e. from the application class pathes, no >>> matter if they are ever used or not). But the latter would be much >>> simpler to implement. >> >> The above discussion and an internal proposal for hybrid archiving seem to converge on a few points. If there is no objection to the hybrid archiving proposal internally, maybe we can shared the details of the proposal on openjdk soon. >> >> Thanks, >> >> Jiangli >> >> >>> >>>> David >>>> ----- >>>> >>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>> >>>>>> On 3/05/2018 4:48 PM, Volker Simonis wrote: >>>>>>> >>>>>>> >>>>>>> On Thu, May 3, 2018 at 6:52 AM, Ioi Lam wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 5/2/18 10:00 AM, Volker Simonis wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, May 1, 2018 at 8:32 PM, Ioi Lam wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> PROBLEM: >>>>>>>>>> >>>>>>>>>> As discussed with Volker and Yumin in previous e-mails, AppCDS has >>>>>>>>>> some >>>>>>>>>> experimental support for custom class loaders. However, it's not very >>>>>>>>>> easy >>>>>>>>>> to use. >>>>>>>>>> >>>>>>>>>> For example, you can write a classlist like this: >>>>>>>>>> >>>>>>>>>> java/lang/Object id: 1 >>>>>>>>>> CustomLoadee id: 2 super: 1 source: /tmp/foo.jar >>>>>>>>>> >>>>>>>>>> The CustomLoadee class will be stored in the shared archive with a >>>>>>>>>> CRC >>>>>>>>>> code. >>>>>>>>>> During runtime, if a customed loader wants to load a class of the >>>>>>>>>> same >>>>>>>>>> name, >>>>>>>>>> and its classfile has the same size and CRC as the archived class, >>>>>>>>>> the >>>>>>>>>> archived version will be loaded. This speeds up class loading by >>>>>>>>>> avoiding >>>>>>>>>> parsing the class file, and saves space by sharing the mmap'ed class >>>>>>>>>> metadata across processes. >>>>>>>>>> >>>>>>>>>> You can see an example test at: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java >>>>>>>>>> >>>>>>>>>> However, the current scheme requires you to specify all the super >>>>>>>>>> classes >>>>>>>>>> and interfaces. There's no support provided by the >>>>>>>>>> -XX:DumpLoadedClassList >>>>>>>>>> option. It can be helped somewhat with Volker's tool: >>>>>>>>>> https://github.com/simonis/cl4cds >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> POSSIBLE SOLUTIONS: >>>>>>>>>> >>>>>>>>>> 1. "Dump-as-you-go". As suggested by Yumin, we can provide a jcmd to >>>>>>>>>> ask >>>>>>>>>> a >>>>>>>>>> running JVM process to dump all of its loaded classes, including >>>>>>>>>> those >>>>>>>>>> loaded by custom loaders, into an archive. An alternative is to dump >>>>>>>>>> the >>>>>>>>>> archive at JVM exit time (or when you press Ctrl-C, etc. >>>>>>>>>> >>>>>>>>>> 2. Add information about the custom classes for >>>>>>>>>> -XX:DumpLoadedClassList. >>>>>>>>>> The >>>>>>>>>> trouble is some class loaders don't specify a code source that can be >>>>>>>>>> understood by the built-in class loaders. For example, the "Fat Jars" >>>>>>>>>> would >>>>>>>>>> have a code source like >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/ >>>>>>>>>> >>>>>>>>>> also, many custom loaders would pre-process the classfile data before >>>>>>>>>> defining the class, so we can't simply archive the version of the >>>>>>>>>> class >>>>>>>>>> on >>>>>>>>>> disk. >>>>>>>>>> >>>>>>>>>> One possible solution for #2 is to include the class file data in the >>>>>>>>>> -XX:DumpLoadedClassList output: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> java/lang/Object id: 1 >>>>>>>>>> CustomLoadee id: 2 super: 1 source: base64 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA >>>>>>>>>> AgAPAAcAAAAKAAEABQADAAYACA== >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Of the 2 solutions: >>>>>>>>>> >>>>>>>>>> #1 seems easier to use, but may require more invasive modifications >>>>>>>>>> in >>>>>>>>>> the >>>>>>>>>> VM, especially if you want to be able to continue execution after >>>>>>>>>> dumping. >>>>>>>>>> >>>>>>>>> Not sure what #1 really proposes: dumping the complete .jsa archive at >>>>>>>>> runtime or dumping just the loaded classes. >>>>>>>>> >>>>>>>>> If it's just about dumping the loaded class without generating the >>>>>>>>> .jsa archive there's the problem that by default the VM doesn't store >>>>>>>>> the exact bytes of a class after the class was loaded (except when >>>>>>>>> class transformers are registered). So the class files would have to >>>>>>>>> be re-assembled from the internal VM structures (in the same way this >>>>>>>>> is done for class redefinition) and the resulting class-file may be >>>>>>>>> different from the original bytes (i.e. some attributes may be >>>>>>>>> missing). >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> #1 is for creating the JSA file, not just dumping the class files. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> If #1 is about creating the whole .jsa archive at runtime (or at VM >>>>>>>>> exit) I think that would be the most attractive solution from a >>>>>>>>> usability point of view although I understand that #2 will be easier >>>>>>>>> to implement in the short term. Regarding the argument that #1 will >>>>>>>>> produce a "binary blob" that's true, but that's already true now when >>>>>>>>> we use "Xshare:dump". I think it should be not to hard to implement a >>>>>>>>> tool based an SA which could introspect a .jsa archive. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> The argument about the binary blob is to compare it against the text >>>>>>>> file >>>>>>>> produced by -XX:DumpLoadedClassList. >>>>>>>> >>>>>>>> One use case to consider is when you have a JAR file that contains >>>>>>>> several >>>>>>>> apps that each load a unique set of classes. Today, (assuming that >>>>>>>> custom >>>>>>>> class loaders are not used), you can run each app once with >>>>>>>> -XX:DumpLoadedClassList, and then do an >>>>>>>> >>>>>>>> cat *.classlist | sort | uniq > combined.classlist >>>>>>>> >>>>>>>> and then create an archive that would work for all these apps. >>>>>>>> >>>>>>> >>>>>>> But is this really y relevant use case? Why would I like to create ONE >>>>>>> archive for several apps? This would actually increase the footprint >>>>>>> of a single instance which uses this archive. If I have several apps I >>>>>>> would expect that users create a specific archive for each app to get >>>>>>> the best out of CDS. >>>>>>> >>>>>>>> With the binary blob, there's no easy way of doing this. It will be >>>>>>>> very >>>>>>>> difficult to write a tool to decipher each blob and then somehow >>>>>>>> combine >>>>>>>> them into a single one. >>>>>>>> >>>>>>> >>>>>>> But if users really wants such a "fat" archive, there's a much easier >>>>>>> way: just dump ALL the classes from the .jar file into the archive. A >>>>>>> class list for this could easily be assembled either with an external >>>>>>> tool like cl4cds (or even a simple shell scripts which converts the >>>>>>> output of `unzip -l ` into the correct format). Or, even >>>>>>> simpler, by adding a new option to the VM similar to >>>>>>> -XX:DumpLoadedClassList which dumps all the classes it can find on the >>>>>>> class path (and potentially other, configurable locations). >>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>>> >>>>>>>> >>>>>>>>>> #2 would be easier to implement, but the classlist might be huge. >>>>>>>>>> >>>>>>>>>> Also, #2 would allow post-processing tools to remove unneeded >>>>>>>>>> classes, >>>>>>>>>> or >>>>>>>>>> merge two runs into a single list. The output of #1 is essentially a >>>>>>>>>> binary >>>>>>>>>> blob that's impossible for off-line analysis/optimizations. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Any comments, or suggestions for alternatives? >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> - Ioi >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>> >>>> >> > From goetz.lindenmaier at sap.com Fri May 4 15:14:08 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 4 May 2018 15:14:08 +0000 Subject: 8202605 ... was RE: [ping] RE: RFR(M): 8199852: Print more information about class loaders in LinkageErrors. Message-ID: Hi Lois, I had a look at the Jira issue you opened. I wanted to address the issue. https://bugs.openjdk.java.net/browse/JDK-8202605 First I was puzzled, but then I realized major parts of it have been done in 8201556: Disallow reading oops in ClassLoaderData if unloading http://hg.openjdk.java.net/jdk/jdk/rev/e242740a92b8 For replacing java_lang_ClassLoader::name() by ClassLoaderData::loader_name(): I don't think this makes sense, as the first is the mere accessor for the field specified in the Java class java.lang.Classloader, and the second (respective SystemDictionary::loader_name()) is supposed to return some useful information in any case, also if the name field is not set. Nevertheless I would like to see the content of the name field printed in SystemDictionary::loader_name() if it is set. And probably SystemDictionary::loader_name() should be replaced by ClassLoaderData::loader_name(). What do you think? Best regards, Goetz. > -----Original Message----- > From: Lois Foltan [mailto:lois.foltan at oracle.com] > Sent: Donnerstag, 3. Mai 2018 20:25 > To: Lindenmaier, Goetz ; David Holmes > ; hotspot-runtime-dev at openjdk.java.net > Subject: Re: [ping] RE: RFR(M): 8199852: Print more information about class > loaders in LinkageErrors. > > On 5/2/2018 11:19 AM, Lindenmaier, Goetz wrote: > > > Hi Lois, > > > > Thanks for your approval of my change! > > > >> Hi Goetz, > >> > >> I'm ok with going forward with this change, you have my review. > However, > >> I would like to record some concerns I do have about this change which I > >> hope RFEs will be created for to clean up. > >> > >> 1. the change introduces another duplicate way to obtain the class > >> loader's name via java_lang_ClassLoader::describe_external, where > >> Klass::class_loader_and_module_name() exists today and could be > modified > > As stated before, I think it would be quite complicated to design one > method > > that delivers a name/string as complex as that of these methods to fit all > purposes. > > I appreciate though to find a common way to put the messages, common > > wording etc.. I think class_loader_and_module_name() could use > > describe_external() for the class_loader name. > > > >> 2. the method describe_external() obtains the class loader's name in a > >> different manner than SystemDictionary::loader_name().? How to obtain > >> the class loader's name should be standardized within the VM. > > In some intermediate design, I had called describe_external within > loader_name. > > As I understand, loader_name does not report the name of the loader at > all, > > only the class name. This really should be improved. > Hi Goetz, > > I've entered an RFE to standardize on the use of > ClassLoaderData::loader_name() as the preferred way within the VM to > obtain a class loader's name.? See > https://bugs.openjdk.java.net/browse/JDK-8202605.? The class loader's > name is set within ClassLoaderData::initialize_name_and_klass().??? The > aim is to remove SystemDictionary::loader_name() and all independent > calls to java_lang_ClassLoader::name() in favor of this method. > > Thanks, > Lois > > > > >> 3. previously I expressed concerns about adding the parent loader's > >> information into the LinkageError message since I think this makes the > >> message wordy and to me this seems like information that is more > >> appropriate for logging. > > I think error messages should contain as much information as possible > > to track down the error. > > Thanks for approving my change anyways. > > > > Best regards, > > Goetz. > > > >> Thanks, > >> Lois > >> > >> On 5/2/2018 7:21 AM, Lindenmaier, Goetz wrote: > >>> Hi David, > >>> > >>>> Sorry really strapped for time these days ... > >>> This is ok, but I will keep pinging ... ?? > >>> A great, general thanks to you, as you really spend a lot of effort on > >>> reviewing changes by SAP. That helps us a lot and we appreciate > >>> your efforts! > >>> > >>>> src/hotspot/share/classfile/javaClasses.cpp > >>>> src/hotspot/share/classfile/javaClasses.hpp > >>>> The description of describe_external in the hpp doesn't match the > >>>> implementation in the cpp: parent: x versus child of x > >>> I updated the comment: > >>> - // Prints "" (instance of , parent: "" > >> ) > >>> - // or (parent: "" ). > >>> + // Prints "" (instance of , child of "" > >> ). > >>> + // If a classloader has no name, it prints instead. The > output > >>> + // for well known loaders (system/platform) is abbreviated. > >>> > >>>> src/hotspot/share/classfile/systemDictionary.cpp > >>>> > >>>> 2208 throwException = true; > >>>> 2209 ss.print("loader constraint violation: loader %s", > >>>> 2210 > >>>> java_lang_ClassLoader::describe_external(class_loader())); > >>>> 2211 ss.print(" wants to load %s %s.", > >>>> 2212 k->external_kind(), k->external_name()); > >>>> 2213 Klass *existing_klass = > >>>> constraints()->find_constrained_klass(name, class_loader); > >>>> 2214 if (existing_klass->class_loader() != class_loader()) { > >>>> 2215 ss.print(" A different %s with the same name was > >>>> previously loaded by %s.", > >>>> 2216 existing_klass->external_kind(), > >>>> 2217 > >>>> java_lang_ClassLoader::describe_external(existing_klass- > >>> class_loader())); > >>>> 2218 } > >>>> > >>>> I still find this way too verbose. I won't oppose it but I don't like > >>>> it. I know you're trying to solve your remote support problem, but I > >>>> worry about the newbie trying to learn and experiment and being > totally > >>>> bamboozled by the exception message. (I thought the old ones were > bad > >>>> enough - but necessary in their level of preciseness.) > >>> I understand. Thanks for not opposing. > >>> > >>> > >>>> 3125 // Caller needs ResourceMark. > >>>> > >>>> Nit: Why change this line but not line 3131? In general single-line > >>>> comments don't need to be written as grammatically correct sentences > >>>> with full punctuation. Only if it is a multi-sentence comment should you > >>>> need to do that. > >>> Reverted, leftover from my other edits in that code that were removed. > >>> > >>>> Only comment I'll make on the tests is that being too specific about the > >>>> exact message makes the tests more fragile. > >>> Yes, but there are so many details in the message I would like to > >>> assure (like the proper name of classes with '.') that I think this is > >>> necessary. > >>> > >>> I'll push the change through the jdk-submit forest. (It's through all > >>> our tests, anyways). After pushing, I will also update the bug to > >>> list the final exception messages. > >>> Can I consider it reviewed by you, then? > >>> > >>> Best regards, > >>> Goetz. > >>> > >>> > >>>> Thanks, > >>>> David > >>>> ----- > >>>> > >>>> > >>>> On 26/04/2018 1:30 AM, Lindenmaier, Goetz wrote: > >>>>> Hi Lois, David, George or others, > >>>>> > >>>>> I would like to finish this one up, could I please get some > >>>>> (hopefully final) reviews? > >>>>> > >>>>> Thanks, > >>>>> Goetz. > >>>>> > >>>>>> -----Original Message----- > >>>>>> From: Lindenmaier, Goetz > >>>>>> Sent: Mittwoch, 18. April 2018 09:55 > >>>>>> To: 'David Holmes' ; 'hotspot-runtime- > >>>>>> dev at openjdk.java.net' > >>>>>> Subject: RE: RFR(M): 8199852: Print more information about class > >> loaders > >>>> in > >>>>>> LinkageErrors. > >>>>>> > >>>>>> Hi, > >>>>>> > >>>>>> I rebased the webrev to the jdk repo: > >>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852- > exMsg_Linkage/03- > >> jdk/ > >>>>>> Could I please get the final go? > >>>>>> > >>>>>> Thanks and best regards, > >>>>>> Goetz. > >>>>>> > >>>>>> > >>>>>>> -----Original Message----- > >>>>>>> From: Lindenmaier, Goetz > >>>>>>> Sent: Freitag, 13. April 2018 10:10 > >>>>>>> To: 'David Holmes' ; hotspot-runtime- > >>>>>>> dev at openjdk.java.net > >>>>>>> Subject: RE: RFR(M): 8199852: Print more information about class > >> loaders > >>>> in > >>>>>>> LinkageErrors. > >>>>>>> > >>>>>>> Hi David, > >>>>>>> > >>>>>>> I fixed what you mentioned here. I also found an older mail > >>>>>>> of yours with some comments I implemented now. I copied > >>>>>>> that mail's content here to have only one mail ... find references > >>>>>>> to some incremental webrevs in my replies below. > >>>>>>> > >>>>>>> Further I moved the test classes of test differentLE/ into a package. > >>>>>>> > >>>>>>> Comprehensive new webrev: > >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852- > exMsg_Linkage/03 > >>>>>>> > >>>>>>> > >>>>>>>> -----Original Message----- > >>>>>>>> From: David Holmes > >>>>>>>> Sent: Thursday, April 12, 2018 8:50 AM > >>>>>>>> To: Lindenmaier, Goetz ; hotspot- > >>>>>> runtime- > >>>>>>>> dev at openjdk.java.net > >>>>>>> ... > >>>>>>>> This looks reasonable to me. Two minor comments: > >>>>>>>> - updated tests need updated copyright notice > >>>>>>>> - I think you can avoid duplicating test1() to test2() by passing in > the > >>>>>>>> loader name and the expected message as parameters > >>>>>>> Fixed both. Incremental webrev: > >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852- > exMsg_Linkage/03- > >>>>>>> incremental2/ > >>>>>>> > >>>>>>> > >>>>>>>> -----Original Message----- > >>>>>>>> From: David Holmes > >>>>>>>> Sent: Sunday, March 25, 2018 11:49 PM > >>>>>>>> To: Lindenmaier, Goetz ; hotspot- > >>>>>> runtime- > >>>>>>>> dev at openjdk.java.net > >>>>>>> ... > >>>>>>>>> Maybe I should suppress printing verbose information about > these > >>>>>>>>> well known loaders. (There is is_builtin_class_loader_data() to > >>>>>>>>> identify such.) > >>>>>>>> Yes it might be good to simplify things for the predefined loaders. > >>>>>>>> Though I'd use: > >>>>>>>> SystemDictionary::is_platform_class_loader > >>>>>>>> SystemDictionary::is_system_class_loader > >>>>>>>> for that check. > >>>>>>> I implemented omitting the parent for these loaders. > >>>>>>> > >>>>>>>> Maybe it makes sense to have a set pattern for this descriptive > text, > >>>>>>>> and use "unnamed" if the loader has no name: > >>>>>>>> > >>>>>>>> loader , instanceof , child of loader ... > >>>>>>> I implemented printing "" for empty names. I would > like > >>>>>>> To keep the format with brackets, because the enclosing text might > >> use > >>>>>>> commas, too, and then the sentence structure might get confusing. > >>>>>>> > >>>>>>> See this incremental webrev for these two changes: > >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852- > exMsg_Linkage/03- > >>>>>>> incremental1a/ > >>>>>>> > >>>>>>>> Add this bug to the @bug in the test(s). > >>>>>>> Done. > >>>>>>> > >>>>>>>>> I'm not clear where you distinguish type and class. > >>>>>>>> Using "class" is inaccurate as it can be a class, interface or array > type. > >>>>>>>> > >>>>>>>> In the above the entity is a "type" as we are referring to a >> name, > >>>>>>>> classloader> pair. But you can also just read it as "class or interface > >>>>>>>> or array type" > >>>>>>> So why not print what it really is? I had added external_kind() for > this > >> in > >>>>>>> My previous exception message change. > >>>>>>> Unfortunately only in SystemDictionary::check_constraints() this is > >> easily > >>>>>>> possible, and it improves the message I think. > >>>>>>> > >>>>>>> Incremental webrev for these two changes: > >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852- > exMsg_Linkage/03- > >>>>>>> incremental1b/ > >>>>>>> > >>>>>>> I also implemented a test for the call to external_kind(), but I > missed > >> to > >>>> put > >>>>>>> that into > >>>>>>> the incremental webrev. > >>>>>>> > >>>>>>> Best regards, > >>>>>>> Goetz. From martin.doerr at sap.com Fri May 4 15:32:48 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 4 May 2018 15:32:48 +0000 Subject: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics In-Reply-To: References: <4a3b4d59c9f54234b0c0d325a3238cbc@sap.com> <372edaad-f92c-7071-036b-703c86ad25c1@oracle.com> <3c0af6430a8e43cbbd4178edc88a0512@sap.com> Message-ID: <0309694e4cf84438ab20b84cae51150a@sap.com> Hi David, thanks for your time. I understand that there's a lot of work at the moment. I have removed memory_order_consume again. We don't need it for our platforms and I guess nobody will miss it. I like your comment proposals, so I've updated them. New webrev: http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.02/ > Though that raises the question why you haven't added memory_order_seq_cst? I think that one could easily be used incorrectly within hotspot which would lead to problems on PPC. "Atomic operations tagged memory_order_seq_cst ... establish a single total modification order of all atomic operations that are so tagged." [1] It should work fine if all involved accesses are "so tagged". But hotspot doesn't use seq_cst loads, it uses things like OrderAccess::load_acquire. On PPC, GCC and xlC don't use a trailing sync for seq_cst stores and RMW-atomics so they only work well with leading sync loads which are not used in hotspot code. I believe that there's a substantial amount of work to be done to get this right. Thanks, Martin [1] https://en.cppreference.com/w/cpp/atomic/memory_order#Sequentially-consistent_ordering -----Original Message----- From: David Holmes [mailto:david.holmes at oracle.com] Sent: Freitag, 4. Mai 2018 08:09 To: Doerr, Martin ; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics Hi Martin, On 3/05/2018 11:03 PM, Doerr, Martin wrote: > Hi Robbin, > > thank you for reviewing. > > Submission forest tests have passed in addition to SAP's nightly builds. I have 2 reviews, now, but I'll wait a little bit before pushing. Thanks for waiting. I think it important there is a general buy-in to this effort, not just the official "get a review and push". Apologies that it's taken me a while to get to this. memory_order_consume has been deprecated in C++17: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0371r1.html I would suggest we not add it to the VM. 40 enum atomic_memory_order { 41 // We support most semantics like in C++11. I would say something like: // The modes that align with C++11 are intended to // follow the same semantics. 47 // We need to be more conservative than seq_cst on PPC64. Not sure the PPC64 comment is relevant here. memory_order_conservative represents the pre-existing strong fencing requirements. It's not intended to map to anything else, nor intended to be different across platforms. Though that raises the question why you haven't added memory_order_seq_cst? Looking at all the platform code that adds the unused "order" parameter ... I would not be surprised if the forthcoming compiler upgrades result in some additional "unused" warnings. But I guess we'll just deal with them as they are discovered. Otherwise all seems okay. Thanks, David > Best regards, > Martin > > > -----Original Message----- > From: Robbin Ehn [mailto:robbin.ehn at oracle.com] > Sent: Donnerstag, 3. Mai 2018 14:29 > To: Doerr, Martin ; Schmidt, Lutz ; David Holmes ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley (aph at redhat.com) ; John Paul Adrian Glaubitz > Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics > > On 2018-05-03 12:50, Doerr, Martin wrote: >> Hi Lutz, >> >> thanks for reviewing. >> >> Can I get more reviews, please? > > I think this looks good, thanks for fixing! > > /Robbin > >> >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: Schmidt, Lutz >> Sent: Mittwoch, 2. Mai 2018 16:20 >> To: Doerr, Martin ; David Holmes ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley (aph at redhat.com) ; John Paul Adrian Glaubitz >> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics >> >> Hi Martin, >> >> your changes look good. >> >> For the additional synchronization on s390 in particular, I do not expect a serious performance impact. The checkpoint synchronization part has always been the expensive, but rarely required, part. PPC may be another story. We knoow that a full-blown sync really hurts. We will see... >> >> Thanks, >> Lutz >> >> ?On 26.04.18, 16:23, "hotspot-runtime-dev on behalf of Doerr, Martin" wrote: >> >> Hi, >> >> I have renamed "cmpxchg_memory_order" to "atomic_memory_order" and added support to all Read-Modify-Write atomics with support for all C++11 semantics except seq_cst. The latter has issues on PPC64 so we're currently using our own memory_order_conservative instead, which is the default. >> >> Please review my new webrev: >> http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.01/ >> >> We'll test Windows, MacOS, linux x86+ppc64+s390, AIX and Solaris SPARC. >> It'd be great if somebody could check arm/aarch64 and zero. >> >> This change may enable optimizations for arm/aarch64 (not yet included). >> >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: Doerr, Martin >> Sent: Donnerstag, 26. April 2018 11:20 >> To: 'David Holmes' ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) >> Subject: RE: RFR(M): 8202080: Introduce ordering semantics for Atomic::add >> >> Hi David, >> >> sounds like we are on the same page, now. I think we should have some kind of summary of what was analyzed. But that belongs to the other thread (JDK- 8154736). >> >> > The bugs in this kind of code are very subtle and rarely exposed through normal levels of testing. >> Right. We have experienced this often enough. So it is definitely in our interest to have correct memory barriers. Mistakes will hit us (PPC64). So I appreciate that you insist on careful analysis. >> >> I'll prepare a new webrev with conservative semantics for all read-modify-write atomics by default. >> >> Best regards, >> Martin >> >> >> From jiangli.zhou at oracle.com Fri May 4 17:42:41 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 4 May 2018 10:42:41 -0700 Subject: Improving AppCDS for Custom Loaders In-Reply-To: References: <8e0d891a-c00c-8f57-362a-6a4684a6cc21@oracle.com> <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> <732ce957-b07c-bff2-277b-ca335c348753@oracle.com> <660A1644-3A08-455D-931D-D95069C39853@oracle.com> Message-ID: Hi Volker, Thank you so much for the feedbacks! You comments made me realize that some clarifications are need for the top-layer dynamic archiving. The top-layer would include all application classes, middleware classes, system library classes that are not in the bottom layer. Apologizing for not stating that clearly in the proposal. With the top layer including all necessary classes (excluding the classes in bottom layer) for a specific application, multiple instances running the same application can achieve maximum memory sharing. Using your example, the extra AWT/Swing classes are archived along with the application classes in the top layer. With the dynamic archiving ability, it?s natural to think there is no need for the two-layer scheme. In fact, I originally had the thoughts. However, after exploring different possible use cases, I agree two-layer archiving do bring benefits that are lacking in the single archive design. One example of such use case is tens/hundreds different applications running on the same machine. Without a common system layer archive, it is difficult to achieve memory sharing for common system classes, String objects, etc between different applications. Another reason why we should go with a two-layer & static/dynamic archiving is some runtime information may be sensitive and should not be archived. Strings can be a good example in this case. Currently, we archive interned String objects without running application. The Strings include resolved CONSTANT_strings, class names, method names, etc, which are all static information from class files. During execution of an application, interned Strings might contain sensitive user information, which is not safe to be included in the archive. It?s difficult to distinguish between the static information and runtime information if we go with the single layer dynamic-only archiving. We had many discussions internally over long period of time on how to improve usability with different approaches. In the end, dynamic-only or static-only two-layer archiving all have their own disadvantages and fail to meet requirements in some use cases. The hybrid archiving combines benefits/advantages of different approaches and seem to be flexible enough to fit most usages. Further comments and feedbacks are much appreciated. Best regards, Jiangli > On May 4, 2018, at 3:01 AM, Volker Simonis wrote: > > Hi Jiangli, > > thanks for sharing the hybrid archiving proposal. I think it is very > interesting and useful! > > One issue I see is that the "system library classes" which should go > into the "static" archive are inherently application specific (i.e. if > my application uses Swing, it will use most of the AWT/Swing classes). > So how will you cope with this problem. Either you put ALL the system > classes into the static archive which will be a waste for most > applications. Or you just put a small commonly used subset of classes > into the static archive which may miss many classes for specific > applications. > > If we would add the possibility to create a dynamic archive at runtime > / program end (which I think would be great from a usability > perspective) I don't see a big need for two different archives. Two > archives will further complicate (and slow down) things like Symbol > lookup (we already have two SymbolTable now and we'd probably need a > third one if we would have two archives). > > I don't think that running a few different Java applications on one > host is the most important use case for CDS. In such a scenario the > current, build time generated archive is probably the best we can do > anyway. Instead, I think the most important use case is if we have > many instances of the same Java application running on the same host. > And this is becoming more common with the raise of containerization. > For the latter use case, a dynamically generated, application specific > archive would be the optimal solution. > > Thank you and best regards, > Volker > > > > On Fri, May 4, 2018 at 3:42 AM, Jiangli Zhou wrote: >> Hi Volker, >> >> Here are some details about the hybrid archiving. The goal is to harvest the benefit of archiving by default and improve its usability. The hybrid approach combines a two-layer archiving (proposed by Ioi internally) and static & dynamic archiving techniques: >> >> - Statically archive system library classes from a provided classlist using the existing method. The archiving includes class metadata, interned string objects, constant pool resolved_references arrays, class mirror objects, etc. Static archiving can be done at the JDK image build time and shipped together with JDK binary. Following need to be addressed: >> *Relaxing the runtime CDS/AppCDS boot path check, so the packaged archive can be used after the JDK binary is installed on the target device. JDK-8199807 was created to address this issue and is targeted for JDK 11. >> *Add the static archiving generation in JDK build steps and package the generated archive with JDK image. The archive can only be generated for the same target (both OS can CPU architecture) as the build platform. I will create a RFE. >> >> - Dynamic archiving can done for application classes at the first execution of a specific application >> * The archive is created on top of the default system archive shipped with the JDK image. A separate top-layer archive file is generated for each different application. >> * Archiving is done at the end of the application execution before VM exists by relocating the class metadata to the archive spaces. Cleanup also needs to be done for copied class meta data to remove any runtime information. Most of the required functionality already exists today. For example, class metadata relocation was implemented by Ioi in JDK 10. >> * Only archive class metadata for application in the top layer initially. Archiving java heap objects in the top-layer requires more investigations. >> >> Benefits of the hybrid archiving: >> * The system archive can be shared by different applications and provides memory saving. >> * Archive for application is created and used transparently. No more profiling step and class list are required! >> * Separating the system archiving from application archiving reduces the cost of archiving at application execution time. The overhead added to the first execution time is reduced. >> >> Thanks, >> >> Jiangli >> >> >>> On May 3, 2018, at 10:34 AM, Jiangli Zhou wrote: >>> >>> >>>> On May 3, 2018, at 2:14 AM, Volker Simonis wrote: >>>> >>>> On Thu, May 3, 2018 at 11:01 AM, David Holmes wrote: >>>>> On 3/05/2018 5:16 PM, Volker Simonis wrote: >>>>>> >>>>>> On Thu, May 3, 2018 at 8:55 AM, David Holmes >>>>>> wrote: >>>>>>> >>>>>>> Just lurking here but ... >>>>>>> >>>>>>>> But is this really y relevant use case? Why would I like to create ONE >>>>>>>> archive for several apps? This would actually increase the footprint >>>>>>>> of a single instance which uses this archive. If I have several apps I >>>>>>>> would expect that users create a specific archive for each app to get >>>>>>>> the best out of CDS. >>>>>>> >>>>>>> >>>>>>> >>>>>>> One app instance may get increased footprint but you presumably use CDS >>>>>>> because you have multiple apps running (whether the same or not). These >>>>>>> apps >>>>>>> all share the core JDK classes from the archive so the overall footprint >>>>>>> per >>>>>>> instance is less. >>>>>>> >>>>>> >>>>>> If we just want to share the core JDK classes that's easy. For that we >>>>>> could mostly use the default class list (or a slightly extended one) >>>>>> which is generated at JDK build time (at JAVA_HOME/lib/classlist). >>>>> >>>>> >>>>> The point is that you are presumably running multiple instances of multiple >>>>> apps, hence you want to share one set of core classes across all, and share >>>>> the app classes across each app instance. >>>>> >>>> >>>> But that would require two archives: a general one with the core >>>> classes and an application specific one for each application. >>>> Combining the core classes and the application of various applications >>>> will not be optimal because the application classes will be all mixed >>>> in the same archive. The archive is being mapped page-wise into the >>>> java process so you'll probably end up mapping the whole archive into >>>> each process although you'll only use a fraction of the classes in the >>>> archive. >>>> >>>>>> If we want to use ONE archive for several applications and we can >>>>>> accept to have a bigger footprint if running a single (or just a few) >>>>>> applications in parallel I suppose the overhead of simply dumping all >>>>>> the classes from the classpathes of the various applications compared >>>>>> to an accurate solution where we only dump the actually used classes >>>>>> of all applications would be not that big. >>>>> >>>>> >>>>> But those "accurate" solutions duplicate the core classes and that's a waste >>>>> of footprint. >>>>> >>>> >>>> By "accurate" I meant one "fat" archive which contains all the classes >>>> USED by several applications plus the core classes. My argument was >>>> that such an "accurate" "fat" archive won't be much smaller compared >>>> to a "fat" archive which simply contains all the core classes plus all >>>> the application classes (i.e. from the application class pathes, no >>>> matter if they are ever used or not). But the latter would be much >>>> simpler to implement. >>> >>> The above discussion and an internal proposal for hybrid archiving seem to converge on a few points. If there is no objection to the hybrid archiving proposal internally, maybe we can shared the details of the proposal on openjdk soon. >>> >>> Thanks, >>> >>> Jiangli >>> >>> >>>> >>>>> David >>>>> ----- >>>>> >>>>> >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>> >>>>>>> On 3/05/2018 4:48 PM, Volker Simonis wrote: >>>>>>>> >>>>>>>> >>>>>>>> On Thu, May 3, 2018 at 6:52 AM, Ioi Lam wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 5/2/18 10:00 AM, Volker Simonis wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, May 1, 2018 at 8:32 PM, Ioi Lam wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> PROBLEM: >>>>>>>>>>> >>>>>>>>>>> As discussed with Volker and Yumin in previous e-mails, AppCDS has >>>>>>>>>>> some >>>>>>>>>>> experimental support for custom class loaders. However, it's not very >>>>>>>>>>> easy >>>>>>>>>>> to use. >>>>>>>>>>> >>>>>>>>>>> For example, you can write a classlist like this: >>>>>>>>>>> >>>>>>>>>>> java/lang/Object id: 1 >>>>>>>>>>> CustomLoadee id: 2 super: 1 source: /tmp/foo.jar >>>>>>>>>>> >>>>>>>>>>> The CustomLoadee class will be stored in the shared archive with a >>>>>>>>>>> CRC >>>>>>>>>>> code. >>>>>>>>>>> During runtime, if a customed loader wants to load a class of the >>>>>>>>>>> same >>>>>>>>>>> name, >>>>>>>>>>> and its classfile has the same size and CRC as the archived class, >>>>>>>>>>> the >>>>>>>>>>> archived version will be loaded. This speeds up class loading by >>>>>>>>>>> avoiding >>>>>>>>>>> parsing the class file, and saves space by sharing the mmap'ed class >>>>>>>>>>> metadata across processes. >>>>>>>>>>> >>>>>>>>>>> You can see an example test at: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java >>>>>>>>>>> >>>>>>>>>>> However, the current scheme requires you to specify all the super >>>>>>>>>>> classes >>>>>>>>>>> and interfaces. There's no support provided by the >>>>>>>>>>> -XX:DumpLoadedClassList >>>>>>>>>>> option. It can be helped somewhat with Volker's tool: >>>>>>>>>>> https://github.com/simonis/cl4cds >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> POSSIBLE SOLUTIONS: >>>>>>>>>>> >>>>>>>>>>> 1. "Dump-as-you-go". As suggested by Yumin, we can provide a jcmd to >>>>>>>>>>> ask >>>>>>>>>>> a >>>>>>>>>>> running JVM process to dump all of its loaded classes, including >>>>>>>>>>> those >>>>>>>>>>> loaded by custom loaders, into an archive. An alternative is to dump >>>>>>>>>>> the >>>>>>>>>>> archive at JVM exit time (or when you press Ctrl-C, etc. >>>>>>>>>>> >>>>>>>>>>> 2. Add information about the custom classes for >>>>>>>>>>> -XX:DumpLoadedClassList. >>>>>>>>>>> The >>>>>>>>>>> trouble is some class loaders don't specify a code source that can be >>>>>>>>>>> understood by the built-in class loaders. For example, the "Fat Jars" >>>>>>>>>>> would >>>>>>>>>>> have a code source like >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/ >>>>>>>>>>> >>>>>>>>>>> also, many custom loaders would pre-process the classfile data before >>>>>>>>>>> defining the class, so we can't simply archive the version of the >>>>>>>>>>> class >>>>>>>>>>> on >>>>>>>>>>> disk. >>>>>>>>>>> >>>>>>>>>>> One possible solution for #2 is to include the class file data in the >>>>>>>>>>> -XX:DumpLoadedClassList output: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> java/lang/Object id: 1 >>>>>>>>>>> CustomLoadee id: 2 super: 1 source: base64 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA >>>>>>>>>>> AgAPAAcAAAAKAAEABQADAAYACA== >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Of the 2 solutions: >>>>>>>>>>> >>>>>>>>>>> #1 seems easier to use, but may require more invasive modifications >>>>>>>>>>> in >>>>>>>>>>> the >>>>>>>>>>> VM, especially if you want to be able to continue execution after >>>>>>>>>>> dumping. >>>>>>>>>>> >>>>>>>>>> Not sure what #1 really proposes: dumping the complete .jsa archive at >>>>>>>>>> runtime or dumping just the loaded classes. >>>>>>>>>> >>>>>>>>>> If it's just about dumping the loaded class without generating the >>>>>>>>>> .jsa archive there's the problem that by default the VM doesn't store >>>>>>>>>> the exact bytes of a class after the class was loaded (except when >>>>>>>>>> class transformers are registered). So the class files would have to >>>>>>>>>> be re-assembled from the internal VM structures (in the same way this >>>>>>>>>> is done for class redefinition) and the resulting class-file may be >>>>>>>>>> different from the original bytes (i.e. some attributes may be >>>>>>>>>> missing). >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> #1 is for creating the JSA file, not just dumping the class files. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> If #1 is about creating the whole .jsa archive at runtime (or at VM >>>>>>>>>> exit) I think that would be the most attractive solution from a >>>>>>>>>> usability point of view although I understand that #2 will be easier >>>>>>>>>> to implement in the short term. Regarding the argument that #1 will >>>>>>>>>> produce a "binary blob" that's true, but that's already true now when >>>>>>>>>> we use "Xshare:dump". I think it should be not to hard to implement a >>>>>>>>>> tool based an SA which could introspect a .jsa archive. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> The argument about the binary blob is to compare it against the text >>>>>>>>> file >>>>>>>>> produced by -XX:DumpLoadedClassList. >>>>>>>>> >>>>>>>>> One use case to consider is when you have a JAR file that contains >>>>>>>>> several >>>>>>>>> apps that each load a unique set of classes. Today, (assuming that >>>>>>>>> custom >>>>>>>>> class loaders are not used), you can run each app once with >>>>>>>>> -XX:DumpLoadedClassList, and then do an >>>>>>>>> >>>>>>>>> cat *.classlist | sort | uniq > combined.classlist >>>>>>>>> >>>>>>>>> and then create an archive that would work for all these apps. >>>>>>>>> >>>>>>>> >>>>>>>> But is this really y relevant use case? Why would I like to create ONE >>>>>>>> archive for several apps? This would actually increase the footprint >>>>>>>> of a single instance which uses this archive. If I have several apps I >>>>>>>> would expect that users create a specific archive for each app to get >>>>>>>> the best out of CDS. >>>>>>>> >>>>>>>>> With the binary blob, there's no easy way of doing this. It will be >>>>>>>>> very >>>>>>>>> difficult to write a tool to decipher each blob and then somehow >>>>>>>>> combine >>>>>>>>> them into a single one. >>>>>>>>> >>>>>>>> >>>>>>>> But if users really wants such a "fat" archive, there's a much easier >>>>>>>> way: just dump ALL the classes from the .jar file into the archive. A >>>>>>>> class list for this could easily be assembled either with an external >>>>>>>> tool like cl4cds (or even a simple shell scripts which converts the >>>>>>>> output of `unzip -l ` into the correct format). Or, even >>>>>>>> simpler, by adding a new option to the VM similar to >>>>>>>> -XX:DumpLoadedClassList which dumps all the classes it can find on the >>>>>>>> class path (and potentially other, configurable locations). >>>>>>>> >>>>>>>>> Thanks >>>>>>>>> - Ioi >>>>>>>>> >>>>>>>>> >>>>>>>>>>> #2 would be easier to implement, but the classlist might be huge. >>>>>>>>>>> >>>>>>>>>>> Also, #2 would allow post-processing tools to remove unneeded >>>>>>>>>>> classes, >>>>>>>>>>> or >>>>>>>>>>> merge two runs into a single list. The output of #1 is essentially a >>>>>>>>>>> binary >>>>>>>>>>> blob that's impossible for off-line analysis/optimizations. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Any comments, or suggestions for alternatives? >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> - Ioi >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>> >> From ioi.lam at oracle.com Fri May 4 18:05:13 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 4 May 2018 11:05:13 -0700 Subject: RFR 8197954 Remove unnecessary intermediary APIs from AppCDS implementation In-Reply-To: <508E3B72-3D09-4971-AFBD-E979E3ADA646@oracle.com> References: <508E3B72-3D09-4971-AFBD-E979E3ADA646@oracle.com> Message-ID: Hi Jiangli, Thank you so much for the review. I've made changes according to your and Calvin's suggestions: http://cr.openjdk.java.net/~iklam/jdk11/8197954-remove-unnecessary-appcds-api.v02.diff/ On 5/3/18 12:43 PM, Jiangli Zhou wrote: > Hi Ioi, > > Thank you for removing the _ext files and merging the code together. > It?s very helpful. > > - sharedPathsMiscInfo.cpp > > 191 // FIXME: why is there no MODULE check? > > The path check for archived module classes is handled differently than > the classes from the -cp or -Xbootclasspath/a because we can quickly > determine if an archived module class? origin matches with the runtime > module source path. That?s why we don?t ?do string comparison of the > dump time module path and the runtime module path in > SharedPathsMiscInfo::check(), which is too restrictive and limited. > Could you please replace the FIXME comment? > Fixed. I replaced the FIXME with your comments. > - sharedPathsMiscInfo.hpp > > 131 const char* type_name(int type) { > 132 switch (type) { > 133 case BOOT: return "BOOT"; > 134 case NON_EXIST: return "NON_EXIST"; > 135 case APP: return "APP"; > 136 case MODULE: return "MODULE"; > > The types are not well-defined. That?s not introduced by your change. > The ?APP? type became inaccurate when we added the ?MODULE? type. It's > noticeable now when types are merged together. To avoid overlapping, > ?APP' can be changed to ?CLASS_PATH? or ?CP?. How about renaming the > types to following: > > BOOT_PATH > CLASS_PATH > MODULE_PATH > NON_EXIST > > > As your current change does not involve larger scale restructuring, > it?s okay if we follow up with a separate RFE for the type renaming. > I changed the names it as you suggested. > - systemDictionaryShared.cpp > > 490 // [c] At this point, if the named class was loaded by the > 491 // AppClassLoader during archive dump time, we know that it must be > 492 // loaded by the AppClassLoader during run time, and will not be > loaded > 493 // by a delegated class loader. > The above only applies to the application classes loaded from -cp, but > not modules. An archived module class may not be used at runtime if > the module has a different source path at runtime. Those archived > module classes are filtered out by the runtime shared class > ?visibility? check. > > 493 // by a delegated class loader. This is true because we have > checked the > 494 // CLASSPATH and module path to ensure compatibility between dump > time and > 495 // run time. > > Please remove ?module path? from above comment. We don?t do string > comparison of the runtime module path and dump time module path as > CLASSPATH. > > The comment [C] probably should be split into two part, one for > application classes from -cp and the other part for application module > classes. > > 508 // An alternative is to modify the Java code of > BuiltinClassLoader.loadClassOrNull(). > Should above be removed? Otherwise, more details are needed. > I've updated the comments. Because there's already comments around is_shared_class_visible_for_classloader(), I just put a reference to that, so we can avoid duplication. What do you think? Thanks - Ioi > Thanks, > Jiangli > >> On May 1, 2018, at 10:17 PM, Ioi Lam > > wrote: >> >> https://bugs.openjdk.java.net/browse/JDK-8197954 >> http://cr.openjdk.java.net/~iklam/jdk11/8197954-remove-unnecessary-appcds-api.v01/ >> >> Summary: >> >> Before AppCDS was open sourced, we had a few "Ext" classes in >> sharedClassUtil.hpp >> that abstracted away the CDS operations related to the non-boot class >> loaders. >> This API made it possible to build the Oracle JDK with AppCDS >> included, or build >> the Open JDK with AppCDS removed. >> >> With the open sourcing of AppCDS, this abstraction layer is no longer >> necessary. I >> have moved the contents of the "Ext" classes into their proper >> locations and removed >> the sharedClassUtil.hpp/cpp files. >> >> Most of the changes are just moving things around. There shouldn't be >> behavioral >> changes. >> >> The files classLoaderExt.hpp/cpp still exists -- it encapsulates the >> classpath >> management code and is not strictly unnecessary. Some clean up may be >> needed there, but >> I'll probably do that in a separate RFE. >> >> Testing with hotspot tiers1~4. >> >> Thanks >> - Ioi >> >> > From coleen.phillimore at oracle.com Fri May 4 19:04:15 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 4 May 2018 15:04:15 -0400 Subject: 8202605 ... was RE: [ping] RE: RFR(M): 8199852: Print more information about class loaders in LinkageErrors. In-Reply-To: References: Message-ID: <95d3a798-7384-a9d5-aaef-12168a8d9af9@oracle.com> On 5/4/18 11:14 AM, Lindenmaier, Goetz wrote: > Hi Lois, > > I had a look at the Jira issue you opened. I wanted to address the issue. > https://bugs.openjdk.java.net/browse/JDK-8202605 > First I was puzzled, but then I realized major parts of it have been done > in 8201556: Disallow reading oops in ClassLoaderData if unloading > http://hg.openjdk.java.net/jdk/jdk/rev/e242740a92b8 Yes, I have to admit that I didn't carry my change far enough as I was focused on not touching _class_loader oop after it's been unloading. > > For replacing java_lang_ClassLoader::name() by ClassLoaderData::loader_name(): > I don't think this makes sense, as the first is the mere accessor for the field > specified in the Java class java.lang.Classloader, and the second (respective > SystemDictionary::loader_name()) is supposed to return some useful > information in any case, also if the name field is not set. ClassLoaderData::loader_name() gets the java_lang_ClassLoader::name() if set.? It's stored in the _class_loader_name field.? The !is_unloading() case should just be removed, I think. > > Nevertheless I would like to see the content of the name field > printed in SystemDictionary::loader_name() if it is set. > And probably SystemDictionary::loader_name() should be replaced > by ClassLoaderData::loader_name(). Yes, I think that's the intent.? The code has gone back and forth on this, but now ClassLoaderData::loader_name() has the best info. One word about your error message change.? For CLD::loader_name() I think printing the parent loader and "" is too wordy for logging and I don't want to see it printed.? I look at this output a lot! Thanks, Coleen > What do you think? > > Best regards, > Goetz. > > > > >> -----Original Message----- >> From: Lois Foltan [mailto:lois.foltan at oracle.com] >> Sent: Donnerstag, 3. Mai 2018 20:25 >> To: Lindenmaier, Goetz ; David Holmes >> ; hotspot-runtime-dev at openjdk.java.net >> Subject: Re: [ping] RE: RFR(M): 8199852: Print more information about class >> loaders in LinkageErrors. >> >> On 5/2/2018 11:19 AM, Lindenmaier, Goetz wrote: >> >>> Hi Lois, >>> >>> Thanks for your approval of my change! >>> >>>> Hi Goetz, >>>> >>>> I'm ok with going forward with this change, you have my review. >> However, >>>> I would like to record some concerns I do have about this change which I >>>> hope RFEs will be created for to clean up. >>>> >>>> 1. the change introduces another duplicate way to obtain the class >>>> loader's name via java_lang_ClassLoader::describe_external, where >>>> Klass::class_loader_and_module_name() exists today and could be >> modified >>> As stated before, I think it would be quite complicated to design one >> method >>> that delivers a name/string as complex as that of these methods to fit all >> purposes. >>> I appreciate though to find a common way to put the messages, common >>> wording etc.. I think class_loader_and_module_name() could use >>> describe_external() for the class_loader name. >>> >>>> 2. the method describe_external() obtains the class loader's name in a >>>> different manner than SystemDictionary::loader_name().? How to obtain >>>> the class loader's name should be standardized within the VM. >>> In some intermediate design, I had called describe_external within >> loader_name. >>> As I understand, loader_name does not report the name of the loader at >> all, >>> only the class name. This really should be improved. >> Hi Goetz, >> >> I've entered an RFE to standardize on the use of >> ClassLoaderData::loader_name() as the preferred way within the VM to >> obtain a class loader's name.? See >> https://bugs.openjdk.java.net/browse/JDK-8202605.? The class loader's >> name is set within ClassLoaderData::initialize_name_and_klass().??? The >> aim is to remove SystemDictionary::loader_name() and all independent >> calls to java_lang_ClassLoader::name() in favor of this method. >> >> Thanks, >> Lois >> >>>> 3. previously I expressed concerns about adding the parent loader's >>>> information into the LinkageError message since I think this makes the >>>> message wordy and to me this seems like information that is more >>>> appropriate for logging. >>> I think error messages should contain as much information as possible >>> to track down the error. >>> Thanks for approving my change anyways. >>> >>> Best regards, >>> Goetz. >>> >>>> Thanks, >>>> Lois >>>> >>>> On 5/2/2018 7:21 AM, Lindenmaier, Goetz wrote: >>>>> Hi David, >>>>> >>>>>> Sorry really strapped for time these days ... >>>>> This is ok, but I will keep pinging ... ?? >>>>> A great, general thanks to you, as you really spend a lot of effort on >>>>> reviewing changes by SAP. That helps us a lot and we appreciate >>>>> your efforts! >>>>> >>>>>> src/hotspot/share/classfile/javaClasses.cpp >>>>>> src/hotspot/share/classfile/javaClasses.hpp >>>>>> The description of describe_external in the hpp doesn't match the >>>>>> implementation in the cpp: parent: x versus child of x >>>>> I updated the comment: >>>>> - // Prints "" (instance of , parent: "" >>>> ) >>>>> - // or (parent: "" ). >>>>> + // Prints "" (instance of , child of "" >>>> ). >>>>> + // If a classloader has no name, it prints instead. The >> output >>>>> + // for well known loaders (system/platform) is abbreviated. >>>>> >>>>>> src/hotspot/share/classfile/systemDictionary.cpp >>>>>> >>>>>> 2208 throwException = true; >>>>>> 2209 ss.print("loader constraint violation: loader %s", >>>>>> 2210 >>>>>> java_lang_ClassLoader::describe_external(class_loader())); >>>>>> 2211 ss.print(" wants to load %s %s.", >>>>>> 2212 k->external_kind(), k->external_name()); >>>>>> 2213 Klass *existing_klass = >>>>>> constraints()->find_constrained_klass(name, class_loader); >>>>>> 2214 if (existing_klass->class_loader() != class_loader()) { >>>>>> 2215 ss.print(" A different %s with the same name was >>>>>> previously loaded by %s.", >>>>>> 2216 existing_klass->external_kind(), >>>>>> 2217 >>>>>> java_lang_ClassLoader::describe_external(existing_klass- >>>>> class_loader())); >>>>>> 2218 } >>>>>> >>>>>> I still find this way too verbose. I won't oppose it but I don't like >>>>>> it. I know you're trying to solve your remote support problem, but I >>>>>> worry about the newbie trying to learn and experiment and being >> totally >>>>>> bamboozled by the exception message. (I thought the old ones were >> bad >>>>>> enough - but necessary in their level of preciseness.) >>>>> I understand. Thanks for not opposing. >>>>> >>>>> >>>>>> 3125 // Caller needs ResourceMark. >>>>>> >>>>>> Nit: Why change this line but not line 3131? In general single-line >>>>>> comments don't need to be written as grammatically correct sentences >>>>>> with full punctuation. Only if it is a multi-sentence comment should you >>>>>> need to do that. >>>>> Reverted, leftover from my other edits in that code that were removed. >>>>> >>>>>> Only comment I'll make on the tests is that being too specific about the >>>>>> exact message makes the tests more fragile. >>>>> Yes, but there are so many details in the message I would like to >>>>> assure (like the proper name of classes with '.') that I think this is >>>>> necessary. >>>>> >>>>> I'll push the change through the jdk-submit forest. (It's through all >>>>> our tests, anyways). After pushing, I will also update the bug to >>>>> list the final exception messages. >>>>> Can I consider it reviewed by you, then? >>>>> >>>>> Best regards, >>>>> Goetz. >>>>> >>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>> >>>>>> On 26/04/2018 1:30 AM, Lindenmaier, Goetz wrote: >>>>>>> Hi Lois, David, George or others, >>>>>>> >>>>>>> I would like to finish this one up, could I please get some >>>>>>> (hopefully final) reviews? >>>>>>> >>>>>>> Thanks, >>>>>>> Goetz. >>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Lindenmaier, Goetz >>>>>>>> Sent: Mittwoch, 18. April 2018 09:55 >>>>>>>> To: 'David Holmes' ; 'hotspot-runtime- >>>>>>>> dev at openjdk.java.net' >>>>>>>> Subject: RE: RFR(M): 8199852: Print more information about class >>>> loaders >>>>>> in >>>>>>>> LinkageErrors. >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I rebased the webrev to the jdk repo: >>>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852- >> exMsg_Linkage/03- >>>> jdk/ >>>>>>>> Could I please get the final go? >>>>>>>> >>>>>>>> Thanks and best regards, >>>>>>>> Goetz. >>>>>>>> >>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Lindenmaier, Goetz >>>>>>>>> Sent: Freitag, 13. April 2018 10:10 >>>>>>>>> To: 'David Holmes' ; hotspot-runtime- >>>>>>>>> dev at openjdk.java.net >>>>>>>>> Subject: RE: RFR(M): 8199852: Print more information about class >>>> loaders >>>>>> in >>>>>>>>> LinkageErrors. >>>>>>>>> >>>>>>>>> Hi David, >>>>>>>>> >>>>>>>>> I fixed what you mentioned here. I also found an older mail >>>>>>>>> of yours with some comments I implemented now. I copied >>>>>>>>> that mail's content here to have only one mail ... find references >>>>>>>>> to some incremental webrevs in my replies below. >>>>>>>>> >>>>>>>>> Further I moved the test classes of test differentLE/ into a package. >>>>>>>>> >>>>>>>>> Comprehensive new webrev: >>>>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852- >> exMsg_Linkage/03 >>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: David Holmes >>>>>>>>>> Sent: Thursday, April 12, 2018 8:50 AM >>>>>>>>>> To: Lindenmaier, Goetz ; hotspot- >>>>>>>> runtime- >>>>>>>>>> dev at openjdk.java.net >>>>>>>>> ... >>>>>>>>>> This looks reasonable to me. Two minor comments: >>>>>>>>>> - updated tests need updated copyright notice >>>>>>>>>> - I think you can avoid duplicating test1() to test2() by passing in >> the >>>>>>>>>> loader name and the expected message as parameters >>>>>>>>> Fixed both. Incremental webrev: >>>>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852- >> exMsg_Linkage/03- >>>>>>>>> incremental2/ >>>>>>>>> >>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: David Holmes >>>>>>>>>> Sent: Sunday, March 25, 2018 11:49 PM >>>>>>>>>> To: Lindenmaier, Goetz ; hotspot- >>>>>>>> runtime- >>>>>>>>>> dev at openjdk.java.net >>>>>>>>> ... >>>>>>>>>>> Maybe I should suppress printing verbose information about >> these >>>>>>>>>>> well known loaders. (There is is_builtin_class_loader_data() to >>>>>>>>>>> identify such.) >>>>>>>>>> Yes it might be good to simplify things for the predefined loaders. >>>>>>>>>> Though I'd use: >>>>>>>>>> SystemDictionary::is_platform_class_loader >>>>>>>>>> SystemDictionary::is_system_class_loader >>>>>>>>>> for that check. >>>>>>>>> I implemented omitting the parent for these loaders. >>>>>>>>> >>>>>>>>>> Maybe it makes sense to have a set pattern for this descriptive >> text, >>>>>>>>>> and use "unnamed" if the loader has no name: >>>>>>>>>> >>>>>>>>>> loader , instanceof , child of loader ... >>>>>>>>> I implemented printing "" for empty names. I would >> like >>>>>>>>> To keep the format with brackets, because the enclosing text might >>>> use >>>>>>>>> commas, too, and then the sentence structure might get confusing. >>>>>>>>> >>>>>>>>> See this incremental webrev for these two changes: >>>>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852- >> exMsg_Linkage/03- >>>>>>>>> incremental1a/ >>>>>>>>> >>>>>>>>>> Add this bug to the @bug in the test(s). >>>>>>>>> Done. >>>>>>>>> >>>>>>>>>>> I'm not clear where you distinguish type and class. >>>>>>>>>> Using "class" is inaccurate as it can be a class, interface or array >> type. >>>>>>>>>> In the above the entity is a "type" as we are referring to a >>> name, >>>>>>>>>> classloader> pair. But you can also just read it as "class or interface >>>>>>>>>> or array type" >>>>>>>>> So why not print what it really is? I had added external_kind() for >> this >>>> in >>>>>>>>> My previous exception message change. >>>>>>>>> Unfortunately only in SystemDictionary::check_constraints() this is >>>> easily >>>>>>>>> possible, and it improves the message I think. >>>>>>>>> >>>>>>>>> Incremental webrev for these two changes: >>>>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852- >> exMsg_Linkage/03- >>>>>>>>> incremental1b/ >>>>>>>>> >>>>>>>>> I also implemented a test for the call to external_kind(), but I >> missed >>>> to >>>>>> put >>>>>>>>> that into >>>>>>>>> the incremental webrev. >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> Goetz. From jiangli.zhou at oracle.com Fri May 4 19:54:21 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 4 May 2018 12:54:21 -0700 Subject: RFR 8197954 Remove unnecessary intermediary APIs from AppCDS implementation In-Reply-To: References: <508E3B72-3D09-4971-AFBD-E979E3ADA646@oracle.com> Message-ID: Looks good. My only question is why not using CLASS_PATH, but I think probably your concern is naming conflict. Using ?APP_PATH? is ok. Thanks, Jiangli > On May 4, 2018, at 11:05 AM, Ioi Lam wrote: > > Hi Jiangli, > > Thank you so much for the review. I've made changes according to your and Calvin's suggestions: > > http://cr.openjdk.java.net/~iklam/jdk11/8197954-remove-unnecessary-appcds-api.v02.diff/ > > On 5/3/18 12:43 PM, Jiangli Zhou wrote: >> Hi Ioi, >> >> Thank you for removing the _ext files and merging the code together. It?s very helpful. >> >> - sharedPathsMiscInfo.cpp >> >> 191 // FIXME: why is there no MODULE check? >> >> The path check for archived module classes is handled differently than the classes from the -cp or -Xbootclasspath/a because we can quickly determine if an archived module class? origin matches with the runtime module source path. That?s why we don?t do string comparison of the dump time module path and the runtime module path in SharedPathsMiscInfo::check(), which is too restrictive and limited. Could you please replace the FIXME comment? >> > Fixed. I replaced the FIXME with your comments. >> - sharedPathsMiscInfo.hpp >> >> 131 const char* type_name(int type) { >> 132 switch (type) { >> 133 case BOOT: return "BOOT"; >> 134 case NON_EXIST: return "NON_EXIST"; >> <> 135 case APP: return "APP"; >> 136 case MODULE: return "MODULE"; >> >> The types are not well-defined. That?s not introduced by your change. The ?APP? type became inaccurate when we added the ?MODULE? type. It's noticeable now when types are merged together. To avoid overlapping, ?APP' can be changed to ?CLASS_PATH? or ?CP?. How about renaming the types to following: >> >> BOOT_PATH >> CLASS_PATH >> MODULE_PATH >> NON_EXIST >> >> As your current change does not involve larger scale restructuring, it?s okay if we follow up with a separate RFE for the type renaming. >> > I changed the names it as you suggested. > >> - systemDictionaryShared.cpp >> >> 490 // [c] At this point, if the named class was loaded by the >> 491 // AppClassLoader during archive dump time, we know that it must be >> 492 // loaded by the AppClassLoader during run time, and will not be loaded >> 493 // by a delegated class loader. >> The above only applies to the application classes loaded from -cp, but not modules. An archived module class may not be used at runtime if the module has a different source path at runtime. Those archived module classes are filtered out by the runtime shared class ?visibility? check. >> >> 493 // by a delegated class loader. This is true because we have checked the >> 494 // CLASSPATH and module path to ensure compatibility between dump time and >> 495 // run time. >> >> Please remove ?module path? from above comment. We don?t do string comparison of the runtime module path and dump time module path as CLASSPATH. >> >> The comment [C] probably should be split into two part, one for application classes from -cp and the other part for application module classes. >> >> 508 // An alternative is to modify the Java code of BuiltinClassLoader.loadClassOrNull(). >> Should above be removed? Otherwise, more details are needed. >> > I've updated the comments. Because there's already comments around is_shared_class_visible_for_classloader(), I just put a reference to that, so we can avoid duplication. What do you think? > > Thanks > - Ioi > >> Thanks, >> Jiangli >> >>> On May 1, 2018, at 10:17 PM, Ioi Lam > wrote: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8197954 >>> http://cr.openjdk.java.net/~iklam/jdk11/8197954-remove-unnecessary-appcds-api.v01/ >>> >>> Summary: >>> >>> Before AppCDS was open sourced, we had a few "Ext" classes in sharedClassUtil.hpp >>> that abstracted away the CDS operations related to the non-boot class loaders. >>> This API made it possible to build the Oracle JDK with AppCDS included, or build >>> the Open JDK with AppCDS removed. >>> >>> With the open sourcing of AppCDS, this abstraction layer is no longer necessary. I >>> have moved the contents of the "Ext" classes into their proper locations and removed >>> the sharedClassUtil.hpp/cpp files. >>> >>> Most of the changes are just moving things around. There shouldn't be behavioral >>> changes. >>> >>> The files classLoaderExt.hpp/cpp still exists -- it encapsulates the classpath >>> management code and is not strictly unnecessary. Some clean up may be needed there, but >>> I'll probably do that in a separate RFE. >>> >>> Testing with hotspot tiers1~4. >>> >>> Thanks >>> - Ioi >>> >>> >> > From kim.barrett at oracle.com Fri May 4 20:00:00 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 4 May 2018 16:00:00 -0400 Subject: RFR: JDK-8202319: Fix compilation warnings in Solaris debug builds for DevStudio 12.6 In-Reply-To: <95043b95-1427-3576-4ad9-30aa18d016c7@oracle.com> References: <5AE1FFD0.1090803@oracle.com> <95043b95-1427-3576-4ad9-30aa18d016c7@oracle.com> Message-ID: Patch looks good. I will sponsor. From lois.foltan at oracle.com Fri May 4 20:35:59 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Fri, 4 May 2018 16:35:59 -0400 Subject: 8202605 ... was RE: [ping] RE: RFR(M): 8199852: Print more information about class loaders in LinkageErrors. In-Reply-To: <95d3a798-7384-a9d5-aaef-12168a8d9af9@oracle.com> References: <95d3a798-7384-a9d5-aaef-12168a8d9af9@oracle.com> Message-ID: <82e35505-04ff-9afd-57bb-c93d06aff8d7@oracle.com> On 5/4/2018 3:04 PM, coleen.phillimore at oracle.com wrote: > > > On 5/4/18 11:14 AM, Lindenmaier, Goetz wrote: >> Hi Lois, >> >> I had a look at the Jira issue you opened. I wanted to address the >> issue. >> https://bugs.openjdk.java.net/browse/JDK-8202605 >> First I was puzzled, but then I realized major parts of it have been >> done >> in 8201556: Disallow reading oops in ClassLoaderData if unloading >> http://hg.openjdk.java.net/jdk/jdk/rev/e242740a92b8 > > Yes, I have to admit that I didn't carry my change far enough as I was > focused on not touching _class_loader oop after it's been unloading. >> >> For replacing java_lang_ClassLoader::name() by >> ClassLoaderData::loader_name(): >> I don't think this makes sense, as the first is the mere accessor for >> the field >> specified in the Java class java.lang.Classloader, and the second >> (respective >> SystemDictionary::loader_name()) is supposed to return some useful >> information in any case, also if the name field is not set. > > ClassLoaderData::loader_name() gets the java_lang_ClassLoader::name() > if set.? It's stored in the _class_loader_name field.? The > !is_unloading() case should just be removed, I think. I agree and did add this as an additional comment to make sure that !is_unloading case is removed as part of the RFE. >> >> Nevertheless I would like to see the content of the name field >> printed in SystemDictionary::loader_name() if it is set. >> And probably SystemDictionary::loader_name() should be replaced >> by ClassLoaderData::loader_name(). > > Yes, I think that's the intent.? The code has gone back and forth on > this, but now ClassLoaderData::loader_name() has the best info. I agree with this as well, the intent of the RFE is to prefer ClassLoaderData::loader_name() over SystemDictionary::loader_name(). Thanks, Lois > > One word about your error message change.? For CLD::loader_name() I > think printing the parent loader and "" is too wordy for > logging and I don't want to see it printed.? I look at this output a lot! > > Thanks, > Coleen >> What do you think? >> >> Best regards, >> ?? Goetz. >> >> >> >> >>> -----Original Message----- >>> From: Lois Foltan [mailto:lois.foltan at oracle.com] >>> Sent: Donnerstag, 3. Mai 2018 20:25 >>> To: Lindenmaier, Goetz ; David Holmes >>> ; hotspot-runtime-dev at openjdk.java.net >>> Subject: Re: [ping] RE: RFR(M): 8199852: Print more information >>> about class >>> loaders in LinkageErrors. >>> >>> On 5/2/2018 11:19 AM, Lindenmaier, Goetz wrote: >>> >>>> Hi Lois, >>>> >>>> Thanks for your approval of my change! >>>> >>>>> Hi Goetz, >>>>> >>>>> I'm ok with going forward with this change, you have my review. >>> However, >>>>> I would like to record some concerns I do have about this change >>>>> which I >>>>> hope RFEs will be created for to clean up. >>>>> >>>>> 1. the change introduces another duplicate way to obtain the class >>>>> loader's name via java_lang_ClassLoader::describe_external, where >>>>> Klass::class_loader_and_module_name() exists today and could be >>> modified >>>> As stated before, I think it would be quite complicated to design one >>> method >>>> that delivers a name/string as complex as that of these methods to >>>> fit all >>> purposes. >>>> I appreciate though to find a common way to put the messages, common >>>> wording etc.. I think class_loader_and_module_name() could use >>>> describe_external() for the class_loader name. >>>> >>>>> 2. the method describe_external() obtains the class loader's name >>>>> in a >>>>> different manner than SystemDictionary::loader_name(). How to obtain >>>>> the class loader's name should be standardized within the VM. >>>> In some intermediate design, I had called describe_external within >>> loader_name. >>>> As I understand, loader_name does not report the name of the loader at >>> all, >>>> only the class name.? This really should be improved. >>> Hi Goetz, >>> >>> I've entered an RFE to standardize on the use of >>> ClassLoaderData::loader_name() as the preferred way within the VM to >>> obtain a class loader's name.? See >>> https://bugs.openjdk.java.net/browse/JDK-8202605.? The class loader's >>> name is set within ClassLoaderData::initialize_name_and_klass().??? The >>> aim is to remove SystemDictionary::loader_name() and all independent >>> calls to java_lang_ClassLoader::name() in favor of this method. >>> >>> Thanks, >>> Lois >>> >>>>> 3. previously I expressed concerns about adding the parent loader's >>>>> information into the LinkageError message since I think this makes >>>>> the >>>>> message wordy and to me this seems like information that is more >>>>> appropriate for logging. >>>> I think error messages should contain as much information as possible >>>> to track down the error. >>>> Thanks for approving my change anyways. >>>> >>>> Best regards, >>>> ??? Goetz. >>>> >>>>> Thanks, >>>>> Lois >>>>> >>>>> On 5/2/2018 7:21 AM, Lindenmaier, Goetz wrote: >>>>>> Hi David, >>>>>> >>>>>>> Sorry really strapped for time these days ... >>>>>> This is ok, but I will keep pinging ... ?? >>>>>> A great, general thanks to you, as you really spend a lot of >>>>>> effort on >>>>>> reviewing changes by SAP.? That helps us a lot and we appreciate >>>>>> your efforts! >>>>>> >>>>>>> src/hotspot/share/classfile/javaClasses.cpp >>>>>>> src/hotspot/share/classfile/javaClasses.hpp >>>>>>> The description of describe_external in the hpp doesn't match the >>>>>>> implementation in the cpp: parent: x versus child of x >>>>>> I updated the comment: >>>>>> -? // Prints "" (instance of , parent: "" >>>>> ) >>>>>> -? // or???? (parent: "" ). >>>>>> +? // Prints "" (instance of , child of "" >>>>> ). >>>>>> +? // If a classloader has no name, it prints instead. The >>> output >>>>>> +? // for well known loaders (system/platform) is abbreviated. >>>>>> >>>>>>> src/hotspot/share/classfile/systemDictionary.cpp >>>>>>> >>>>>>> 2208???????? throwException = true; >>>>>>> 2209???????? ss.print("loader constraint violation: loader %s", >>>>>>> 2210 >>>>>>> java_lang_ClassLoader::describe_external(class_loader())); >>>>>>> 2211???????? ss.print(" wants to load %s %s.", >>>>>>> 2212????????????????? k->external_kind(), k->external_name()); >>>>>>> 2213???????? Klass *existing_klass = >>>>>>> constraints()->find_constrained_klass(name, class_loader); >>>>>>> 2214???????? if (existing_klass->class_loader() != >>>>>>> class_loader()) { >>>>>>> 2215?????????? ss.print(" A different %s with the same name was >>>>>>> previously loaded by %s.", >>>>>>> 2216 existing_klass->external_kind(), >>>>>>> 2217 >>>>>>> java_lang_ClassLoader::describe_external(existing_klass- >>>>>> class_loader())); >>>>>>> 2218???????? } >>>>>>> >>>>>>> I still find this way too verbose. I won't oppose it but I don't >>>>>>> like >>>>>>> it. I know you're trying to solve your remote support problem, >>>>>>> but I >>>>>>> worry about the newbie trying to learn and experiment and being >>> totally >>>>>>> bamboozled by the exception message. (I thought the old ones were >>> bad >>>>>>> enough - but necessary in their level of preciseness.) >>>>>> I understand. Thanks for not opposing. >>>>>> >>>>>> >>>>>>> 3125 // Caller needs ResourceMark. >>>>>>> >>>>>>> Nit: Why change this line but not line 3131? In general single-line >>>>>>> comments don't need to be written as grammatically correct >>>>>>> sentences >>>>>>> with full punctuation. Only if it is a multi-sentence comment >>>>>>> should you >>>>>>> need to do that. >>>>>> Reverted, leftover from my other edits in that code that were >>>>>> removed. >>>>>> >>>>>>> Only comment I'll make on the tests is that being too specific >>>>>>> about the >>>>>>> exact message makes the tests more fragile. >>>>>> Yes, but there are so many details in the message I would like to >>>>>> assure (like the proper name of classes with '.') that I think >>>>>> this is >>>>>> necessary. >>>>>> >>>>>> I'll push the change through the jdk-submit forest. (It's through >>>>>> all >>>>>> our tests, anyways).? After pushing, I will also update the bug to >>>>>> list the final exception messages. >>>>>> Can I consider it reviewed by you, then? >>>>>> >>>>>> Best regards, >>>>>> ???? Goetz. >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>> >>>>>>> On 26/04/2018 1:30 AM, Lindenmaier, Goetz wrote: >>>>>>>> Hi Lois, David, George or others, >>>>>>>> >>>>>>>> I would like to finish this one up, could I please get some >>>>>>>> (hopefully final) reviews? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> ????? Goetz. >>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Lindenmaier, Goetz >>>>>>>>> Sent: Mittwoch, 18. April 2018 09:55 >>>>>>>>> To: 'David Holmes' ; 'hotspot-runtime- >>>>>>>>> dev at openjdk.java.net' >>>>>>>>> Subject: RE: RFR(M): 8199852: Print more information about class >>>>> loaders >>>>>>> in >>>>>>>>> LinkageErrors. >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I rebased the webrev to the jdk repo: >>>>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852- >>> exMsg_Linkage/03- >>>>> jdk/ >>>>>>>>> Could I please get the final go? >>>>>>>>> >>>>>>>>> Thanks and best regards, >>>>>>>>> ????? Goetz. >>>>>>>>> >>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Lindenmaier, Goetz >>>>>>>>>> Sent: Freitag, 13. April 2018 10:10 >>>>>>>>>> To: 'David Holmes' ; hotspot-runtime- >>>>>>>>>> dev at openjdk.java.net >>>>>>>>>> Subject: RE: RFR(M): 8199852: Print more information about class >>>>> loaders >>>>>>> in >>>>>>>>>> LinkageErrors. >>>>>>>>>> >>>>>>>>>> Hi David, >>>>>>>>>> >>>>>>>>>> I fixed what you mentioned here. I also found an older mail >>>>>>>>>> of yours with some comments I implemented now. I copied >>>>>>>>>> that mail's content here to have only one mail ... find >>>>>>>>>> references >>>>>>>>>> to some incremental webrevs in my replies below. >>>>>>>>>> >>>>>>>>>> Further I moved the test classes of test differentLE/ into a >>>>>>>>>> package. >>>>>>>>>> >>>>>>>>>> Comprehensive new webrev: >>>>>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852- >>> exMsg_Linkage/03 >>>>>>>>>> >>>>>>>>>>> -----Original Message----- >>>>>>>>>>> From: David Holmes >>>>>>>>>>> Sent: Thursday, April 12, 2018 8:50 AM >>>>>>>>>>> To: Lindenmaier, Goetz ; hotspot- >>>>>>>>> runtime- >>>>>>>>>>> dev at openjdk.java.net >>>>>>>>>> ... >>>>>>>>>>> This looks reasonable to me. Two minor comments: >>>>>>>>>>> - updated tests need updated copyright notice >>>>>>>>>>> - I think you can avoid duplicating test1() to test2() by >>>>>>>>>>> passing in >>> the >>>>>>>>>>> ?????? loader name and the expected message as parameters >>>>>>>>>> Fixed both. Incremental webrev: >>>>>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852- >>> exMsg_Linkage/03- >>>>>>>>>> incremental2/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> -----Original Message----- >>>>>>>>>>> From: David Holmes >>>>>>>>>>> Sent: Sunday, March 25, 2018 11:49 PM >>>>>>>>>>> To: Lindenmaier, Goetz ; hotspot- >>>>>>>>> runtime- >>>>>>>>>>> dev at openjdk.java.net >>>>>>>>>> ... >>>>>>>>>>>> Maybe I should suppress printing verbose information about >>> these >>>>>>>>>>>> well known loaders. (There is >>>>>>>>>>>> is_builtin_class_loader_data() to >>>>>>>>>>>> identify such.) >>>>>>>>>>> Yes it might be good to simplify things for the predefined >>>>>>>>>>> loaders. >>>>>>>>>>> Though I'd use: >>>>>>>>>>> SystemDictionary::is_platform_class_loader >>>>>>>>>>> SystemDictionary::is_system_class_loader >>>>>>>>>>> for that check. >>>>>>>>>> I implemented omitting the parent for these loaders. >>>>>>>>>> >>>>>>>>>>> Maybe it makes sense to have a set pattern for this descriptive >>> text, >>>>>>>>>>> and use "unnamed" if the loader has no name: >>>>>>>>>>> >>>>>>>>>>> loader , instanceof , child of loader ... >>>>>>>>>> I implemented printing "" for empty names.? I would >>> like >>>>>>>>>> To keep the format with brackets, because the enclosing text >>>>>>>>>> might >>>>> use >>>>>>>>>> commas, too, and then the sentence structure might get >>>>>>>>>> confusing. >>>>>>>>>> >>>>>>>>>> See this incremental webrev for these two changes: >>>>>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852- >>> exMsg_Linkage/03- >>>>>>>>>> incremental1a/ >>>>>>>>>> >>>>>>>>>>> Add this bug to the @bug in the test(s). >>>>>>>>>> Done. >>>>>>>>>> >>>>>>>>>>>> I'm not clear where you distinguish type and class. >>>>>>>>>>> Using "class" is inaccurate as it can be a class, interface >>>>>>>>>>> or array >>> type. >>>>>>>>>>> In the above the entity is a "type" as we are referring to a >>>>>>>>>>> >>>> name, >>>>>>>>>>> classloader> pair. But you can also just read it as "class >>>>>>>>>>> or interface >>>>>>>>>>> or array type" >>>>>>>>>> So why not print what it really is? I had added >>>>>>>>>> external_kind() for >>> this >>>>> in >>>>>>>>>> My previous exception message change. >>>>>>>>>> Unfortunately only in SystemDictionary::check_constraints() >>>>>>>>>> this is >>>>> easily >>>>>>>>>> possible, and it improves the message I think. >>>>>>>>>> >>>>>>>>>> Incremental webrev for these two changes: >>>>>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8199852- >>> exMsg_Linkage/03- >>>>>>>>>> incremental1b/ >>>>>>>>>> >>>>>>>>>> I also implemented a test for the call to external_kind(), but I >>> missed >>>>> to >>>>>>> put >>>>>>>>>> that into >>>>>>>>>> the incremental webrev. >>>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> ????? Goetz. > From ioi.lam at oracle.com Fri May 4 21:09:48 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 4 May 2018 14:09:48 -0700 Subject: RFR 8197954 Remove unnecessary intermediary APIs from AppCDS implementation In-Reply-To: References: <508E3B72-3D09-4971-AFBD-E979E3ADA646@oracle.com> Message-ID: <3e3c5229-cea3-7dd6-dcb1-c0d58a5fa6e0@oracle.com> On 5/4/18 12:54 PM, Jiangli Zhou wrote: > Looks good. My only question is why not using CLASS_PATH, but I think > probably your concern is naming conflict. Using ?APP_PATH? is ok. > One reason is we have many places that refer to "app" vs "boot" paths, so if we just say "CLASS_PATH" it is not consist with the rest of the AppCDS code. By the way, the CLASSPATH has traditionally been called many different names -- the classpath, the app classpath, the system classpath. Maybe we should clean all that up in a separate RFE. Also, I found out that SharedPathsMiscInfo::MODULE_PATH is not used anywhere, so I will delete it. Thanks - Ioi > Thanks, > > Jiangli > >> On May 4, 2018, at 11:05 AM, Ioi Lam > > wrote: >> >> Hi Jiangli, >> >> Thank you so much for the review. I've made changes according to your >> and Calvin's suggestions: >> >> http://cr.openjdk.java.net/~iklam/jdk11/8197954-remove-unnecessary-appcds-api.v02.diff/ >> >> >> On 5/3/18 12:43 PM, Jiangli Zhou wrote: >>> Hi Ioi, >>> >>> Thank you for removing the _ext files and merging the code together. >>> It?s very helpful. >>> >>> - sharedPathsMiscInfo.cpp >>> >>> 191 // FIXME: why is there no MODULE check? >>> >>> The path check for archived module classes is handled differently >>> than the classes from the -cp or -Xbootclasspath/a because we can >>> quickly determine if an archived module class? origin matches with >>> the runtime module source path. That?s why we don?t ?do string >>> comparison of the dump time module path and the runtime module path >>> in SharedPathsMiscInfo::check(), which is too restrictive and >>> limited. Could you please replace the FIXME comment? >>> >> Fixed. I replaced the FIXME with your comments. >>> - sharedPathsMiscInfo.hpp >>> >>> 131 const char* type_name(int type) { >>> 132 switch (type) { >>> 133 case BOOT: return "BOOT"; >>> 134 case NON_EXIST: return "NON_EXIST"; >>> 135 case APP: return "APP"; >>> 136 case MODULE: return "MODULE"; >>> >>> The types are not well-defined. That?s not introduced by your >>> change. The ?APP? type became inaccurate when we added the ?MODULE? >>> type. It's noticeable now when types are merged together. To avoid >>> overlapping, ?APP' can be changed to ?CLASS_PATH? or ?CP?. How about >>> renaming the types to following: >>> >>> BOOT_PATH >>> CLASS_PATH >>> MODULE_PATH >>> NON_EXIST >>> >>> >>> As your current change does not involve larger scale restructuring, >>> it?s okay if we follow up with a separate RFE for the type renaming. >>> >> I changed the names it as you suggested. >> >>> - systemDictionaryShared.cpp >>> >>> 490 // [c] At this point, if the named class was loaded by the >>> 491 // AppClassLoader during archive dump time, we know that it must be >>> 492 // loaded by the AppClassLoader during run time, and will not be >>> loaded >>> 493 // by a delegated class loader. >>> The above only applies to the application classes loaded from -cp, >>> but not modules. An archived module class may not be used at runtime >>> if the module has a different source path at runtime. Those archived >>> module classes are filtered out by the runtime shared class >>> ?visibility? check. >>> >>> 493 // by a delegated class loader. This is true because we have >>> checked the >>> 494 // CLASSPATH and module path to ensure compatibility between >>> dump time and >>> 495 // run time. >>> >>> Please remove ?module path? from above comment. We don?t do string >>> comparison of the runtime module path and dump time module path as >>> CLASSPATH. >>> >>> The comment [C] probably should be split into two part, one for >>> application classes from -cp and the other part for application >>> module classes. >>> >>> 508 // An alternative is to modify the Java code of >>> BuiltinClassLoader.loadClassOrNull(). >>> Should above be removed? Otherwise, more details are needed. >>> >> I've updated the comments. Because there's already comments around >> is_shared_class_visible_for_classloader(), I just put a reference to >> that, so we can avoid duplication. What do you think? >> >> Thanks >> - Ioi >> >>> Thanks, >>> Jiangli >>> >>>> On May 1, 2018, at 10:17 PM, Ioi Lam >>> > wrote: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8197954 >>>> http://cr.openjdk.java.net/~iklam/jdk11/8197954-remove-unnecessary-appcds-api.v01/ >>>> >>>> Summary: >>>> >>>> Before AppCDS was open sourced, we had a few "Ext" classes in >>>> sharedClassUtil.hpp >>>> that abstracted away the CDS operations related to the non-boot >>>> class loaders. >>>> This API made it possible to build the Oracle JDK with AppCDS >>>> included, or build >>>> the Open JDK with AppCDS removed. >>>> >>>> With the open sourcing of AppCDS, this abstraction layer is no >>>> longer necessary. I >>>> have moved the contents of the "Ext" classes into their proper >>>> locations and removed >>>> the sharedClassUtil.hpp/cpp files. >>>> >>>> Most of the changes are just moving things around. There shouldn't >>>> be behavioral >>>> changes. >>>> >>>> The files classLoaderExt.hpp/cpp still exists -- it encapsulates >>>> the classpath >>>> management code and is not strictly unnecessary. Some clean up may >>>> be needed there, but >>>> I'll probably do that in a separate RFE. >>>> >>>> Testing with hotspot tiers1~4. >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>> >> > From roger.riggs at oracle.com Fri May 4 21:13:27 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Fri, 4 May 2018 17:13:27 -0400 Subject: RFR(M): 8201593: Print array length in ArrayIndexOutOfBoundsException. In-Reply-To: <819d942132204f58992c236518e0c344@sap.com> References: <504bdd8469bb45cc8c5cf790d1f4b3fc@sap.com> <4ca91dfb-c576-98e2-1222-ca9750a64177@oracle.com> <5be0f86df2834aa8a881206ba285011e@sap.com> <0b881c69-79af-2a8b-c34c-5929357ca8fa@oracle.com> <5ADF5897.8070809@cjnash.com> <819d942132204f58992c236518e0c344@sap.com> Message-ID: Hi Goetz, Just comments on the test/message format.? (found in the 04 version of the webrev) The "." at the end of the exception messages should be removed. The text is not a sentence (its is just a fragment without a verb) and it is more flexible to print the exception message in various contexts (in which the "." would seem to terminate a sentence). I'm not a big fan of the hyphens in out-of-bounds and it would be more consistent across the new messages. It would be more consistent if the "arraycopy:" prefix always included the ":". Thanks, Roger On 4/24/18 12:25 PM, Lindenmaier, Goetz wrote: > Hi Simon, > > Because as stated here, > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-April/052665.html > it is used in other places like that, too. > > Later mails agreed on that usage to keep it consistent. > > Best regards, > Goetz. > >> -----Original Message----- >> From: Simon Nash [mailto:simon at cjnash.com] >> Sent: Dienstag, 24. April 2018 18:17 >> To: Lindenmaier, Goetz >> Cc: David Holmes ; hotspot-runtime- >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; aarch64- >> port-dev at openjdk.java.net; aarch32-port-dev at openjdk.java.net; core-libs- >> dev Libs >> Subject: Re: RFR(M): 8201593: Print array length in >> ArrayIndexOutOfBoundsException. >> >> On 24/04/2018 15:08, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> I implemented what we figured out in >>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018- >> April/027555.html >>> Some examples: >>> "Index 12 out-of-bounds for length 10." >>> "arraycopy source index -17 out of bounds for object array[10]." >>> "arraycopy destination index -18 out of bounds for double[5]." >>> "arraycopy length -19 is negative." >>> "arraycopy: last source index 13 out of bounds for double[10]." >>> "arraycopy: last destination index 7 out of bounds for long[5]." >>> >> Is there a reason why the first message says "out-of-bounds" but all others >> say "out of bounds"? >> >> Simon >> >>> Incremental webrev: >>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/03- >> incremental/ >>> Full webrev: >>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/03/ >>> >>> I'll push it through our tests tonight. >>> >>> See further comments in line: >>> >>>> -----Original Message----- >>>> From: David Holmes [mailto:david.holmes at oracle.com] >>>> Sent: Freitag, 20. April 2018 09:25 >>>> To: Lindenmaier, Goetz ; hotspot-runtime- >>>> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; >> aarch64- >>>> port-dev at openjdk.java.net; aarch32-port-dev at openjdk.java.net; core- >> libs- >>>> dev Libs >>>> Subject: Re: RFR(M): 8201593: Print array length in >>>> ArrayIndexOutOfBoundsException. >>>> >>>> Hi Goetz, >>>> >>>> This is not a file by file review ... >>>> >>>> On 19/04/2018 10:24 PM, Lindenmaier, Goetz wrote: >>>>> Hi, >>>>> >>>>> New webrev: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/02/ >>>>> >>>>> I admit my wording is not optimal. >>>> src/hotspot/share/oops/typeArrayKlass.cpp >>>> >>>> Sorry but this is still far too verbose for my tastes. The type of the >>>> array is not relevant. For the array copy its okay to indicate src or >>>> dst array. But the message should be clear and succinct not prose-like. >>>> Plus you have a lot of repetition in the ss.print statements when only >>>> one thing is changing. >>> We discussed this in some further mails. Implemented as proposed there. >>> >>>> src/hotspot/cpu/x86/c1_CodeStubs_x86.cpp >>>> >>>> I'm not seeing why the throw_index_out_of_bounds_exception should >> be >>>> tied to whether or not you have an array reference. Certainly I hate >>>> seeing the array ref being used as an implicit bool! >>> I split the constructor into two, one for ArrayIndexOutOfBounds, the other >>> for IndexOutOfBounds. >>> >>> ... >>> >>>>> I extended the test to cover the exception thrown in arraycopy better >>>> The test seems somewhat excessive, and I now see it is because of the >>>> more elaborate error messages you have at SAP. But with only the index >>>> and the array length of interest here the test can be considerably smaller. >>>> >>>> The creation tests for ArrayIndexOutOfBoundsException don't seem >>>> relevant in this context either. This looks more TCK like. >>> Yes, the constructor tests are for the code not yet contributed. >>> I simplified the tests to only check the messages. >>> >>> Best regards, >>> Goetz. >>> >>> >>> >>> >>> >>>> David >>>> ----- >>>> >>>>> and added the elementary type to the message text. This probably >>>>> needs improvement in the text, too. There are (currently) these cases: >>>>> >>>>> bject[] oa1 = new Object[10]; >>>>> Object[] oa2 = new Object[5]; >>>>> System.arraycopy(oa1, -17, oa2, 0, 5); >>>>> "while trying to copy from index -17 of an object array with length 10"); >>>>> System.arraycopy(oa1, 2, oa2, -18, 5); >>>>> "while trying to copy to index -18 of an object array with length 5"); >>>>> System.arraycopy(oa1, 2, oa2, 0, -19); >>>>> "while trying to copy a negative range -19 from an object array with >> length >>>> 10 to an object array with length 5"); >>>>> System.arraycopy(oa1, 8, oa2, 0, 5); >>>>> "while trying to copy from index 13 of an object array with length 10"); >>>>> System.arraycopy(oa1, 1, oa2, 0, 7); >>>>> "while trying to copy to index 7 of an object array with length 5"); >>>>> double[] ta1 = new double[10]; >>>>> double[] ta2 = new double[5]; >>>>> System.arraycopy(ta1, -17, ta2, 0, 5); >>>>> "while trying to copy from index -17 of a doubl array with length 10"); >>>>> System.arraycopy(ta1, 2, ta2, -18, 5); >>>>> "while trying to copy to index -18 of a double array with length 5"); >>>>> System.arraycopy(ta1, 2, ta2, 0, -19); >>>>> "while trying to copy a negative range -19 from a double array with >> length >>>> 10 to a double array with length 5"); >>>>> System.arraycopy(ta1, 8, ta2, 0, 5); >>>>> "while trying to copy from index 13 of a double array with length 10"); >>>>> System.arraycopy(ta1, 1, ta2, 0, 7); >>>>> "while trying to copy to index 7 of a double array with length 5"); >>>>> >>>>> Maybe it should say: >>>>> Arraycopy source index -1 out-of-bounds for double array of length 10. >>>>> Arraycopy target index 10 out-of-bounds for object array of length 10. >>>>> Negative range -19 when copying from an object array of length 10 to an >>>> object array of length 5. >>>>> Best regards, >>>>> Goetz. >>>>> >>>>>> -----Original Message----- >>>>>> From: David Holmes [mailto:david.holmes at oracle.com] >>>>>> Sent: Mittwoch, 18. April 2018 10:55 >>>>>> To: Lindenmaier, Goetz ; hotspot- >> runtime- >>>>>> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; >>>> aarch64- >>>>>> port-dev at openjdk.java.net; aarch32-port-dev at openjdk.java.net; >> core- >>>> libs- >>>>>> dev Libs >>>>>> Subject: Re: RFR(M): 8201593: Print array length in >>>>>> ArrayIndexOutOfBoundsException. >>>>>> >>>>>> Adding core-libs-dev as you're changing >>>>>> java.lang.ArrayIndexOutOfBoundsException. >>>>>> >>>>>> I appreciate the intent here but I find the messages excessively >>>>>> verbose. The basic error is: >>>>>> >>>>>> index N is outside range [0, length-1] >>>>>> >>>>>> David >>>>>> >>>>>> On 18/04/2018 6:09 PM, Lindenmaier, Goetz wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I would like to print a more verbose text on ArrayIndexOutOfBounds >>>>>> exception >>>>>>> that not only mentions the index, but also the length of the array >>>> accessed. >>>>>>> See the bug for documentation of the change of the message. >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/01/ >>>>>>> >>>>>>> @aarch/arm people: >>>>>>> I edited the aarch/arm files. Could you please verify this is correct? >>>>>>> I can not build on these platforms. >>>>>>> >>>>>>> The code on all the other platforms is tested with all the jtreg and jck >>>> tests >>>>>> etc. >>>>>>> Best regards, >>>>>>> Goetz. >>>>>>> >>>>>>> From jiangli.zhou at oracle.com Fri May 4 21:39:11 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 4 May 2018 14:39:11 -0700 Subject: RFR 8197954 Remove unnecessary intermediary APIs from AppCDS implementation In-Reply-To: <3e3c5229-cea3-7dd6-dcb1-c0d58a5fa6e0@oracle.com> References: <508E3B72-3D09-4971-AFBD-E979E3ADA646@oracle.com> <3e3c5229-cea3-7dd6-dcb1-c0d58a5fa6e0@oracle.com> Message-ID: <15684A64-24C2-4678-8D70-AE8A18E46CBA@oracle.com> > On May 4, 2018, at 2:09 PM, Ioi Lam wrote: > > On 5/4/18 12:54 PM, Jiangli Zhou wrote: >> Looks good. My only question is why not using CLASS_PATH, but I think probably your concern is naming conflict. Using ?APP_PATH? is ok. >> > One reason is we have many places that refer to "app" vs "boot" paths, so if we just say "CLASS_PATH" it is not consist with the rest of the AppCDS code. > > By the way, the CLASSPATH has traditionally been called many different names -- the classpath, the app classpath, the system classpath. Maybe we should clean all that up in a separate RFE. > > Also, I found out that SharedPathsMiscInfo::MODULE_PATH is not used anywhere, so I will delete it. Ok. Thanks, Jiangli > > Thanks > - Ioi > >> Thanks, >> >> Jiangli >> >>> On May 4, 2018, at 11:05 AM, Ioi Lam > wrote: >>> >>> Hi Jiangli, >>> >>> Thank you so much for the review. I've made changes according to your and Calvin's suggestions: >>> >>> http://cr.openjdk.java.net/~iklam/jdk11/8197954-remove-unnecessary-appcds-api.v02.diff/ >>> >>> On 5/3/18 12:43 PM, Jiangli Zhou wrote: >>>> Hi Ioi, >>>> >>>> Thank you for removing the _ext files and merging the code together. It?s very helpful. >>>> >>>> - sharedPathsMiscInfo.cpp >>>> >>>> 191 // FIXME: why is there no MODULE check? >>>> >>>> The path check for archived module classes is handled differently than the classes from the -cp or -Xbootclasspath/a because we can quickly determine if an archived module class? origin matches with the runtime module source path. That?s why we don?t do string comparison of the dump time module path and the runtime module path in SharedPathsMiscInfo::check(), which is too restrictive and limited. Could you please replace the FIXME comment? >>>> >>> Fixed. I replaced the FIXME with your comments. >>>> - sharedPathsMiscInfo.hpp >>>> >>>> 131 const char* type_name(int type) { >>>> 132 switch (type) { >>>> 133 case BOOT: return "BOOT"; >>>> 134 case NON_EXIST: return "NON_EXIST"; >>>> <> 135 case APP: return "APP"; >>>> 136 case MODULE: return "MODULE"; >>>> >>>> The types are not well-defined. That?s not introduced by your change. The ?APP? type became inaccurate when we added the ?MODULE? type. It's noticeable now when types are merged together. To avoid overlapping, ?APP' can be changed to ?CLASS_PATH? or ?CP?. How about renaming the types to following: >>>> >>>> BOOT_PATH >>>> CLASS_PATH >>>> MODULE_PATH >>>> NON_EXIST >>>> >>>> As your current change does not involve larger scale restructuring, it?s okay if we follow up with a separate RFE for the type renaming. >>>> >>> I changed the names it as you suggested. >>> >>>> - systemDictionaryShared.cpp >>>> >>>> 490 // [c] At this point, if the named class was loaded by the >>>> 491 // AppClassLoader during archive dump time, we know that it must be >>>> 492 // loaded by the AppClassLoader during run time, and will not be loaded >>>> 493 // by a delegated class loader. >>>> The above only applies to the application classes loaded from -cp, but not modules. An archived module class may not be used at runtime if the module has a different source path at runtime. Those archived module classes are filtered out by the runtime shared class ?visibility? check. >>>> >>>> 493 // by a delegated class loader. This is true because we have checked the >>>> 494 // CLASSPATH and module path to ensure compatibility between dump time and >>>> 495 // run time. >>>> >>>> Please remove ?module path? from above comment. We don?t do string comparison of the runtime module path and dump time module path as CLASSPATH. >>>> >>>> The comment [C] probably should be split into two part, one for application classes from -cp and the other part for application module classes. >>>> >>>> 508 // An alternative is to modify the Java code of BuiltinClassLoader.loadClassOrNull(). >>>> Should above be removed? Otherwise, more details are needed. >>>> >>> I've updated the comments. Because there's already comments around is_shared_class_visible_for_classloader(), I just put a reference to that, so we can avoid duplication. What do you think? >>> >>> Thanks >>> - Ioi >>> >>>> Thanks, >>>> Jiangli >>>> >>>>> On May 1, 2018, at 10:17 PM, Ioi Lam > wrote: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8197954 >>>>> http://cr.openjdk.java.net/~iklam/jdk11/8197954-remove-unnecessary-appcds-api.v01/ >>>>> >>>>> Summary: >>>>> >>>>> Before AppCDS was open sourced, we had a few "Ext" classes in sharedClassUtil.hpp >>>>> that abstracted away the CDS operations related to the non-boot class loaders. >>>>> This API made it possible to build the Oracle JDK with AppCDS included, or build >>>>> the Open JDK with AppCDS removed. >>>>> >>>>> With the open sourcing of AppCDS, this abstraction layer is no longer necessary. I >>>>> have moved the contents of the "Ext" classes into their proper locations and removed >>>>> the sharedClassUtil.hpp/cpp files. >>>>> >>>>> Most of the changes are just moving things around. There shouldn't be behavioral >>>>> changes. >>>>> >>>>> The files classLoaderExt.hpp/cpp still exists -- it encapsulates the classpath >>>>> management code and is not strictly unnecessary. Some clean up may be needed there, but >>>>> I'll probably do that in a separate RFE. >>>>> >>>>> Testing with hotspot tiers1~4. >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>> >>> >> > From david.holmes at oracle.com Fri May 4 22:38:44 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 5 May 2018 08:38:44 +1000 Subject: RFR(M): 8201593: Print array length in ArrayIndexOutOfBoundsException. In-Reply-To: References: <504bdd8469bb45cc8c5cf790d1f4b3fc@sap.com> <4ca91dfb-c576-98e2-1222-ca9750a64177@oracle.com> <5be0f86df2834aa8a881206ba285011e@sap.com> <0b881c69-79af-2a8b-c34c-5929357ca8fa@oracle.com> <5ADF5897.8070809@cjnash.com> <819d942132204f58992c236518e0c344@sap.com> Message-ID: <85837aa4-c053-1a81-d02e-6ecaf98e0ccf@oracle.com> Not my final review just piggybacking on Roger's comments ... On 5/05/2018 7:13 AM, Roger Riggs wrote: > Hi Goetz, > > Just comments on the test/message format.? (found in the 04 version of > the webrev) > > The "." at the end of the exception messages should be removed. > The text is not a sentence (its is just a fragment without a verb) and > it is more flexible to print the exception message in various contexts > (in which the "." would seem to terminate a sentence). +1 > I'm not a big fan of the hyphens in out-of-bounds and it would be more > consistent across the new messages. I also see hyphens used a lot in: java/util/Objects.java But yes consistency within this patch is required. > It would be more consistent if the "arraycopy:" prefix always included > the ":". +1 David > Thanks, Roger > > > On 4/24/18 12:25 PM, Lindenmaier, Goetz wrote: >> Hi Simon, >> >> Because as stated here, >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-April/052665.html >> >> it is used in other places like that, too. >> >> Later mails agreed on that usage to keep it consistent. >> >> Best regards, >> ?? Goetz. >> >>> -----Original Message----- >>> From: Simon Nash [mailto:simon at cjnash.com] >>> Sent: Dienstag, 24. April 2018 18:17 >>> To: Lindenmaier, Goetz >>> Cc: David Holmes ; hotspot-runtime- >>> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; aarch64- >>> port-dev at openjdk.java.net; aarch32-port-dev at openjdk.java.net; core-libs- >>> dev Libs >>> Subject: Re: RFR(M): 8201593: Print array length in >>> ArrayIndexOutOfBoundsException. >>> >>> On 24/04/2018 15:08, Lindenmaier, Goetz wrote: >>>> Hi, >>>> >>>> I implemented what we figured out in >>>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018- >>> April/027555.html >>>> Some examples: >>>> "Index 12 out-of-bounds for length 10." >>>> "arraycopy source index -17 out of bounds for object array[10]." >>>> "arraycopy destination index -18 out of bounds for double[5]." >>>> "arraycopy length -19 is negative." >>>> "arraycopy: last source index 13 out of bounds for double[10]." >>>> "arraycopy: last destination index 7 out of bounds for long[5]." >>>> >>> Is there a reason why the first message says "out-of-bounds" but all >>> others >>> say "out of bounds"? >>> >>> Simon >>> >>>> Incremental webrev: >>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/03- >>> incremental/ >>>> Full webrev: >>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/03/ >>>> >>>> I'll push it through our tests tonight. >>>> >>>> See further comments in line: >>>> >>>>> -----Original Message----- >>>>> From: David Holmes [mailto:david.holmes at oracle.com] >>>>> Sent: Freitag, 20. April 2018 09:25 >>>>> To: Lindenmaier, Goetz ; hotspot-runtime- >>>>> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; >>> aarch64- >>>>> port-dev at openjdk.java.net; aarch32-port-dev at openjdk.java.net; core- >>> libs- >>>>> dev Libs >>>>> Subject: Re: RFR(M): 8201593: Print array length in >>>>> ArrayIndexOutOfBoundsException. >>>>> >>>>> Hi Goetz, >>>>> >>>>> This is not a file by file review ... >>>>> >>>>> On 19/04/2018 10:24 PM, Lindenmaier, Goetz wrote: >>>>>> Hi, >>>>>> >>>>>> New webrev: >>>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/02/ >>>>>> >>>>>> I admit my wording is not optimal. >>>>> src/hotspot/share/oops/typeArrayKlass.cpp >>>>> >>>>> Sorry but this is still far too verbose for my tastes. The type of the >>>>> array is not relevant. For the array copy its okay to indicate src or >>>>> dst array. But the message should be clear and succinct not >>>>> prose-like. >>>>> Plus you have a lot of repetition in the ss.print statements when only >>>>> one thing is changing. >>>> We discussed this in some further mails. Implemented as proposed there. >>>> >>>>> src/hotspot/cpu/x86/c1_CodeStubs_x86.cpp >>>>> >>>>> I'm not seeing why the throw_index_out_of_bounds_exception should >>> be >>>>> tied to whether or not you have an array reference. Certainly I hate >>>>> seeing the array ref being used as an implicit bool! >>>> I split the constructor into two, one for ArrayIndexOutOfBounds, the >>>> other >>>> for IndexOutOfBounds. >>>> >>>> ... >>>> >>>>>> I extended the test to cover the exception thrown in arraycopy better >>>>> The test seems somewhat excessive, and I now see it is because of the >>>>> more elaborate error messages you have at SAP. But with only the index >>>>> and the array length of interest here the test can be considerably >>>>> smaller. >>>>> >>>>> The creation tests for ArrayIndexOutOfBoundsException don't seem >>>>> relevant in this context either. This looks more TCK like. >>>> Yes, the constructor tests are for the code not yet contributed. >>>> I simplified the tests to only check the messages. >>>> >>>> Best regards, >>>> ?? Goetz. >>>> >>>> >>>> >>>> >>>> >>>>> David >>>>> ----- >>>>> >>>>>> and added the elementary type to the message text.? This probably >>>>>> needs improvement in the text, too.? There are (currently) these >>>>>> cases: >>>>>> >>>>>> bject[]? oa1 = new Object[10]; >>>>>> Object[]? oa2 = new Object[5]; >>>>>> System.arraycopy(oa1, -17, oa2, 0, 5); >>>>>> "while trying to copy from index -17 of an object array with >>>>>> length 10"); >>>>>> System.arraycopy(oa1, 2, oa2, -18, 5); >>>>>> "while trying to copy to index -18 of an object array with length >>>>>> 5"); >>>>>> System.arraycopy(oa1, 2, oa2, 0, -19); >>>>>> "while trying to copy a negative range -19 from an object array with >>> length >>>>> 10 to an object array with length 5"); >>>>>> System.arraycopy(oa1, 8, oa2, 0, 5); >>>>>> "while trying to copy from index 13 of an object array with length >>>>>> 10"); >>>>>> System.arraycopy(oa1, 1, oa2, 0, 7); >>>>>> "while trying to copy to index 7 of an object array with length 5"); >>>>>> double[]? ta1 = new double[10]; >>>>>> double[]? ta2 = new double[5]; >>>>>> System.arraycopy(ta1, -17, ta2, 0, 5); >>>>>> "while trying to copy from index -17 of a doubl array with length >>>>>> 10"); >>>>>> System.arraycopy(ta1, 2, ta2, -18, 5); >>>>>> "while trying to copy to index -18 of a double array with length 5"); >>>>>> System.arraycopy(ta1, 2, ta2, 0, -19); >>>>>> "while trying to copy a negative range -19 from a double array with >>> length >>>>> 10 to a double array with length 5"); >>>>>> System.arraycopy(ta1, 8, ta2, 0, 5); >>>>>> "while trying to copy from index 13 of a double array with length >>>>>> 10"); >>>>>> System.arraycopy(ta1, 1, ta2, 0, 7); >>>>>> "while trying to copy to index 7 of a double array with length 5"); >>>>>> >>>>>> Maybe it should say: >>>>>> Arraycopy source index -1 out-of-bounds for double array of length >>>>>> 10. >>>>>> Arraycopy target index 10 out-of-bounds for object array of length >>>>>> 10. >>>>>> Negative range -19 when copying from an object array of length 10 >>>>>> to an >>>>> object array of length 5. >>>>>> Best regards, >>>>>> ??? Goetz. >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: David Holmes [mailto:david.holmes at oracle.com] >>>>>>> Sent: Mittwoch, 18. April 2018 10:55 >>>>>>> To: Lindenmaier, Goetz ; hotspot- >>> runtime- >>>>>>> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; >>>>> aarch64- >>>>>>> port-dev at openjdk.java.net; aarch32-port-dev at openjdk.java.net; >>> core- >>>>> libs- >>>>>>> dev Libs >>>>>>> Subject: Re: RFR(M): 8201593: Print array length in >>>>>>> ArrayIndexOutOfBoundsException. >>>>>>> >>>>>>> Adding core-libs-dev as you're changing >>>>>>> java.lang.ArrayIndexOutOfBoundsException. >>>>>>> >>>>>>> I appreciate the intent here but I find the messages excessively >>>>>>> verbose. The basic error is: >>>>>>> >>>>>>> index N is outside range [0, length-1] >>>>>>> >>>>>>> David >>>>>>> >>>>>>> On 18/04/2018 6:09 PM, Lindenmaier, Goetz wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I would like to print a more verbose text on ArrayIndexOutOfBounds >>>>>>> exception >>>>>>>> that not only mentions the index, but also the length of the array >>>>> accessed. >>>>>>>> See the bug for documentation of the change of the message. >>>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/01/ >>>>>>>> >>>>>>>> @aarch/arm people: >>>>>>>> I edited the aarch/arm files. Could you please verify this is >>>>>>>> correct? >>>>>>>> I can not build on these platforms. >>>>>>>> >>>>>>>> The code on all the other platforms is tested with all the jtreg >>>>>>>> and jck >>>>> tests >>>>>>> etc. >>>>>>>> Best regards, >>>>>>>> ???? Goetz. >>>>>>>> >>>>>>>> > From calvin.cheung at oracle.com Sat May 5 01:47:45 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Fri, 04 May 2018 18:47:45 -0700 Subject: RFR 8197954 Remove unnecessary intermediary APIs from AppCDS implementation In-Reply-To: <3e3c5229-cea3-7dd6-dcb1-c0d58a5fa6e0@oracle.com> References: <508E3B72-3D09-4971-AFBD-E979E3ADA646@oracle.com> <3e3c5229-cea3-7dd6-dcb1-c0d58a5fa6e0@oracle.com> Message-ID: <5AED0D41.7090404@oracle.com> On 5/4/18, 2:09 PM, Ioi Lam wrote: > On 5/4/18 12:54 PM, Jiangli Zhou wrote: >> Looks good. My only question is why not using CLASS_PATH, but I think >> probably your concern is naming conflict. Using ?APP_PATH? is ok. >> > One reason is we have many places that refer to "app" vs "boot" paths, > so if we just say "CLASS_PATH" it is not consist with the rest of the > AppCDS code. > > By the way, the CLASSPATH has traditionally been called many different > names -- the classpath, the app classpath, the system classpath. Maybe > we should clean all that up in a separate RFE. > > Also, I found out that SharedPathsMiscInfo::MODULE_PATH is not used > anywhere, so I will delete it. Do you mean the MODULE_PATH in the enum? I saw it's being referenced in SharedPathsMiscInfo::print_path(). Looks good otherwise. thanks, Calvin > > Thanks > - Ioi > >> Thanks, >> >> Jiangli >> >>> On May 4, 2018, at 11:05 AM, Ioi Lam >> > wrote: >>> >>> Hi Jiangli, >>> >>> Thank you so much for the review. I've made changes according to >>> your and Calvin's suggestions: >>> >>> http://cr.openjdk.java.net/~iklam/jdk11/8197954-remove-unnecessary-appcds-api.v02.diff/ >>> >>> >>> >>> On 5/3/18 12:43 PM, Jiangli Zhou wrote: >>>> Hi Ioi, >>>> >>>> Thank you for removing the _ext files and merging the code >>>> together. It?s very helpful. >>>> >>>> - sharedPathsMiscInfo.cpp >>>> >>>> 191 // FIXME: why is there no MODULE check? >>>> >>>> The path check for archived module classes is handled differently >>>> than the classes from the -cp or -Xbootclasspath/a because we can >>>> quickly determine if an archived module class? origin matches with >>>> the runtime module source path. That?s why we don?t do string >>>> comparison of the dump time module path and the runtime module path >>>> in SharedPathsMiscInfo::check(), which is too restrictive and >>>> limited. Could you please replace the FIXME comment? >>>> >>> Fixed. I replaced the FIXME with your comments. >>>> - sharedPathsMiscInfo.hpp >>>> >>>> 131 const char* type_name(int type) { >>>> 132 switch (type) { >>>> 133 case BOOT: return "BOOT"; >>>> 134 case NON_EXIST: return "NON_EXIST"; >>>> 135 case APP: return "APP"; >>>> 136 case MODULE: return "MODULE"; >>>> >>>> The types are not well-defined. That?s not introduced by your >>>> change. The ?APP? type became inaccurate when we added the ?MODULE? >>>> type. It's noticeable now when types are merged together. To avoid >>>> overlapping, ?APP' can be changed to ?CLASS_PATH? or ?CP?. How >>>> about renaming the types to following: >>>> >>>> BOOT_PATH >>>> CLASS_PATH >>>> MODULE_PATH >>>> NON_EXIST >>>> >>>> >>>> As your current change does not involve larger scale restructuring, >>>> it?s okay if we follow up with a separate RFE for the type renaming. >>>> >>> I changed the names it as you suggested. >>> >>>> - systemDictionaryShared.cpp >>>> >>>> 490 // [c] At this point, if the named class was loaded by the >>>> 491 // AppClassLoader during archive dump time, we know that it >>>> must be >>>> 492 // loaded by the AppClassLoader during run time, and will not >>>> be loaded >>>> 493 // by a delegated class loader. >>>> The above only applies to the application classes loaded from -cp, >>>> but not modules. An archived module class may not be used at >>>> runtime if the module has a different source path at runtime. Those >>>> archived module classes are filtered out by the runtime shared >>>> class ?visibility? check. >>>> >>>> 493 // by a delegated class loader. This is true because we have >>>> checked the >>>> 494 // CLASSPATH and module path to ensure compatibility between >>>> dump time and >>>> 495 // run time. >>>> >>>> Please remove ?module path? from above comment. We don?t do string >>>> comparison of the runtime module path and dump time module path as >>>> CLASSPATH. >>>> >>>> The comment [C] probably should be split into two part, one for >>>> application classes from -cp and the other part for application >>>> module classes. >>>> >>>> 508 // An alternative is to modify the Java code of >>>> BuiltinClassLoader.loadClassOrNull(). >>>> Should above be removed? Otherwise, more details are needed. >>>> >>> I've updated the comments. Because there's already comments around >>> is_shared_class_visible_for_classloader(), I just put a reference to >>> that, so we can avoid duplication. What do you think? >>> >>> Thanks >>> - Ioi >>> >>>> Thanks, >>>> Jiangli >>>> >>>>> On May 1, 2018, at 10:17 PM, Ioi Lam >>>> > wrote: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8197954 >>>>> http://cr.openjdk.java.net/~iklam/jdk11/8197954-remove-unnecessary-appcds-api.v01/ >>>>> >>>>> >>>>> Summary: >>>>> >>>>> Before AppCDS was open sourced, we had a few "Ext" classes in >>>>> sharedClassUtil.hpp >>>>> that abstracted away the CDS operations related to the non-boot >>>>> class loaders. >>>>> This API made it possible to build the Oracle JDK with AppCDS >>>>> included, or build >>>>> the Open JDK with AppCDS removed. >>>>> >>>>> With the open sourcing of AppCDS, this abstraction layer is no >>>>> longer necessary. I >>>>> have moved the contents of the "Ext" classes into their proper >>>>> locations and removed >>>>> the sharedClassUtil.hpp/cpp files. >>>>> >>>>> Most of the changes are just moving things around. There shouldn't >>>>> be behavioral >>>>> changes. >>>>> >>>>> The files classLoaderExt.hpp/cpp still exists -- it encapsulates >>>>> the classpath >>>>> management code and is not strictly unnecessary. Some clean up may >>>>> be needed there, but >>>>> I'll probably do that in a separate RFE. >>>>> >>>>> Testing with hotspot tiers1~4. >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>> >>> >> > From ioi.lam at oracle.com Sat May 5 03:22:08 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 4 May 2018 20:22:08 -0700 Subject: RFR 8197954 Remove unnecessary intermediary APIs from AppCDS implementation In-Reply-To: <5AED0D41.7090404@oracle.com> References: <508E3B72-3D09-4971-AFBD-E979E3ADA646@oracle.com> <3e3c5229-cea3-7dd6-dcb1-c0d58a5fa6e0@oracle.com> <5AED0D41.7090404@oracle.com> Message-ID: <22b9dd2c-7724-f76e-77bb-f6160922947c@oracle.com> On 5/4/18 6:47 PM, Calvin Cheung wrote: > > > On 5/4/18, 2:09 PM, Ioi Lam wrote: >> On 5/4/18 12:54 PM, Jiangli Zhou wrote: >>> Looks good. My only question is why not using CLASS_PATH, but I >>> think probably your concern is naming conflict. Using ?APP_PATH? is ok. >>> >> One reason is we have many places that refer to "app" vs "boot" >> paths, so if we just say "CLASS_PATH" it is not consist with the rest >> of the AppCDS code. >> >> By the way, the CLASSPATH has traditionally been called many >> different names -- the classpath, the app classpath, the system >> classpath. Maybe we should clean all that up in a separate RFE. >> >> Also, I found out that SharedPathsMiscInfo::MODULE_PATH is not used >> anywhere, so I will delete it. > Do you mean the MODULE_PATH in the enum? > I saw it's being referenced in SharedPathsMiscInfo::print_path(). > Yes, it was used there, but there was no code that would add a MODULE_PATH, so this printing code is actually never used. Thanks - Ioi > Looks good otherwise. > > thanks, > Calvin >> >> Thanks >> - Ioi >> >>> Thanks, >>> >>> Jiangli >>> >>>> On May 4, 2018, at 11:05 AM, Ioi Lam >>> > wrote: >>>> >>>> Hi Jiangli, >>>> >>>> Thank you so much for the review. I've made changes according to >>>> your and Calvin's suggestions: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk11/8197954-remove-unnecessary-appcds-api.v02.diff/ >>>> >>>> >>>> >>>> On 5/3/18 12:43 PM, Jiangli Zhou wrote: >>>>> Hi Ioi, >>>>> >>>>> Thank you for removing the _ext files and merging the code >>>>> together. It?s very helpful. >>>>> >>>>> - sharedPathsMiscInfo.cpp >>>>> >>>>> 191 // FIXME: why is there no MODULE check? >>>>> >>>>> The path check for archived module classes is handled differently >>>>> than the classes from the -cp or -Xbootclasspath/a because we can >>>>> quickly determine if an archived module class? origin matches with >>>>> the runtime module source path. That?s why we don?t? do string >>>>> comparison of the dump time module path and the runtime module >>>>> path in SharedPathsMiscInfo::check(), which is too restrictive and >>>>> limited. Could you please replace the FIXME comment? >>>>> >>>> Fixed. I replaced the FIXME with your comments. >>>>> - sharedPathsMiscInfo.hpp >>>>> >>>>> 131 const char* type_name(int type) { >>>>> ? 132???? switch (type) { >>>>> ? 133???? case BOOT:????? return "BOOT"; >>>>> ? 134???? case NON_EXIST: return "NON_EXIST"; >>>>> 135 case APP: return "APP"; >>>>> 136 case MODULE: return "MODULE"; >>>>> >>>>> The types are not well-defined. That?s not introduced by your >>>>> change. The ?APP? type became inaccurate when we added the >>>>> ?MODULE? type. It's noticeable now when types are merged together. >>>>> To avoid overlapping, ?APP' can be changed to ?CLASS_PATH? or >>>>> ?CP?. How about renaming the types to following: >>>>> >>>>> ??? BOOT_PATH >>>>> ??? CLASS_PATH >>>>> ??? MODULE_PATH >>>>> ??? NON_EXIST >>>>> >>>>> >>>>> As your current change does not involve larger scale >>>>> restructuring, it?s okay if we follow up with a separate RFE for >>>>> the type renaming. >>>>> >>>> I changed the names it as you suggested. >>>> >>>>> - systemDictionaryShared.cpp >>>>> >>>>> 490 // [c] At this point, if the named class was loaded by the >>>>> 491 // AppClassLoader during archive dump time, we know that it >>>>> must be >>>>> 492 // loaded by the AppClassLoader during run time, and will not >>>>> be loaded >>>>> 493 // by a delegated class loader. >>>>> The above only applies to the application classes loaded from -cp, >>>>> but not modules. An archived module class may not be used at >>>>> runtime if the module has a different source path at runtime. >>>>> Those archived module classes are filtered out by the runtime >>>>> shared class ?visibility? check. >>>>> >>>>> 493 // by a delegated class loader. This is true because we have >>>>> checked the >>>>> 494 // CLASSPATH and module path to ensure compatibility between >>>>> dump time and >>>>> 495 // run time. >>>>> >>>>> Please remove ?module path? from above comment. We don?t do string >>>>> comparison of the runtime module path and dump time module path as >>>>> CLASSPATH. >>>>> >>>>> The comment [C] probably should be split into two part, one for >>>>> application classes from -cp and the other part for application >>>>> module classes. >>>>> >>>>> 508 // An alternative is to modify the Java code of >>>>> BuiltinClassLoader.loadClassOrNull(). >>>>> Should above be removed? Otherwise, more details are needed. >>>>> >>>> I've updated the comments. Because there's already comments around >>>> is_shared_class_visible_for_classloader(), I just put a reference >>>> to that, so we can avoid duplication. What do you think? >>>> >>>> Thanks >>>> - Ioi >>>> >>>>> Thanks, >>>>> Jiangli >>>>> >>>>>> On May 1, 2018, at 10:17 PM, Ioi Lam >>>>> > wrote: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8197954 >>>>>> http://cr.openjdk.java.net/~iklam/jdk11/8197954-remove-unnecessary-appcds-api.v01/ >>>>>> >>>>>> >>>>>> Summary: >>>>>> >>>>>> Before AppCDS was open sourced, we had a few "Ext" classes in >>>>>> sharedClassUtil.hpp >>>>>> that abstracted away the CDS operations related to the non-boot >>>>>> class loaders. >>>>>> This API made it possible to build the Oracle JDK with AppCDS >>>>>> included, or build >>>>>> the Open JDK with AppCDS removed. >>>>>> >>>>>> With the open sourcing of AppCDS, this abstraction layer is no >>>>>> longer necessary. I >>>>>> have moved the contents of the "Ext" classes into their proper >>>>>> locations and removed >>>>>> the sharedClassUtil.hpp/cpp files. >>>>>> >>>>>> Most of the changes are just moving things around. There >>>>>> shouldn't be behavioral >>>>>> changes. >>>>>> >>>>>> The files classLoaderExt.hpp/cpp still exists -- it encapsulates >>>>>> the classpath >>>>>> management code and is not strictly unnecessary. Some clean up >>>>>> may be needed there, but >>>>>> I'll probably do that in a separate RFE. >>>>>> >>>>>> Testing with hotspot tiers1~4. >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>>> >>>> >>> >> From thomas.stuefe at gmail.com Sat May 5 19:58:36 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 5 May 2018 21:58:36 +0200 Subject: RFR(xxs): 8185034: Cleanup and consolidate Metaspace coding Message-ID: Hi all, May I get reviews please? Bug: https://bugs.openjdk.java.net/browse/JDK-8185034 Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8185034-cleanup-and-consolidate-metaspace-coding/webrev.00/webrev/ Despite the title this item is quite unexciting. Most of the cleanups originally targeted by this item did already happen as part of other enhancements (https://bugs.openjdk.java.net/browse/JDK-8199430, https://bugs.openjdk.java.net/browse/JDK-8202638). What is left are some smallish cleanups: - removed some functions which were unused - small changes to ~SpaceManager to avoid logging the same info twice (chunk manager state) - removed all LogStream::cr() calls which were needed because ~LogStream() did not autoflush; but 8202303 did fix that, so the cr() can go. Note that there are more things one could clean up, which may happen in the future. Especially the split-up of metaspace.cpp, which is still open (https://bugs.openjdk.java.net/browse/JDK-8176808). But for now I would like to close this particular issue. Thanks! ..Thomas From volker.simonis at gmail.com Sat May 5 20:22:44 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Sat, 05 May 2018 20:22:44 +0000 Subject: Improving AppCDS for Custom Loaders In-Reply-To: References: <8e0d891a-c00c-8f57-362a-6a4684a6cc21@oracle.com> <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> <732ce957-b07c-bff2-277b-ca335c348753@oracle.com> <660A1644-3A08-455D-931D-D95069C39853@oracle.com> Message-ID: Hi Jiangli, thanks a lot explaining your plans more precisely. I think the approach makes much more sense now. While I?m all for it in general, I still have some questions and remarks :) 1. Will it be possible to use the two archives independantly ? I.e. will it be possible to choose just one of the two archives as well as a combination of both? 2. We should make sure to not introduce any performance regressions if we have two archives (and potentially three SymbolTables and three ?Metaspaces?) 3. In general I think we?re moving away from scenarios where we have one single, central Java installation on a system which is used by the ?tens/hundreds different applications? you mentioned in your mail towards a world where every of these ?tens/hundreds different applications? comes with its own, bundled JDK/JRE version (and as far as I understand Oracle is propagating this new distribution model after the deprecation of Applets/WebStart). In such a world it would be not easy to have a static, common system layer archive. We could try to keep the format of the archived classes as compatible between Java versions as possible, although I understand that this will be quite complicated. Do you already have any thoughts on this specific problem? 4. Finally we should design the new model in a way which allows it to easily integrate AOT compiled code into the same one or two archives in the future. Because with AOT we face the exactly same problems and having to maintain four different archives will be not very user friendly. OpenJ9 already uses a single archive for CDT/AOT/ProfilingData. I don?t say we have to unify the AOT and CDT archives in HotSpot now, I just want to suggest that if we?re redesigning the CDS archive anyway to perhaps think twice before we come to a new model such that we at least don?t exclude the possibility of a unification with the AOT archive in the future. Thanks, Volker Jiangli Zhou schrieb am Fr. 4. Mai 2018 um 19:42: > Hi Volker, > > Thank you so much for the feedbacks! > > You comments made me realize that some clarifications are need for the > top-layer dynamic archiving. The top-layer would include all application > classes, middleware classes, system library classes that are not in the > bottom layer. Apologizing for not stating that clearly in the proposal. > With the top layer including all necessary classes (excluding the classes > in bottom layer) for a specific application, multiple instances running the > same application can achieve maximum memory sharing. Using your example, > the extra AWT/Swing classes are archived along with the application classes > in the top layer. > > With the dynamic archiving ability, it?s natural to think there is no need > for the two-layer scheme. In fact, I originally had the thoughts. However, > after exploring different possible use cases, I agree two-layer archiving > do bring benefits that are lacking in the single archive design. One > example of such use case is tens/hundreds different applications running on > the same machine. Without a common system layer archive, it is difficult to > achieve memory sharing for common system classes, String objects, etc > between different applications. > > Another reason why we should go with a two-layer & static/dynamic > archiving is some runtime information may be sensitive and should not be > archived. Strings can be a good example in this case. Currently, we archive > interned String objects without running application. The Strings include > resolved CONSTANT_strings, class names, method names, etc, which are all > static information from class files. During execution of an application, > interned Strings might contain sensitive user information, which is not > safe to be included in the archive. It?s difficult to distinguish between > the static information and runtime information if we go with the single > layer dynamic-only archiving. > > We had many discussions internally over long period of time on how to > improve usability with different approaches. In the end, dynamic-only or > static-only two-layer archiving all have their own disadvantages and fail > to meet requirements in some use cases. The hybrid archiving combines > benefits/advantages of different approaches and seem to be flexible enough > to fit most usages. > > Further comments and feedbacks are much appreciated. > > Best regards, > > Jiangli > > > On May 4, 2018, at 3:01 AM, Volker Simonis > wrote: > > > > Hi Jiangli, > > > > thanks for sharing the hybrid archiving proposal. I think it is very > > interesting and useful! > > > > One issue I see is that the "system library classes" which should go > > into the "static" archive are inherently application specific (i.e. if > > my application uses Swing, it will use most of the AWT/Swing classes). > > So how will you cope with this problem. Either you put ALL the system > > classes into the static archive which will be a waste for most > > applications. Or you just put a small commonly used subset of classes > > into the static archive which may miss many classes for specific > > applications. > > > > If we would add the possibility to create a dynamic archive at runtime > > / program end (which I think would be great from a usability > > perspective) I don't see a big need for two different archives. Two > > archives will further complicate (and slow down) things like Symbol > > lookup (we already have two SymbolTable now and we'd probably need a > > third one if we would have two archives). > > > > I don't think that running a few different Java applications on one > > host is the most important use case for CDS. In such a scenario the > > current, build time generated archive is probably the best we can do > > anyway. Instead, I think the most important use case is if we have > > many instances of the same Java application running on the same host. > > And this is becoming more common with the raise of containerization. > > For the latter use case, a dynamically generated, application specific > > archive would be the optimal solution. > > > > Thank you and best regards, > > Volker > > > > > > > > On Fri, May 4, 2018 at 3:42 AM, Jiangli Zhou > wrote: > >> Hi Volker, > >> > >> Here are some details about the hybrid archiving. The goal is to > harvest the benefit of archiving by default and improve its usability. The > hybrid approach combines a two-layer archiving (proposed by Ioi internally) > and static & dynamic archiving techniques: > >> > >> - Statically archive system library classes from a provided classlist > using the existing method. The archiving includes class metadata, interned > string objects, constant pool resolved_references arrays, class mirror > objects, etc. Static archiving can be done at the JDK image build time and > shipped together with JDK binary. Following need to be addressed: > >> *Relaxing the runtime CDS/AppCDS boot path check, so the > packaged archive can be used after the JDK binary is installed on the > target device. JDK-8199807 was created to address this issue and is > targeted for JDK 11. > >> *Add the static archiving generation in JDK build steps and > package the generated archive with JDK image. The archive can only be > generated for the same target (both OS can CPU architecture) as the build > platform. I will create a RFE. > >> > >> - Dynamic archiving can done for application classes at the first > execution of a specific application > >> * The archive is created on top of the default system archive > shipped with the JDK image. A separate top-layer archive file is generated > for each different application. > >> * Archiving is done at the end of the application execution > before VM exists by relocating the class metadata to the archive spaces. > Cleanup also needs to be done for copied class meta data to remove any > runtime information. Most of the required functionality already exists > today. For example, class metadata relocation was implemented by Ioi in JDK > 10. > >> * Only archive class metadata for application in the top layer > initially. Archiving java heap objects in the top-layer requires more > investigations. > >> > >> Benefits of the hybrid archiving: > >> * The system archive can be shared by different applications and > provides memory saving. > >> * Archive for application is created and used transparently. No more > profiling step and class list are required! > >> * Separating the system archiving from application archiving reduces > the cost of archiving at application execution time. The overhead added to > the first execution time is reduced. > >> > >> Thanks, > >> > >> Jiangli > >> > >> > >>> On May 3, 2018, at 10:34 AM, Jiangli Zhou > wrote: > >>> > >>> > >>>> On May 3, 2018, at 2:14 AM, Volker Simonis > wrote: > >>>> > >>>> On Thu, May 3, 2018 at 11:01 AM, David Holmes < > david.holmes at oracle.com> wrote: > >>>>> On 3/05/2018 5:16 PM, Volker Simonis wrote: > >>>>>> > >>>>>> On Thu, May 3, 2018 at 8:55 AM, David Holmes < > david.holmes at oracle.com> > >>>>>> wrote: > >>>>>>> > >>>>>>> Just lurking here but ... > >>>>>>> > >>>>>>>> But is this really y relevant use case? Why would I like to > create ONE > >>>>>>>> archive for several apps? This would actually increase the > footprint > >>>>>>>> of a single instance which uses this archive. If I have several > apps I > >>>>>>>> would expect that users create a specific archive for each app to > get > >>>>>>>> the best out of CDS. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> One app instance may get increased footprint but you presumably > use CDS > >>>>>>> because you have multiple apps running (whether the same or not). > These > >>>>>>> apps > >>>>>>> all share the core JDK classes from the archive so the overall > footprint > >>>>>>> per > >>>>>>> instance is less. > >>>>>>> > >>>>>> > >>>>>> If we just want to share the core JDK classes that's easy. For that > we > >>>>>> could mostly use the default class list (or a slightly extended one) > >>>>>> which is generated at JDK build time (at JAVA_HOME/lib/classlist). > >>>>> > >>>>> > >>>>> The point is that you are presumably running multiple instances of > multiple > >>>>> apps, hence you want to share one set of core classes across all, > and share > >>>>> the app classes across each app instance. > >>>>> > >>>> > >>>> But that would require two archives: a general one with the core > >>>> classes and an application specific one for each application. > >>>> Combining the core classes and the application of various applications > >>>> will not be optimal because the application classes will be all mixed > >>>> in the same archive. The archive is being mapped page-wise into the > >>>> java process so you'll probably end up mapping the whole archive into > >>>> each process although you'll only use a fraction of the classes in the > >>>> archive. > >>>> > >>>>>> If we want to use ONE archive for several applications and we can > >>>>>> accept to have a bigger footprint if running a single (or just a > few) > >>>>>> applications in parallel I suppose the overhead of simply dumping > all > >>>>>> the classes from the classpathes of the various applications > compared > >>>>>> to an accurate solution where we only dump the actually used classes > >>>>>> of all applications would be not that big. > >>>>> > >>>>> > >>>>> But those "accurate" solutions duplicate the core classes and that's > a waste > >>>>> of footprint. > >>>>> > >>>> > >>>> By "accurate" I meant one "fat" archive which contains all the classes > >>>> USED by several applications plus the core classes. My argument was > >>>> that such an "accurate" "fat" archive won't be much smaller compared > >>>> to a "fat" archive which simply contains all the core classes plus all > >>>> the application classes (i.e. from the application class pathes, no > >>>> matter if they are ever used or not). But the latter would be much > >>>> simpler to implement. > >>> > >>> The above discussion and an internal proposal for hybrid archiving > seem to converge on a few points. If there is no objection to the hybrid > archiving proposal internally, maybe we can shared the details of the > proposal on openjdk soon. > >>> > >>> Thanks, > >>> > >>> Jiangli > >>> > >>> > >>>> > >>>>> David > >>>>> ----- > >>>>> > >>>>> > >>>>>>> David > >>>>>>> ----- > >>>>>>> > >>>>>>> > >>>>>>> On 3/05/2018 4:48 PM, Volker Simonis wrote: > >>>>>>>> > >>>>>>>> > >>>>>>>> On Thu, May 3, 2018 at 6:52 AM, Ioi Lam > wrote: > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On 5/2/18 10:00 AM, Volker Simonis wrote: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On Tue, May 1, 2018 at 8:32 PM, Ioi Lam > wrote: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> PROBLEM: > >>>>>>>>>>> > >>>>>>>>>>> As discussed with Volker and Yumin in previous e-mails, AppCDS > has > >>>>>>>>>>> some > >>>>>>>>>>> experimental support for custom class loaders. However, it's > not very > >>>>>>>>>>> easy > >>>>>>>>>>> to use. > >>>>>>>>>>> > >>>>>>>>>>> For example, you can write a classlist like this: > >>>>>>>>>>> > >>>>>>>>>>> java/lang/Object id: 1 > >>>>>>>>>>> CustomLoadee id: 2 super: 1 source: /tmp/foo.jar > >>>>>>>>>>> > >>>>>>>>>>> The CustomLoadee class will be stored in the shared archive > with a > >>>>>>>>>>> CRC > >>>>>>>>>>> code. > >>>>>>>>>>> During runtime, if a customed loader wants to load a class of > the > >>>>>>>>>>> same > >>>>>>>>>>> name, > >>>>>>>>>>> and its classfile has the same size and CRC as the archived > class, > >>>>>>>>>>> the > >>>>>>>>>>> archived version will be loaded. This speeds up class loading > by > >>>>>>>>>>> avoiding > >>>>>>>>>>> parsing the class file, and saves space by sharing the mmap'ed > class > >>>>>>>>>>> metadata across processes. > >>>>>>>>>>> > >>>>>>>>>>> You can see an example test at: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java > >>>>>>>>>>> > >>>>>>>>>>> However, the current scheme requires you to specify all the > super > >>>>>>>>>>> classes > >>>>>>>>>>> and interfaces. There's no support provided by the > >>>>>>>>>>> -XX:DumpLoadedClassList > >>>>>>>>>>> option. It can be helped somewhat with Volker's tool: > >>>>>>>>>>> https://github.com/simonis/cl4cds > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> POSSIBLE SOLUTIONS: > >>>>>>>>>>> > >>>>>>>>>>> 1. "Dump-as-you-go". As suggested by Yumin, we can provide a > jcmd to > >>>>>>>>>>> ask > >>>>>>>>>>> a > >>>>>>>>>>> running JVM process to dump all of its loaded classes, > including > >>>>>>>>>>> those > >>>>>>>>>>> loaded by custom loaders, into an archive. An alternative is > to dump > >>>>>>>>>>> the > >>>>>>>>>>> archive at JVM exit time (or when you press Ctrl-C, etc. > >>>>>>>>>>> > >>>>>>>>>>> 2. Add information about the custom classes for > >>>>>>>>>>> -XX:DumpLoadedClassList. > >>>>>>>>>>> The > >>>>>>>>>>> trouble is some class loaders don't specify a code source that > can be > >>>>>>>>>>> understood by the built-in class loaders. For example, the > "Fat Jars" > >>>>>>>>>>> would > >>>>>>>>>>> have a code source like > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/ > >>>>>>>>>>> > >>>>>>>>>>> also, many custom loaders would pre-process the classfile data > before > >>>>>>>>>>> defining the class, so we can't simply archive the version of > the > >>>>>>>>>>> class > >>>>>>>>>>> on > >>>>>>>>>>> disk. > >>>>>>>>>>> > >>>>>>>>>>> One possible solution for #2 is to include the class file data > in the > >>>>>>>>>>> -XX:DumpLoadedClassList output: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> java/lang/Object id: 1 > >>>>>>>>>>> CustomLoadee id: 2 super: 1 source: base64 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA > >>>>>>>>>>> AgAPAAcAAAAKAAEABQADAAYACA== > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Of the 2 solutions: > >>>>>>>>>>> > >>>>>>>>>>> #1 seems easier to use, but may require more invasive > modifications > >>>>>>>>>>> in > >>>>>>>>>>> the > >>>>>>>>>>> VM, especially if you want to be able to continue execution > after > >>>>>>>>>>> dumping. > >>>>>>>>>>> > >>>>>>>>>> Not sure what #1 really proposes: dumping the complete .jsa > archive at > >>>>>>>>>> runtime or dumping just the loaded classes. > >>>>>>>>>> > >>>>>>>>>> If it's just about dumping the loaded class without generating > the > >>>>>>>>>> .jsa archive there's the problem that by default the VM doesn't > store > >>>>>>>>>> the exact bytes of a class after the class was loaded (except > when > >>>>>>>>>> class transformers are registered). So the class files would > have to > >>>>>>>>>> be re-assembled from the internal VM structures (in the same > way this > >>>>>>>>>> is done for class redefinition) and the resulting class-file > may be > >>>>>>>>>> different from the original bytes (i.e. some attributes may be > >>>>>>>>>> missing). > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> #1 is for creating the JSA file, not just dumping the class > files. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> If #1 is about creating the whole .jsa archive at runtime (or > at VM > >>>>>>>>>> exit) I think that would be the most attractive solution from a > >>>>>>>>>> usability point of view although I understand that #2 will be > easier > >>>>>>>>>> to implement in the short term. Regarding the argument that #1 > will > >>>>>>>>>> produce a "binary blob" that's true, but that's already true > now when > >>>>>>>>>> we use "Xshare:dump". I think it should be not to hard to > implement a > >>>>>>>>>> tool based an SA which could introspect a .jsa archive. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> The argument about the binary blob is to compare it against the > text > >>>>>>>>> file > >>>>>>>>> produced by -XX:DumpLoadedClassList. > >>>>>>>>> > >>>>>>>>> One use case to consider is when you have a JAR file that > contains > >>>>>>>>> several > >>>>>>>>> apps that each load a unique set of classes. Today, (assuming > that > >>>>>>>>> custom > >>>>>>>>> class loaders are not used), you can run each app once with > >>>>>>>>> -XX:DumpLoadedClassList, and then do an > >>>>>>>>> > >>>>>>>>> cat *.classlist | sort | uniq > combined.classlist > >>>>>>>>> > >>>>>>>>> and then create an archive that would work for all these apps. > >>>>>>>>> > >>>>>>>> > >>>>>>>> But is this really y relevant use case? Why would I like to > create ONE > >>>>>>>> archive for several apps? This would actually increase the > footprint > >>>>>>>> of a single instance which uses this archive. If I have several > apps I > >>>>>>>> would expect that users create a specific archive for each app to > get > >>>>>>>> the best out of CDS. > >>>>>>>> > >>>>>>>>> With the binary blob, there's no easy way of doing this. It will > be > >>>>>>>>> very > >>>>>>>>> difficult to write a tool to decipher each blob and then somehow > >>>>>>>>> combine > >>>>>>>>> them into a single one. > >>>>>>>>> > >>>>>>>> > >>>>>>>> But if users really wants such a "fat" archive, there's a much > easier > >>>>>>>> way: just dump ALL the classes from the .jar file into the > archive. A > >>>>>>>> class list for this could easily be assembled either with an > external > >>>>>>>> tool like cl4cds (or even a simple shell scripts which converts > the > >>>>>>>> output of `unzip -l ` into the correct format). Or, even > >>>>>>>> simpler, by adding a new option to the VM similar to > >>>>>>>> -XX:DumpLoadedClassList which dumps all the classes it can find > on the > >>>>>>>> class path (and potentially other, configurable locations). > >>>>>>>> > >>>>>>>>> Thanks > >>>>>>>>> - Ioi > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>>> #2 would be easier to implement, but the classlist might be > huge. > >>>>>>>>>>> > >>>>>>>>>>> Also, #2 would allow post-processing tools to remove unneeded > >>>>>>>>>>> classes, > >>>>>>>>>>> or > >>>>>>>>>>> merge two runs into a single list. The output of #1 is > essentially a > >>>>>>>>>>> binary > >>>>>>>>>>> blob that's impossible for off-line analysis/optimizations. > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Any comments, or suggestions for alternatives? > >>>>>>>>>>> > >>>>>>>>>>> Thanks > >>>>>>>>>>> - Ioi > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>> > >>> > >> > > From jiangli.zhou at oracle.com Sun May 6 18:46:32 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Sun, 6 May 2018 11:46:32 -0700 Subject: Improving AppCDS for Custom Loaders In-Reply-To: References: <8e0d891a-c00c-8f57-362a-6a4684a6cc21@oracle.com> <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> <732ce957-b07c-bff2-277b-ca335c348753@oracle.com> <660A1644-3A08-455D-931D-D95069C39853@oracle.com> Message-ID: <2E67F760-EC67-4657-9A98-1C5C275F5597@oracle.com> Hi Volker, Thanks for the additional remarks! > On May 5, 2018, at 1:22 PM, Volker Simonis wrote: > > Hi Jiangli, > > thanks a lot explaining your plans more precisely. I think the approach makes much more sense now. > > While I?m all for it in general, I still have some questions and remarks :) > > 1. Will it be possible to use the two archives independantly ? I.e. will it be possible to choose just one of the two archives as well as a combination of both? The system archive (bottom archive) can be used independently, while the top layer archive would have dependency on the bottom layer. In general cases, the top layer archive must be used with the system archive. If the bottom layer does not exist (system archive deleted?) at the time during dynamic archiving, the top layer would contain metadata for all classes loaded during the execution of a specific application (no Java heap object archived). In that case, there is only a single archive for an application. It seems this is a more interesting usage to you. If it is useful, we could consider a command line option to indicate if the bottom layer should be ignored during the dynamic archiving. > > 2. We should make sure to not introduce any performance regressions if we have two archives (and potentially three SymbolTables and three ?Metaspaces?) With dynamic archiving at the end of the application execution, I imagine most of the symbols (except the temporary ones that are freed) created during the application life time would be captured in the top layer archive. Most symbol lookup probably would only involve one table (the top layer one) and at most two (if the symbol is in the system archive). If performance measurement does show regression, John Rose has a very interesting idea that could be used to reduce overhead. For system dictionary, I think there would be no added overhead. As all loaded classes would be in the system dictionaries within one of the two archives, the runtime system dictionary would contain no entry. > > 3. In general I think we?re moving away from scenarios where we have one single, central Java installation on a system which is used by the ?tens/hundreds different applications? you mentioned in your mail towards a world where every of these ?tens/hundreds different applications? comes with its own, bundled JDK/JRE version (and as far as I understand Oracle is propagating this new distribution model after the deprecation of Applets/WebStart). In such a world it would be not easy to have a static, common system layer archive. We could try to keep the format of the archived classes as compatible between Java versions as possible, although I understand that this will be quite complicated. Do you already have any thoughts on this specific problem? The system archive can be shared across different JVM instances as long as the same JDK version is used. You are absolutely right that there would be no sharing when different JDK versions are used. Yes, implementing a format that?s compatible between different Java versions is quite complicated. We haven?t seriously explored in that direction. > > 4. Finally we should design the new model in a way which allows it to easily integrate AOT compiled code into the same one or two archives in the future. Because with AOT we face the exactly same problems and having to maintain four different archives will be not very user friendly. OpenJ9 already uses a single archive for CDT/AOT/ProfilingData. I don?t say we have to unify the AOT and CDT archives in HotSpot now, I just want to suggest that if we?re redesigning the CDS archive anyway to perhaps think twice before we come to a new model such that we at least don?t exclude the possibility of a unification with the AOT archive in the future. You have a very good point about the tight integration of CDS and AOT. We have regular discussions with our compiler team on this topic. :) Thanks! Jiangli > > Thanks, > Volker > > > > Jiangli Zhou > schrieb am Fr. 4. Mai 2018 um 19:42: > Hi Volker, > > Thank you so much for the feedbacks! > > You comments made me realize that some clarifications are need for the top-layer dynamic archiving. The top-layer would include all application classes, middleware classes, system library classes that are not in the bottom layer. Apologizing for not stating that clearly in the proposal. With the top layer including all necessary classes (excluding the classes in bottom layer) for a specific application, multiple instances running the same application can achieve maximum memory sharing. Using your example, the extra AWT/Swing classes are archived along with the application classes in the top layer. > > With the dynamic archiving ability, it?s natural to think there is no need for the two-layer scheme. In fact, I originally had the thoughts. However, after exploring different possible use cases, I agree two-layer archiving do bring benefits that are lacking in the single archive design. One example of such use case is tens/hundreds different applications running on the same machine. Without a common system layer archive, it is difficult to achieve memory sharing for common system classes, String objects, etc between different applications. > > Another reason why we should go with a two-layer & static/dynamic archiving is some runtime information may be sensitive and should not be archived. Strings can be a good example in this case. Currently, we archive interned String objects without running application. The Strings include resolved CONSTANT_strings, class names, method names, etc, which are all static information from class files. During execution of an application, interned Strings might contain sensitive user information, which is not safe to be included in the archive. It?s difficult to distinguish between the static information and runtime information if we go with the single layer dynamic-only archiving. > > We had many discussions internally over long period of time on how to improve usability with different approaches. In the end, dynamic-only or static-only two-layer archiving all have their own disadvantages and fail to meet requirements in some use cases. The hybrid archiving combines benefits/advantages of different approaches and seem to be flexible enough to fit most usages. > > Further comments and feedbacks are much appreciated. > > Best regards, > > Jiangli > > > On May 4, 2018, at 3:01 AM, Volker Simonis > wrote: > > > > Hi Jiangli, > > > > thanks for sharing the hybrid archiving proposal. I think it is very > > interesting and useful! > > > > One issue I see is that the "system library classes" which should go > > into the "static" archive are inherently application specific (i.e. if > > my application uses Swing, it will use most of the AWT/Swing classes). > > So how will you cope with this problem. Either you put ALL the system > > classes into the static archive which will be a waste for most > > applications. Or you just put a small commonly used subset of classes > > into the static archive which may miss many classes for specific > > applications. > > > > If we would add the possibility to create a dynamic archive at runtime > > / program end (which I think would be great from a usability > > perspective) I don't see a big need for two different archives. Two > > archives will further complicate (and slow down) things like Symbol > > lookup (we already have two SymbolTable now and we'd probably need a > > third one if we would have two archives). > > > > I don't think that running a few different Java applications on one > > host is the most important use case for CDS. In such a scenario the > > current, build time generated archive is probably the best we can do > > anyway. Instead, I think the most important use case is if we have > > many instances of the same Java application running on the same host. > > And this is becoming more common with the raise of containerization. > > For the latter use case, a dynamically generated, application specific > > archive would be the optimal solution. > > > > Thank you and best regards, > > Volker > > > > > > > > On Fri, May 4, 2018 at 3:42 AM, Jiangli Zhou > wrote: > >> Hi Volker, > >> > >> Here are some details about the hybrid archiving. The goal is to harvest the benefit of archiving by default and improve its usability. The hybrid approach combines a two-layer archiving (proposed by Ioi internally) and static & dynamic archiving techniques: > >> > >> - Statically archive system library classes from a provided classlist using the existing method. The archiving includes class metadata, interned string objects, constant pool resolved_references arrays, class mirror objects, etc. Static archiving can be done at the JDK image build time and shipped together with JDK binary. Following need to be addressed: > >> *Relaxing the runtime CDS/AppCDS boot path check, so the packaged archive can be used after the JDK binary is installed on the target device. JDK-8199807 was created to address this issue and is targeted for JDK 11. > >> *Add the static archiving generation in JDK build steps and package the generated archive with JDK image. The archive can only be generated for the same target (both OS can CPU architecture) as the build platform. I will create a RFE. > >> > >> - Dynamic archiving can done for application classes at the first execution of a specific application > >> * The archive is created on top of the default system archive shipped with the JDK image. A separate top-layer archive file is generated for each different application. > >> * Archiving is done at the end of the application execution before VM exists by relocating the class metadata to the archive spaces. Cleanup also needs to be done for copied class meta data to remove any runtime information. Most of the required functionality already exists today. For example, class metadata relocation was implemented by Ioi in JDK 10. > >> * Only archive class metadata for application in the top layer initially. Archiving java heap objects in the top-layer requires more investigations. > >> > >> Benefits of the hybrid archiving: > >> * The system archive can be shared by different applications and provides memory saving. > >> * Archive for application is created and used transparently. No more profiling step and class list are required! > >> * Separating the system archiving from application archiving reduces the cost of archiving at application execution time. The overhead added to the first execution time is reduced. > >> > >> Thanks, > >> > >> Jiangli > >> > >> > >>> On May 3, 2018, at 10:34 AM, Jiangli Zhou wrote: > >>> > >>> > >>>> On May 3, 2018, at 2:14 AM, Volker Simonis > wrote: > >>>> > >>>> On Thu, May 3, 2018 at 11:01 AM, David Holmes > wrote: > >>>>> On 3/05/2018 5:16 PM, Volker Simonis wrote: > >>>>>> > >>>>>> On Thu, May 3, 2018 at 8:55 AM, David Holmes > > >>>>>> wrote: > >>>>>>> > >>>>>>> Just lurking here but ... > >>>>>>> > >>>>>>>> But is this really y relevant use case? Why would I like to create ONE > >>>>>>>> archive for several apps? This would actually increase the footprint > >>>>>>>> of a single instance which uses this archive. If I have several apps I > >>>>>>>> would expect that users create a specific archive for each app to get > >>>>>>>> the best out of CDS. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> One app instance may get increased footprint but you presumably use CDS > >>>>>>> because you have multiple apps running (whether the same or not). These > >>>>>>> apps > >>>>>>> all share the core JDK classes from the archive so the overall footprint > >>>>>>> per > >>>>>>> instance is less. > >>>>>>> > >>>>>> > >>>>>> If we just want to share the core JDK classes that's easy. For that we > >>>>>> could mostly use the default class list (or a slightly extended one) > >>>>>> which is generated at JDK build time (at JAVA_HOME/lib/classlist). > >>>>> > >>>>> > >>>>> The point is that you are presumably running multiple instances of multiple > >>>>> apps, hence you want to share one set of core classes across all, and share > >>>>> the app classes across each app instance. > >>>>> > >>>> > >>>> But that would require two archives: a general one with the core > >>>> classes and an application specific one for each application. > >>>> Combining the core classes and the application of various applications > >>>> will not be optimal because the application classes will be all mixed > >>>> in the same archive. The archive is being mapped page-wise into the > >>>> java process so you'll probably end up mapping the whole archive into > >>>> each process although you'll only use a fraction of the classes in the > >>>> archive. > >>>> > >>>>>> If we want to use ONE archive for several applications and we can > >>>>>> accept to have a bigger footprint if running a single (or just a few) > >>>>>> applications in parallel I suppose the overhead of simply dumping all > >>>>>> the classes from the classpathes of the various applications compared > >>>>>> to an accurate solution where we only dump the actually used classes > >>>>>> of all applications would be not that big. > >>>>> > >>>>> > >>>>> But those "accurate" solutions duplicate the core classes and that's a waste > >>>>> of footprint. > >>>>> > >>>> > >>>> By "accurate" I meant one "fat" archive which contains all the classes > >>>> USED by several applications plus the core classes. My argument was > >>>> that such an "accurate" "fat" archive won't be much smaller compared > >>>> to a "fat" archive which simply contains all the core classes plus all > >>>> the application classes (i.e. from the application class pathes, no > >>>> matter if they are ever used or not). But the latter would be much > >>>> simpler to implement. > >>> > >>> The above discussion and an internal proposal for hybrid archiving seem to converge on a few points. If there is no objection to the hybrid archiving proposal internally, maybe we can shared the details of the proposal on openjdk soon. > >>> > >>> Thanks, > >>> > >>> Jiangli > >>> > >>> > >>>> > >>>>> David > >>>>> ----- > >>>>> > >>>>> > >>>>>>> David > >>>>>>> ----- > >>>>>>> > >>>>>>> > >>>>>>> On 3/05/2018 4:48 PM, Volker Simonis wrote: > >>>>>>>> > >>>>>>>> > >>>>>>>> On Thu, May 3, 2018 at 6:52 AM, Ioi Lam > wrote: > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On 5/2/18 10:00 AM, Volker Simonis wrote: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On Tue, May 1, 2018 at 8:32 PM, Ioi Lam > wrote: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> PROBLEM: > >>>>>>>>>>> > >>>>>>>>>>> As discussed with Volker and Yumin in previous e-mails, AppCDS has > >>>>>>>>>>> some > >>>>>>>>>>> experimental support for custom class loaders. However, it's not very > >>>>>>>>>>> easy > >>>>>>>>>>> to use. > >>>>>>>>>>> > >>>>>>>>>>> For example, you can write a classlist like this: > >>>>>>>>>>> > >>>>>>>>>>> java/lang/Object id: 1 > >>>>>>>>>>> CustomLoadee id: 2 super: 1 source: /tmp/foo.jar > >>>>>>>>>>> > >>>>>>>>>>> The CustomLoadee class will be stored in the shared archive with a > >>>>>>>>>>> CRC > >>>>>>>>>>> code. > >>>>>>>>>>> During runtime, if a customed loader wants to load a class of the > >>>>>>>>>>> same > >>>>>>>>>>> name, > >>>>>>>>>>> and its classfile has the same size and CRC as the archived class, > >>>>>>>>>>> the > >>>>>>>>>>> archived version will be loaded. This speeds up class loading by > >>>>>>>>>>> avoiding > >>>>>>>>>>> parsing the class file, and saves space by sharing the mmap'ed class > >>>>>>>>>>> metadata across processes. > >>>>>>>>>>> > >>>>>>>>>>> You can see an example test at: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java > >>>>>>>>>>> > >>>>>>>>>>> However, the current scheme requires you to specify all the super > >>>>>>>>>>> classes > >>>>>>>>>>> and interfaces. There's no support provided by the > >>>>>>>>>>> -XX:DumpLoadedClassList > >>>>>>>>>>> option. It can be helped somewhat with Volker's tool: > >>>>>>>>>>> https://github.com/simonis/cl4cds > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> POSSIBLE SOLUTIONS: > >>>>>>>>>>> > >>>>>>>>>>> 1. "Dump-as-you-go". As suggested by Yumin, we can provide a jcmd to > >>>>>>>>>>> ask > >>>>>>>>>>> a > >>>>>>>>>>> running JVM process to dump all of its loaded classes, including > >>>>>>>>>>> those > >>>>>>>>>>> loaded by custom loaders, into an archive. An alternative is to dump > >>>>>>>>>>> the > >>>>>>>>>>> archive at JVM exit time (or when you press Ctrl-C, etc. > >>>>>>>>>>> > >>>>>>>>>>> 2. Add information about the custom classes for > >>>>>>>>>>> -XX:DumpLoadedClassList. > >>>>>>>>>>> The > >>>>>>>>>>> trouble is some class loaders don't specify a code source that can be > >>>>>>>>>>> understood by the built-in class loaders. For example, the "Fat Jars" > >>>>>>>>>>> would > >>>>>>>>>>> have a code source like > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/ > >>>>>>>>>>> > >>>>>>>>>>> also, many custom loaders would pre-process the classfile data before > >>>>>>>>>>> defining the class, so we can't simply archive the version of the > >>>>>>>>>>> class > >>>>>>>>>>> on > >>>>>>>>>>> disk. > >>>>>>>>>>> > >>>>>>>>>>> One possible solution for #2 is to include the class file data in the > >>>>>>>>>>> -XX:DumpLoadedClassList output: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> java/lang/Object id: 1 > >>>>>>>>>>> CustomLoadee id: 2 super: 1 source: base64 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA > >>>>>>>>>>> AgAPAAcAAAAKAAEABQADAAYACA== > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Of the 2 solutions: > >>>>>>>>>>> > >>>>>>>>>>> #1 seems easier to use, but may require more invasive modifications > >>>>>>>>>>> in > >>>>>>>>>>> the > >>>>>>>>>>> VM, especially if you want to be able to continue execution after > >>>>>>>>>>> dumping. > >>>>>>>>>>> > >>>>>>>>>> Not sure what #1 really proposes: dumping the complete .jsa archive at > >>>>>>>>>> runtime or dumping just the loaded classes. > >>>>>>>>>> > >>>>>>>>>> If it's just about dumping the loaded class without generating the > >>>>>>>>>> .jsa archive there's the problem that by default the VM doesn't store > >>>>>>>>>> the exact bytes of a class after the class was loaded (except when > >>>>>>>>>> class transformers are registered). So the class files would have to > >>>>>>>>>> be re-assembled from the internal VM structures (in the same way this > >>>>>>>>>> is done for class redefinition) and the resulting class-file may be > >>>>>>>>>> different from the original bytes (i.e. some attributes may be > >>>>>>>>>> missing). > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> #1 is for creating the JSA file, not just dumping the class files. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> If #1 is about creating the whole .jsa archive at runtime (or at VM > >>>>>>>>>> exit) I think that would be the most attractive solution from a > >>>>>>>>>> usability point of view although I understand that #2 will be easier > >>>>>>>>>> to implement in the short term. Regarding the argument that #1 will > >>>>>>>>>> produce a "binary blob" that's true, but that's already true now when > >>>>>>>>>> we use "Xshare:dump". I think it should be not to hard to implement a > >>>>>>>>>> tool based an SA which could introspect a .jsa archive. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> The argument about the binary blob is to compare it against the text > >>>>>>>>> file > >>>>>>>>> produced by -XX:DumpLoadedClassList. > >>>>>>>>> > >>>>>>>>> One use case to consider is when you have a JAR file that contains > >>>>>>>>> several > >>>>>>>>> apps that each load a unique set of classes. Today, (assuming that > >>>>>>>>> custom > >>>>>>>>> class loaders are not used), you can run each app once with > >>>>>>>>> -XX:DumpLoadedClassList, and then do an > >>>>>>>>> > >>>>>>>>> cat *.classlist | sort | uniq > combined.classlist > >>>>>>>>> > >>>>>>>>> and then create an archive that would work for all these apps. > >>>>>>>>> > >>>>>>>> > >>>>>>>> But is this really y relevant use case? Why would I like to create ONE > >>>>>>>> archive for several apps? This would actually increase the footprint > >>>>>>>> of a single instance which uses this archive. If I have several apps I > >>>>>>>> would expect that users create a specific archive for each app to get > >>>>>>>> the best out of CDS. > >>>>>>>> > >>>>>>>>> With the binary blob, there's no easy way of doing this. It will be > >>>>>>>>> very > >>>>>>>>> difficult to write a tool to decipher each blob and then somehow > >>>>>>>>> combine > >>>>>>>>> them into a single one. > >>>>>>>>> > >>>>>>>> > >>>>>>>> But if users really wants such a "fat" archive, there's a much easier > >>>>>>>> way: just dump ALL the classes from the .jar file into the archive. A > >>>>>>>> class list for this could easily be assembled either with an external > >>>>>>>> tool like cl4cds (or even a simple shell scripts which converts the > >>>>>>>> output of `unzip -l ` into the correct format). Or, even > >>>>>>>> simpler, by adding a new option to the VM similar to > >>>>>>>> -XX:DumpLoadedClassList which dumps all the classes it can find on the > >>>>>>>> class path (and potentially other, configurable locations). > >>>>>>>> > >>>>>>>>> Thanks > >>>>>>>>> - Ioi > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>>> #2 would be easier to implement, but the classlist might be huge. > >>>>>>>>>>> > >>>>>>>>>>> Also, #2 would allow post-processing tools to remove unneeded > >>>>>>>>>>> classes, > >>>>>>>>>>> or > >>>>>>>>>>> merge two runs into a single list. The output of #1 is essentially a > >>>>>>>>>>> binary > >>>>>>>>>>> blob that's impossible for off-line analysis/optimizations. > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Any comments, or suggestions for alternatives? > >>>>>>>>>>> > >>>>>>>>>>> Thanks > >>>>>>>>>>> - Ioi > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>> > >>> > >> > From david.holmes at oracle.com Mon May 7 07:06:35 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 7 May 2018 17:06:35 +1000 Subject: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics In-Reply-To: <0309694e4cf84438ab20b84cae51150a@sap.com> References: <4a3b4d59c9f54234b0c0d325a3238cbc@sap.com> <372edaad-f92c-7071-036b-703c86ad25c1@oracle.com> <3c0af6430a8e43cbbd4178edc88a0512@sap.com> <0309694e4cf84438ab20b84cae51150a@sap.com> Message-ID: Hi Martin, On 5/05/2018 1:32 AM, Doerr, Martin wrote: > Hi David, > > thanks for your time. I understand that there's a lot of work at the moment. > > I have removed memory_order_consume again. We don't need it for our platforms and I guess nobody will miss it. > > I like your comment proposals, so I've updated them. > > New webrev: > http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.02/ Changes seem fine. Thanks. >> Though that raises the question why you haven't added memory_order_seq_cst? > I think that one could easily be used incorrectly within hotspot which would lead to problems on PPC. > "Atomic operations tagged memory_order_seq_cst ... establish a single total modification order of all atomic operations that are so tagged." [1] > It should work fine if all involved accesses are "so tagged". But hotspot doesn't use seq_cst loads, it uses things like OrderAccess::load_acquire. On PPC, GCC and xlC don't use a trailing sync for seq_cst stores and RMW-atomics so they only work well with leading sync loads which are not used in hotspot code. > I believe that there's a substantial amount of work to be done to get this right. I find this somewhat ironic/amusing. My main concern with adding in any access modes weaker than the existing "conservative" is precisely because it can be very hard to establish that they are in fact correct. :) But I don't follow your concern. The "total modification order" is in addition to the acquire/release semantics that are inherent in seq-cst. So if the seq-cst atomic-rmw operation is used on a variable that is otherwise accessed via load_acquire/release_store, then I would expect everything to be okay - else it serves no point to define seq-cst rmw in terms of acquire and release IMHO. I would expect these access modes to interact in the "obvious" way. Cheers, David > Thanks, > Martin > > > [1] https://en.cppreference.com/w/cpp/atomic/memory_order#Sequentially-consistent_ordering > > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Freitag, 4. Mai 2018 08:09 > To: Doerr, Martin ; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics > > Hi Martin, > > On 3/05/2018 11:03 PM, Doerr, Martin wrote: >> Hi Robbin, >> >> thank you for reviewing. >> >> Submission forest tests have passed in addition to SAP's nightly builds. I have 2 reviews, now, but I'll wait a little bit before pushing. > > Thanks for waiting. I think it important there is a general buy-in to > this effort, not just the official "get a review and push". Apologies > that it's taken me a while to get to this. > > memory_order_consume has been deprecated in C++17: > > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0371r1.html > > I would suggest we not add it to the VM. > > 40 enum atomic_memory_order { > 41 // We support most semantics like in C++11. > > I would say something like: > > // The modes that align with C++11 are intended to > // follow the same semantics. > > > 47 // We need to be more conservative than seq_cst on PPC64. > > Not sure the PPC64 comment is relevant here. memory_order_conservative > represents the pre-existing strong fencing requirements. It's not > intended to map to anything else, nor intended to be different across > platforms. > > Though that raises the question why you haven't added memory_order_seq_cst? > > Looking at all the platform code that adds the unused "order" parameter > ... I would not be surprised if the forthcoming compiler upgrades result > in some additional "unused" warnings. But I guess we'll just deal with > them as they are discovered. > > Otherwise all seems okay. > > Thanks, > David > > >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: Robbin Ehn [mailto:robbin.ehn at oracle.com] >> Sent: Donnerstag, 3. Mai 2018 14:29 >> To: Doerr, Martin ; Schmidt, Lutz ; David Holmes ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley (aph at redhat.com) ; John Paul Adrian Glaubitz >> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics >> >> On 2018-05-03 12:50, Doerr, Martin wrote: >>> Hi Lutz, >>> >>> thanks for reviewing. >>> >>> Can I get more reviews, please? >> >> I think this looks good, thanks for fixing! >> >> /Robbin >> >>> >>> Best regards, >>> Martin >>> >>> >>> -----Original Message----- >>> From: Schmidt, Lutz >>> Sent: Mittwoch, 2. Mai 2018 16:20 >>> To: Doerr, Martin ; David Holmes ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley (aph at redhat.com) ; John Paul Adrian Glaubitz >>> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics >>> >>> Hi Martin, >>> >>> your changes look good. >>> >>> For the additional synchronization on s390 in particular, I do not expect a serious performance impact. The checkpoint synchronization part has always been the expensive, but rarely required, part. PPC may be another story. We knoow that a full-blown sync really hurts. We will see... >>> >>> Thanks, >>> Lutz >>> >>> ?On 26.04.18, 16:23, "hotspot-runtime-dev on behalf of Doerr, Martin" wrote: >>> >>> Hi, >>> >>> I have renamed "cmpxchg_memory_order" to "atomic_memory_order" and added support to all Read-Modify-Write atomics with support for all C++11 semantics except seq_cst. The latter has issues on PPC64 so we're currently using our own memory_order_conservative instead, which is the default. >>> >>> Please review my new webrev: >>> http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.01/ >>> >>> We'll test Windows, MacOS, linux x86+ppc64+s390, AIX and Solaris SPARC. >>> It'd be great if somebody could check arm/aarch64 and zero. >>> >>> This change may enable optimizations for arm/aarch64 (not yet included). >>> >>> Best regards, >>> Martin >>> >>> >>> -----Original Message----- >>> From: Doerr, Martin >>> Sent: Donnerstag, 26. April 2018 11:20 >>> To: 'David Holmes' ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) >>> Subject: RE: RFR(M): 8202080: Introduce ordering semantics for Atomic::add >>> >>> Hi David, >>> >>> sounds like we are on the same page, now. I think we should have some kind of summary of what was analyzed. But that belongs to the other thread (JDK- 8154736). >>> >>> > The bugs in this kind of code are very subtle and rarely exposed through normal levels of testing. >>> Right. We have experienced this often enough. So it is definitely in our interest to have correct memory barriers. Mistakes will hit us (PPC64). So I appreciate that you insist on careful analysis. >>> >>> I'll prepare a new webrev with conservative semantics for all read-modify-write atomics by default. >>> >>> Best regards, >>> Martin >>> >>> >>> From david.holmes at oracle.com Mon May 7 07:30:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 7 May 2018 17:30:12 +1000 Subject: RFR(M): 8201593: Print array length in ArrayIndexOutOfBoundsException. In-Reply-To: <14107212c2664aee9965e9f9da8006ad@sap.com> References: <504bdd8469bb45cc8c5cf790d1f4b3fc@sap.com> <12e8ad2367484b0b88bcee4bcb77ac47@sap.com> <033adcb3-1ab0-094e-78cf-4062a8868d3e@redhat.com> <7767af56c6744e8aa5479a5dd9626936@sap.com> <9d2753f6975c4477912420a2fe459ef3@sap.com> <14107212c2664aee9965e9f9da8006ad@sap.com> Message-ID: <54e1796c-46e8-18be-2773-69636c4b4c0d@oracle.com> Hi Goetz, On 4/05/2018 7:22 PM, Lindenmaier, Goetz wrote: > Hi, > > thanks to Aleksey and Boris this is now also tested on arm. > This final webrev contains some fixes needed in the arm files: > http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06-arm/ > > David, can I consider this as finally reviewed? This follows on top of my +1 comments to Roger about the message consistency and punctuation etc. Aside: Took me a while to realize the throw_index_out_of_bounds_exception field was not a throw/don't-throw flag, but a throw AIOOBE or IOOBE flag! Again note I can't comment on the detailed CPU specific code. One further nit: src/hotspot/cpu/aarch64/templateTable_aarch64.cpp I don't like the // ??? convention comments. It suggests the code is not understood. I don't expect you to fix existing ones but adding new ones doesn't seem good. Thanks, David > Best regards, > Goetz > > > >> -----Original Message----- >> From: aarch64-port-dev [mailto:aarch64-port-dev- >> bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz >> Sent: Mittwoch, 2. Mai 2018 17:57 >> To: Stuart Monteith >> Cc: hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- >> dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; aarch32-port- >> dev at openjdk.java.net >> Subject: [CAUTION] Re: [aarch64-port-dev ] RFR(M): 8201593: Print array >> length in ArrayIndexOutOfBoundsException. >> >> Hi, >> >> I needed to move the edit from c1_LIRGenerator_.cpp to >> the shared file after "8201543: Modularize C1 GC barriers" >> New webrev: >> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06/ >> >> @Stuart >> Thanks for testing! >>> so as to accommodate the array pointer you are pushing onto the stack? >> Yes, what you are pointing out seems to be wrong, I changed it to '2'. >> >> Best regards, >> Goetz. >> >> >>> -----Original Message----- >>> From: Stuart Monteith [mailto:stuart.monteith at linaro.org] >>> Sent: Freitag, 27. April 2018 16:37 >>> To: Lindenmaier, Goetz >>> Cc: hotspot-compiler-dev at openjdk.java.net; aarch64-port- >>> dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; aarch32- >>> port-dev at openjdk.java.net >>> Subject: Re: RFR(M): 8201593: Print array length in >>> ArrayIndexOutOfBoundsException. >>> >>> Hi, >>> JTregs hasn't flagged any issues, so it should be ok. >>> >>> Regarding the 32-bit arm code, in "void >>> RangeCheckStub::emit_code(LIR_Assembler* ce)" should: >>> ce->verify_reserved_argument_area_size(1); >>> be >>> ce->verify_reserved_argument_area_size(2); >>> >>> so as to accommodate the array pointer you are pushing onto the stack? >>> >>> I've not tested 32-bit arm. >>> >>> >>> BR, >>> Stuart >>> >>> On 26 April 2018 at 15:31, Stuart Monteith >>> wrote: >>>> Thanks, I'm happy with that. >>>> >>>> The registers have a clean path to call_RT - r22 and r23 aren't used >>>> inbetween. They are an arbitrary choice - c_rarg0 and c_rarg1 were >>>> always going to cause problems. If _array->as_pointer_register() >>>> and/or _index->as_register() or _index->as_jint() were the registers >>>> we were using as parameters there would be trouble. However, with >>>> pd_last_allocatable_cpu_reg = 16, that shouldn't happen with r22/23, >>>> or indeed anything else in the range r17 to r28. >>>> >>>> I'm going to run all of JTRegs and seem what that produces now. >>>> >>>> BR, >>>> Stuart >>>> >>>> >>>> >>>> >>>> On 26 April 2018 at 15:14, Lindenmaier, Goetz >>> wrote: >>>>> Hi Stuart, >>>>> >>>>> thanks for fixing this! Webrev with your changes: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/05/ >>>>> >>>>>> There is the possibility of overwriting live values though, aren't >>>>>> there? The registers are saved by call_RT. Should I be concerned about >>>>>> deopt and debugging going wrong? Furthermore, won't there be >> issues >>> in >>>>>> exception handlers? >>>>> As I understand, this just has to survive the far_call. >>>>> The call_RT in c1_Runtime then moves it into the >>>>> proper argument registers. This is just the handling of an >>>>> exception, and in these few instructions no java code is >>>>> executed, no safepoint is passed, so this should be fine. >>>>> >>>>> incremental diff: >>>>> iff -r 874f2b999ff6 src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp >>>>> --- a/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Mon Apr 16 >>> 15:17:20 2018 +0200 >>>>> +++ b/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Thu Apr 26 >>> 15:55:18 2018 +0200 >>>>> @@ -75,16 +75,16 @@ >>>>> } >>>>> >>>>> if (_index->is_cpu_register()) { >>>>> - __ mov(rscratch1, _index->as_register()); >>>>> + __ mov(r22, _index->as_register()); >>>>> } else { >>>>> - __ mov(rscratch1, _index->as_jint()); >>>>> + __ mov(r22, _index->as_jint()); >>>>> } >>>>> Runtime1::StubID stub_id; >>>>> if (_throw_index_out_of_bounds_exception) { >>>>> stub_id = Runtime1::throw_index_exception_id; >>>>> } else { >>>>> assert(_array != NULL, "sanity"); >>>>> - __ mov(rscratch2, _array->as_pointer_register()); >>>>> + __ mov(r23, _array->as_pointer_register()); >>>>> stub_id = Runtime1::throw_range_check_failed_id; >>>>> } >>>>> __ far_call(RuntimeAddress(Runtime1::entry_for(stub_id)), NULL, >>> rscratch2); >>>>> diff -r 874f2b999ff6 src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp >>>>> --- a/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Mon Apr 16 >>> 15:17:20 2018 +0200 >>>>> +++ b/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Thu Apr 26 >>> 15:55:18 2018 +0200 >>>>> @@ -327,7 +327,7 @@ >>>>> >>>>> >>>>> // target: the entry point of the method that creates and posts the >>> exception oop >>>>> -// has_argument: true if the exception needs an argument (passed in >>> rscratch1) >>>>> +// has_argument: true if the exception needs arguments (passed in >> r22 >>> and r23) >>>>> >>>>> OopMapSet* Runtime1::generate_exception_throw(StubAssembler* >>> sasm, address target, bool has_argument) { >>>>> // make a frame and preserve the caller's caller-save registers >>>>> @@ -336,7 +336,7 @@ >>>>> if (!has_argument) { >>>>> call_offset = __ call_RT(noreg, noreg, target); >>>>> } else { >>>>> - call_offset = __ call_RT(noreg, noreg, target, rscratch1, rscratch2); >>>>> + call_offset = __ call_RT(noreg, noreg, target, r22, r23); >>>>> } >>>>> OopMapSet* oop_maps = new OopMapSet(); >>>>> oop_maps->add_gc_map(call_offset, oop_map); >>>>> >>>>> Best regards, >>>>> Goetz. >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: Stuart Monteith [mailto:stuart.monteith at linaro.org] >>>>>> Sent: Donnerstag, 26. April 2018 12:52 >>>>>> To: Andrew Haley >>>>>> Cc: Lindenmaier, Goetz ; hotspot- >>> compiler- >>>>>> dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; hotspot- >>>>>> runtime-dev at openjdk.java.net; aarch32-port-dev at openjdk.java.net >>>>>> Subject: Re: RFR(M): 8201593: Print array length in >>>>>> ArrayIndexOutOfBoundsException. >>>>>> >>>>>> Hi, >>>>>> Using c_rarg1 and c_rarg2 instead of rscratch1 and overwriting >>>>>> rscratch2 causes a SIGSEGV. >>>>>> Using r22 and r23 instead, the test ran successfully. >>>>>> >>>>>> In c1_CodeStubs_aarch64.cpp >>>>>> : >>>>>> 77 if (_index->is_cpu_register()) { >>>>>> 78 __ mov(r22, _index->as_register()); >>>>>> 79 } else { >>>>>> 80 __ mov(r22, _index->as_jint()); >>>>>> 81 } >>>>>> 82 Runtime1::StubID stub_id; >>>>>> 83 if (_throw_index_out_of_bounds_exception) { >>>>>> 84 stub_id = Runtime1::throw_index_exception_id; >>>>>> 85 } else { >>>>>> 86 assert(_array != NULL, "sanity"); >>>>>> 87 __ mov(r23, _array->as_pointer_register()); >>>>>> 88 stub_id = Runtime1::throw_range_check_failed_id; >>>>>> 89 } >>>>>> >>>>>> in c1_Runtime_aarch64.cpp: >>>>>> >>>>>> 336 if (!has_argument) { >>>>>> 337 call_offset = __ call_RT(noreg, noreg, target); >>>>>> 338 } else { >>>>>> 339 call_offset = __ call_RT(noreg, noreg, target, r22, r23); >>>>>> 340 } >>>>>> >>>>>> There is the possibility of overwriting live values though, aren't >>>>>> there? The registers are saved by call_RT. Should I be concerned about >>>>>> deopt and debugging going wrong? Furthermore, won't there be >> issues >>> in >>>>>> exception handlers? >>>>>> >>>>>> BR, >>>>>> Stuart >>>>>> >>>>>> >>>>>> On 25 April 2018 at 16:49, Stuart Monteith >> >>>>>> wrote: >>>>>>> Indeed - and that is what I am seeing. Usually no parameters are >> being >>>>>>> called with this pattern, or rscratch1, with the temporary variable >>>>>>> being changed to use rscratch2 in such circumstances. >>>>>>> I'll try c_rarg1 and c_rarg2 - they should pass straight through,if I >>>>>>> interpret the code correcting. >>>>>>> >>>>>>> On 25 April 2018 at 16:26, Andrew Haley wrote: >>>>>>>> On 04/25/2018 04:00 PM, Stuart Monteith wrote: >>>>>>>>> I'm not quite sure to solve this yet - we'll need to use the stack in >>>>>>>>> some safe way. >>>>>>>> >>>>>>>> It's not a great idea to pass arguments in rscratch1 or rscratch2. >> These >>>>>>>> registers are for use in macros and should be treated as volatile. >>> Given >>>>>>>> that you're throwing an exception, registers will be clobbered >>> anyway. >>>>>>>> >>>>>>>> -- >>>>>>>> Andrew Haley >>>>>>>> Java Platform Lead Engineer >>>>>>>> Red Hat UK Ltd. >>>>>>>> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From goetz.lindenmaier at sap.com Mon May 7 08:10:57 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 7 May 2018 08:10:57 +0000 Subject: RFR(M): 8201593: Print array length in ArrayIndexOutOfBoundsException. In-Reply-To: References: <504bdd8469bb45cc8c5cf790d1f4b3fc@sap.com> <4ca91dfb-c576-98e2-1222-ca9750a64177@oracle.com> <5be0f86df2834aa8a881206ba285011e@sap.com> <0b881c69-79af-2a8b-c34c-5929357ca8fa@oracle.com> <5ADF5897.8070809@cjnash.com> <819d942132204f58992c236518e0c344@sap.com> Message-ID: Hi Roger, new webrev: http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/07/ See my comments inline. > -----Original Message----- > From: Roger Riggs [mailto:roger.riggs at oracle.com] > > Hi Goetz, > > Just comments on the test/message format.? (found in the 04 version of > the webrev) > > The "." at the end of the exception messages should be removed. > The text is not a sentence (its is just a fragment without a verb) and > it is more flexible to print the exception message in various contexts > (in which the "." would seem to terminate a sentence). ok, I removed the "." > I'm not a big fan of the hyphens in out-of-bounds and it would be more > consistent across the new messages. I'll do a follow-up and remove the hyphens everywhere. Also removed here. > It would be more consistent if the "arraycopy:" prefix always included > the ":". Added. Best regards, Goetz. > > Thanks, Roger > > > On 4/24/18 12:25 PM, Lindenmaier, Goetz wrote: > > Hi Simon, > > > > Because as stated here, > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2018- > April/052665.html > > it is used in other places like that, too. > > > > Later mails agreed on that usage to keep it consistent. > > > > Best regards, > > Goetz. > > > >> -----Original Message----- > >> From: Simon Nash [mailto:simon at cjnash.com] > >> Sent: Dienstag, 24. April 2018 18:17 > >> To: Lindenmaier, Goetz > >> Cc: David Holmes ; hotspot-runtime- > >> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; > aarch64- > >> port-dev at openjdk.java.net; aarch32-port-dev at openjdk.java.net; core- > libs- > >> dev Libs > >> Subject: Re: RFR(M): 8201593: Print array length in > >> ArrayIndexOutOfBoundsException. > >> > >> On 24/04/2018 15:08, Lindenmaier, Goetz wrote: > >>> Hi, > >>> > >>> I implemented what we figured out in > >>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018- > >> April/027555.html > >>> Some examples: > >>> "Index 12 out-of-bounds for length 10." > >>> "arraycopy source index -17 out of bounds for object array[10]." > >>> "arraycopy destination index -18 out of bounds for double[5]." > >>> "arraycopy length -19 is negative." > >>> "arraycopy: last source index 13 out of bounds for double[10]." > >>> "arraycopy: last destination index 7 out of bounds for long[5]." > >>> > >> Is there a reason why the first message says "out-of-bounds" but all > others > >> say "out of bounds"? > >> > >> Simon > >> > >>> Incremental webrev: > >>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/03- > >> incremental/ > >>> Full webrev: > >>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/03/ > >>> > >>> I'll push it through our tests tonight. > >>> > >>> See further comments in line: > >>> > >>>> -----Original Message----- > >>>> From: David Holmes [mailto:david.holmes at oracle.com] > >>>> Sent: Freitag, 20. April 2018 09:25 > >>>> To: Lindenmaier, Goetz ; hotspot- > runtime- > >>>> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; > >> aarch64- > >>>> port-dev at openjdk.java.net; aarch32-port-dev at openjdk.java.net; > core- > >> libs- > >>>> dev Libs > >>>> Subject: Re: RFR(M): 8201593: Print array length in > >>>> ArrayIndexOutOfBoundsException. > >>>> > >>>> Hi Goetz, > >>>> > >>>> This is not a file by file review ... > >>>> > >>>> On 19/04/2018 10:24 PM, Lindenmaier, Goetz wrote: > >>>>> Hi, > >>>>> > >>>>> New webrev: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/02/ > >>>>> > >>>>> I admit my wording is not optimal. > >>>> src/hotspot/share/oops/typeArrayKlass.cpp > >>>> > >>>> Sorry but this is still far too verbose for my tastes. The type of the > >>>> array is not relevant. For the array copy its okay to indicate src or > >>>> dst array. But the message should be clear and succinct not prose-like. > >>>> Plus you have a lot of repetition in the ss.print statements when only > >>>> one thing is changing. > >>> We discussed this in some further mails. Implemented as proposed > there. > >>> > >>>> src/hotspot/cpu/x86/c1_CodeStubs_x86.cpp > >>>> > >>>> I'm not seeing why the throw_index_out_of_bounds_exception > should > >> be > >>>> tied to whether or not you have an array reference. Certainly I hate > >>>> seeing the array ref being used as an implicit bool! > >>> I split the constructor into two, one for ArrayIndexOutOfBounds, the > other > >>> for IndexOutOfBounds. > >>> > >>> ... > >>> > >>>>> I extended the test to cover the exception thrown in arraycopy better > >>>> The test seems somewhat excessive, and I now see it is because of the > >>>> more elaborate error messages you have at SAP. But with only the > index > >>>> and the array length of interest here the test can be considerably > smaller. > >>>> > >>>> The creation tests for ArrayIndexOutOfBoundsException don't seem > >>>> relevant in this context either. This looks more TCK like. > >>> Yes, the constructor tests are for the code not yet contributed. > >>> I simplified the tests to only check the messages. > >>> > >>> Best regards, > >>> Goetz. > >>> > >>> > >>> > >>> > >>> > >>>> David > >>>> ----- > >>>> > >>>>> and added the elementary type to the message text. This probably > >>>>> needs improvement in the text, too. There are (currently) these > cases: > >>>>> > >>>>> bject[] oa1 = new Object[10]; > >>>>> Object[] oa2 = new Object[5]; > >>>>> System.arraycopy(oa1, -17, oa2, 0, 5); > >>>>> "while trying to copy from index -17 of an object array with length > 10"); > >>>>> System.arraycopy(oa1, 2, oa2, -18, 5); > >>>>> "while trying to copy to index -18 of an object array with length 5"); > >>>>> System.arraycopy(oa1, 2, oa2, 0, -19); > >>>>> "while trying to copy a negative range -19 from an object array with > >> length > >>>> 10 to an object array with length 5"); > >>>>> System.arraycopy(oa1, 8, oa2, 0, 5); > >>>>> "while trying to copy from index 13 of an object array with length 10"); > >>>>> System.arraycopy(oa1, 1, oa2, 0, 7); > >>>>> "while trying to copy to index 7 of an object array with length 5"); > >>>>> double[] ta1 = new double[10]; > >>>>> double[] ta2 = new double[5]; > >>>>> System.arraycopy(ta1, -17, ta2, 0, 5); > >>>>> "while trying to copy from index -17 of a doubl array with length 10"); > >>>>> System.arraycopy(ta1, 2, ta2, -18, 5); > >>>>> "while trying to copy to index -18 of a double array with length 5"); > >>>>> System.arraycopy(ta1, 2, ta2, 0, -19); > >>>>> "while trying to copy a negative range -19 from a double array with > >> length > >>>> 10 to a double array with length 5"); > >>>>> System.arraycopy(ta1, 8, ta2, 0, 5); > >>>>> "while trying to copy from index 13 of a double array with length 10"); > >>>>> System.arraycopy(ta1, 1, ta2, 0, 7); > >>>>> "while trying to copy to index 7 of a double array with length 5"); > >>>>> > >>>>> Maybe it should say: > >>>>> Arraycopy source index -1 out-of-bounds for double array of length > 10. > >>>>> Arraycopy target index 10 out-of-bounds for object array of length 10. > >>>>> Negative range -19 when copying from an object array of length 10 to > an > >>>> object array of length 5. > >>>>> Best regards, > >>>>> Goetz. > >>>>> > >>>>>> -----Original Message----- > >>>>>> From: David Holmes [mailto:david.holmes at oracle.com] > >>>>>> Sent: Mittwoch, 18. April 2018 10:55 > >>>>>> To: Lindenmaier, Goetz ; hotspot- > >> runtime- > >>>>>> dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; > >>>> aarch64- > >>>>>> port-dev at openjdk.java.net; aarch32-port-dev at openjdk.java.net; > >> core- > >>>> libs- > >>>>>> dev Libs > >>>>>> Subject: Re: RFR(M): 8201593: Print array length in > >>>>>> ArrayIndexOutOfBoundsException. > >>>>>> > >>>>>> Adding core-libs-dev as you're changing > >>>>>> java.lang.ArrayIndexOutOfBoundsException. > >>>>>> > >>>>>> I appreciate the intent here but I find the messages excessively > >>>>>> verbose. The basic error is: > >>>>>> > >>>>>> index N is outside range [0, length-1] > >>>>>> > >>>>>> David > >>>>>> > >>>>>> On 18/04/2018 6:09 PM, Lindenmaier, Goetz wrote: > >>>>>>> Hi, > >>>>>>> > >>>>>>> I would like to print a more verbose text on > ArrayIndexOutOfBounds > >>>>>> exception > >>>>>>> that not only mentions the index, but also the length of the array > >>>> accessed. > >>>>>>> See the bug for documentation of the change of the message. > >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/01/ > >>>>>>> > >>>>>>> @aarch/arm people: > >>>>>>> I edited the aarch/arm files. Could you please verify this is correct? > >>>>>>> I can not build on these platforms. > >>>>>>> > >>>>>>> The code on all the other platforms is tested with all the jtreg and > jck > >>>> tests > >>>>>> etc. > >>>>>>> Best regards, > >>>>>>> Goetz. > >>>>>>> > >>>>>>> From goetz.lindenmaier at sap.com Mon May 7 08:16:23 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 7 May 2018 08:16:23 +0000 Subject: RFR(M): 8201593: Print array length in ArrayIndexOutOfBoundsException. In-Reply-To: <54e1796c-46e8-18be-2773-69636c4b4c0d@oracle.com> References: <504bdd8469bb45cc8c5cf790d1f4b3fc@sap.com> <12e8ad2367484b0b88bcee4bcb77ac47@sap.com> <033adcb3-1ab0-094e-78cf-4062a8868d3e@redhat.com> <7767af56c6744e8aa5479a5dd9626936@sap.com> <9d2753f6975c4477912420a2fe459ef3@sap.com> <14107212c2664aee9965e9f9da8006ad@sap.com> <54e1796c-46e8-18be-2773-69636c4b4c0d@oracle.com> Message-ID: <7887adc187914fef9b4a77de28b1745f@sap.com> Hi David, New webrev with the punctuation changed: http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/07/ For the punctuation see also my mail reply to Roger's mail. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Montag, 7. Mai 2018 09:30 > > Hi Goetz, > > On 4/05/2018 7:22 PM, Lindenmaier, Goetz wrote: > > Hi, > > > > thanks to Aleksey and Boris this is now also tested on arm. > > This final webrev contains some fixes needed in the arm files: > > http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06-arm/ > > > > David, can I consider this as finally reviewed? > > This follows on top of my +1 comments to Roger about the message > consistency and punctuation etc. > > Aside: Took me a while to realize the > throw_index_out_of_bounds_exception field was not a throw/don't-throw > flag, but a throw AIOOBE or IOOBE flag! Yes, this is not very intuitive ... > Again note I can't comment on the detailed CPU specific code. The CPU code was reviewed by Martin Doerr, Stuart Monteith, Aleksey Shipilev and Boris Ulasevich. > One further nit: > src/hotspot/cpu/aarch64/templateTable_aarch64.cpp > I don't like the > // ??? convention > comments. It suggests the code is not understood. I don't expect you to > fix existing ones but adding new ones doesn't seem good. I don't like that either, nor the design of using a convention here. The reviewers had trouble with that, too. But as the two values are handled similarly, I would to like to document it similarly, following the existing format. Stuart was fine with that, anyways. An improvement of the design how these values are handled would require changes on all platforms (it's similarly bad everywhere) and I don't like to do that in this scope. Best regards, Goetz. > > > Thanks, > David > > > Best regards, > > Goetz > > > > > > > >> -----Original Message----- > >> From: aarch64-port-dev [mailto:aarch64-port-dev- > >> bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz > >> Sent: Mittwoch, 2. Mai 2018 17:57 > >> To: Stuart Monteith > >> Cc: hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > >> dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; aarch32- > port- > >> dev at openjdk.java.net > >> Subject: [CAUTION] Re: [aarch64-port-dev ] RFR(M): 8201593: Print array > >> length in ArrayIndexOutOfBoundsException. > >> > >> Hi, > >> > >> I needed to move the edit from c1_LIRGenerator_.cpp to > >> the shared file after "8201543: Modularize C1 GC barriers" > >> New webrev: > >> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06/ > >> > >> @Stuart > >> Thanks for testing! > >>> so as to accommodate the array pointer you are pushing onto the stack? > >> Yes, what you are pointing out seems to be wrong, I changed it to '2'. > >> > >> Best regards, > >> Goetz. > >> > >> > >>> -----Original Message----- > >>> From: Stuart Monteith [mailto:stuart.monteith at linaro.org] > >>> Sent: Freitag, 27. April 2018 16:37 > >>> To: Lindenmaier, Goetz > >>> Cc: hotspot-compiler-dev at openjdk.java.net; aarch64-port- > >>> dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; > aarch32- > >>> port-dev at openjdk.java.net > >>> Subject: Re: RFR(M): 8201593: Print array length in > >>> ArrayIndexOutOfBoundsException. > >>> > >>> Hi, > >>> JTregs hasn't flagged any issues, so it should be ok. > >>> > >>> Regarding the 32-bit arm code, in "void > >>> RangeCheckStub::emit_code(LIR_Assembler* ce)" should: > >>> ce->verify_reserved_argument_area_size(1); > >>> be > >>> ce->verify_reserved_argument_area_size(2); > >>> > >>> so as to accommodate the array pointer you are pushing onto the stack? > >>> > >>> I've not tested 32-bit arm. > >>> > >>> > >>> BR, > >>> Stuart > >>> > >>> On 26 April 2018 at 15:31, Stuart Monteith > >>> wrote: > >>>> Thanks, I'm happy with that. > >>>> > >>>> The registers have a clean path to call_RT - r22 and r23 aren't used > >>>> inbetween. They are an arbitrary choice - c_rarg0 and c_rarg1 were > >>>> always going to cause problems. If _array->as_pointer_register() > >>>> and/or _index->as_register() or _index->as_jint() were the registers > >>>> we were using as parameters there would be trouble. However, with > >>>> pd_last_allocatable_cpu_reg = 16, that shouldn't happen with r22/23, > >>>> or indeed anything else in the range r17 to r28. > >>>> > >>>> I'm going to run all of JTRegs and seem what that produces now. > >>>> > >>>> BR, > >>>> Stuart > >>>> > >>>> > >>>> > >>>> > >>>> On 26 April 2018 at 15:14, Lindenmaier, Goetz > >>> wrote: > >>>>> Hi Stuart, > >>>>> > >>>>> thanks for fixing this! Webrev with your changes: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/05/ > >>>>> > >>>>>> There is the possibility of overwriting live values though, aren't > >>>>>> there? The registers are saved by call_RT. Should I be concerned > about > >>>>>> deopt and debugging going wrong? Furthermore, won't there be > >> issues > >>> in > >>>>>> exception handlers? > >>>>> As I understand, this just has to survive the far_call. > >>>>> The call_RT in c1_Runtime then moves it into the > >>>>> proper argument registers. This is just the handling of an > >>>>> exception, and in these few instructions no java code is > >>>>> executed, no safepoint is passed, so this should be fine. > >>>>> > >>>>> incremental diff: > >>>>> iff -r 874f2b999ff6 > src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp > >>>>> --- a/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Mon Apr > 16 > >>> 15:17:20 2018 +0200 > >>>>> +++ b/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Thu Apr > 26 > >>> 15:55:18 2018 +0200 > >>>>> @@ -75,16 +75,16 @@ > >>>>> } > >>>>> > >>>>> if (_index->is_cpu_register()) { > >>>>> - __ mov(rscratch1, _index->as_register()); > >>>>> + __ mov(r22, _index->as_register()); > >>>>> } else { > >>>>> - __ mov(rscratch1, _index->as_jint()); > >>>>> + __ mov(r22, _index->as_jint()); > >>>>> } > >>>>> Runtime1::StubID stub_id; > >>>>> if (_throw_index_out_of_bounds_exception) { > >>>>> stub_id = Runtime1::throw_index_exception_id; > >>>>> } else { > >>>>> assert(_array != NULL, "sanity"); > >>>>> - __ mov(rscratch2, _array->as_pointer_register()); > >>>>> + __ mov(r23, _array->as_pointer_register()); > >>>>> stub_id = Runtime1::throw_range_check_failed_id; > >>>>> } > >>>>> __ far_call(RuntimeAddress(Runtime1::entry_for(stub_id)), NULL, > >>> rscratch2); > >>>>> diff -r 874f2b999ff6 > src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp > >>>>> --- a/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Mon Apr > 16 > >>> 15:17:20 2018 +0200 > >>>>> +++ b/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Thu Apr > 26 > >>> 15:55:18 2018 +0200 > >>>>> @@ -327,7 +327,7 @@ > >>>>> > >>>>> > >>>>> // target: the entry point of the method that creates and posts the > >>> exception oop > >>>>> -// has_argument: true if the exception needs an argument (passed > in > >>> rscratch1) > >>>>> +// has_argument: true if the exception needs arguments (passed in > >> r22 > >>> and r23) > >>>>> > >>>>> OopMapSet* > Runtime1::generate_exception_throw(StubAssembler* > >>> sasm, address target, bool has_argument) { > >>>>> // make a frame and preserve the caller's caller-save registers > >>>>> @@ -336,7 +336,7 @@ > >>>>> if (!has_argument) { > >>>>> call_offset = __ call_RT(noreg, noreg, target); > >>>>> } else { > >>>>> - call_offset = __ call_RT(noreg, noreg, target, rscratch1, rscratch2); > >>>>> + call_offset = __ call_RT(noreg, noreg, target, r22, r23); > >>>>> } > >>>>> OopMapSet* oop_maps = new OopMapSet(); > >>>>> oop_maps->add_gc_map(call_offset, oop_map); > >>>>> > >>>>> Best regards, > >>>>> Goetz. > >>>>> > >>>>> > >>>>>> -----Original Message----- > >>>>>> From: Stuart Monteith [mailto:stuart.monteith at linaro.org] > >>>>>> Sent: Donnerstag, 26. April 2018 12:52 > >>>>>> To: Andrew Haley > >>>>>> Cc: Lindenmaier, Goetz ; hotspot- > >>> compiler- > >>>>>> dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; > hotspot- > >>>>>> runtime-dev at openjdk.java.net; aarch32-port- > dev at openjdk.java.net > >>>>>> Subject: Re: RFR(M): 8201593: Print array length in > >>>>>> ArrayIndexOutOfBoundsException. > >>>>>> > >>>>>> Hi, > >>>>>> Using c_rarg1 and c_rarg2 instead of rscratch1 and overwriting > >>>>>> rscratch2 causes a SIGSEGV. > >>>>>> Using r22 and r23 instead, the test ran successfully. > >>>>>> > >>>>>> In c1_CodeStubs_aarch64.cpp > >>>>>> : > >>>>>> 77 if (_index->is_cpu_register()) { > >>>>>> 78 __ mov(r22, _index->as_register()); > >>>>>> 79 } else { > >>>>>> 80 __ mov(r22, _index->as_jint()); > >>>>>> 81 } > >>>>>> 82 Runtime1::StubID stub_id; > >>>>>> 83 if (_throw_index_out_of_bounds_exception) { > >>>>>> 84 stub_id = Runtime1::throw_index_exception_id; > >>>>>> 85 } else { > >>>>>> 86 assert(_array != NULL, "sanity"); > >>>>>> 87 __ mov(r23, _array->as_pointer_register()); > >>>>>> 88 stub_id = Runtime1::throw_range_check_failed_id; > >>>>>> 89 } > >>>>>> > >>>>>> in c1_Runtime_aarch64.cpp: > >>>>>> > >>>>>> 336 if (!has_argument) { > >>>>>> 337 call_offset = __ call_RT(noreg, noreg, target); > >>>>>> 338 } else { > >>>>>> 339 call_offset = __ call_RT(noreg, noreg, target, r22, r23); > >>>>>> 340 } > >>>>>> > >>>>>> There is the possibility of overwriting live values though, aren't > >>>>>> there? The registers are saved by call_RT. Should I be concerned > about > >>>>>> deopt and debugging going wrong? Furthermore, won't there be > >> issues > >>> in > >>>>>> exception handlers? > >>>>>> > >>>>>> BR, > >>>>>> Stuart > >>>>>> > >>>>>> > >>>>>> On 25 April 2018 at 16:49, Stuart Monteith > >> > >>>>>> wrote: > >>>>>>> Indeed - and that is what I am seeing. Usually no parameters are > >> being > >>>>>>> called with this pattern, or rscratch1, with the temporary variable > >>>>>>> being changed to use rscratch2 in such circumstances. > >>>>>>> I'll try c_rarg1 and c_rarg2 - they should pass straight through,if I > >>>>>>> interpret the code correcting. > >>>>>>> > >>>>>>> On 25 April 2018 at 16:26, Andrew Haley wrote: > >>>>>>>> On 04/25/2018 04:00 PM, Stuart Monteith wrote: > >>>>>>>>> I'm not quite sure to solve this yet - we'll need to use the stack in > >>>>>>>>> some safe way. > >>>>>>>> > >>>>>>>> It's not a great idea to pass arguments in rscratch1 or rscratch2. > >> These > >>>>>>>> registers are for use in macros and should be treated as volatile. > >>> Given > >>>>>>>> that you're throwing an exception, registers will be clobbered > >>> anyway. > >>>>>>>> > >>>>>>>> -- > >>>>>>>> Andrew Haley > >>>>>>>> Java Platform Lead Engineer > >>>>>>>> Red Hat UK Ltd. > >>>>>>>> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From david.holmes at oracle.com Mon May 7 09:00:07 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 7 May 2018 19:00:07 +1000 Subject: RFR(M): 8201593: Print array length in ArrayIndexOutOfBoundsException. In-Reply-To: <7887adc187914fef9b4a77de28b1745f@sap.com> References: <504bdd8469bb45cc8c5cf790d1f4b3fc@sap.com> <12e8ad2367484b0b88bcee4bcb77ac47@sap.com> <033adcb3-1ab0-094e-78cf-4062a8868d3e@redhat.com> <7767af56c6744e8aa5479a5dd9626936@sap.com> <9d2753f6975c4477912420a2fe459ef3@sap.com> <14107212c2664aee9965e9f9da8006ad@sap.com> <54e1796c-46e8-18be-2773-69636c4b4c0d@oracle.com> <7887adc187914fef9b4a77de28b1745f@sap.com> Message-ID: <243f21b7-ebb6-a372-1536-7d55c76acf01@oracle.com> Hi Goetz, On 7/05/2018 6:16 PM, Lindenmaier, Goetz wrote: > Hi David, > > New webrev with the punctuation changed: > http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/07/ > For the punctuation see also my mail reply to Roger's mail. Okay. That seems okay. Only further oddity I noticed is the use of %i rather than %d. Use of %i is very rare in hotspot. (I had to go and lookup what it means :) ). Thanks, David >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Montag, 7. Mai 2018 09:30 >> >> Hi Goetz, >> >> On 4/05/2018 7:22 PM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> thanks to Aleksey and Boris this is now also tested on arm. >>> This final webrev contains some fixes needed in the arm files: >>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06-arm/ >>> >>> David, can I consider this as finally reviewed? >> >> This follows on top of my +1 comments to Roger about the message >> consistency and punctuation etc. >> >> Aside: Took me a while to realize the >> throw_index_out_of_bounds_exception field was not a throw/don't-throw >> flag, but a throw AIOOBE or IOOBE flag! > Yes, this is not very intuitive ... > >> Again note I can't comment on the detailed CPU specific code. > The CPU code was reviewed by Martin Doerr, Stuart Monteith, Aleksey Shipilev and Boris Ulasevich. > >> One further nit: >> src/hotspot/cpu/aarch64/templateTable_aarch64.cpp >> I don't like the >> // ??? convention >> comments. It suggests the code is not understood. I don't expect you to >> fix existing ones but adding new ones doesn't seem good. > I don't like that either, nor the design of using a convention here. > The reviewers had trouble with that, too. > But as the two values are handled similarly, I would to like to > document it similarly, following the existing format. Stuart > was fine with that, anyways. > > An improvement of the design how these values are handled > would require changes on all platforms (it's similarly bad everywhere) > and I don't like to do that in this scope. > > Best regards, > Goetz. > > >> >> >> Thanks, >> David >> >>> Best regards, >>> Goetz >>> >>> >>> >>>> -----Original Message----- >>>> From: aarch64-port-dev [mailto:aarch64-port-dev- >>>> bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz >>>> Sent: Mittwoch, 2. Mai 2018 17:57 >>>> To: Stuart Monteith >>>> Cc: hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- >>>> dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; aarch32- >> port- >>>> dev at openjdk.java.net >>>> Subject: [CAUTION] Re: [aarch64-port-dev ] RFR(M): 8201593: Print array >>>> length in ArrayIndexOutOfBoundsException. >>>> >>>> Hi, >>>> >>>> I needed to move the edit from c1_LIRGenerator_.cpp to >>>> the shared file after "8201543: Modularize C1 GC barriers" >>>> New webrev: >>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06/ >>>> >>>> @Stuart >>>> Thanks for testing! >>>>> so as to accommodate the array pointer you are pushing onto the stack? >>>> Yes, what you are pointing out seems to be wrong, I changed it to '2'. >>>> >>>> Best regards, >>>> Goetz. >>>> >>>> >>>>> -----Original Message----- >>>>> From: Stuart Monteith [mailto:stuart.monteith at linaro.org] >>>>> Sent: Freitag, 27. April 2018 16:37 >>>>> To: Lindenmaier, Goetz >>>>> Cc: hotspot-compiler-dev at openjdk.java.net; aarch64-port- >>>>> dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; >> aarch32- >>>>> port-dev at openjdk.java.net >>>>> Subject: Re: RFR(M): 8201593: Print array length in >>>>> ArrayIndexOutOfBoundsException. >>>>> >>>>> Hi, >>>>> JTregs hasn't flagged any issues, so it should be ok. >>>>> >>>>> Regarding the 32-bit arm code, in "void >>>>> RangeCheckStub::emit_code(LIR_Assembler* ce)" should: >>>>> ce->verify_reserved_argument_area_size(1); >>>>> be >>>>> ce->verify_reserved_argument_area_size(2); >>>>> >>>>> so as to accommodate the array pointer you are pushing onto the stack? >>>>> >>>>> I've not tested 32-bit arm. >>>>> >>>>> >>>>> BR, >>>>> Stuart >>>>> >>>>> On 26 April 2018 at 15:31, Stuart Monteith >>>>> wrote: >>>>>> Thanks, I'm happy with that. >>>>>> >>>>>> The registers have a clean path to call_RT - r22 and r23 aren't used >>>>>> inbetween. They are an arbitrary choice - c_rarg0 and c_rarg1 were >>>>>> always going to cause problems. If _array->as_pointer_register() >>>>>> and/or _index->as_register() or _index->as_jint() were the registers >>>>>> we were using as parameters there would be trouble. However, with >>>>>> pd_last_allocatable_cpu_reg = 16, that shouldn't happen with r22/23, >>>>>> or indeed anything else in the range r17 to r28. >>>>>> >>>>>> I'm going to run all of JTRegs and seem what that produces now. >>>>>> >>>>>> BR, >>>>>> Stuart >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On 26 April 2018 at 15:14, Lindenmaier, Goetz >>>>> wrote: >>>>>>> Hi Stuart, >>>>>>> >>>>>>> thanks for fixing this! Webrev with your changes: >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/05/ >>>>>>> >>>>>>>> There is the possibility of overwriting live values though, aren't >>>>>>>> there? The registers are saved by call_RT. Should I be concerned >> about >>>>>>>> deopt and debugging going wrong? Furthermore, won't there be >>>> issues >>>>> in >>>>>>>> exception handlers? >>>>>>> As I understand, this just has to survive the far_call. >>>>>>> The call_RT in c1_Runtime then moves it into the >>>>>>> proper argument registers. This is just the handling of an >>>>>>> exception, and in these few instructions no java code is >>>>>>> executed, no safepoint is passed, so this should be fine. >>>>>>> >>>>>>> incremental diff: >>>>>>> iff -r 874f2b999ff6 >> src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp >>>>>>> --- a/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Mon Apr >> 16 >>>>> 15:17:20 2018 +0200 >>>>>>> +++ b/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Thu Apr >> 26 >>>>> 15:55:18 2018 +0200 >>>>>>> @@ -75,16 +75,16 @@ >>>>>>> } >>>>>>> >>>>>>> if (_index->is_cpu_register()) { >>>>>>> - __ mov(rscratch1, _index->as_register()); >>>>>>> + __ mov(r22, _index->as_register()); >>>>>>> } else { >>>>>>> - __ mov(rscratch1, _index->as_jint()); >>>>>>> + __ mov(r22, _index->as_jint()); >>>>>>> } >>>>>>> Runtime1::StubID stub_id; >>>>>>> if (_throw_index_out_of_bounds_exception) { >>>>>>> stub_id = Runtime1::throw_index_exception_id; >>>>>>> } else { >>>>>>> assert(_array != NULL, "sanity"); >>>>>>> - __ mov(rscratch2, _array->as_pointer_register()); >>>>>>> + __ mov(r23, _array->as_pointer_register()); >>>>>>> stub_id = Runtime1::throw_range_check_failed_id; >>>>>>> } >>>>>>> __ far_call(RuntimeAddress(Runtime1::entry_for(stub_id)), NULL, >>>>> rscratch2); >>>>>>> diff -r 874f2b999ff6 >> src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp >>>>>>> --- a/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Mon Apr >> 16 >>>>> 15:17:20 2018 +0200 >>>>>>> +++ b/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Thu Apr >> 26 >>>>> 15:55:18 2018 +0200 >>>>>>> @@ -327,7 +327,7 @@ >>>>>>> >>>>>>> >>>>>>> // target: the entry point of the method that creates and posts the >>>>> exception oop >>>>>>> -// has_argument: true if the exception needs an argument (passed >> in >>>>> rscratch1) >>>>>>> +// has_argument: true if the exception needs arguments (passed in >>>> r22 >>>>> and r23) >>>>>>> >>>>>>> OopMapSet* >> Runtime1::generate_exception_throw(StubAssembler* >>>>> sasm, address target, bool has_argument) { >>>>>>> // make a frame and preserve the caller's caller-save registers >>>>>>> @@ -336,7 +336,7 @@ >>>>>>> if (!has_argument) { >>>>>>> call_offset = __ call_RT(noreg, noreg, target); >>>>>>> } else { >>>>>>> - call_offset = __ call_RT(noreg, noreg, target, rscratch1, rscratch2); >>>>>>> + call_offset = __ call_RT(noreg, noreg, target, r22, r23); >>>>>>> } >>>>>>> OopMapSet* oop_maps = new OopMapSet(); >>>>>>> oop_maps->add_gc_map(call_offset, oop_map); >>>>>>> >>>>>>> Best regards, >>>>>>> Goetz. >>>>>>> >>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Stuart Monteith [mailto:stuart.monteith at linaro.org] >>>>>>>> Sent: Donnerstag, 26. April 2018 12:52 >>>>>>>> To: Andrew Haley >>>>>>>> Cc: Lindenmaier, Goetz ; hotspot- >>>>> compiler- >>>>>>>> dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; >> hotspot- >>>>>>>> runtime-dev at openjdk.java.net; aarch32-port- >> dev at openjdk.java.net >>>>>>>> Subject: Re: RFR(M): 8201593: Print array length in >>>>>>>> ArrayIndexOutOfBoundsException. >>>>>>>> >>>>>>>> Hi, >>>>>>>> Using c_rarg1 and c_rarg2 instead of rscratch1 and overwriting >>>>>>>> rscratch2 causes a SIGSEGV. >>>>>>>> Using r22 and r23 instead, the test ran successfully. >>>>>>>> >>>>>>>> In c1_CodeStubs_aarch64.cpp >>>>>>>> : >>>>>>>> 77 if (_index->is_cpu_register()) { >>>>>>>> 78 __ mov(r22, _index->as_register()); >>>>>>>> 79 } else { >>>>>>>> 80 __ mov(r22, _index->as_jint()); >>>>>>>> 81 } >>>>>>>> 82 Runtime1::StubID stub_id; >>>>>>>> 83 if (_throw_index_out_of_bounds_exception) { >>>>>>>> 84 stub_id = Runtime1::throw_index_exception_id; >>>>>>>> 85 } else { >>>>>>>> 86 assert(_array != NULL, "sanity"); >>>>>>>> 87 __ mov(r23, _array->as_pointer_register()); >>>>>>>> 88 stub_id = Runtime1::throw_range_check_failed_id; >>>>>>>> 89 } >>>>>>>> >>>>>>>> in c1_Runtime_aarch64.cpp: >>>>>>>> >>>>>>>> 336 if (!has_argument) { >>>>>>>> 337 call_offset = __ call_RT(noreg, noreg, target); >>>>>>>> 338 } else { >>>>>>>> 339 call_offset = __ call_RT(noreg, noreg, target, r22, r23); >>>>>>>> 340 } >>>>>>>> >>>>>>>> There is the possibility of overwriting live values though, aren't >>>>>>>> there? The registers are saved by call_RT. Should I be concerned >> about >>>>>>>> deopt and debugging going wrong? Furthermore, won't there be >>>> issues >>>>> in >>>>>>>> exception handlers? >>>>>>>> >>>>>>>> BR, >>>>>>>> Stuart >>>>>>>> >>>>>>>> >>>>>>>> On 25 April 2018 at 16:49, Stuart Monteith >>>> >>>>>>>> wrote: >>>>>>>>> Indeed - and that is what I am seeing. Usually no parameters are >>>> being >>>>>>>>> called with this pattern, or rscratch1, with the temporary variable >>>>>>>>> being changed to use rscratch2 in such circumstances. >>>>>>>>> I'll try c_rarg1 and c_rarg2 - they should pass straight through,if I >>>>>>>>> interpret the code correcting. >>>>>>>>> >>>>>>>>> On 25 April 2018 at 16:26, Andrew Haley wrote: >>>>>>>>>> On 04/25/2018 04:00 PM, Stuart Monteith wrote: >>>>>>>>>>> I'm not quite sure to solve this yet - we'll need to use the stack in >>>>>>>>>>> some safe way. >>>>>>>>>> >>>>>>>>>> It's not a great idea to pass arguments in rscratch1 or rscratch2. >>>> These >>>>>>>>>> registers are for use in macros and should be treated as volatile. >>>>> Given >>>>>>>>>> that you're throwing an exception, registers will be clobbered >>>>> anyway. >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Andrew Haley >>>>>>>>>> Java Platform Lead Engineer >>>>>>>>>> Red Hat UK Ltd. >>>>>>>>>> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From martin.doerr at sap.com Mon May 7 10:47:48 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 7 May 2018 10:47:48 +0000 Subject: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics In-Reply-To: References: <4a3b4d59c9f54234b0c0d325a3238cbc@sap.com> <372edaad-f92c-7071-036b-703c86ad25c1@oracle.com> <3c0af6430a8e43cbbd4178edc88a0512@sap.com> <0309694e4cf84438ab20b84cae51150a@sap.com> Message-ID: Hi David, thanks for the review. I've pushed it. About the seq_cst topic: I could imagine supporting memory_order_seq_cst, but IMHO it will only be really useful if we add something equivalent to C++11's x.load(std::memory_order_seq_cst) and x.store(y, std::memory_order_seq_cst). Having it only for the RMW-atomics without being able to tag load/store the same way sounds very poor to me. In contrast, the release/acquire semantics already make sense to me because they respect all other accesses, not only the "so tagged" ones. Best regards, Martin -----Original Message----- From: David Holmes [mailto:david.holmes at oracle.com] Sent: Montag, 7. Mai 2018 09:07 To: Doerr, Martin ; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics Hi Martin, On 5/05/2018 1:32 AM, Doerr, Martin wrote: > Hi David, > > thanks for your time. I understand that there's a lot of work at the moment. > > I have removed memory_order_consume again. We don't need it for our platforms and I guess nobody will miss it. > > I like your comment proposals, so I've updated them. > > New webrev: > http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.02/ Changes seem fine. Thanks. >> Though that raises the question why you haven't added memory_order_seq_cst? > I think that one could easily be used incorrectly within hotspot which would lead to problems on PPC. > "Atomic operations tagged memory_order_seq_cst ... establish a single total modification order of all atomic operations that are so tagged." [1] > It should work fine if all involved accesses are "so tagged". But hotspot doesn't use seq_cst loads, it uses things like OrderAccess::load_acquire. On PPC, GCC and xlC don't use a trailing sync for seq_cst stores and RMW-atomics so they only work well with leading sync loads which are not used in hotspot code. > I believe that there's a substantial amount of work to be done to get this right. I find this somewhat ironic/amusing. My main concern with adding in any access modes weaker than the existing "conservative" is precisely because it can be very hard to establish that they are in fact correct. :) But I don't follow your concern. The "total modification order" is in addition to the acquire/release semantics that are inherent in seq-cst. So if the seq-cst atomic-rmw operation is used on a variable that is otherwise accessed via load_acquire/release_store, then I would expect everything to be okay - else it serves no point to define seq-cst rmw in terms of acquire and release IMHO. I would expect these access modes to interact in the "obvious" way. Cheers, David > Thanks, > Martin > > > [1] https://en.cppreference.com/w/cpp/atomic/memory_order#Sequentially-consistent_ordering > > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Freitag, 4. Mai 2018 08:09 > To: Doerr, Martin ; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics > > Hi Martin, > > On 3/05/2018 11:03 PM, Doerr, Martin wrote: >> Hi Robbin, >> >> thank you for reviewing. >> >> Submission forest tests have passed in addition to SAP's nightly builds. I have 2 reviews, now, but I'll wait a little bit before pushing. > > Thanks for waiting. I think it important there is a general buy-in to > this effort, not just the official "get a review and push". Apologies > that it's taken me a while to get to this. > > memory_order_consume has been deprecated in C++17: > > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0371r1.html > > I would suggest we not add it to the VM. > > 40 enum atomic_memory_order { > 41 // We support most semantics like in C++11. > > I would say something like: > > // The modes that align with C++11 are intended to > // follow the same semantics. > > > 47 // We need to be more conservative than seq_cst on PPC64. > > Not sure the PPC64 comment is relevant here. memory_order_conservative > represents the pre-existing strong fencing requirements. It's not > intended to map to anything else, nor intended to be different across > platforms. > > Though that raises the question why you haven't added memory_order_seq_cst? > > Looking at all the platform code that adds the unused "order" parameter > ... I would not be surprised if the forthcoming compiler upgrades result > in some additional "unused" warnings. But I guess we'll just deal with > them as they are discovered. > > Otherwise all seems okay. > > Thanks, > David > > >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: Robbin Ehn [mailto:robbin.ehn at oracle.com] >> Sent: Donnerstag, 3. Mai 2018 14:29 >> To: Doerr, Martin ; Schmidt, Lutz ; David Holmes ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley (aph at redhat.com) ; John Paul Adrian Glaubitz >> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics >> >> On 2018-05-03 12:50, Doerr, Martin wrote: >>> Hi Lutz, >>> >>> thanks for reviewing. >>> >>> Can I get more reviews, please? >> >> I think this looks good, thanks for fixing! >> >> /Robbin >> >>> >>> Best regards, >>> Martin >>> >>> >>> -----Original Message----- >>> From: Schmidt, Lutz >>> Sent: Mittwoch, 2. Mai 2018 16:20 >>> To: Doerr, Martin ; David Holmes ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley (aph at redhat.com) ; John Paul Adrian Glaubitz >>> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics >>> >>> Hi Martin, >>> >>> your changes look good. >>> >>> For the additional synchronization on s390 in particular, I do not expect a serious performance impact. The checkpoint synchronization part has always been the expensive, but rarely required, part. PPC may be another story. We knoow that a full-blown sync really hurts. We will see... >>> >>> Thanks, >>> Lutz >>> >>> ?On 26.04.18, 16:23, "hotspot-runtime-dev on behalf of Doerr, Martin" wrote: >>> >>> Hi, >>> >>> I have renamed "cmpxchg_memory_order" to "atomic_memory_order" and added support to all Read-Modify-Write atomics with support for all C++11 semantics except seq_cst. The latter has issues on PPC64 so we're currently using our own memory_order_conservative instead, which is the default. >>> >>> Please review my new webrev: >>> http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.01/ >>> >>> We'll test Windows, MacOS, linux x86+ppc64+s390, AIX and Solaris SPARC. >>> It'd be great if somebody could check arm/aarch64 and zero. >>> >>> This change may enable optimizations for arm/aarch64 (not yet included). >>> >>> Best regards, >>> Martin >>> >>> >>> -----Original Message----- >>> From: Doerr, Martin >>> Sent: Donnerstag, 26. April 2018 11:20 >>> To: 'David Holmes' ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) >>> Subject: RE: RFR(M): 8202080: Introduce ordering semantics for Atomic::add >>> >>> Hi David, >>> >>> sounds like we are on the same page, now. I think we should have some kind of summary of what was analyzed. But that belongs to the other thread (JDK- 8154736). >>> >>> > The bugs in this kind of code are very subtle and rarely exposed through normal levels of testing. >>> Right. We have experienced this often enough. So it is definitely in our interest to have correct memory barriers. Mistakes will hit us (PPC64). So I appreciate that you insist on careful analysis. >>> >>> I'll prepare a new webrev with conservative semantics for all read-modify-write atomics by default. >>> >>> Best regards, >>> Martin >>> >>> >>> From david.holmes at oracle.com Mon May 7 12:16:25 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 7 May 2018 22:16:25 +1000 Subject: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics In-Reply-To: References: <4a3b4d59c9f54234b0c0d325a3238cbc@sap.com> <372edaad-f92c-7071-036b-703c86ad25c1@oracle.com> <3c0af6430a8e43cbbd4178edc88a0512@sap.com> <0309694e4cf84438ab20b84cae51150a@sap.com> Message-ID: <4f94006f-d83f-a4c3-8947-6afb898c99bf@oracle.com> On 7/05/2018 8:47 PM, Doerr, Martin wrote: > Hi David, > > thanks for the review. I've pushed it. > > About the seq_cst topic: > I could imagine supporting memory_order_seq_cst, but IMHO it will only be really useful if we add something equivalent to C++11's x.load(std::memory_order_seq_cst) and x.store(y, std::memory_order_seq_cst). > Having it only for the RMW-atomics without being able to tag load/store the same way sounds very poor to me. > In contrast, the release/acquire semantics already make sense to me because they respect all other accesses, not only the "so tagged" ones. I'm not sure how to reconcile that with the existing status-quo where we have "conservative" Atomic-rmw combined with acquire/release accesses. Is seq-cst in some sense stronger than our "conservative" ordering ?? Cheers, David > Best regards, > Martin > > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Montag, 7. Mai 2018 09:07 > To: Doerr, Martin ; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics > > Hi Martin, > > On 5/05/2018 1:32 AM, Doerr, Martin wrote: >> Hi David, >> >> thanks for your time. I understand that there's a lot of work at the moment. >> >> I have removed memory_order_consume again. We don't need it for our platforms and I guess nobody will miss it. >> >> I like your comment proposals, so I've updated them. >> >> New webrev: >> http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.02/ > > Changes seem fine. Thanks. > >>> Though that raises the question why you haven't added memory_order_seq_cst? >> I think that one could easily be used incorrectly within hotspot which would lead to problems on PPC. >> "Atomic operations tagged memory_order_seq_cst ... establish a single total modification order of all atomic operations that are so tagged." [1] >> It should work fine if all involved accesses are "so tagged". But hotspot doesn't use seq_cst loads, it uses things like OrderAccess::load_acquire. On PPC, GCC and xlC don't use a trailing sync for seq_cst stores and RMW-atomics so they only work well with leading sync loads which are not used in hotspot code. >> I believe that there's a substantial amount of work to be done to get this right. > > I find this somewhat ironic/amusing. My main concern with adding in any > access modes weaker than the existing "conservative" is precisely > because it can be very hard to establish that they are in fact correct. :) > > But I don't follow your concern. The "total modification order" is in > addition to the acquire/release semantics that are inherent in seq-cst. > So if the seq-cst atomic-rmw operation is used on a variable that is > otherwise accessed via load_acquire/release_store, then I would expect > everything to be okay - else it serves no point to define seq-cst rmw in > terms of acquire and release IMHO. I would expect these access modes to > interact in the "obvious" way. > > Cheers, > David > >> Thanks, >> Martin >> >> >> [1] https://en.cppreference.com/w/cpp/atomic/memory_order#Sequentially-consistent_ordering >> >> >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Freitag, 4. Mai 2018 08:09 >> To: Doerr, Martin ; hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics >> >> Hi Martin, >> >> On 3/05/2018 11:03 PM, Doerr, Martin wrote: >>> Hi Robbin, >>> >>> thank you for reviewing. >>> >>> Submission forest tests have passed in addition to SAP's nightly builds. I have 2 reviews, now, but I'll wait a little bit before pushing. >> >> Thanks for waiting. I think it important there is a general buy-in to >> this effort, not just the official "get a review and push". Apologies >> that it's taken me a while to get to this. >> >> memory_order_consume has been deprecated in C++17: >> >> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0371r1.html >> >> I would suggest we not add it to the VM. >> >> 40 enum atomic_memory_order { >> 41 // We support most semantics like in C++11. >> >> I would say something like: >> >> // The modes that align with C++11 are intended to >> // follow the same semantics. >> >> >> 47 // We need to be more conservative than seq_cst on PPC64. >> >> Not sure the PPC64 comment is relevant here. memory_order_conservative >> represents the pre-existing strong fencing requirements. It's not >> intended to map to anything else, nor intended to be different across >> platforms. >> >> Though that raises the question why you haven't added memory_order_seq_cst? >> >> Looking at all the platform code that adds the unused "order" parameter >> ... I would not be surprised if the forthcoming compiler upgrades result >> in some additional "unused" warnings. But I guess we'll just deal with >> them as they are discovered. >> >> Otherwise all seems okay. >> >> Thanks, >> David >> >> >>> Best regards, >>> Martin >>> >>> >>> -----Original Message----- >>> From: Robbin Ehn [mailto:robbin.ehn at oracle.com] >>> Sent: Donnerstag, 3. Mai 2018 14:29 >>> To: Doerr, Martin ; Schmidt, Lutz ; David Holmes ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley (aph at redhat.com) ; John Paul Adrian Glaubitz >>> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics >>> >>> On 2018-05-03 12:50, Doerr, Martin wrote: >>>> Hi Lutz, >>>> >>>> thanks for reviewing. >>>> >>>> Can I get more reviews, please? >>> >>> I think this looks good, thanks for fixing! >>> >>> /Robbin >>> >>>> >>>> Best regards, >>>> Martin >>>> >>>> >>>> -----Original Message----- >>>> From: Schmidt, Lutz >>>> Sent: Mittwoch, 2. Mai 2018 16:20 >>>> To: Doerr, Martin ; David Holmes ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley (aph at redhat.com) ; John Paul Adrian Glaubitz >>>> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics >>>> >>>> Hi Martin, >>>> >>>> your changes look good. >>>> >>>> For the additional synchronization on s390 in particular, I do not expect a serious performance impact. The checkpoint synchronization part has always been the expensive, but rarely required, part. PPC may be another story. We knoow that a full-blown sync really hurts. We will see... >>>> >>>> Thanks, >>>> Lutz >>>> >>>> ?On 26.04.18, 16:23, "hotspot-runtime-dev on behalf of Doerr, Martin" wrote: >>>> >>>> Hi, >>>> >>>> I have renamed "cmpxchg_memory_order" to "atomic_memory_order" and added support to all Read-Modify-Write atomics with support for all C++11 semantics except seq_cst. The latter has issues on PPC64 so we're currently using our own memory_order_conservative instead, which is the default. >>>> >>>> Please review my new webrev: >>>> http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.01/ >>>> >>>> We'll test Windows, MacOS, linux x86+ppc64+s390, AIX and Solaris SPARC. >>>> It'd be great if somebody could check arm/aarch64 and zero. >>>> >>>> This change may enable optimizations for arm/aarch64 (not yet included). >>>> >>>> Best regards, >>>> Martin >>>> >>>> >>>> -----Original Message----- >>>> From: Doerr, Martin >>>> Sent: Donnerstag, 26. April 2018 11:20 >>>> To: 'David Holmes' ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev ; Lindenmaier, Goetz ; Michihiro Horie (HORIE at jp.ibm.com) >>>> Subject: RE: RFR(M): 8202080: Introduce ordering semantics for Atomic::add >>>> >>>> Hi David, >>>> >>>> sounds like we are on the same page, now. I think we should have some kind of summary of what was analyzed. But that belongs to the other thread (JDK- 8154736). >>>> >>>> > The bugs in this kind of code are very subtle and rarely exposed through normal levels of testing. >>>> Right. We have experienced this often enough. So it is definitely in our interest to have correct memory barriers. Mistakes will hit us (PPC64). So I appreciate that you insist on careful analysis. >>>> >>>> I'll prepare a new webrev with conservative semantics for all read-modify-write atomics by default. >>>> >>>> Best regards, >>>> Martin >>>> >>>> >>>> From goetz.lindenmaier at sap.com Mon May 7 12:20:25 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 7 May 2018 12:20:25 +0000 Subject: RFR(M): 8201593: Print array length in ArrayIndexOutOfBoundsException. In-Reply-To: <243f21b7-ebb6-a372-1536-7d55c76acf01@oracle.com> References: <504bdd8469bb45cc8c5cf790d1f4b3fc@sap.com> <12e8ad2367484b0b88bcee4bcb77ac47@sap.com> <033adcb3-1ab0-094e-78cf-4062a8868d3e@redhat.com> <7767af56c6744e8aa5479a5dd9626936@sap.com> <9d2753f6975c4477912420a2fe459ef3@sap.com> <14107212c2664aee9965e9f9da8006ad@sap.com> <54e1796c-46e8-18be-2773-69636c4b4c0d@oracle.com> <7887adc187914fef9b4a77de28b1745f@sap.com> <243f21b7-ebb6-a372-1536-7d55c76acf01@oracle.com> Message-ID: Hi, Webrev withoug %i: http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/08/ it passed the jdk/submit testing. Best regards, Goetz. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Montag, 7. Mai 2018 11:00 > To: Lindenmaier, Goetz ; Stuart Monteith > > Cc: hotspot-compiler-dev at openjdk.java.net; aarch64-port- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; aarch32- > port-dev at openjdk.java.net > Subject: Re: RFR(M): 8201593: Print array length in > ArrayIndexOutOfBoundsException. > > Hi Goetz, > > On 7/05/2018 6:16 PM, Lindenmaier, Goetz wrote: > > Hi David, > > > > New webrev with the punctuation changed: > > http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/07/ > > For the punctuation see also my mail reply to Roger's mail. > > Okay. That seems okay. > > Only further oddity I noticed is the use of %i rather than %d. Use of %i > is very rare in hotspot. (I had to go and lookup what it means :) ). > > Thanks, > David > > >> -----Original Message----- > >> From: David Holmes [mailto:david.holmes at oracle.com] > >> Sent: Montag, 7. Mai 2018 09:30 > >> > >> Hi Goetz, > >> > >> On 4/05/2018 7:22 PM, Lindenmaier, Goetz wrote: > >>> Hi, > >>> > >>> thanks to Aleksey and Boris this is now also tested on arm. > >>> This final webrev contains some fixes needed in the arm files: > >>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06-arm/ > >>> > >>> David, can I consider this as finally reviewed? > >> > >> This follows on top of my +1 comments to Roger about the message > >> consistency and punctuation etc. > >> > >> Aside: Took me a while to realize the > >> throw_index_out_of_bounds_exception field was not a throw/don't- > throw > >> flag, but a throw AIOOBE or IOOBE flag! > > Yes, this is not very intuitive ... > > > >> Again note I can't comment on the detailed CPU specific code. > > The CPU code was reviewed by Martin Doerr, Stuart Monteith, Aleksey > Shipilev and Boris Ulasevich. > > > >> One further nit: > >> src/hotspot/cpu/aarch64/templateTable_aarch64.cpp > >> I don't like the > >> // ??? convention > >> comments. It suggests the code is not understood. I don't expect you to > >> fix existing ones but adding new ones doesn't seem good. > > I don't like that either, nor the design of using a convention here. > > The reviewers had trouble with that, too. > > But as the two values are handled similarly, I would to like to > > document it similarly, following the existing format. Stuart > > was fine with that, anyways. > > > > An improvement of the design how these values are handled > > would require changes on all platforms (it's similarly bad everywhere) > > and I don't like to do that in this scope. > > > > Best regards, > > Goetz. > > > > > >> > >> > >> Thanks, > >> David > >> > >>> Best regards, > >>> Goetz > >>> > >>> > >>> > >>>> -----Original Message----- > >>>> From: aarch64-port-dev [mailto:aarch64-port-dev- > >>>> bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz > >>>> Sent: Mittwoch, 2. Mai 2018 17:57 > >>>> To: Stuart Monteith > >>>> Cc: hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > >>>> dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; aarch32- > >> port- > >>>> dev at openjdk.java.net > >>>> Subject: [CAUTION] Re: [aarch64-port-dev ] RFR(M): 8201593: Print > array > >>>> length in ArrayIndexOutOfBoundsException. > >>>> > >>>> Hi, > >>>> > >>>> I needed to move the edit from c1_LIRGenerator_.cpp to > >>>> the shared file after "8201543: Modularize C1 GC barriers" > >>>> New webrev: > >>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06/ > >>>> > >>>> @Stuart > >>>> Thanks for testing! > >>>>> so as to accommodate the array pointer you are pushing onto the > stack? > >>>> Yes, what you are pointing out seems to be wrong, I changed it to '2'. > >>>> > >>>> Best regards, > >>>> Goetz. > >>>> > >>>> > >>>>> -----Original Message----- > >>>>> From: Stuart Monteith [mailto:stuart.monteith at linaro.org] > >>>>> Sent: Freitag, 27. April 2018 16:37 > >>>>> To: Lindenmaier, Goetz > >>>>> Cc: hotspot-compiler-dev at openjdk.java.net; aarch64-port- > >>>>> dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; > >> aarch32- > >>>>> port-dev at openjdk.java.net > >>>>> Subject: Re: RFR(M): 8201593: Print array length in > >>>>> ArrayIndexOutOfBoundsException. > >>>>> > >>>>> Hi, > >>>>> JTregs hasn't flagged any issues, so it should be ok. > >>>>> > >>>>> Regarding the 32-bit arm code, in "void > >>>>> RangeCheckStub::emit_code(LIR_Assembler* ce)" should: > >>>>> ce->verify_reserved_argument_area_size(1); > >>>>> be > >>>>> ce->verify_reserved_argument_area_size(2); > >>>>> > >>>>> so as to accommodate the array pointer you are pushing onto the > stack? > >>>>> > >>>>> I've not tested 32-bit arm. > >>>>> > >>>>> > >>>>> BR, > >>>>> Stuart > >>>>> > >>>>> On 26 April 2018 at 15:31, Stuart Monteith > > >>>>> wrote: > >>>>>> Thanks, I'm happy with that. > >>>>>> > >>>>>> The registers have a clean path to call_RT - r22 and r23 aren't used > >>>>>> inbetween. They are an arbitrary choice - c_rarg0 and c_rarg1 were > >>>>>> always going to cause problems. If _array->as_pointer_register() > >>>>>> and/or _index->as_register() or _index->as_jint() were the registers > >>>>>> we were using as parameters there would be trouble. However, > with > >>>>>> pd_last_allocatable_cpu_reg = 16, that shouldn't happen with > r22/23, > >>>>>> or indeed anything else in the range r17 to r28. > >>>>>> > >>>>>> I'm going to run all of JTRegs and seem what that produces now. > >>>>>> > >>>>>> BR, > >>>>>> Stuart > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> On 26 April 2018 at 15:14, Lindenmaier, Goetz > >>>>> wrote: > >>>>>>> Hi Stuart, > >>>>>>> > >>>>>>> thanks for fixing this! Webrev with your changes: > >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/05/ > >>>>>>> > >>>>>>>> There is the possibility of overwriting live values though, aren't > >>>>>>>> there? The registers are saved by call_RT. Should I be concerned > >> about > >>>>>>>> deopt and debugging going wrong? Furthermore, won't there be > >>>> issues > >>>>> in > >>>>>>>> exception handlers? > >>>>>>> As I understand, this just has to survive the far_call. > >>>>>>> The call_RT in c1_Runtime then moves it into the > >>>>>>> proper argument registers. This is just the handling of an > >>>>>>> exception, and in these few instructions no java code is > >>>>>>> executed, no safepoint is passed, so this should be fine. > >>>>>>> > >>>>>>> incremental diff: > >>>>>>> iff -r 874f2b999ff6 > >> src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp > >>>>>>> --- a/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Mon > Apr > >> 16 > >>>>> 15:17:20 2018 +0200 > >>>>>>> +++ b/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Thu > Apr > >> 26 > >>>>> 15:55:18 2018 +0200 > >>>>>>> @@ -75,16 +75,16 @@ > >>>>>>> } > >>>>>>> > >>>>>>> if (_index->is_cpu_register()) { > >>>>>>> - __ mov(rscratch1, _index->as_register()); > >>>>>>> + __ mov(r22, _index->as_register()); > >>>>>>> } else { > >>>>>>> - __ mov(rscratch1, _index->as_jint()); > >>>>>>> + __ mov(r22, _index->as_jint()); > >>>>>>> } > >>>>>>> Runtime1::StubID stub_id; > >>>>>>> if (_throw_index_out_of_bounds_exception) { > >>>>>>> stub_id = Runtime1::throw_index_exception_id; > >>>>>>> } else { > >>>>>>> assert(_array != NULL, "sanity"); > >>>>>>> - __ mov(rscratch2, _array->as_pointer_register()); > >>>>>>> + __ mov(r23, _array->as_pointer_register()); > >>>>>>> stub_id = Runtime1::throw_range_check_failed_id; > >>>>>>> } > >>>>>>> __ far_call(RuntimeAddress(Runtime1::entry_for(stub_id)), > NULL, > >>>>> rscratch2); > >>>>>>> diff -r 874f2b999ff6 > >> src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp > >>>>>>> --- a/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Mon > Apr > >> 16 > >>>>> 15:17:20 2018 +0200 > >>>>>>> +++ b/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Thu > Apr > >> 26 > >>>>> 15:55:18 2018 +0200 > >>>>>>> @@ -327,7 +327,7 @@ > >>>>>>> > >>>>>>> > >>>>>>> // target: the entry point of the method that creates and posts > the > >>>>> exception oop > >>>>>>> -// has_argument: true if the exception needs an argument > (passed > >> in > >>>>> rscratch1) > >>>>>>> +// has_argument: true if the exception needs arguments (passed > in > >>>> r22 > >>>>> and r23) > >>>>>>> > >>>>>>> OopMapSet* > >> Runtime1::generate_exception_throw(StubAssembler* > >>>>> sasm, address target, bool has_argument) { > >>>>>>> // make a frame and preserve the caller's caller-save registers > >>>>>>> @@ -336,7 +336,7 @@ > >>>>>>> if (!has_argument) { > >>>>>>> call_offset = __ call_RT(noreg, noreg, target); > >>>>>>> } else { > >>>>>>> - call_offset = __ call_RT(noreg, noreg, target, rscratch1, > rscratch2); > >>>>>>> + call_offset = __ call_RT(noreg, noreg, target, r22, r23); > >>>>>>> } > >>>>>>> OopMapSet* oop_maps = new OopMapSet(); > >>>>>>> oop_maps->add_gc_map(call_offset, oop_map); > >>>>>>> > >>>>>>> Best regards, > >>>>>>> Goetz. > >>>>>>> > >>>>>>> > >>>>>>>> -----Original Message----- > >>>>>>>> From: Stuart Monteith [mailto:stuart.monteith at linaro.org] > >>>>>>>> Sent: Donnerstag, 26. April 2018 12:52 > >>>>>>>> To: Andrew Haley > >>>>>>>> Cc: Lindenmaier, Goetz ; hotspot- > >>>>> compiler- > >>>>>>>> dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; > >> hotspot- > >>>>>>>> runtime-dev at openjdk.java.net; aarch32-port- > >> dev at openjdk.java.net > >>>>>>>> Subject: Re: RFR(M): 8201593: Print array length in > >>>>>>>> ArrayIndexOutOfBoundsException. > >>>>>>>> > >>>>>>>> Hi, > >>>>>>>> Using c_rarg1 and c_rarg2 instead of rscratch1 and overwriting > >>>>>>>> rscratch2 causes a SIGSEGV. > >>>>>>>> Using r22 and r23 instead, the test ran successfully. > >>>>>>>> > >>>>>>>> In c1_CodeStubs_aarch64.cpp > >>>>>>>> : > >>>>>>>> 77 if (_index->is_cpu_register()) { > >>>>>>>> 78 __ mov(r22, _index->as_register()); > >>>>>>>> 79 } else { > >>>>>>>> 80 __ mov(r22, _index->as_jint()); > >>>>>>>> 81 } > >>>>>>>> 82 Runtime1::StubID stub_id; > >>>>>>>> 83 if (_throw_index_out_of_bounds_exception) { > >>>>>>>> 84 stub_id = Runtime1::throw_index_exception_id; > >>>>>>>> 85 } else { > >>>>>>>> 86 assert(_array != NULL, "sanity"); > >>>>>>>> 87 __ mov(r23, _array->as_pointer_register()); > >>>>>>>> 88 stub_id = Runtime1::throw_range_check_failed_id; > >>>>>>>> 89 } > >>>>>>>> > >>>>>>>> in c1_Runtime_aarch64.cpp: > >>>>>>>> > >>>>>>>> 336 if (!has_argument) { > >>>>>>>> 337 call_offset = __ call_RT(noreg, noreg, target); > >>>>>>>> 338 } else { > >>>>>>>> 339 call_offset = __ call_RT(noreg, noreg, target, r22, r23); > >>>>>>>> 340 } > >>>>>>>> > >>>>>>>> There is the possibility of overwriting live values though, aren't > >>>>>>>> there? The registers are saved by call_RT. Should I be concerned > >> about > >>>>>>>> deopt and debugging going wrong? Furthermore, won't there be > >>>> issues > >>>>> in > >>>>>>>> exception handlers? > >>>>>>>> > >>>>>>>> BR, > >>>>>>>> Stuart > >>>>>>>> > >>>>>>>> > >>>>>>>> On 25 April 2018 at 16:49, Stuart Monteith > >>>> > >>>>>>>> wrote: > >>>>>>>>> Indeed - and that is what I am seeing. Usually no parameters are > >>>> being > >>>>>>>>> called with this pattern, or rscratch1, with the temporary variable > >>>>>>>>> being changed to use rscratch2 in such circumstances. > >>>>>>>>> I'll try c_rarg1 and c_rarg2 - they should pass straight through,if I > >>>>>>>>> interpret the code correcting. > >>>>>>>>> > >>>>>>>>> On 25 April 2018 at 16:26, Andrew Haley > wrote: > >>>>>>>>>> On 04/25/2018 04:00 PM, Stuart Monteith wrote: > >>>>>>>>>>> I'm not quite sure to solve this yet - we'll need to use the > stack in > >>>>>>>>>>> some safe way. > >>>>>>>>>> > >>>>>>>>>> It's not a great idea to pass arguments in rscratch1 or rscratch2. > >>>> These > >>>>>>>>>> registers are for use in macros and should be treated as > volatile. > >>>>> Given > >>>>>>>>>> that you're throwing an exception, registers will be clobbered > >>>>> anyway. > >>>>>>>>>> > >>>>>>>>>> -- > >>>>>>>>>> Andrew Haley > >>>>>>>>>> Java Platform Lead Engineer > >>>>>>>>>> Red Hat UK Ltd. > >>>>>>>>>> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From erik.osterlund at oracle.com Mon May 7 12:32:14 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 7 May 2018 14:32:14 +0200 Subject: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics In-Reply-To: <4f94006f-d83f-a4c3-8947-6afb898c99bf@oracle.com> References: <4a3b4d59c9f54234b0c0d325a3238cbc@sap.com> <372edaad-f92c-7071-036b-703c86ad25c1@oracle.com> <3c0af6430a8e43cbbd4178edc88a0512@sap.com> <0309694e4cf84438ab20b84cae51150a@sap.com> <4f94006f-d83f-a4c3-8947-6afb898c99bf@oracle.com> Message-ID: <5AF0474E.6090302@oracle.com> Hi David, On 2018-05-07 14:16, David Holmes wrote: > On 7/05/2018 8:47 PM, Doerr, Martin wrote: >> Hi David, >> >> thanks for the review. I've pushed it. >> >> About the seq_cst topic: >> I could imagine supporting memory_order_seq_cst, but IMHO it will >> only be really useful if we add something equivalent to C++11's >> x.load(std::memory_order_seq_cst) and x.store(y, >> std::memory_order_seq_cst). >> Having it only for the RMW-atomics without being able to tag >> load/store the same way sounds very poor to me. >> In contrast, the release/acquire semantics already make sense to me >> because they respect all other accesses, not only the "so tagged" ones. > > I'm not sure how to reconcile that with the existing status-quo where > we have "conservative" Atomic-rmw combined with acquire/release > accesses. Is seq-cst in some sense stronger than our "conservative" > ordering ?? Seq cst is strictly weaker than "conservative". Conservative has full trailing and leading fence. Seq cst basically has either full leading or trailing fence, depending on choice of bindings. That makes pairs of seq cst accesses totally ordered, but has potentially unintuitive consequences when mixing with weaker accesses. So ideally, we would have seq_cst loads and stores that could be used with those RMW operations. I personally don't see a problem with introducing seq cst as an ordering you can choose for both RMW and loads/stores, that binds to precisely what we already have today for JMM volatile. That way, my Access API can call straight into OrderAccess/Atomic for such uses, instead of doing the conditional fence dance outside. But I don't mind if we do not either. Naturally, as with every other access, we have to take care that we know what we are doing when introducing this in lock-free code. But that arguably goes for the other ordering semantics as well. Thanks, /Erik > Cheers, > David > >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Montag, 7. Mai 2018 09:07 >> To: Doerr, Martin ; >> hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for >> Atomic::add and other RMW atomics >> >> Hi Martin, >> >> On 5/05/2018 1:32 AM, Doerr, Martin wrote: >>> Hi David, >>> >>> thanks for your time. I understand that there's a lot of work at the >>> moment. >>> >>> I have removed memory_order_consume again. We don't need it for our >>> platforms and I guess nobody will miss it. >>> >>> I like your comment proposals, so I've updated them. >>> >>> New webrev: >>> http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.02/ >> >> Changes seem fine. Thanks. >> >>>> Though that raises the question why you haven't added >>>> memory_order_seq_cst? >>> I think that one could easily be used incorrectly within hotspot >>> which would lead to problems on PPC. >>> "Atomic operations tagged memory_order_seq_cst ... establish a >>> single total modification order of all atomic operations that are so >>> tagged." [1] >>> It should work fine if all involved accesses are "so tagged". But >>> hotspot doesn't use seq_cst loads, it uses things like >>> OrderAccess::load_acquire. On PPC, GCC and xlC don't use a trailing >>> sync for seq_cst stores and RMW-atomics so they only work well with >>> leading sync loads which are not used in hotspot code. >>> I believe that there's a substantial amount of work to be done to >>> get this right. >> >> I find this somewhat ironic/amusing. My main concern with adding in any >> access modes weaker than the existing "conservative" is precisely >> because it can be very hard to establish that they are in fact >> correct. :) >> >> But I don't follow your concern. The "total modification order" is in >> addition to the acquire/release semantics that are inherent in seq-cst. >> So if the seq-cst atomic-rmw operation is used on a variable that is >> otherwise accessed via load_acquire/release_store, then I would expect >> everything to be okay - else it serves no point to define seq-cst rmw in >> terms of acquire and release IMHO. I would expect these access modes to >> interact in the "obvious" way. >> >> Cheers, >> David >> >>> Thanks, >>> Martin >>> >>> >>> [1] >>> https://en.cppreference.com/w/cpp/atomic/memory_order#Sequentially-consistent_ordering >>> >>> >>> -----Original Message----- >>> From: David Holmes [mailto:david.holmes at oracle.com] >>> Sent: Freitag, 4. Mai 2018 08:09 >>> To: Doerr, Martin ; >>> hotspot-runtime-dev at openjdk.java.net >>> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for >>> Atomic::add and other RMW atomics >>> >>> Hi Martin, >>> >>> On 3/05/2018 11:03 PM, Doerr, Martin wrote: >>>> Hi Robbin, >>>> >>>> thank you for reviewing. >>>> >>>> Submission forest tests have passed in addition to SAP's nightly >>>> builds. I have 2 reviews, now, but I'll wait a little bit before >>>> pushing. >>> >>> Thanks for waiting. I think it important there is a general buy-in to >>> this effort, not just the official "get a review and push". Apologies >>> that it's taken me a while to get to this. >>> >>> memory_order_consume has been deprecated in C++17: >>> >>> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0371r1.html >>> >>> I would suggest we not add it to the VM. >>> >>> 40 enum atomic_memory_order { >>> 41 // We support most semantics like in C++11. >>> >>> I would say something like: >>> >>> // The modes that align with C++11 are intended to >>> // follow the same semantics. >>> >>> >>> 47 // We need to be more conservative than seq_cst on PPC64. >>> >>> Not sure the PPC64 comment is relevant here. memory_order_conservative >>> represents the pre-existing strong fencing requirements. It's not >>> intended to map to anything else, nor intended to be different across >>> platforms. >>> >>> Though that raises the question why you haven't added >>> memory_order_seq_cst? >>> >>> Looking at all the platform code that adds the unused "order" parameter >>> ... I would not be surprised if the forthcoming compiler upgrades >>> result >>> in some additional "unused" warnings. But I guess we'll just deal with >>> them as they are discovered. >>> >>> Otherwise all seems okay. >>> >>> Thanks, >>> David >>> >>> >>>> Best regards, >>>> Martin >>>> >>>> >>>> -----Original Message----- >>>> From: Robbin Ehn [mailto:robbin.ehn at oracle.com] >>>> Sent: Donnerstag, 3. Mai 2018 14:29 >>>> To: Doerr, Martin ; Schmidt, Lutz >>>> ; David Holmes ; >>>> Erik ?sterlund ; >>>> hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev >>>> ; Lindenmaier, Goetz ; >>>> Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley >>>> (aph at redhat.com) ; John Paul Adrian Glaubitz >>>> >>>> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for >>>> Atomic::add and other RMW atomics >>>> >>>> On 2018-05-03 12:50, Doerr, Martin wrote: >>>>> Hi Lutz, >>>>> >>>>> thanks for reviewing. >>>>> >>>>> Can I get more reviews, please? >>>> >>>> I think this looks good, thanks for fixing! >>>> >>>> /Robbin >>>> >>>>> >>>>> Best regards, >>>>> Martin >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: Schmidt, Lutz >>>>> Sent: Mittwoch, 2. Mai 2018 16:20 >>>>> To: Doerr, Martin ; David Holmes >>>>> ; Erik ?sterlund >>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>> Aleksey Shipilev ; Lindenmaier, Goetz >>>>> ; Michihiro Horie (HORIE at jp.ibm.com) >>>>> ; Andrew Haley (aph at redhat.com) >>>>> ; John Paul Adrian Glaubitz >>>>> >>>>> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for >>>>> Atomic::add and other RMW atomics >>>>> >>>>> Hi Martin, >>>>> >>>>> your changes look good. >>>>> >>>>> For the additional synchronization on s390 in particular, I do not >>>>> expect a serious performance impact. The checkpoint >>>>> synchronization part has always been the expensive, but rarely >>>>> required, part. PPC may be another story. We knoow that a >>>>> full-blown sync really hurts. We will see... >>>>> >>>>> Thanks, >>>>> Lutz >>>>> >>>>> ?On 26.04.18, 16:23, "hotspot-runtime-dev on behalf of Doerr, >>>>> Martin" >>>> martin.doerr at sap.com> wrote: >>>>> >>>>> Hi, >>>>> I have renamed "cmpxchg_memory_order" to >>>>> "atomic_memory_order" and added support to all Read-Modify-Write >>>>> atomics with support for all C++11 semantics except seq_cst. The >>>>> latter has issues on PPC64 so we're currently using our own >>>>> memory_order_conservative instead, which is the default. >>>>> Please review my new webrev: >>>>> http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.01/ >>>>> We'll test Windows, MacOS, linux x86+ppc64+s390, >>>>> AIX and Solaris SPARC. >>>>> It'd be great if somebody could check arm/aarch64 and zero. >>>>> This change may enable optimizations for >>>>> arm/aarch64 (not yet included). >>>>> Best regards, >>>>> Martin >>>>> -----Original Message----- >>>>> From: Doerr, Martin >>>>> Sent: Donnerstag, 26. April 2018 11:20 >>>>> To: 'David Holmes' ; Erik >>>>> ?sterlund ; >>>>> hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev >>>>> ; Lindenmaier, Goetz >>>>> ; Michihiro Horie (HORIE at jp.ibm.com) >>>>> >>>>> Subject: RE: RFR(M): 8202080: Introduce ordering semantics >>>>> for Atomic::add >>>>> Hi David, >>>>> sounds like we are on the same page, now. I think >>>>> we should have some kind of summary of what was analyzed. But that >>>>> belongs to the other thread (JDK- 8154736). >>>>> > The bugs in this kind of code are very subtle >>>>> and rarely exposed through normal levels of testing. >>>>> Right. We have experienced this often enough. So it is >>>>> definitely in our interest to have correct memory barriers. >>>>> Mistakes will hit us (PPC64). So I appreciate that you insist on >>>>> careful analysis. >>>>> I'll prepare a new webrev with conservative >>>>> semantics for all read-modify-write atomics by default. >>>>> Best regards, >>>>> Martin >>>>> From david.holmes at oracle.com Mon May 7 12:50:18 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 7 May 2018 22:50:18 +1000 Subject: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics In-Reply-To: <5AF0474E.6090302@oracle.com> References: <4a3b4d59c9f54234b0c0d325a3238cbc@sap.com> <372edaad-f92c-7071-036b-703c86ad25c1@oracle.com> <3c0af6430a8e43cbbd4178edc88a0512@sap.com> <0309694e4cf84438ab20b84cae51150a@sap.com> <4f94006f-d83f-a4c3-8947-6afb898c99bf@oracle.com> <5AF0474E.6090302@oracle.com> Message-ID: <79c15cb1-d5f6-989d-0056-0b69b8d5f962@oracle.com> Hi Erik, On 7/05/2018 10:32 PM, Erik ?sterlund wrote: > Hi David, > > On 2018-05-07 14:16, David Holmes wrote: >> On 7/05/2018 8:47 PM, Doerr, Martin wrote: >>> Hi David, >>> >>> thanks for the review. I've pushed it. >>> >>> About the seq_cst topic: >>> I could imagine supporting memory_order_seq_cst, but IMHO it will >>> only be really useful if we add something equivalent to C++11's >>> x.load(std::memory_order_seq_cst) and x.store(y, >>> std::memory_order_seq_cst). >>> Having it only for the RMW-atomics without being able to tag >>> load/store the same way sounds very poor to me. >>> In contrast, the release/acquire semantics already make sense to me >>> because they respect all other accesses, not only the "so tagged" ones. >> >> I'm not sure how to reconcile that with the existing status-quo where >> we have "conservative" Atomic-rmw combined with acquire/release >> accesses. Is seq-cst in some sense stronger than our "conservative" >> ordering ?? > > Seq cst is strictly weaker than "conservative". Conservative has full > trailing and leading fence. Seq cst basically has either full leading or > trailing fence, depending on choice of bindings. That makes pairs of seq > cst accesses totally ordered, but has potentially unintuitive > consequences when mixing with weaker accesses. So ideally, we would have > seq_cst loads and stores that could be used with those RMW operations. Well I'm confused. If seq-cst is weaker than what we have and seq-cst would be broken with only load-acquire/release-store, then doesn't that imply what we have with "conservative" is also broken?? > I personally don't see a problem with introducing seq cst as an ordering > you can choose for both RMW and loads/stores, that binds to precisely > what we already have today for JMM volatile. That way, my Access API can > call straight into OrderAccess/Atomic for such uses, instead of doing > the conditional fence dance outside. But I don't mind if we do not either. I'm not following what you are saying about JMM volatile here. The semantics for JMM volatile are best expressed as barriers required between pairs of accesses not the current way we deal with them as treating each access as if it were preceded and followed by another access that required the strongest barrier between them. David > Naturally, as with every other access, we have to take care that we know > what we are doing when introducing this in lock-free code. But that > arguably goes for the other ordering semantics as well. > > Thanks, > /Erik > >> Cheers, >> David >> >>> Best regards, >>> Martin >>> >>> >>> -----Original Message----- >>> From: David Holmes [mailto:david.holmes at oracle.com] >>> Sent: Montag, 7. Mai 2018 09:07 >>> To: Doerr, Martin ; >>> hotspot-runtime-dev at openjdk.java.net >>> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for >>> Atomic::add and other RMW atomics >>> >>> Hi Martin, >>> >>> On 5/05/2018 1:32 AM, Doerr, Martin wrote: >>>> Hi David, >>>> >>>> thanks for your time. I understand that there's a lot of work at the >>>> moment. >>>> >>>> I have removed memory_order_consume again. We don't need it for our >>>> platforms and I guess nobody will miss it. >>>> >>>> I like your comment proposals, so I've updated them. >>>> >>>> New webrev: >>>> http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.02/ >>> >>> Changes seem fine. Thanks. >>> >>>>> Though that raises the question why you haven't added >>>>> memory_order_seq_cst? >>>> I think that one could easily be used incorrectly within hotspot >>>> which would lead to problems on PPC. >>>> "Atomic operations tagged memory_order_seq_cst ... establish a >>>> single total modification order of all atomic operations that are so >>>> tagged." [1] >>>> It should work fine if all involved accesses are "so tagged". But >>>> hotspot doesn't use seq_cst loads, it uses things like >>>> OrderAccess::load_acquire. On PPC, GCC and xlC don't use a trailing >>>> sync for seq_cst stores and RMW-atomics so they only work well with >>>> leading sync loads which are not used in hotspot code. >>>> I believe that there's a substantial amount of work to be done to >>>> get this right. >>> >>> I find this somewhat ironic/amusing. My main concern with adding in any >>> access modes weaker than the existing "conservative" is precisely >>> because it can be very hard to establish that they are in fact >>> correct. :) >>> >>> But I don't follow your concern. The "total modification order" is in >>> addition to the acquire/release semantics that are inherent in seq-cst. >>> So if the seq-cst atomic-rmw operation is used on a variable that is >>> otherwise accessed via load_acquire/release_store, then I would expect >>> everything to be okay - else it serves no point to define seq-cst rmw in >>> terms of acquire and release IMHO. I would expect these access modes to >>> interact in the "obvious" way. >>> >>> Cheers, >>> David >>> >>>> Thanks, >>>> Martin >>>> >>>> >>>> [1] >>>> https://en.cppreference.com/w/cpp/atomic/memory_order#Sequentially-consistent_ordering >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: David Holmes [mailto:david.holmes at oracle.com] >>>> Sent: Freitag, 4. Mai 2018 08:09 >>>> To: Doerr, Martin ; >>>> hotspot-runtime-dev at openjdk.java.net >>>> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for >>>> Atomic::add and other RMW atomics >>>> >>>> Hi Martin, >>>> >>>> On 3/05/2018 11:03 PM, Doerr, Martin wrote: >>>>> Hi Robbin, >>>>> >>>>> thank you for reviewing. >>>>> >>>>> Submission forest tests have passed in addition to SAP's nightly >>>>> builds. I have 2 reviews, now, but I'll wait a little bit before >>>>> pushing. >>>> >>>> Thanks for waiting. I think it important there is a general buy-in to >>>> this effort, not just the official "get a review and push". Apologies >>>> that it's taken me a while to get to this. >>>> >>>> memory_order_consume has been deprecated in C++17: >>>> >>>> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0371r1.html >>>> >>>> I would suggest we not add it to the VM. >>>> >>>> ???? 40 enum atomic_memory_order { >>>> ???? 41?? // We support most semantics like in C++11. >>>> >>>> I would say something like: >>>> >>>> // The modes that align with C++11 are intended to >>>> // follow the same semantics. >>>> >>>> >>>> 47?? // We need to be more conservative than seq_cst on PPC64. >>>> >>>> Not sure the PPC64 comment is relevant here. memory_order_conservative >>>> represents the pre-existing strong fencing requirements. It's not >>>> intended to map to anything else, nor intended to be different across >>>> platforms. >>>> >>>> Though that raises the question why you haven't added >>>> memory_order_seq_cst? >>>> >>>> Looking at all the platform code that adds the unused "order" parameter >>>> ... I would not be surprised if the forthcoming compiler upgrades >>>> result >>>> in some additional "unused" warnings. But I guess we'll just deal with >>>> them as they are discovered. >>>> >>>> Otherwise all seems okay. >>>> >>>> Thanks, >>>> David >>>> >>>> >>>>> Best regards, >>>>> Martin >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: Robbin Ehn [mailto:robbin.ehn at oracle.com] >>>>> Sent: Donnerstag, 3. Mai 2018 14:29 >>>>> To: Doerr, Martin ; Schmidt, Lutz >>>>> ; David Holmes ; >>>>> Erik ?sterlund ; >>>>> hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev >>>>> ; Lindenmaier, Goetz ; >>>>> Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley >>>>> (aph at redhat.com) ; John Paul Adrian Glaubitz >>>>> >>>>> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for >>>>> Atomic::add and other RMW atomics >>>>> >>>>> On 2018-05-03 12:50, Doerr, Martin wrote: >>>>>> Hi Lutz, >>>>>> >>>>>> thanks for reviewing. >>>>>> >>>>>> Can I get more reviews, please? >>>>> >>>>> I think this looks good, thanks for fixing! >>>>> >>>>> /Robbin >>>>> >>>>>> >>>>>> Best regards, >>>>>> Martin >>>>>> >>>>>> >>>>>> -----Original Message----- >>>>>> From: Schmidt, Lutz >>>>>> Sent: Mittwoch, 2. Mai 2018 16:20 >>>>>> To: Doerr, Martin ; David Holmes >>>>>> ; Erik ?sterlund >>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>> Aleksey Shipilev ; Lindenmaier, Goetz >>>>>> ; Michihiro Horie (HORIE at jp.ibm.com) >>>>>> ; Andrew Haley (aph at redhat.com) >>>>>> ; John Paul Adrian Glaubitz >>>>>> >>>>>> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for >>>>>> Atomic::add and other RMW atomics >>>>>> >>>>>> Hi Martin, >>>>>> >>>>>> your changes look good. >>>>>> >>>>>> For the additional synchronization on s390 in particular, I do not >>>>>> expect a serious performance impact. The checkpoint >>>>>> synchronization part has always been the expensive, but rarely >>>>>> required, part. PPC may be another story. We knoow that a >>>>>> full-blown sync really hurts. We will see... >>>>>> >>>>>> Thanks, >>>>>> Lutz >>>>>> >>>>>> ?On 26.04.18, 16:23, "hotspot-runtime-dev on behalf of Doerr, >>>>>> Martin" >>>>> martin.doerr at sap.com> wrote: >>>>>> >>>>>> ??????? Hi, >>>>>> ??????????????? I have renamed "cmpxchg_memory_order" to >>>>>> "atomic_memory_order" and added support to all Read-Modify-Write >>>>>> atomics with support for all C++11 semantics except seq_cst. The >>>>>> latter has issues on PPC64 so we're currently using our own >>>>>> memory_order_conservative instead, which is the default. >>>>>> ??????????????? Please review my new webrev: >>>>>> http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.01/ >>>>>> ??????????????? We'll test Windows, MacOS, linux x86+ppc64+s390, >>>>>> AIX and Solaris SPARC. >>>>>> ??????? It'd be great if somebody could check arm/aarch64 and zero. >>>>>> ??????????????? This change may enable optimizations for >>>>>> arm/aarch64 (not yet included). >>>>>> ??????????????? Best regards, >>>>>> ??????? Martin >>>>>> ??????????????????????? -----Original Message----- >>>>>> ??????? From: Doerr, Martin >>>>>> ??????? Sent: Donnerstag, 26. April 2018 11:20 >>>>>> ??????? To: 'David Holmes' ; Erik >>>>>> ?sterlund ; >>>>>> hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev >>>>>> ; Lindenmaier, Goetz >>>>>> ; Michihiro Horie (HORIE at jp.ibm.com) >>>>>> >>>>>> ??????? Subject: RE: RFR(M): 8202080: Introduce ordering semantics >>>>>> for Atomic::add >>>>>> ??????????????? Hi David, >>>>>> ??????????????? sounds like we are on the same page, now. I think >>>>>> we should have some kind of summary of what was analyzed. But that >>>>>> belongs to the other thread (JDK- 8154736). >>>>>> ??????????????? > The bugs in this kind of code are very subtle >>>>>> and rarely exposed through normal levels of testing. >>>>>> ??????? Right. We have experienced this often enough. So it is >>>>>> definitely in our interest to have correct memory barriers. >>>>>> Mistakes will hit us (PPC64). So I appreciate that you insist on >>>>>> careful analysis. >>>>>> ??????????????? I'll prepare a new webrev with conservative >>>>>> semantics for all read-modify-write atomics by default. >>>>>> ??????????????? Best regards, >>>>>> ??????? Martin >>>>>> > From coleen.phillimore at oracle.com Mon May 7 13:29:15 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 7 May 2018 09:29:15 -0400 Subject: RFR(xxs): 8185034: Cleanup and consolidate Metaspace coding In-Reply-To: References: Message-ID: <8f5456bb-303a-6b1e-cb3e-174c2b2de629@oracle.com> This looks good to me.? Thank you for all the metaspace cleanups and for looking at the logging in detail. Coleen On 5/5/18 3:58 PM, Thomas St?fe wrote: > Hi all, > > May I get reviews please? > > Bug: https://bugs.openjdk.java.net/browse/JDK-8185034 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8185034-cleanup-and-consolidate-metaspace-coding/webrev.00/webrev/ > > Despite the title this item is quite unexciting. > > Most of the cleanups originally targeted by this item did already > happen as part of other enhancements > (https://bugs.openjdk.java.net/browse/JDK-8199430, > https://bugs.openjdk.java.net/browse/JDK-8202638). > > What is left are some smallish cleanups: > - removed some functions which were unused > - small changes to ~SpaceManager to avoid logging the same info twice > (chunk manager state) > - removed all LogStream::cr() calls which were needed because > ~LogStream() did not autoflush; but 8202303 did fix that, so the cr() > can go. > > Note that there are more things one could clean up, which may happen > in the future. Especially the split-up of metaspace.cpp, which is > still open (https://bugs.openjdk.java.net/browse/JDK-8176808). But for > now I would like to close this particular issue. > > Thanks! > > ..Thomas From Roger.Riggs at Oracle.com Mon May 7 13:44:36 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 7 May 2018 09:44:36 -0400 Subject: RFR(M): 8201593: Print array length in ArrayIndexOutOfBoundsException. In-Reply-To: References: <504bdd8469bb45cc8c5cf790d1f4b3fc@sap.com> <12e8ad2367484b0b88bcee4bcb77ac47@sap.com> <033adcb3-1ab0-094e-78cf-4062a8868d3e@redhat.com> <7767af56c6744e8aa5479a5dd9626936@sap.com> <9d2753f6975c4477912420a2fe459ef3@sap.com> <14107212c2664aee9965e9f9da8006ad@sap.com> <54e1796c-46e8-18be-2773-69636c4b4c0d@oracle.com> <7887adc187914fef9b4a77de28b1745f@sap.com> <243f21b7-ebb6-a372-1536-7d55c76acf01@oracle.com> Message-ID: <95137992-18f2-fee2-81f2-aec67ccf0e2e@Oracle.com> Hi Goetz, Looks good to me (as to the exception message format). Thanks, Roger On 5/7/2018 8:20 AM, Lindenmaier, Goetz wrote: > Hi, > > Webrev withoug %i: > http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/08/ > it passed the jdk/submit testing. > > Best regards, > Goetz. > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Montag, 7. Mai 2018 11:00 >> To: Lindenmaier, Goetz ; Stuart Monteith >> >> Cc: hotspot-compiler-dev at openjdk.java.net; aarch64-port- >> dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; aarch32- >> port-dev at openjdk.java.net >> Subject: Re: RFR(M): 8201593: Print array length in >> ArrayIndexOutOfBoundsException. >> >> Hi Goetz, >> >> On 7/05/2018 6:16 PM, Lindenmaier, Goetz wrote: >>> Hi David, >>> >>> New webrev with the punctuation changed: >>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/07/ >>> For the punctuation see also my mail reply to Roger's mail. >> Okay. That seems okay. >> >> Only further oddity I noticed is the use of %i rather than %d. Use of %i >> is very rare in hotspot. (I had to go and lookup what it means :) ). >> >> Thanks, >> David >> >>>> -----Original Message----- >>>> From: David Holmes [mailto:david.holmes at oracle.com] >>>> Sent: Montag, 7. Mai 2018 09:30 >>>> >>>> Hi Goetz, >>>> >>>> On 4/05/2018 7:22 PM, Lindenmaier, Goetz wrote: >>>>> Hi, >>>>> >>>>> thanks to Aleksey and Boris this is now also tested on arm. >>>>> This final webrev contains some fixes needed in the arm files: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06-arm/ >>>>> >>>>> David, can I consider this as finally reviewed? >>>> This follows on top of my +1 comments to Roger about the message >>>> consistency and punctuation etc. >>>> >>>> Aside: Took me a while to realize the >>>> throw_index_out_of_bounds_exception field was not a throw/don't- >> throw >>>> flag, but a throw AIOOBE or IOOBE flag! >>> Yes, this is not very intuitive ... >>> >>>> Again note I can't comment on the detailed CPU specific code. >>> The CPU code was reviewed by Martin Doerr, Stuart Monteith, Aleksey >> Shipilev and Boris Ulasevich. >>>> One further nit: >>>> src/hotspot/cpu/aarch64/templateTable_aarch64.cpp >>>> I don't like the >>>> // ??? convention >>>> comments. It suggests the code is not understood. I don't expect you to >>>> fix existing ones but adding new ones doesn't seem good. >>> I don't like that either, nor the design of using a convention here. >>> The reviewers had trouble with that, too. >>> But as the two values are handled similarly, I would to like to >>> document it similarly, following the existing format. Stuart >>> was fine with that, anyways. >>> >>> An improvement of the design how these values are handled >>> would require changes on all platforms (it's similarly bad everywhere) >>> and I don't like to do that in this scope. >>> >>> Best regards, >>> Goetz. >>> >>> >>>> >>>> Thanks, >>>> David >>>> >>>>> Best regards, >>>>> Goetz >>>>> >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: aarch64-port-dev [mailto:aarch64-port-dev- >>>>>> bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz >>>>>> Sent: Mittwoch, 2. Mai 2018 17:57 >>>>>> To: Stuart Monteith >>>>>> Cc: hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- >>>>>> dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; aarch32- >>>> port- >>>>>> dev at openjdk.java.net >>>>>> Subject: [CAUTION] Re: [aarch64-port-dev ] RFR(M): 8201593: Print >> array >>>>>> length in ArrayIndexOutOfBoundsException. >>>>>> >>>>>> Hi, >>>>>> >>>>>> I needed to move the edit from c1_LIRGenerator_.cpp to >>>>>> the shared file after "8201543: Modularize C1 GC barriers" >>>>>> New webrev: >>>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06/ >>>>>> >>>>>> @Stuart >>>>>> Thanks for testing! >>>>>>> so as to accommodate the array pointer you are pushing onto the >> stack? >>>>>> Yes, what you are pointing out seems to be wrong, I changed it to '2'. >>>>>> >>>>>> Best regards, >>>>>> Goetz. >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Stuart Monteith [mailto:stuart.monteith at linaro.org] >>>>>>> Sent: Freitag, 27. April 2018 16:37 >>>>>>> To: Lindenmaier, Goetz >>>>>>> Cc: hotspot-compiler-dev at openjdk.java.net; aarch64-port- >>>>>>> dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; >>>> aarch32- >>>>>>> port-dev at openjdk.java.net >>>>>>> Subject: Re: RFR(M): 8201593: Print array length in >>>>>>> ArrayIndexOutOfBoundsException. >>>>>>> >>>>>>> Hi, >>>>>>> JTregs hasn't flagged any issues, so it should be ok. >>>>>>> >>>>>>> Regarding the 32-bit arm code, in "void >>>>>>> RangeCheckStub::emit_code(LIR_Assembler* ce)" should: >>>>>>> ce->verify_reserved_argument_area_size(1); >>>>>>> be >>>>>>> ce->verify_reserved_argument_area_size(2); >>>>>>> >>>>>>> so as to accommodate the array pointer you are pushing onto the >> stack? >>>>>>> I've not tested 32-bit arm. >>>>>>> >>>>>>> >>>>>>> BR, >>>>>>> Stuart >>>>>>> >>>>>>> On 26 April 2018 at 15:31, Stuart Monteith >> >>>>>>> wrote: >>>>>>>> Thanks, I'm happy with that. >>>>>>>> >>>>>>>> The registers have a clean path to call_RT - r22 and r23 aren't used >>>>>>>> inbetween. They are an arbitrary choice - c_rarg0 and c_rarg1 were >>>>>>>> always going to cause problems. If _array->as_pointer_register() >>>>>>>> and/or _index->as_register() or _index->as_jint() were the registers >>>>>>>> we were using as parameters there would be trouble. However, >> with >>>>>>>> pd_last_allocatable_cpu_reg = 16, that shouldn't happen with >> r22/23, >>>>>>>> or indeed anything else in the range r17 to r28. >>>>>>>> >>>>>>>> I'm going to run all of JTRegs and seem what that produces now. >>>>>>>> >>>>>>>> BR, >>>>>>>> Stuart >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 26 April 2018 at 15:14, Lindenmaier, Goetz >>>>>>> wrote: >>>>>>>>> Hi Stuart, >>>>>>>>> >>>>>>>>> thanks for fixing this! Webrev with your changes: >>>>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/05/ >>>>>>>>> >>>>>>>>>> There is the possibility of overwriting live values though, aren't >>>>>>>>>> there? The registers are saved by call_RT. Should I be concerned >>>> about >>>>>>>>>> deopt and debugging going wrong? Furthermore, won't there be >>>>>> issues >>>>>>> in >>>>>>>>>> exception handlers? >>>>>>>>> As I understand, this just has to survive the far_call. >>>>>>>>> The call_RT in c1_Runtime then moves it into the >>>>>>>>> proper argument registers. This is just the handling of an >>>>>>>>> exception, and in these few instructions no java code is >>>>>>>>> executed, no safepoint is passed, so this should be fine. >>>>>>>>> >>>>>>>>> incremental diff: >>>>>>>>> iff -r 874f2b999ff6 >>>> src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp >>>>>>>>> --- a/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Mon >> Apr >>>> 16 >>>>>>> 15:17:20 2018 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Thu >> Apr >>>> 26 >>>>>>> 15:55:18 2018 +0200 >>>>>>>>> @@ -75,16 +75,16 @@ >>>>>>>>> } >>>>>>>>> >>>>>>>>> if (_index->is_cpu_register()) { >>>>>>>>> - __ mov(rscratch1, _index->as_register()); >>>>>>>>> + __ mov(r22, _index->as_register()); >>>>>>>>> } else { >>>>>>>>> - __ mov(rscratch1, _index->as_jint()); >>>>>>>>> + __ mov(r22, _index->as_jint()); >>>>>>>>> } >>>>>>>>> Runtime1::StubID stub_id; >>>>>>>>> if (_throw_index_out_of_bounds_exception) { >>>>>>>>> stub_id = Runtime1::throw_index_exception_id; >>>>>>>>> } else { >>>>>>>>> assert(_array != NULL, "sanity"); >>>>>>>>> - __ mov(rscratch2, _array->as_pointer_register()); >>>>>>>>> + __ mov(r23, _array->as_pointer_register()); >>>>>>>>> stub_id = Runtime1::throw_range_check_failed_id; >>>>>>>>> } >>>>>>>>> __ far_call(RuntimeAddress(Runtime1::entry_for(stub_id)), >> NULL, >>>>>>> rscratch2); >>>>>>>>> diff -r 874f2b999ff6 >>>> src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp >>>>>>>>> --- a/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Mon >> Apr >>>> 16 >>>>>>> 15:17:20 2018 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Thu >> Apr >>>> 26 >>>>>>> 15:55:18 2018 +0200 >>>>>>>>> @@ -327,7 +327,7 @@ >>>>>>>>> >>>>>>>>> >>>>>>>>> // target: the entry point of the method that creates and posts >> the >>>>>>> exception oop >>>>>>>>> -// has_argument: true if the exception needs an argument >> (passed >>>> in >>>>>>> rscratch1) >>>>>>>>> +// has_argument: true if the exception needs arguments (passed >> in >>>>>> r22 >>>>>>> and r23) >>>>>>>>> OopMapSet* >>>> Runtime1::generate_exception_throw(StubAssembler* >>>>>>> sasm, address target, bool has_argument) { >>>>>>>>> // make a frame and preserve the caller's caller-save registers >>>>>>>>> @@ -336,7 +336,7 @@ >>>>>>>>> if (!has_argument) { >>>>>>>>> call_offset = __ call_RT(noreg, noreg, target); >>>>>>>>> } else { >>>>>>>>> - call_offset = __ call_RT(noreg, noreg, target, rscratch1, >> rscratch2); >>>>>>>>> + call_offset = __ call_RT(noreg, noreg, target, r22, r23); >>>>>>>>> } >>>>>>>>> OopMapSet* oop_maps = new OopMapSet(); >>>>>>>>> oop_maps->add_gc_map(call_offset, oop_map); >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> Goetz. >>>>>>>>> >>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Stuart Monteith [mailto:stuart.monteith at linaro.org] >>>>>>>>>> Sent: Donnerstag, 26. April 2018 12:52 >>>>>>>>>> To: Andrew Haley >>>>>>>>>> Cc: Lindenmaier, Goetz ; hotspot- >>>>>>> compiler- >>>>>>>>>> dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; >>>> hotspot- >>>>>>>>>> runtime-dev at openjdk.java.net; aarch32-port- >>>> dev at openjdk.java.net >>>>>>>>>> Subject: Re: RFR(M): 8201593: Print array length in >>>>>>>>>> ArrayIndexOutOfBoundsException. >>>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> Using c_rarg1 and c_rarg2 instead of rscratch1 and overwriting >>>>>>>>>> rscratch2 causes a SIGSEGV. >>>>>>>>>> Using r22 and r23 instead, the test ran successfully. >>>>>>>>>> >>>>>>>>>> In c1_CodeStubs_aarch64.cpp >>>>>>>>>> : >>>>>>>>>> 77 if (_index->is_cpu_register()) { >>>>>>>>>> 78 __ mov(r22, _index->as_register()); >>>>>>>>>> 79 } else { >>>>>>>>>> 80 __ mov(r22, _index->as_jint()); >>>>>>>>>> 81 } >>>>>>>>>> 82 Runtime1::StubID stub_id; >>>>>>>>>> 83 if (_throw_index_out_of_bounds_exception) { >>>>>>>>>> 84 stub_id = Runtime1::throw_index_exception_id; >>>>>>>>>> 85 } else { >>>>>>>>>> 86 assert(_array != NULL, "sanity"); >>>>>>>>>> 87 __ mov(r23, _array->as_pointer_register()); >>>>>>>>>> 88 stub_id = Runtime1::throw_range_check_failed_id; >>>>>>>>>> 89 } >>>>>>>>>> >>>>>>>>>> in c1_Runtime_aarch64.cpp: >>>>>>>>>> >>>>>>>>>> 336 if (!has_argument) { >>>>>>>>>> 337 call_offset = __ call_RT(noreg, noreg, target); >>>>>>>>>> 338 } else { >>>>>>>>>> 339 call_offset = __ call_RT(noreg, noreg, target, r22, r23); >>>>>>>>>> 340 } >>>>>>>>>> >>>>>>>>>> There is the possibility of overwriting live values though, aren't >>>>>>>>>> there? The registers are saved by call_RT. Should I be concerned >>>> about >>>>>>>>>> deopt and debugging going wrong? Furthermore, won't there be >>>>>> issues >>>>>>> in >>>>>>>>>> exception handlers? >>>>>>>>>> >>>>>>>>>> BR, >>>>>>>>>> Stuart >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 25 April 2018 at 16:49, Stuart Monteith >>>>>> >>>>>>>>>> wrote: >>>>>>>>>>> Indeed - and that is what I am seeing. Usually no parameters are >>>>>> being >>>>>>>>>>> called with this pattern, or rscratch1, with the temporary variable >>>>>>>>>>> being changed to use rscratch2 in such circumstances. >>>>>>>>>>> I'll try c_rarg1 and c_rarg2 - they should pass straight through,if I >>>>>>>>>>> interpret the code correcting. >>>>>>>>>>> >>>>>>>>>>> On 25 April 2018 at 16:26, Andrew Haley >> wrote: >>>>>>>>>>>> On 04/25/2018 04:00 PM, Stuart Monteith wrote: >>>>>>>>>>>>> I'm not quite sure to solve this yet - we'll need to use the >> stack in >>>>>>>>>>>>> some safe way. >>>>>>>>>>>> It's not a great idea to pass arguments in rscratch1 or rscratch2. >>>>>> These >>>>>>>>>>>> registers are for use in macros and should be treated as >> volatile. >>>>>>> Given >>>>>>>>>>>> that you're throwing an exception, registers will be clobbered >>>>>>> anyway. >>>>>>>>>>>> -- >>>>>>>>>>>> Andrew Haley >>>>>>>>>>>> Java Platform Lead Engineer >>>>>>>>>>>> Red Hat UK Ltd. >>>>>>>>>>>> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From thomas.stuefe at gmail.com Mon May 7 13:48:42 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 7 May 2018 15:48:42 +0200 Subject: RFR(xxs): 8185034: Cleanup and consolidate Metaspace coding In-Reply-To: <8f5456bb-303a-6b1e-cb3e-174c2b2de629@oracle.com> References: <8f5456bb-303a-6b1e-cb3e-174c2b2de629@oracle.com> Message-ID: Thank you Coleen! On Mon, May 7, 2018 at 3:29 PM, wrote: > This looks good to me. Thank you for all the metaspace cleanups and for > looking at the logging in detail. > Coleen > > > > On 5/5/18 3:58 PM, Thomas St?fe wrote: >> >> Hi all, >> >> May I get reviews please? >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8185034 >> Webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/8185034-cleanup-and-consolidate-metaspace-coding/webrev.00/webrev/ >> >> Despite the title this item is quite unexciting. >> >> Most of the cleanups originally targeted by this item did already >> happen as part of other enhancements >> (https://bugs.openjdk.java.net/browse/JDK-8199430, >> https://bugs.openjdk.java.net/browse/JDK-8202638). >> >> What is left are some smallish cleanups: >> - removed some functions which were unused >> - small changes to ~SpaceManager to avoid logging the same info twice >> (chunk manager state) >> - removed all LogStream::cr() calls which were needed because >> ~LogStream() did not autoflush; but 8202303 did fix that, so the cr() >> can go. >> >> Note that there are more things one could clean up, which may happen >> in the future. Especially the split-up of metaspace.cpp, which is >> still open (https://bugs.openjdk.java.net/browse/JDK-8176808). But for >> now I would like to close this particular issue. >> >> Thanks! >> >> ..Thomas > > From goetz.lindenmaier at sap.com Mon May 7 14:08:26 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 7 May 2018 14:08:26 +0000 Subject: RFR(M): 8201593: Print array length in ArrayIndexOutOfBoundsException. In-Reply-To: <95137992-18f2-fee2-81f2-aec67ccf0e2e@Oracle.com> References: <504bdd8469bb45cc8c5cf790d1f4b3fc@sap.com> <12e8ad2367484b0b88bcee4bcb77ac47@sap.com> <033adcb3-1ab0-094e-78cf-4062a8868d3e@redhat.com> <7767af56c6744e8aa5479a5dd9626936@sap.com> <9d2753f6975c4477912420a2fe459ef3@sap.com> <14107212c2664aee9965e9f9da8006ad@sap.com> <54e1796c-46e8-18be-2773-69636c4b4c0d@oracle.com> <7887adc187914fef9b4a77de28b1745f@sap.com> <243f21b7-ebb6-a372-1536-7d55c76acf01@oracle.com> <95137992-18f2-fee2-81f2-aec67ccf0e2e@Oracle.com> Message-ID: <884d007c11f4457e98660c80434133ae@sap.com> Hi Roger, thanks a lot! Best regards, Goetz. > -----Original Message----- > From: Roger Riggs [mailto:Roger.Riggs at Oracle.com] > Sent: Montag, 7. Mai 2018 15:45 > To: Lindenmaier, Goetz > Cc: hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; aarch32-port- > dev at openjdk.java.net > Subject: Re: RFR(M): 8201593: Print array length in > ArrayIndexOutOfBoundsException. > > Hi Goetz, > > Looks good to me (as to the exception message format). > > Thanks, Roger > > > On 5/7/2018 8:20 AM, Lindenmaier, Goetz wrote: > > > Hi, > > Webrev withoug %i: > http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/08/ > it passed the jdk/submit testing. > > Best regards, > Goetz. > > > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Montag, 7. Mai 2018 11:00 > To: Lindenmaier, Goetz > ; Stuart Monteith > > > Cc: hotspot-compiler-dev at openjdk.java.net > ; aarch64-port- > dev at openjdk.java.net ; > hotspot-runtime-dev at openjdk.java.net dev at openjdk.java.net> ; aarch32- > port-dev at openjdk.java.net dev at openjdk.java.net> > Subject: Re: RFR(M): 8201593: Print array length in > ArrayIndexOutOfBoundsException. > > Hi Goetz, > > On 7/05/2018 6:16 PM, Lindenmaier, Goetz wrote: > > Hi David, > > New webrev with the punctuation changed: > http://cr.openjdk.java.net/~goetz/wr18/8201593- > lenInAIOOB/07/ > For the punctuation see also my mail reply to Roger's > mail. > > > Okay. That seems okay. > > Only further oddity I noticed is the use of %i rather than %d. > Use of %i > is very rare in hotspot. (I had to go and lookup what it means > :) ). > > Thanks, > David > > > -----Original Message----- > From: David Holmes > [mailto:david.holmes at oracle.com] > Sent: Montag, 7. Mai 2018 09:30 > > Hi Goetz, > > On 4/05/2018 7:22 PM, Lindenmaier, Goetz > wrote: > > Hi, > > thanks to Aleksey and Boris this is now > also tested on arm. > This final webrev contains some fixes > needed in the arm files: > > http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06- > arm/ > > David, can I consider this as finally > reviewed? > > > This follows on top of my +1 comments to > Roger about the message > consistency and punctuation etc. > > Aside: Took me a while to realize the > throw_index_out_of_bounds_exception field > was not a throw/don't- > > throw > > flag, but a throw AIOOBE or IOOBE flag! > > Yes, this is not very intuitive ... > > > Again note I can't comment on the detailed > CPU specific code. > > The CPU code was reviewed by Martin Doerr, Stuart > Monteith, Aleksey > > Shipilev and Boris Ulasevich. > > > > One further nit: > > src/hotspot/cpu/aarch64/templateTable_aarch64.cpp > I don't like the > // ??? convention > comments. It suggests the code is not > understood. I don't expect you to > fix existing ones but adding new ones doesn't > seem good. > > I don't like that either, nor the design of using a > convention here. > The reviewers had trouble with that, too. > But as the two values are handled similarly, I would to > like to > document it similarly, following the existing format. > Stuart > was fine with that, anyways. > > An improvement of the design how these values are > handled > would require changes on all platforms (it's similarly > bad everywhere) > and I don't like to do that in this scope. > > Best regards, > Goetz. > > > > > > Thanks, > David > > > Best regards, > Goetz > > > > > -----Original Message----- > From: aarch64-port-dev > [mailto:aarch64-port-dev- > bounces at openjdk.java.net > ] On Behalf Of Lindenmaier, Goetz > Sent: Mittwoch, 2. Mai 2018 17:57 > To: Stuart Monteith > > Cc: hotspot-compiler- > dev at openjdk.java.net ; > hotspot-runtime- > dev at openjdk.java.net > ; aarch64-port-dev at openjdk.java.net > ; aarch32- > > port- > > dev at openjdk.java.net > > Subject: [CAUTION] Re: [aarch64-port- > dev ] RFR(M): 8201593: Print > > array > > length in > ArrayIndexOutOfBoundsException. > > Hi, > > I needed to move the edit from > c1_LIRGenerator_.cpp to > the shared file after "8201543: > Modularize C1 GC barriers" > New webrev: > > http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06/ > > @Stuart > Thanks for testing! > > so as to accommodate the array > pointer you are pushing onto the > > stack? > > Yes, what you are pointing out seems > to be wrong, I changed it to '2'. > > Best regards, > Goetz. > > > > -----Original Message----- > From: Stuart Monteith > [mailto:stuart.monteith at linaro.org] > Sent: Freitag, 27. April 2018 16:37 > To: Lindenmaier, Goetz > > Cc: hotspot-compiler- > dev at openjdk.java.net ; > aarch64-port- > dev at openjdk.java.net > ; hotspot-runtime-dev at openjdk.java.net > ; > > aarch32- > > port-dev at openjdk.java.net > > Subject: Re: RFR(M): 8201593: Print > array length in > ArrayIndexOutOfBoundsException. > > Hi, > JTregs hasn't flagged any issues, so > it should be ok. > > Regarding the 32-bit arm code, in > "void > > RangeCheckStub::emit_code(LIR_Assembler* ce)" should: > ce- > >verify_reserved_argument_area_size(1); > be > ce- > >verify_reserved_argument_area_size(2); > > so as to accommodate the array > pointer you are pushing onto the > > stack? > > > I've not tested 32-bit arm. > > > BR, > Stuart > > On 26 April 2018 at 15:31, Stuart > Monteith > > > > > wrote: > > Thanks, I'm happy with that. > > The registers have a clean path to > call_RT - r22 and r23 aren't used > inbetween. They are an arbitrary > choice - c_rarg0 and c_rarg1 were > always going to cause problems. If > _array->as_pointer_register() > and/or _index->as_register() or > _index->as_jint() were the registers > we were using as parameters there > would be trouble. However, > > with > > pd_last_allocatable_cpu_reg = 16, > that shouldn't happen with > > r22/23, > > or indeed anything else in the range > r17 to r28. > > I'm going to run all of JTRegs and seem > what that produces now. > > BR, > Stuart > > > > > On 26 April 2018 at 15:14, Lindenmaier, > Goetz > > > wrote: > > Hi Stuart, > > thanks for fixing this! Webrev with > your changes: > > http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/05/ > > > There is the possibility of overwriting > live values though, aren't > there? The registers are saved by > call_RT. Should I be concerned > > about > > deopt and debugging going wrong? > Furthermore, won't there be > > issues > > in > > exception handlers? > > As I understand, this just has to > survive the far_call. > The call_RT in c1_Runtime then moves > it into the > proper argument registers. This is just > the handling of an > exception, and in these few > instructions no java code is > executed, no safepoint is passed, so > this should be fine. > > incremental diff: > iff -r 874f2b999ff6 > > > src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp > > --- > a/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Mon > > Apr > > 16 > > 15:17:20 2018 +0200 > > +++ > b/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Thu > > Apr > > 26 > > 15:55:18 2018 +0200 > > @@ -75,16 +75,16 @@ > } > > if (_index->is_cpu_register()) { > - __ mov(rscratch1, _index- > >as_register()); > + __ mov(r22, _index- > >as_register()); > } else { > - __ mov(rscratch1, _index- > >as_jint()); > + __ mov(r22, _index->as_jint()); > } > Runtime1::StubID stub_id; > if > (_throw_index_out_of_bounds_exception) { > stub_id = > Runtime1::throw_index_exception_id; > } else { > assert(_array != NULL, "sanity"); > - __ mov(rscratch2, _array- > >as_pointer_register()); > + __ mov(r23, _array- > >as_pointer_register()); > stub_id = > Runtime1::throw_range_check_failed_id; > } > __ > far_call(RuntimeAddress(Runtime1::entry_for(stub_id)), > > NULL, > > rscratch2); > > diff -r 874f2b999ff6 > > > src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp > > --- > a/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Mon > > Apr > > 16 > > 15:17:20 2018 +0200 > > +++ > b/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Thu > > Apr > > 26 > > 15:55:18 2018 +0200 > > @@ -327,7 +327,7 @@ > > > // target: the entry point of the > method that creates and posts > > the > > exception oop > > -// has_argument: true if the > exception needs an argument > > (passed > > in > > rscratch1) > > +// has_argument: true if the > exception needs arguments (passed > > in > > r22 > > and r23) > > > OopMapSet* > > > Runtime1::generate_exception_throw(StubAssembler* > > sasm, address target, bool > has_argument) { > > // make a frame and preserve the > caller's caller-save registers > @@ -336,7 +336,7 @@ > if (!has_argument) { > call_offset = __ call_RT(noreg, > noreg, target); > } else { > - call_offset = __ call_RT(noreg, > noreg, target, rscratch1, > > rscratch2); > > + call_offset = __ call_RT(noreg, > noreg, target, r22, r23); > } > OopMapSet* oop_maps = new > OopMapSet(); > oop_maps- > >add_gc_map(call_offset, oop_map); > > Best regards, > Goetz. > > > > -----Original Message----- > From: Stuart Monteith > [mailto:stuart.monteith at linaro.org] > Sent: Donnerstag, 26. April 2018 12:52 > To: Andrew Haley > > Cc: Lindenmaier, Goetz > ; > hotspot- > > compiler- > > dev at openjdk.java.net > ; aarch64-port-dev at openjdk.java.net > ; > > hotspot- > > runtime-dev at openjdk.java.net > ; aarch32-port- > > dev at openjdk.java.net > > > Subject: Re: RFR(M): 8201593: Print > array length in > ArrayIndexOutOfBoundsException. > > Hi, > Using c_rarg1 and c_rarg2 instead of > rscratch1 and overwriting > rscratch2 causes a SIGSEGV. > Using r22 and r23 instead, the test ran > successfully. > > In c1_CodeStubs_aarch64.cpp > : > 77 if (_index->is_cpu_register()) { > 78 __ mov(r22, _index- > >as_register()); > 79 } else { > 80 __ mov(r22, _index->as_jint()); > 81 } > 82 Runtime1::StubID stub_id; > 83 if > (_throw_index_out_of_bounds_exception) { > 84 stub_id = > Runtime1::throw_index_exception_id; > 85 } else { > 86 assert(_array != NULL, "sanity"); > 87 __ mov(r23, _array- > >as_pointer_register()); > 88 stub_id = > Runtime1::throw_range_check_failed_id; > 89 } > > in c1_Runtime_aarch64.cpp: > > 336 if (!has_argument) { > 337 call_offset = __ call_RT(noreg, > noreg, target); > 338 } else { > 339 call_offset = __ call_RT(noreg, > noreg, target, r22, r23); > 340 } > > There is the possibility of overwriting > live values though, aren't > there? The registers are saved by > call_RT. Should I be concerned > > about > > deopt and debugging going wrong? > Furthermore, won't there be > > issues > > in > > exception handlers? > > BR, > Stuart > > > On 25 April 2018 at 16:49, Stuart > Monteith > > > > > wrote: > > Indeed - and that is what I am seeing. > Usually no parameters are > > being > > called with this pattern, or rscratch1, > with the temporary variable > being changed to use rscratch2 in such > circumstances. > I'll try c_rarg1 and c_rarg2 - they > should pass straight through,if I > interpret the code correcting. > > On 25 April 2018 at 16:26, Andrew > Haley > > wrote: > > On 04/25/2018 04:00 PM, Stuart > Monteith wrote: > > I'm not quite sure to solve this yet - > we'll need to use the > > stack in > > some safe way. > > > It's not a great idea to pass arguments > in rscratch1 or rscratch2. > > These > > registers are for use in macros and > should be treated as > > volatile. > > Given > > that you're throwing an exception, > registers will be clobbered > > anyway. > > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > > EAC8 43EB D3EF DB98 CC77 2FAD A5CD > 6035 332F A671 > From martin.doerr at sap.com Mon May 7 15:01:55 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 7 May 2018 15:01:55 +0000 Subject: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics In-Reply-To: <79c15cb1-d5f6-989d-0056-0b69b8d5f962@oracle.com> References: <4a3b4d59c9f54234b0c0d325a3238cbc@sap.com> <372edaad-f92c-7071-036b-703c86ad25c1@oracle.com> <3c0af6430a8e43cbbd4178edc88a0512@sap.com> <0309694e4cf84438ab20b84cae51150a@sap.com> <4f94006f-d83f-a4c3-8947-6afb898c99bf@oracle.com> <5AF0474E.6090302@oracle.com> <79c15cb1-d5f6-989d-0056-0b69b8d5f962@oracle.com> Message-ID: <9fe5604e7cda460b80005a632d1e8a69@sap.com> Hi David, "conservative" is not broken because the two-way sync makes it compatible with any loads and stores. "seq_cst" doesn't generally specify if leading or trailing sync is used so it only has clear store-load ordering semantics with respect to other "seq_cst" accesses. I think Erik's comparison with the JMM is nice. Seq_cst accesses behave like Java volatile accesses with respect to other accesses. (Atomic RMW operations are both, read and write.) Java volatiles can be implemented as leading or trailing sync, too. At least until "JEP 188: Java Memory Model Update" clarifies this. Best regards, Martin -----Original Message----- From: David Holmes [mailto:david.holmes at oracle.com] Sent: Montag, 7. Mai 2018 14:50 To: Erik ?sterlund ; Doerr, Martin ; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(M): 8202080: Introduce ordering semantics for Atomic::add and other RMW atomics Hi Erik, On 7/05/2018 10:32 PM, Erik ?sterlund wrote: > Hi David, > > On 2018-05-07 14:16, David Holmes wrote: >> On 7/05/2018 8:47 PM, Doerr, Martin wrote: >>> Hi David, >>> >>> thanks for the review. I've pushed it. >>> >>> About the seq_cst topic: >>> I could imagine supporting memory_order_seq_cst, but IMHO it will >>> only be really useful if we add something equivalent to C++11's >>> x.load(std::memory_order_seq_cst) and x.store(y, >>> std::memory_order_seq_cst). >>> Having it only for the RMW-atomics without being able to tag >>> load/store the same way sounds very poor to me. >>> In contrast, the release/acquire semantics already make sense to me >>> because they respect all other accesses, not only the "so tagged" ones. >> >> I'm not sure how to reconcile that with the existing status-quo where >> we have "conservative" Atomic-rmw combined with acquire/release >> accesses. Is seq-cst in some sense stronger than our "conservative" >> ordering ?? > > Seq cst is strictly weaker than "conservative". Conservative has full > trailing and leading fence. Seq cst basically has either full leading or > trailing fence, depending on choice of bindings. That makes pairs of seq > cst accesses totally ordered, but has potentially unintuitive > consequences when mixing with weaker accesses. So ideally, we would have > seq_cst loads and stores that could be used with those RMW operations. Well I'm confused. If seq-cst is weaker than what we have and seq-cst would be broken with only load-acquire/release-store, then doesn't that imply what we have with "conservative" is also broken?? > I personally don't see a problem with introducing seq cst as an ordering > you can choose for both RMW and loads/stores, that binds to precisely > what we already have today for JMM volatile. That way, my Access API can > call straight into OrderAccess/Atomic for such uses, instead of doing > the conditional fence dance outside. But I don't mind if we do not either. I'm not following what you are saying about JMM volatile here. The semantics for JMM volatile are best expressed as barriers required between pairs of accesses not the current way we deal with them as treating each access as if it were preceded and followed by another access that required the strongest barrier between them. David > Naturally, as with every other access, we have to take care that we know > what we are doing when introducing this in lock-free code. But that > arguably goes for the other ordering semantics as well. > > Thanks, > /Erik > >> Cheers, >> David >> >>> Best regards, >>> Martin >>> >>> >>> -----Original Message----- >>> From: David Holmes [mailto:david.holmes at oracle.com] >>> Sent: Montag, 7. Mai 2018 09:07 >>> To: Doerr, Martin ; >>> hotspot-runtime-dev at openjdk.java.net >>> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for >>> Atomic::add and other RMW atomics >>> >>> Hi Martin, >>> >>> On 5/05/2018 1:32 AM, Doerr, Martin wrote: >>>> Hi David, >>>> >>>> thanks for your time. I understand that there's a lot of work at the >>>> moment. >>>> >>>> I have removed memory_order_consume again. We don't need it for our >>>> platforms and I guess nobody will miss it. >>>> >>>> I like your comment proposals, so I've updated them. >>>> >>>> New webrev: >>>> http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.02/ >>> >>> Changes seem fine. Thanks. >>> >>>>> Though that raises the question why you haven't added >>>>> memory_order_seq_cst? >>>> I think that one could easily be used incorrectly within hotspot >>>> which would lead to problems on PPC. >>>> "Atomic operations tagged memory_order_seq_cst ... establish a >>>> single total modification order of all atomic operations that are so >>>> tagged." [1] >>>> It should work fine if all involved accesses are "so tagged". But >>>> hotspot doesn't use seq_cst loads, it uses things like >>>> OrderAccess::load_acquire. On PPC, GCC and xlC don't use a trailing >>>> sync for seq_cst stores and RMW-atomics so they only work well with >>>> leading sync loads which are not used in hotspot code. >>>> I believe that there's a substantial amount of work to be done to >>>> get this right. >>> >>> I find this somewhat ironic/amusing. My main concern with adding in any >>> access modes weaker than the existing "conservative" is precisely >>> because it can be very hard to establish that they are in fact >>> correct. :) >>> >>> But I don't follow your concern. The "total modification order" is in >>> addition to the acquire/release semantics that are inherent in seq-cst. >>> So if the seq-cst atomic-rmw operation is used on a variable that is >>> otherwise accessed via load_acquire/release_store, then I would expect >>> everything to be okay - else it serves no point to define seq-cst rmw in >>> terms of acquire and release IMHO. I would expect these access modes to >>> interact in the "obvious" way. >>> >>> Cheers, >>> David >>> >>>> Thanks, >>>> Martin >>>> >>>> >>>> [1] >>>> https://en.cppreference.com/w/cpp/atomic/memory_order#Sequentially-consistent_ordering >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: David Holmes [mailto:david.holmes at oracle.com] >>>> Sent: Freitag, 4. Mai 2018 08:09 >>>> To: Doerr, Martin ; >>>> hotspot-runtime-dev at openjdk.java.net >>>> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for >>>> Atomic::add and other RMW atomics >>>> >>>> Hi Martin, >>>> >>>> On 3/05/2018 11:03 PM, Doerr, Martin wrote: >>>>> Hi Robbin, >>>>> >>>>> thank you for reviewing. >>>>> >>>>> Submission forest tests have passed in addition to SAP's nightly >>>>> builds. I have 2 reviews, now, but I'll wait a little bit before >>>>> pushing. >>>> >>>> Thanks for waiting. I think it important there is a general buy-in to >>>> this effort, not just the official "get a review and push". Apologies >>>> that it's taken me a while to get to this. >>>> >>>> memory_order_consume has been deprecated in C++17: >>>> >>>> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0371r1.html >>>> >>>> I would suggest we not add it to the VM. >>>> >>>> ???? 40 enum atomic_memory_order { >>>> ???? 41?? // We support most semantics like in C++11. >>>> >>>> I would say something like: >>>> >>>> // The modes that align with C++11 are intended to >>>> // follow the same semantics. >>>> >>>> >>>> 47?? // We need to be more conservative than seq_cst on PPC64. >>>> >>>> Not sure the PPC64 comment is relevant here. memory_order_conservative >>>> represents the pre-existing strong fencing requirements. It's not >>>> intended to map to anything else, nor intended to be different across >>>> platforms. >>>> >>>> Though that raises the question why you haven't added >>>> memory_order_seq_cst? >>>> >>>> Looking at all the platform code that adds the unused "order" parameter >>>> ... I would not be surprised if the forthcoming compiler upgrades >>>> result >>>> in some additional "unused" warnings. But I guess we'll just deal with >>>> them as they are discovered. >>>> >>>> Otherwise all seems okay. >>>> >>>> Thanks, >>>> David >>>> >>>> >>>>> Best regards, >>>>> Martin >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: Robbin Ehn [mailto:robbin.ehn at oracle.com] >>>>> Sent: Donnerstag, 3. Mai 2018 14:29 >>>>> To: Doerr, Martin ; Schmidt, Lutz >>>>> ; David Holmes ; >>>>> Erik ?sterlund ; >>>>> hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev >>>>> ; Lindenmaier, Goetz ; >>>>> Michihiro Horie (HORIE at jp.ibm.com) ; Andrew Haley >>>>> (aph at redhat.com) ; John Paul Adrian Glaubitz >>>>> >>>>> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for >>>>> Atomic::add and other RMW atomics >>>>> >>>>> On 2018-05-03 12:50, Doerr, Martin wrote: >>>>>> Hi Lutz, >>>>>> >>>>>> thanks for reviewing. >>>>>> >>>>>> Can I get more reviews, please? >>>>> >>>>> I think this looks good, thanks for fixing! >>>>> >>>>> /Robbin >>>>> >>>>>> >>>>>> Best regards, >>>>>> Martin >>>>>> >>>>>> >>>>>> -----Original Message----- >>>>>> From: Schmidt, Lutz >>>>>> Sent: Mittwoch, 2. Mai 2018 16:20 >>>>>> To: Doerr, Martin ; David Holmes >>>>>> ; Erik ?sterlund >>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>> Aleksey Shipilev ; Lindenmaier, Goetz >>>>>> ; Michihiro Horie (HORIE at jp.ibm.com) >>>>>> ; Andrew Haley (aph at redhat.com) >>>>>> ; John Paul Adrian Glaubitz >>>>>> >>>>>> Subject: Re: RFR(M): 8202080: Introduce ordering semantics for >>>>>> Atomic::add and other RMW atomics >>>>>> >>>>>> Hi Martin, >>>>>> >>>>>> your changes look good. >>>>>> >>>>>> For the additional synchronization on s390 in particular, I do not >>>>>> expect a serious performance impact. The checkpoint >>>>>> synchronization part has always been the expensive, but rarely >>>>>> required, part. PPC may be another story. We knoow that a >>>>>> full-blown sync really hurts. We will see... >>>>>> >>>>>> Thanks, >>>>>> Lutz >>>>>> >>>>>> ?On 26.04.18, 16:23, "hotspot-runtime-dev on behalf of Doerr, >>>>>> Martin" >>>>> martin.doerr at sap.com> wrote: >>>>>> >>>>>> ??????? Hi, >>>>>> ??????????????? I have renamed "cmpxchg_memory_order" to >>>>>> "atomic_memory_order" and added support to all Read-Modify-Write >>>>>> atomics with support for all C++11 semantics except seq_cst. The >>>>>> latter has issues on PPC64 so we're currently using our own >>>>>> memory_order_conservative instead, which is the default. >>>>>> ??????????????? Please review my new webrev: >>>>>> http://cr.openjdk.java.net/~mdoerr/8202080_atomic_add/webrev.01/ >>>>>> ??????????????? We'll test Windows, MacOS, linux x86+ppc64+s390, >>>>>> AIX and Solaris SPARC. >>>>>> ??????? It'd be great if somebody could check arm/aarch64 and zero. >>>>>> ??????????????? This change may enable optimizations for >>>>>> arm/aarch64 (not yet included). >>>>>> ??????????????? Best regards, >>>>>> ??????? Martin >>>>>> ??????????????????????? -----Original Message----- >>>>>> ??????? From: Doerr, Martin >>>>>> ??????? Sent: Donnerstag, 26. April 2018 11:20 >>>>>> ??????? To: 'David Holmes' ; Erik >>>>>> ?sterlund ; >>>>>> hotspot-runtime-dev at openjdk.java.net; Aleksey Shipilev >>>>>> ; Lindenmaier, Goetz >>>>>> ; Michihiro Horie (HORIE at jp.ibm.com) >>>>>> >>>>>> ??????? Subject: RE: RFR(M): 8202080: Introduce ordering semantics >>>>>> for Atomic::add >>>>>> ??????????????? Hi David, >>>>>> ??????????????? sounds like we are on the same page, now. I think >>>>>> we should have some kind of summary of what was analyzed. But that >>>>>> belongs to the other thread (JDK- 8154736). >>>>>> ??????????????? > The bugs in this kind of code are very subtle >>>>>> and rarely exposed through normal levels of testing. >>>>>> ??????? Right. We have experienced this often enough. So it is >>>>>> definitely in our interest to have correct memory barriers. >>>>>> Mistakes will hit us (PPC64). So I appreciate that you insist on >>>>>> careful analysis. >>>>>> ??????????????? I'll prepare a new webrev with conservative >>>>>> semantics for all read-modify-write atomics by default. >>>>>> ??????????????? Best regards, >>>>>> ??????? Martin >>>>>> > From thomas.stuefe at gmail.com Mon May 7 15:29:39 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 7 May 2018 17:29:39 +0200 Subject: RFR: 8202634: Metaspace: simplify SpaceManager lists Message-ID: Hi all, may I please have reviews for this small simplification. Bug: https://bugs.openjdk.java.net/browse/JDK-8202634 webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8202634-simplify-spacemanager/webrev.00/webrev/ This replaces the four separate in-use-chunk-lists in SpaceManager with a single one. The reasoning behind this is that we do not need to keep the chunks sorted by chunk size, a single list containing all chunks of all sizes in use by this SpaceManager is enough, and it simplifies the coding. There is only one place where we really did use the former by-chunktype lists - that is to count how many chunks per chunk type there are. However, arguably that is better done with running counters, so this is what I did. The added benefit is that in SpaceManager::calc_chunk_size() we do not need to walk the lists anymore to sum up chunks. Changes in detail: - removed the "ChunkIndex" argument in ChunkManager::return_single_chunk() and Chunkmanager::return_chunk_list() - that argument was unnecessary, since the Metachunk we return knows its index. - Unfolded SpaceManager::initialize into Spacemanager constructor (I prefer initializing members in ctor init lists) - replaces SpaceManager::sum_count_chunks_in_use() with running counters. - I dumbed down the logging inside ChunkManager::return_chunk_list() somewhat. Should anyone miss the former details, I can reinstate it. Thank you, Thomas From calvin.cheung at oracle.com Mon May 7 16:12:25 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Mon, 07 May 2018 09:12:25 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time In-Reply-To: <56AA6E39-131E-4D5B-BBEF-9FA8C4A60F5A@oracle.com> References: <5AEB8C04.4040501@oracle.com> <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> <5AEB98B7.9030209@oracle.com> <56AA6E39-131E-4D5B-BBEF-9FA8C4A60F5A@oracle.com> Message-ID: <5AF07AE9.9000700@oracle.com> Hi Jiangli, On 5/3/18, 5:33 PM, Jiangli Zhou wrote: > Hi Calvin, > > >> On May 3, 2018, at 4:18 PM, Calvin Cheung wrote: >> >> >> >> On 5/3/18, 3:42 PM, Jiangli Zhou wrote: >>> Hi Calvin, >>> >>> I think we don?t need to check the module path entries if only boot classes are archived. The ?end? is set to app_class_paths_start_index when there are only boot classes are archived. If there are app module classes loaded from jars/directories at dump time, has_platform_or_app_classes() should return true and ?end? is set to the end of the shared path table, which includes all module path entries. The following loop is not needed. The existing loop in the code covers all cases with different ?end? value based on the type of classes loaded in the archive. >>> 366 // may need to check module path entries. >>> 367 if ((end == ClassLoaderExt::app_class_paths_start_index())&& (ClassLoader::num_module_path_entries()> 0)) { >>> 368 for (int i = ClassLoaderExt::app_module_paths_start_index(); i< _shared_path_table_size; i++) { >>> 369 SharedClassPathEntry* e = shared_path(i); >>> 370 has_nonempty_dir = check_nonempty_dir(e); >>> >>> 371 } >>> 372 } >>> >> The has_platform_or_app_classes() will return false if ClassLoaderExt::record_result() hasn't been called while a class is being loaded during dumping. >> This could happen before this proposed change if one specifies a --module-path with an non-empty directory containing a module. While the class will be loaded during dumping but its classpath_index will still be -1. Therefore, the has_platform_or_app_classes() will return false and the module path entries won't be checked without the above block of code. > > With the module directory path being added to the path table now, it should record the correct path table index instead of -1 for module classes loaded from directory at dump time. > > I applied your patch and stepped through ClassLoader::record_result() with your test case. The following path comparison fails due the the extra ?/? in ?path?. That why it fails to find the correct entry in the path table, which causes the class type cannot be correctly identified. > > if (strcmp(canonical_path, os::native_path((char*)path)) == 0) {os::dir_is_empty > > (gdb) p canonical_path > $74 = 0x7ffff0aea0f0 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple" > (gdb) p os::native_path((char*)path) > $75 = 0x7ffff7fd4a65 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple/" > > The trailing ?/? should be excluded for the path comparison. JDK-8202519 is a related issue in the same area. It might be better to fix it together with this one. Thanks for debugging the issue. I've fixed it by calling get_canonical_path() on the "path" before doing strcmp(). It should also fix the bug in JDK-8202519 since a buffer is being allocated for get_canonical_path() to write into. The os::native_path() on linux does nothing and just returns the same string. I also discovered a problem in os::dir_is_empty() on windows. When a directory path is passed into the function, it may have the following form with forward slashes: C:/somedir The fix is to call os::native_path() which essentially converts the forward slashes to back slashes so that subsequent windows api call returns the correct result. > > Once the above issue is fixed, the class type should be recorded correctly for a module classes loaded from directory at dump time. Then ClassLoaderExt::record_result() can set ?has_app_classes? flag properly. And, no change should be needed in FileMapInfo::check_nonempty_dir_in_shared_path_table(). I don't need to touch filemap.cpp in the following updated webrev: http://cr.openjdk.java.net/~ccheung/8202289/webrev.01/ thanks, Calvin > >>> We also need a unit test for non-empty directory in module path. Please add. >> It is already in my webrev: >> http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/MainModuleOnly.java.sdiff.html > Ok. I was expecting a separate unit test. I?m fine with an added test case in MainModuleOnly.java. > > Thanks, > Jiangli > >> thanks, >> Calvin >>> Thanks, >>> Jiangli >>> >>>> On May 3, 2018, at 3:24 PM, Calvin Cheung wrote: >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8202289 >>>> webrev: http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ >>>> >>>> In ClassLoaderExt::process_module_table(), it adds all entries from the ModuleEntryTable with the "file:" protocol to the _module_path_entries and eventually to the _shared_path_table. >>>> The FileMapInfo::check_nonempty_dir_in_shared_path_table() will check for non-empty directory in the module path. >>>> >>>> Ran all CDS and AppCDS tests locally on linux-x64. >>>> Will run hs-tier{1,2,3} tests. >>>> >>>> thanks, >>>> Calvin From coleen.phillimore at oracle.com Mon May 7 17:07:06 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 7 May 2018 13:07:06 -0400 Subject: RFR(xxs): 8185034: Cleanup and consolidate Metaspace coding In-Reply-To: References: <8f5456bb-303a-6b1e-cb3e-174c2b2de629@oracle.com> Message-ID: <84c230af-aeaa-832b-8eb9-64c83e629c0d@oracle.com> If Zhengyu (or someone else who knows this code) doesn't have time to look at this, I declare it to be a "trivial" change and should only need one review. thanks, Coleen On 5/7/18 9:48 AM, Thomas St?fe wrote: > Thank you Coleen! > > On Mon, May 7, 2018 at 3:29 PM, wrote: >> This looks good to me. Thank you for all the metaspace cleanups and for >> looking at the logging in detail. >> Coleen >> >> >> >> On 5/5/18 3:58 PM, Thomas St?fe wrote: >>> Hi all, >>> >>> May I get reviews please? >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8185034 >>> Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8185034-cleanup-and-consolidate-metaspace-coding/webrev.00/webrev/ >>> >>> Despite the title this item is quite unexciting. >>> >>> Most of the cleanups originally targeted by this item did already >>> happen as part of other enhancements >>> (https://bugs.openjdk.java.net/browse/JDK-8199430, >>> https://bugs.openjdk.java.net/browse/JDK-8202638). >>> >>> What is left are some smallish cleanups: >>> - removed some functions which were unused >>> - small changes to ~SpaceManager to avoid logging the same info twice >>> (chunk manager state) >>> - removed all LogStream::cr() calls which were needed because >>> ~LogStream() did not autoflush; but 8202303 did fix that, so the cr() >>> can go. >>> >>> Note that there are more things one could clean up, which may happen >>> in the future. Especially the split-up of metaspace.cpp, which is >>> still open (https://bugs.openjdk.java.net/browse/JDK-8176808). But for >>> now I would like to close this particular issue. >>> >>> Thanks! >>> >>> ..Thomas >> From zgu at redhat.com Mon May 7 17:10:11 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 7 May 2018 13:10:11 -0400 Subject: RFR(xxs): 8185034: Cleanup and consolidate Metaspace coding In-Reply-To: <84c230af-aeaa-832b-8eb9-64c83e629c0d@oracle.com> References: <8f5456bb-303a-6b1e-cb3e-174c2b2de629@oracle.com> <84c230af-aeaa-832b-8eb9-64c83e629c0d@oracle.com> Message-ID: <4aceecf7-0bd7-aaac-27cb-47efbe33d223@redhat.com> Sorry, I looked at the webrev over the weekend, but forgot to reply. It looks good to me. Thanks, -Zhengyu On 05/07/2018 01:07 PM, coleen.phillimore at oracle.com wrote: > > If Zhengyu (or someone else who knows this code) doesn't have time to > look at this, I declare it to be a "trivial" change and should only need > one review. > thanks, > Coleen > > On 5/7/18 9:48 AM, Thomas St?fe wrote: >> Thank you Coleen! >> >> On Mon, May 7, 2018 at 3:29 PM,? wrote: >>> This looks good to me.? Thank you for all the metaspace cleanups and for >>> looking at the logging in detail. >>> Coleen >>> >>> >>> >>> On 5/5/18 3:58 PM, Thomas St?fe wrote: >>>> Hi all, >>>> >>>> May I get reviews please? >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8185034 >>>> Webrev: >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8185034-cleanup-and-consolidate-metaspace-coding/webrev.00/webrev/ >>>> >>>> >>>> Despite the title this item is quite unexciting. >>>> >>>> Most of the cleanups originally targeted by this item did already >>>> happen as part of other enhancements >>>> (https://bugs.openjdk.java.net/browse/JDK-8199430, >>>> https://bugs.openjdk.java.net/browse/JDK-8202638). >>>> >>>> What is left are some smallish cleanups: >>>> - removed some functions which were unused >>>> - small changes to ~SpaceManager to avoid logging the same info twice >>>> (chunk manager state) >>>> - removed all LogStream::cr() calls which were needed because >>>> ~LogStream() did not autoflush; but? 8202303 did fix that, so the cr() >>>> can go. >>>> >>>> Note that there are more things one could clean up, which may happen >>>> in the future. Especially the split-up of metaspace.cpp, which is >>>> still open (https://bugs.openjdk.java.net/browse/JDK-8176808). But for >>>> now I would like to close this particular issue. >>>> >>>> Thanks! >>>> >>>> ..Thomas >>> > From jiangli.zhou at Oracle.COM Mon May 7 17:28:23 2018 From: jiangli.zhou at Oracle.COM (Jiangli Zhou) Date: Mon, 7 May 2018 10:28:23 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time In-Reply-To: <5AF07AE9.9000700@oracle.com> References: <5AEB8C04.4040501@oracle.com> <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> <5AEB98B7.9030209@oracle.com> <56AA6E39-131E-4D5B-BBEF-9FA8C4A60F5A@oracle.com> <5AF07AE9.9000700@oracle.com> Message-ID: Hi Calvin, -classLoader.cpp The get_canonical_path() call for ?path? should be moved out of the ?for' loop. 1569 get_canonical_path(path, canonical_src_path, JVM_MAXPATHLEN)) { To make the code more readable, please rename ?canonical_path? to ?canonical_path_table_entry? (or ?canonical_path_table_entry_name?) and ?canonical_src_path? to ?canonical_class_src_path?. BTW, when we create the path entries, the paths are already converted to canonical forms. I think there is no need to call get_canonical_path() on ?ent->name()?. Can you please double check? - os_windows.cpp Please fix indentation at line 4390 & 4391. Thanks, Jiangli > On May 7, 2018, at 9:12 AM, Calvin Cheung wrote: > > Hi Jiangli, > > On 5/3/18, 5:33 PM, Jiangli Zhou wrote: >> Hi Calvin, >> >> >>> On May 3, 2018, at 4:18 PM, Calvin Cheung wrote: >>> >>> >>> >>> On 5/3/18, 3:42 PM, Jiangli Zhou wrote: >>>> Hi Calvin, >>>> >>>> I think we don?t need to check the module path entries if only boot classes are archived. The ?end? is set to app_class_paths_start_index when there are only boot classes are archived. If there are app module classes loaded from jars/directories at dump time, has_platform_or_app_classes() should return true and ?end? is set to the end of the shared path table, which includes all module path entries. The following loop is not needed. The existing loop in the code covers all cases with different ?end? value based on the type of classes loaded in the archive. >>>> 366 // may need to check module path entries. >>>> 367 if ((end == ClassLoaderExt::app_class_paths_start_index())&& (ClassLoader::num_module_path_entries()> 0)) { >>>> 368 for (int i = ClassLoaderExt::app_module_paths_start_index(); i< _shared_path_table_size; i++) { >>>> 369 SharedClassPathEntry* e = shared_path(i); >>>> 370 has_nonempty_dir = check_nonempty_dir(e); >>>> >>>> 371 } >>>> 372 } >>>> >>> The has_platform_or_app_classes() will return false if ClassLoaderExt::record_result() hasn't been called while a class is being loaded during dumping. >>> This could happen before this proposed change if one specifies a --module-path with an non-empty directory containing a module. While the class will be loaded during dumping but its classpath_index will still be -1. Therefore, the has_platform_or_app_classes() will return false and the module path entries won't be checked without the above block of code. >> >> With the module directory path being added to the path table now, it should record the correct path table index instead of -1 for module classes loaded from directory at dump time. >> >> I applied your patch and stepped through ClassLoader::record_result() with your test case. The following path comparison fails due the the extra ?/? in ?path?. That why it fails to find the correct entry in the path table, which causes the class type cannot be correctly identified. >> >> if (strcmp(canonical_path, os::native_path((char*)path)) == 0) {os::dir_is_empty >> >> (gdb) p canonical_path >> $74 = 0x7ffff0aea0f0 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple" >> (gdb) p os::native_path((char*)path) >> $75 = 0x7ffff7fd4a65 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple/" >> >> The trailing ?/? should be excluded for the path comparison. JDK-8202519 is a related issue in the same area. It might be better to fix it together with this one. > Thanks for debugging the issue. > I've fixed it by calling get_canonical_path() on the "path" before doing strcmp(). It should also fix the bug in JDK-8202519 since a buffer is being allocated for get_canonical_path() to write into. The os::native_path() on linux does nothing and just returns the same string. > > I also discovered a problem in os::dir_is_empty() on windows. When a directory path is passed into the function, it may have the following form with forward slashes: C:/somedir > The fix is to call os::native_path() which essentially converts the forward slashes to back slashes so that subsequent windows api call returns the correct result. > >> >> Once the above issue is fixed, the class type should be recorded correctly for a module classes loaded from directory at dump time. Then ClassLoaderExt::record_result() can set ?has_app_classes? flag properly. And, no change should be needed in FileMapInfo::check_nonempty_dir_in_shared_path_table(). > I don't need to touch filemap.cpp in the following updated webrev: > > http://cr.openjdk.java.net/~ccheung/8202289/webrev.01/ > > thanks, > Calvin >> >>>> We also need a unit test for non-empty directory in module path. Please add. >>> It is already in my webrev: >>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/MainModuleOnly.java.sdiff.html >> Ok. I was expecting a separate unit test. I?m fine with an added test case in MainModuleOnly.java. >> >> Thanks, >> Jiangli >> >>> thanks, >>> Calvin >>>> Thanks, >>>> Jiangli >>>> >>>>> On May 3, 2018, at 3:24 PM, Calvin Cheung wrote: >>>>> >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8202289 >>>>> webrev: http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ >>>>> >>>>> In ClassLoaderExt::process_module_table(), it adds all entries from the ModuleEntryTable with the "file:" protocol to the _module_path_entries and eventually to the _shared_path_table. >>>>> The FileMapInfo::check_nonempty_dir_in_shared_path_table() will check for non-empty directory in the module path. >>>>> >>>>> Ran all CDS and AppCDS tests locally on linux-x64. >>>>> Will run hs-tier{1,2,3} tests. >>>>> >>>>> thanks, >>>>> Calvin From coleen.phillimore at oracle.com Mon May 7 18:09:19 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 7 May 2018 14:09:19 -0400 Subject: RFR (trivial) 8202606: Deprecate AllowNonVirtualCalls option Message-ID: <3dc8508f-b924-7df7-3d52-1a57fda2081b@oracle.com> The CSR request is approved, see bug for details.? Tested with java -XX:+AllowNonVirtualCalls -version open webrev at http://cr.openjdk.java.net/~coleenp/8202606.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8202606 Thanks, Coleen From thomas.stuefe at gmail.com Mon May 7 18:13:17 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 7 May 2018 20:13:17 +0200 Subject: RFR(xxs): 8185034: Cleanup and consolidate Metaspace coding In-Reply-To: <4aceecf7-0bd7-aaac-27cb-47efbe33d223@redhat.com> References: <8f5456bb-303a-6b1e-cb3e-174c2b2de629@oracle.com> <84c230af-aeaa-832b-8eb9-64c83e629c0d@oracle.com> <4aceecf7-0bd7-aaac-27cb-47efbe33d223@redhat.com> Message-ID: Thank you Zhengyu! On Mon, May 7, 2018 at 7:10 PM, Zhengyu Gu wrote: > Sorry, I looked at the webrev over the weekend, but forgot to reply. > > It looks good to me. > > Thanks, > > -Zhengyu > > > On 05/07/2018 01:07 PM, coleen.phillimore at oracle.com wrote: >> >> >> If Zhengyu (or someone else who knows this code) doesn't have time to look >> at this, I declare it to be a "trivial" change and should only need one >> review. >> thanks, >> Coleen >> >> On 5/7/18 9:48 AM, Thomas St?fe wrote: >>> >>> Thank you Coleen! >>> >>> On Mon, May 7, 2018 at 3:29 PM, wrote: >>>> >>>> This looks good to me. Thank you for all the metaspace cleanups and for >>>> looking at the logging in detail. >>>> Coleen >>>> >>>> >>>> >>>> On 5/5/18 3:58 PM, Thomas St?fe wrote: >>>>> >>>>> Hi all, >>>>> >>>>> May I get reviews please? >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8185034 >>>>> Webrev: >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8185034-cleanup-and-consolidate-metaspace-coding/webrev.00/webrev/ >>>>> >>>>> Despite the title this item is quite unexciting. >>>>> >>>>> Most of the cleanups originally targeted by this item did already >>>>> happen as part of other enhancements >>>>> (https://bugs.openjdk.java.net/browse/JDK-8199430, >>>>> https://bugs.openjdk.java.net/browse/JDK-8202638). >>>>> >>>>> What is left are some smallish cleanups: >>>>> - removed some functions which were unused >>>>> - small changes to ~SpaceManager to avoid logging the same info twice >>>>> (chunk manager state) >>>>> - removed all LogStream::cr() calls which were needed because >>>>> ~LogStream() did not autoflush; but 8202303 did fix that, so the cr() >>>>> can go. >>>>> >>>>> Note that there are more things one could clean up, which may happen >>>>> in the future. Especially the split-up of metaspace.cpp, which is >>>>> still open (https://bugs.openjdk.java.net/browse/JDK-8176808). But for >>>>> now I would like to close this particular issue. >>>>> >>>>> Thanks! >>>>> >>>>> ..Thomas >>>> >>>> >> > From calvin.cheung at oracle.com Mon May 7 18:33:11 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Mon, 07 May 2018 11:33:11 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time In-Reply-To: References: <5AEB8C04.4040501@oracle.com> <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> <5AEB98B7.9030209@oracle.com> <56AA6E39-131E-4D5B-BBEF-9FA8C4A60F5A@oracle.com> <5AF07AE9.9000700@oracle.com> Message-ID: <5AF09BE7.30208@oracle.com> On 5/7/18, 10:28 AM, Jiangli Zhou wrote: > Hi Calvin, > > -classLoader.cpp > > The get_canonical_path() call for ?path? should be moved out of the ?for' loop. > > 1569 get_canonical_path(path, canonical_src_path, JVM_MAXPATHLEN)) { Fixed. > > To make the code more readable, please rename ?canonical_path? to ?canonical_path_table_entry? (or ?canonical_path_table_entry_name?) I chose the shorter one. > and ?canonical_src_path? to ?canonical_class_src_path?. Renamed. > > BTW, when we create the path entries, the paths are already converted to canonical forms. I think there is no need to call get_canonical_path() on ?ent->name()?. Can you please double check? The call to get_canonical_path() is only for the non-directory case (please refer to ClassLoader::create_class_path_entry() for details). I did try get_canonical_path() for the directory case but it didn't work during runtime; it failed in SystemDictionaryShared::is_shared_class_visible_for_classloader() in the following check: (strcmp(ent->name(), ClassLoader::skip_uri_protocol(mod_entry->location()->as_C_string())) == 0)) { The location() in the mod_entry isn't in canonical form. If we convert it to canonical form during runtime, there maybe performance impact. I think the proper way of fixing it is storing the location() in canonical form when a module entry is being defined. I'm not sure if it is possible for the core lib to pass the location in canonical form to jvm when a module entry is being defined. I can file a follow-up RFE to address that. > > - os_windows.cpp > > Please fix indentation at line 4390& 4391. That was done on purpose because the condition in line 4390 is '||' with line 4391. I'm further indenting line 4391 to make it clearer. 4389 if (search_path[1] == ':'&& 4390 (search_path[2] == '\0' || 4391 (search_path[2] == '\\'&& search_path[3] == '\0'))) { Updated webrev: http://cr.openjdk.java.net/~ccheung/8202289/webrev.02/ thanks, Calvin > > Thanks, > Jiangli > > >> On May 7, 2018, at 9:12 AM, Calvin Cheung wrote: >> >> Hi Jiangli, >> >> On 5/3/18, 5:33 PM, Jiangli Zhou wrote: >>> Hi Calvin, >>> >>> >>>> On May 3, 2018, at 4:18 PM, Calvin Cheung wrote: >>>> >>>> >>>> >>>> On 5/3/18, 3:42 PM, Jiangli Zhou wrote: >>>>> Hi Calvin, >>>>> >>>>> I think we don?t need to check the module path entries if only boot classes are archived. The ?end? is set to app_class_paths_start_index when there are only boot classes are archived. If there are app module classes loaded from jars/directories at dump time, has_platform_or_app_classes() should return true and ?end? is set to the end of the shared path table, which includes all module path entries. The following loop is not needed. The existing loop in the code covers all cases with different ?end? value based on the type of classes loaded in the archive. >>>>> 366 // may need to check module path entries. >>>>> 367 if ((end == ClassLoaderExt::app_class_paths_start_index())&& (ClassLoader::num_module_path_entries()> 0)) { >>>>> 368 for (int i = ClassLoaderExt::app_module_paths_start_index(); i< _shared_path_table_size; i++) { >>>>> 369 SharedClassPathEntry* e = shared_path(i); >>>>> 370 has_nonempty_dir = check_nonempty_dir(e); >>>>> >>>>> 371 } >>>>> 372 } >>>>> >>>> The has_platform_or_app_classes() will return false if ClassLoaderExt::record_result() hasn't been called while a class is being loaded during dumping. >>>> This could happen before this proposed change if one specifies a --module-path with an non-empty directory containing a module. While the class will be loaded during dumping but its classpath_index will still be -1. Therefore, the has_platform_or_app_classes() will return false and the module path entries won't be checked without the above block of code. >>> With the module directory path being added to the path table now, it should record the correct path table index instead of -1 for module classes loaded from directory at dump time. >>> >>> I applied your patch and stepped through ClassLoader::record_result() with your test case. The following path comparison fails due the the extra ?/? in ?path?. That why it fails to find the correct entry in the path table, which causes the class type cannot be correctly identified. >>> >>> if (strcmp(canonical_path, os::native_path((char*)path)) == 0) {os::dir_is_empty >>> >>> (gdb) p canonical_path >>> $74 = 0x7ffff0aea0f0 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple" >>> (gdb) p os::native_path((char*)path) >>> $75 = 0x7ffff7fd4a65 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple/" >>> >>> The trailing ?/? should be excluded for the path comparison. JDK-8202519 is a related issue in the same area. It might be better to fix it together with this one. >> Thanks for debugging the issue. >> I've fixed it by calling get_canonical_path() on the "path" before doing strcmp(). It should also fix the bug in JDK-8202519 since a buffer is being allocated for get_canonical_path() to write into. The os::native_path() on linux does nothing and just returns the same string. >> >> I also discovered a problem in os::dir_is_empty() on windows. When a directory path is passed into the function, it may have the following form with forward slashes: C:/somedir >> The fix is to call os::native_path() which essentially converts the forward slashes to back slashes so that subsequent windows api call returns the correct result. >> >>> Once the above issue is fixed, the class type should be recorded correctly for a module classes loaded from directory at dump time. Then ClassLoaderExt::record_result() can set ?has_app_classes? flag properly. And, no change should be needed in FileMapInfo::check_nonempty_dir_in_shared_path_table(). >> I don't need to touch filemap.cpp in the following updated webrev: >> >> http://cr.openjdk.java.net/~ccheung/8202289/webrev.01/ >> >> thanks, >> Calvin >>>>> We also need a unit test for non-empty directory in module path. Please add. >>>> It is already in my webrev: >>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/MainModuleOnly.java.sdiff.html >>> Ok. I was expecting a separate unit test. I?m fine with an added test case in MainModuleOnly.java. >>> >>> Thanks, >>> Jiangli >>> >>>> thanks, >>>> Calvin >>>>> Thanks, >>>>> Jiangli >>>>> >>>>>> On May 3, 2018, at 3:24 PM, Calvin Cheung wrote: >>>>>> >>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8202289 >>>>>> webrev: http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ >>>>>> >>>>>> In ClassLoaderExt::process_module_table(), it adds all entries from the ModuleEntryTable with the "file:" protocol to the _module_path_entries and eventually to the _shared_path_table. >>>>>> The FileMapInfo::check_nonempty_dir_in_shared_path_table() will check for non-empty directory in the module path. >>>>>> >>>>>> Ran all CDS and AppCDS tests locally on linux-x64. >>>>>> Will run hs-tier{1,2,3} tests. >>>>>> >>>>>> thanks, >>>>>> Calvin From harold.seigel at oracle.com Mon May 7 18:35:15 2018 From: harold.seigel at oracle.com (harold seigel) Date: Mon, 7 May 2018 14:35:15 -0400 Subject: RFR (trivial) 8202606: Deprecate AllowNonVirtualCalls option In-Reply-To: <3dc8508f-b924-7df7-3d52-1a57fda2081b@oracle.com> References: <3dc8508f-b924-7df7-3d52-1a57fda2081b@oracle.com> Message-ID: <9cece75c-3379-5e9b-1a43-582ce48fc257@oracle.com> Looks good and trivial. Thanks, Harold On 5/7/2018 2:09 PM, coleen.phillimore at oracle.com wrote: > The CSR request is approved, see bug for details.? Tested with java > -XX:+AllowNonVirtualCalls -version > > open webrev at http://cr.openjdk.java.net/~coleenp/8202606.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8202606 > > Thanks, > Coleen From david.holmes at oracle.com Mon May 7 20:37:31 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 8 May 2018 06:37:31 +1000 Subject: RFR (trivial) 8202606: Deprecate AllowNonVirtualCalls option In-Reply-To: <3dc8508f-b924-7df7-3d52-1a57fda2081b@oracle.com> References: <3dc8508f-b924-7df7-3d52-1a57fda2081b@oracle.com> Message-ID: <24525a32-b3d7-459f-8eec-3b37d53d6a6d@oracle.com> Hi Coleen, On 8/05/2018 4:09 AM, coleen.phillimore at oracle.com wrote: > The CSR request is approved, see bug for details.? Tested with java > -XX:+AllowNonVirtualCalls -version > > open webrev at http://cr.openjdk.java.net/~coleenp/8202606.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8202606 Please also add it to the runtime/CommandLine/VMDeprecatedOptions.java test. Thanks, David > Thanks, > Coleen From ioi.lam at oracle.com Mon May 7 20:39:36 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 7 May 2018 13:39:36 -0700 Subject: Improving AppCDS for Custom Loaders In-Reply-To: References: <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> <732ce957-b07c-bff2-277b-ca335c348753@oracle.com> <660A1644-3A08-455D-931D-D95069C39853@oracle.com> Message-ID: <9abbe65c-93e9-3931-45ce-1688541c29c4@oracle.com> On 5/5/18 1:22 PM, Volker Simonis wrote: > Hi Jiangli, > > thanks a lot explaining your plans more precisely. I think the > approach makes much more sense now. > > While I?m all for it in general, I still have some questions and > remarks :) > > 1. Will it be possible to use the two archives independantly ? I.e. > will it be possible to choose just one of the two archives as well as > a combination of both? > > 2. We should make sure to not introduce any performance regressions if > we have two archives (and potentially three SymbolTables and three > ?Metaspaces?) > > 3. In ?general I think we?re moving away from scenarios where we have > one single, central Java installation on a system which is used by the > ?tens/hundreds different applications? you mentioned in your mail > towards a world where every of these ??tens/hundreds different > applications? comes with its own, bundled ?JDK/JRE version (and as far > as I understand Oracle is propagating this new distribution model > after the deprecation of Applets/WebStart). In such a world it would > be not easy to have a static, common system layer archive. We could > try to keep the format of the archived classes as compatible between > Java versions as possible, although I understand that this will be > quite complicated. Do you already have any thoughts on this specific > problem? > I think the 2 level archive will work very well with containerized app deployment. For example, with docker, you can create a "base docker image" with the JDK and the "base" archive that contains the frequently used JDK and library classes. Then, you create an "app docker image" for each application, which contains the modules of the app, as well as the CDS archive of the classes used by this app. So imagine you have created 2 different app images from the same base image. When you deploy these 2 app images on the same hardware, they can share the files for the JDK and the base CDS archive between them. This could be quite significant (tens or 100s of MB). > 4. Finally we should design the new model in a way which allows it to > easily integrate AOT compiled code into the same one or two archives > in the future. Because with AOT we face the exactly same problems and > having to maintain four different archives will be not very user > friendly. OpenJ9 already uses a single archive for > CDT/AOT/ProfilingData. I don?t say we have to unify the AOT and CDT > archives in HotSpot now, I just want to suggest that if we?re > redesigning the CDS archive anyway to perhaps think twice before we > come to a new model such that we at least don?t exclude the > possibility of a unification with the AOT archive in the future. > There are 2 reasons we have not unified the AOT and CDS storage: 1. If we store AOT code inside the CDS archive, we would need to implement our own native code loader (similar to ld.so). This could be quite involving since we have many supported platforms. 2. If we store CDS data into AOT (e.g., add .text/.data sections to the .so files generated by AOT), we have very large overhead -- CDS has a lot of direct data pointers. E.g., InstanceKlass::_super, etc. Last time I checked, ELF uses 2x8 bytes of relocation data per pointer. All of these pointers are relocated when the .so is loaded, which means all the "read only" CDS data is effectively copied-on-write. So, I think we need a good justification to see why the unification is needed, before we try to resolve the technical issues. Thanks - Ioi > Thanks, > Volker > > > > Jiangli Zhou > schrieb am Fr. 4. Mai 2018 um 19:42: > > Hi Volker, > > Thank you so much for the feedbacks! > > You comments made me realize that some clarifications are need for > the top-layer dynamic archiving. The top-layer would include all > application classes, middleware classes, system library classes > that are not in the bottom layer. Apologizing for not stating that > clearly in the proposal. With the top layer including all > necessary classes (excluding the classes in bottom layer) for a > specific application, multiple instances running the same > application can achieve maximum memory sharing. Using your > example, the extra AWT/Swing classes are archived along with the > application classes in the top layer. > > With the dynamic archiving ability, it?s natural to think there is > no need for the two-layer scheme. In fact, I originally had the > thoughts. However, after exploring different possible use cases, I > agree two-layer archiving do bring benefits that are lacking in > the single archive design. One example of such use case is > tens/hundreds different applications running on the same machine. > Without a common system layer archive, it is difficult to achieve > memory sharing for common system classes, String objects, etc > between different applications. > > Another reason why we should go with a two-layer & static/dynamic > archiving is some runtime information may be sensitive and should > not be archived. Strings can be a good example in this case. > Currently, we archive interned String objects without running > application. The Strings include resolved CONSTANT_strings, class > names, method names, etc, which are all static information from > class files. During execution of an application, interned Strings > might contain sensitive user information, which is not safe to be > included in the archive. It?s difficult to distinguish between the > static information and runtime information if we go with the > single layer dynamic-only archiving. > > We had many discussions internally over long period of time on how > to improve usability with different approaches. In the end, > dynamic-only or static-only two-layer archiving all have their own > disadvantages and fail to meet requirements in some use cases. The > hybrid archiving combines benefits/advantages of different > approaches and seem to be flexible enough to fit most usages. > > Further comments and feedbacks are much appreciated. > > Best regards, > > Jiangli > > > On May 4, 2018, at 3:01 AM, Volker Simonis > > wrote: > > > > Hi Jiangli, > > > > thanks for sharing the hybrid archiving proposal. I think it is very > > interesting and useful! > > > > One issue I see is that the "system library classes" which should go > > into the "static" archive are inherently application specific > (i.e. if > > my application uses Swing, it will use most of the AWT/Swing > classes). > > So how will you cope with this problem. Either you put ALL the > system > > classes into the static archive which will be a waste for most > > applications. Or you just put a small commonly used subset of > classes > > into the static archive which may miss many classes for specific > > applications. > > > > If we would add the possibility to create a dynamic archive at > runtime > > / program end (which I think would be great from a usability > > perspective) I don't see a big need for two different archives. Two > > archives will further complicate (and slow down) things like Symbol > > lookup (we already have two SymbolTable now and we'd probably need a > > third one if we would have two archives). > > > > I don't think that running a few different Java applications on one > > host is the most important use case for CDS. In such a scenario the > > current, build time generated archive is probably the best we can do > > anyway. Instead, I think the most important use case is if we have > > many instances of the same Java application running on the same > host. > > And this is becoming more common with the raise of containerization. > > For the latter use case, a dynamically generated, application > specific > > archive would be the optimal solution. > > > > Thank you and best regards, > > Volker > > > > > > > > On Fri, May 4, 2018 at 3:42 AM, Jiangli Zhou > > wrote: > >> Hi Volker, > >> > >> Here are some details about the hybrid archiving. The goal is > to harvest the benefit of archiving by default and improve its > usability. The hybrid approach combines a two-layer archiving > (proposed by Ioi internally) and static & dynamic archiving > techniques: > >> > >> - Statically archive system library classes from a provided > classlist using the existing method. The archiving includes class > metadata, interned string objects, constant pool > resolved_references arrays, class mirror objects, etc. Static > archiving can be done at the JDK image build time and shipped > together with JDK binary. Following need to be addressed: > >>? ? ? ? *Relaxing the runtime CDS/AppCDS boot path check, so the > packaged archive can be used after the JDK binary is installed on > the target device. JDK-8199807 was created to address this issue > and is targeted for JDK 11. > >>? ? ? ? *Add the static archiving generation in JDK build steps > and package the generated archive with JDK image. The archive can > only be generated for the same target (both OS can CPU > architecture) as the build platform. I will create a RFE. > >> > >> - Dynamic archiving can done for application classes at the > first execution of a specific application > >>? ? ? ? * The archive is created on top of the default system > archive shipped with the JDK image. A separate top-layer archive > file is generated for each different application. > >>? ? ? ? * Archiving is done at the end of the application > execution before VM exists by relocating the class metadata to the > archive spaces. Cleanup also needs to be done for copied class > meta data to remove any runtime information. Most of the required > functionality already exists today. For example, class metadata > relocation was implemented by Ioi in JDK 10. > >>? ? ? ? * Only archive class metadata for application in the top > layer initially. Archiving java heap objects in the top-layer > requires more investigations. > >> > >> Benefits of the hybrid archiving: > >> * The system archive can be shared by different applications > and provides memory saving. > >> * Archive for application is created and used transparently. No > more profiling step and class list are required! > >> * Separating the system archiving from application archiving > reduces the cost of archiving at application execution time. The > overhead added to the first execution time is reduced. > >> > >> Thanks, > >> > >> Jiangli > >> > >> > >>> On May 3, 2018, at 10:34 AM, Jiangli Zhou > wrote: > >>> > >>> > >>>> On May 3, 2018, at 2:14 AM, Volker Simonis > > wrote: > >>>> > >>>> On Thu, May 3, 2018 at 11:01 AM, David Holmes > > wrote: > >>>>> On 3/05/2018 5:16 PM, Volker Simonis wrote: > >>>>>> > >>>>>> On Thu, May 3, 2018 at 8:55 AM, David Holmes > > > >>>>>> wrote: > >>>>>>> > >>>>>>> Just lurking here but ... > >>>>>>> > >>>>>>>> But is this really y relevant use case? Why would I like > to create ONE > >>>>>>>> archive for several apps? This would actually increase > the footprint > >>>>>>>> of a single instance which uses this archive. If I have > several apps I > >>>>>>>> would expect that users create a specific archive for > each app to get > >>>>>>>> the best out of CDS. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> One app instance may get increased footprint but you > presumably use CDS > >>>>>>> because you have multiple apps running (whether the same > or not). These > >>>>>>> apps > >>>>>>> all share the core JDK classes from the archive so the > overall footprint > >>>>>>> per > >>>>>>> instance is less. > >>>>>>> > >>>>>> > >>>>>> If we just want to share the core JDK classes that's easy. > For that we > >>>>>> could mostly use the default class list (or a slightly > extended one) > >>>>>> which is generated at JDK build time (at > JAVA_HOME/lib/classlist). > >>>>> > >>>>> > >>>>> The point is that you are presumably running multiple > instances of multiple > >>>>> apps, hence you want to share one set of core classes across > all, and share > >>>>> the app classes across each app instance. > >>>>> > >>>> > >>>> But that would require two archives: a general one with the core > >>>> classes and an application specific one for each application. > >>>> Combining the core classes and the application of various > applications > >>>> will not be optimal because the application classes will be > all mixed > >>>> in the same archive. The archive is being mapped page-wise > into the > >>>> java process so you'll probably end up mapping the whole > archive into > >>>> each process although you'll only use a fraction of the > classes in the > >>>> archive. > >>>> > >>>>>> If we want to use ONE archive for several applications and > we can > >>>>>> accept to have a bigger footprint if running a single (or > just a few) > >>>>>> applications in parallel I suppose the overhead of simply > dumping all > >>>>>> the classes from the classpathes of the various > applications compared > >>>>>> to an accurate solution where we only dump the actually > used classes > >>>>>> of all applications would be not that big. > >>>>> > >>>>> > >>>>> But those "accurate" solutions duplicate the core classes > and that's a waste > >>>>> of footprint. > >>>>> > >>>> > >>>> By "accurate" I meant one "fat" archive which contains all > the classes > >>>> USED by several applications plus the core classes. My > argument was > >>>> that such an "accurate" "fat" archive won't be much smaller > compared > >>>> to a "fat" archive which simply contains all the core classes > plus all > >>>> the application classes (i.e. from the application class > pathes, no > >>>> matter if they are ever used or not). But the latter would be > much > >>>> simpler to implement. > >>> > >>> The above discussion and an internal proposal for hybrid > archiving seem to converge on a few points. If there is no > objection to the hybrid archiving proposal internally, maybe we > can shared the details of the proposal on openjdk soon. > >>> > >>> Thanks, > >>> > >>> Jiangli > >>> > >>> > >>>> > >>>>> David > >>>>> ----- > >>>>> > >>>>> > >>>>>>> David > >>>>>>> ----- > >>>>>>> > >>>>>>> > >>>>>>> On 3/05/2018 4:48 PM, Volker Simonis wrote: > >>>>>>>> > >>>>>>>> > >>>>>>>> On Thu, May 3, 2018 at 6:52 AM, Ioi Lam > > wrote: > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On 5/2/18 10:00 AM, Volker Simonis wrote: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On Tue, May 1, 2018 at 8:32 PM, Ioi Lam > > wrote: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> PROBLEM: > >>>>>>>>>>> > >>>>>>>>>>> As discussed with Volker and Yumin in previous > e-mails, AppCDS has > >>>>>>>>>>> some > >>>>>>>>>>> experimental support for custom class loaders. > However, it's not very > >>>>>>>>>>> easy > >>>>>>>>>>> to use. > >>>>>>>>>>> > >>>>>>>>>>> For example, you can write a classlist like this: > >>>>>>>>>>> > >>>>>>>>>>> ?java/lang/Object id: 1 > >>>>>>>>>>> ?CustomLoadee id: 2 super: 1 source: /tmp/foo.jar > >>>>>>>>>>> > >>>>>>>>>>> The CustomLoadee class will be stored in the shared > archive with a > >>>>>>>>>>> CRC > >>>>>>>>>>> code. > >>>>>>>>>>> During runtime, if a customed loader wants to load a > class of the > >>>>>>>>>>> same > >>>>>>>>>>> name, > >>>>>>>>>>> and its classfile has the same size and CRC as the > archived class, > >>>>>>>>>>> the > >>>>>>>>>>> archived version will be loaded. This speeds up class > loading by > >>>>>>>>>>> avoiding > >>>>>>>>>>> parsing the class file, and saves space by sharing the > mmap'ed class > >>>>>>>>>>> metadata across processes. > >>>>>>>>>>> > >>>>>>>>>>> You can see an example test at: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java > >>>>>>>>>>> > >>>>>>>>>>> However, the current scheme requires you to specify > all the super > >>>>>>>>>>> classes > >>>>>>>>>>> and interfaces. There's no support provided by the > >>>>>>>>>>> -XX:DumpLoadedClassList > >>>>>>>>>>> option. It can be helped somewhat with Volker's tool: > >>>>>>>>>>> https://github.com/simonis/cl4cds > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> POSSIBLE SOLUTIONS: > >>>>>>>>>>> > >>>>>>>>>>> 1. "Dump-as-you-go". As suggested by Yumin, we can > provide a jcmd to > >>>>>>>>>>> ask > >>>>>>>>>>> a > >>>>>>>>>>> running JVM process to dump all of its loaded classes, > including > >>>>>>>>>>> those > >>>>>>>>>>> loaded by custom loaders, into an archive. An > alternative is to dump > >>>>>>>>>>> the > >>>>>>>>>>> archive at JVM exit time (or when you press Ctrl-C, etc. > >>>>>>>>>>> > >>>>>>>>>>> 2. Add information about the custom classes for > >>>>>>>>>>> -XX:DumpLoadedClassList. > >>>>>>>>>>> The > >>>>>>>>>>> trouble is some class loaders don't specify a code > source that can be > >>>>>>>>>>> understood by the built-in class loaders. For example, > the "Fat Jars" > >>>>>>>>>>> would > >>>>>>>>>>> have a code source like > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/ > >>>>>>>>>>> > >>>>>>>>>>> also, many custom loaders would pre-process the > classfile data before > >>>>>>>>>>> defining the class, so we can't simply archive the > version of the > >>>>>>>>>>> class > >>>>>>>>>>> on > >>>>>>>>>>> disk. > >>>>>>>>>>> > >>>>>>>>>>> One possible solution for #2 is to include the class > file data in the > >>>>>>>>>>> -XX:DumpLoadedClassList output: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> ?java/lang/Object id: 1 > >>>>>>>>>>> ?CustomLoadee id: 2 super: 1 source: base64 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA > >>>>>>>>>>> ?AgAPAAcAAAAKAAEABQADAAYACA== > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Of the 2 solutions: > >>>>>>>>>>> > >>>>>>>>>>> #1 seems easier to use, but may require more invasive > modifications > >>>>>>>>>>> in > >>>>>>>>>>> the > >>>>>>>>>>> VM, especially if you want to be able to continue > execution after > >>>>>>>>>>> dumping. > >>>>>>>>>>> > >>>>>>>>>> Not sure what #1 really proposes: dumping the complete > .jsa archive at > >>>>>>>>>> runtime or dumping just the loaded classes. > >>>>>>>>>> > >>>>>>>>>> If it's just about dumping the loaded class without > generating the > >>>>>>>>>> .jsa archive there's the problem that by default the VM > doesn't store > >>>>>>>>>> the exact bytes of a class after the class was loaded > (except when > >>>>>>>>>> class transformers are registered). So the class files > would have to > >>>>>>>>>> be re-assembled from the internal VM structures (in the > same way this > >>>>>>>>>> is done for class redefinition) and the resulting > class-file may be > >>>>>>>>>> different from the original bytes (i.e. some attributes > may be > >>>>>>>>>> missing). > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> #1 is for creating the JSA file, not just dumping the > class files. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> If #1 is about creating the whole .jsa archive at > runtime (or at VM > >>>>>>>>>> exit) I think that would be the most attractive > solution from a > >>>>>>>>>> usability point of view although I understand that #2 > will be easier > >>>>>>>>>> to implement in the short term. Regarding the argument > that #1 will > >>>>>>>>>> produce a "binary blob" that's true, but that's already > true now when > >>>>>>>>>> we use "Xshare:dump". I think it should be not to hard > to implement a > >>>>>>>>>> tool based an SA which could introspect a .jsa archive. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> The argument about the binary blob is to compare it > against the text > >>>>>>>>> file > >>>>>>>>> produced by -XX:DumpLoadedClassList. > >>>>>>>>> > >>>>>>>>> One use case to consider is when you have a JAR file > that contains > >>>>>>>>> several > >>>>>>>>> apps that each load a unique set of classes. Today, > (assuming that > >>>>>>>>> custom > >>>>>>>>> class loaders are not used), you can run each app once with > >>>>>>>>> -XX:DumpLoadedClassList, and then do an > >>>>>>>>> > >>>>>>>>>? ? cat *.classlist | sort | uniq > combined.classlist > >>>>>>>>> > >>>>>>>>> and then create an archive that would work for all these > apps. > >>>>>>>>> > >>>>>>>> > >>>>>>>> But is this really y relevant use case? Why would I like > to create ONE > >>>>>>>> archive for several apps? This would actually increase > the footprint > >>>>>>>> of a single instance which uses this archive. If I have > several apps I > >>>>>>>> would expect that users create a specific archive for > each app to get > >>>>>>>> the best out of CDS. > >>>>>>>> > >>>>>>>>> With the binary blob, there's no easy way of doing this. > It will be > >>>>>>>>> very > >>>>>>>>> difficult to write a tool to decipher each blob and then > somehow > >>>>>>>>> combine > >>>>>>>>> them into a single one. > >>>>>>>>> > >>>>>>>> > >>>>>>>> But if users really wants such a "fat" archive, there's a > much easier > >>>>>>>> way: just dump ALL the classes from the .jar file into > the archive. A > >>>>>>>> class list for this could easily be assembled either with > an external > >>>>>>>> tool like cl4cds (or even a simple shell scripts which > converts the > >>>>>>>> output of `unzip -l ` into the correct format). > Or, even > >>>>>>>> simpler, by adding a new option to the VM similar to > >>>>>>>> -XX:DumpLoadedClassList which dumps all the classes it > can find on the > >>>>>>>> class path (and potentially other, configurable locations). > >>>>>>>> > >>>>>>>>> Thanks > >>>>>>>>> - Ioi > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>>> #2 would be easier to implement, but the classlist > might be huge. > >>>>>>>>>>> > >>>>>>>>>>> Also, #2 would allow post-processing tools to remove > unneeded > >>>>>>>>>>> classes, > >>>>>>>>>>> or > >>>>>>>>>>> merge two runs into a single list. The output of #1 is > essentially a > >>>>>>>>>>> binary > >>>>>>>>>>> blob that's impossible for off-line > analysis/optimizations. > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Any comments, or suggestions for alternatives? > >>>>>>>>>>> > >>>>>>>>>>> Thanks > >>>>>>>>>>> - Ioi > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>> > >>> > >> > From david.holmes at oracle.com Mon May 7 21:03:05 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 8 May 2018 07:03:05 +1000 Subject: RFR(M): 8201593: Print array length in ArrayIndexOutOfBoundsException. In-Reply-To: References: <504bdd8469bb45cc8c5cf790d1f4b3fc@sap.com> <12e8ad2367484b0b88bcee4bcb77ac47@sap.com> <033adcb3-1ab0-094e-78cf-4062a8868d3e@redhat.com> <7767af56c6744e8aa5479a5dd9626936@sap.com> <9d2753f6975c4477912420a2fe459ef3@sap.com> <14107212c2664aee9965e9f9da8006ad@sap.com> <54e1796c-46e8-18be-2773-69636c4b4c0d@oracle.com> <7887adc187914fef9b4a77de28b1745f@sap.com> <243f21b7-ebb6-a372-1536-7d55c76acf01@oracle.com> Message-ID: Nothing further from me. Thanks, David On 7/05/2018 10:20 PM, Lindenmaier, Goetz wrote: > Hi, > > Webrev withoug %i: > http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/08/ > it passed the jdk/submit testing. > > Best regards, > Goetz. > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Montag, 7. Mai 2018 11:00 >> To: Lindenmaier, Goetz ; Stuart Monteith >> >> Cc: hotspot-compiler-dev at openjdk.java.net; aarch64-port- >> dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; aarch32- >> port-dev at openjdk.java.net >> Subject: Re: RFR(M): 8201593: Print array length in >> ArrayIndexOutOfBoundsException. >> >> Hi Goetz, >> >> On 7/05/2018 6:16 PM, Lindenmaier, Goetz wrote: >>> Hi David, >>> >>> New webrev with the punctuation changed: >>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/07/ >>> For the punctuation see also my mail reply to Roger's mail. >> >> Okay. That seems okay. >> >> Only further oddity I noticed is the use of %i rather than %d. Use of %i >> is very rare in hotspot. (I had to go and lookup what it means :) ). >> >> Thanks, >> David >> >>>> -----Original Message----- >>>> From: David Holmes [mailto:david.holmes at oracle.com] >>>> Sent: Montag, 7. Mai 2018 09:30 >>>> >>>> Hi Goetz, >>>> >>>> On 4/05/2018 7:22 PM, Lindenmaier, Goetz wrote: >>>>> Hi, >>>>> >>>>> thanks to Aleksey and Boris this is now also tested on arm. >>>>> This final webrev contains some fixes needed in the arm files: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06-arm/ >>>>> >>>>> David, can I consider this as finally reviewed? >>>> >>>> This follows on top of my +1 comments to Roger about the message >>>> consistency and punctuation etc. >>>> >>>> Aside: Took me a while to realize the >>>> throw_index_out_of_bounds_exception field was not a throw/don't- >> throw >>>> flag, but a throw AIOOBE or IOOBE flag! >>> Yes, this is not very intuitive ... >>> >>>> Again note I can't comment on the detailed CPU specific code. >>> The CPU code was reviewed by Martin Doerr, Stuart Monteith, Aleksey >> Shipilev and Boris Ulasevich. >>> >>>> One further nit: >>>> src/hotspot/cpu/aarch64/templateTable_aarch64.cpp >>>> I don't like the >>>> // ??? convention >>>> comments. It suggests the code is not understood. I don't expect you to >>>> fix existing ones but adding new ones doesn't seem good. >>> I don't like that either, nor the design of using a convention here. >>> The reviewers had trouble with that, too. >>> But as the two values are handled similarly, I would to like to >>> document it similarly, following the existing format. Stuart >>> was fine with that, anyways. >>> >>> An improvement of the design how these values are handled >>> would require changes on all platforms (it's similarly bad everywhere) >>> and I don't like to do that in this scope. >>> >>> Best regards, >>> Goetz. >>> >>> >>>> >>>> >>>> Thanks, >>>> David >>>> >>>>> Best regards, >>>>> Goetz >>>>> >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: aarch64-port-dev [mailto:aarch64-port-dev- >>>>>> bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz >>>>>> Sent: Mittwoch, 2. Mai 2018 17:57 >>>>>> To: Stuart Monteith >>>>>> Cc: hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- >>>>>> dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; aarch32- >>>> port- >>>>>> dev at openjdk.java.net >>>>>> Subject: [CAUTION] Re: [aarch64-port-dev ] RFR(M): 8201593: Print >> array >>>>>> length in ArrayIndexOutOfBoundsException. >>>>>> >>>>>> Hi, >>>>>> >>>>>> I needed to move the edit from c1_LIRGenerator_.cpp to >>>>>> the shared file after "8201543: Modularize C1 GC barriers" >>>>>> New webrev: >>>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06/ >>>>>> >>>>>> @Stuart >>>>>> Thanks for testing! >>>>>>> so as to accommodate the array pointer you are pushing onto the >> stack? >>>>>> Yes, what you are pointing out seems to be wrong, I changed it to '2'. >>>>>> >>>>>> Best regards, >>>>>> Goetz. >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Stuart Monteith [mailto:stuart.monteith at linaro.org] >>>>>>> Sent: Freitag, 27. April 2018 16:37 >>>>>>> To: Lindenmaier, Goetz >>>>>>> Cc: hotspot-compiler-dev at openjdk.java.net; aarch64-port- >>>>>>> dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; >>>> aarch32- >>>>>>> port-dev at openjdk.java.net >>>>>>> Subject: Re: RFR(M): 8201593: Print array length in >>>>>>> ArrayIndexOutOfBoundsException. >>>>>>> >>>>>>> Hi, >>>>>>> JTregs hasn't flagged any issues, so it should be ok. >>>>>>> >>>>>>> Regarding the 32-bit arm code, in "void >>>>>>> RangeCheckStub::emit_code(LIR_Assembler* ce)" should: >>>>>>> ce->verify_reserved_argument_area_size(1); >>>>>>> be >>>>>>> ce->verify_reserved_argument_area_size(2); >>>>>>> >>>>>>> so as to accommodate the array pointer you are pushing onto the >> stack? >>>>>>> >>>>>>> I've not tested 32-bit arm. >>>>>>> >>>>>>> >>>>>>> BR, >>>>>>> Stuart >>>>>>> >>>>>>> On 26 April 2018 at 15:31, Stuart Monteith >> >>>>>>> wrote: >>>>>>>> Thanks, I'm happy with that. >>>>>>>> >>>>>>>> The registers have a clean path to call_RT - r22 and r23 aren't used >>>>>>>> inbetween. They are an arbitrary choice - c_rarg0 and c_rarg1 were >>>>>>>> always going to cause problems. If _array->as_pointer_register() >>>>>>>> and/or _index->as_register() or _index->as_jint() were the registers >>>>>>>> we were using as parameters there would be trouble. However, >> with >>>>>>>> pd_last_allocatable_cpu_reg = 16, that shouldn't happen with >> r22/23, >>>>>>>> or indeed anything else in the range r17 to r28. >>>>>>>> >>>>>>>> I'm going to run all of JTRegs and seem what that produces now. >>>>>>>> >>>>>>>> BR, >>>>>>>> Stuart >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 26 April 2018 at 15:14, Lindenmaier, Goetz >>>>>>> wrote: >>>>>>>>> Hi Stuart, >>>>>>>>> >>>>>>>>> thanks for fixing this! Webrev with your changes: >>>>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/05/ >>>>>>>>> >>>>>>>>>> There is the possibility of overwriting live values though, aren't >>>>>>>>>> there? The registers are saved by call_RT. Should I be concerned >>>> about >>>>>>>>>> deopt and debugging going wrong? Furthermore, won't there be >>>>>> issues >>>>>>> in >>>>>>>>>> exception handlers? >>>>>>>>> As I understand, this just has to survive the far_call. >>>>>>>>> The call_RT in c1_Runtime then moves it into the >>>>>>>>> proper argument registers. This is just the handling of an >>>>>>>>> exception, and in these few instructions no java code is >>>>>>>>> executed, no safepoint is passed, so this should be fine. >>>>>>>>> >>>>>>>>> incremental diff: >>>>>>>>> iff -r 874f2b999ff6 >>>> src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp >>>>>>>>> --- a/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Mon >> Apr >>>> 16 >>>>>>> 15:17:20 2018 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Thu >> Apr >>>> 26 >>>>>>> 15:55:18 2018 +0200 >>>>>>>>> @@ -75,16 +75,16 @@ >>>>>>>>> } >>>>>>>>> >>>>>>>>> if (_index->is_cpu_register()) { >>>>>>>>> - __ mov(rscratch1, _index->as_register()); >>>>>>>>> + __ mov(r22, _index->as_register()); >>>>>>>>> } else { >>>>>>>>> - __ mov(rscratch1, _index->as_jint()); >>>>>>>>> + __ mov(r22, _index->as_jint()); >>>>>>>>> } >>>>>>>>> Runtime1::StubID stub_id; >>>>>>>>> if (_throw_index_out_of_bounds_exception) { >>>>>>>>> stub_id = Runtime1::throw_index_exception_id; >>>>>>>>> } else { >>>>>>>>> assert(_array != NULL, "sanity"); >>>>>>>>> - __ mov(rscratch2, _array->as_pointer_register()); >>>>>>>>> + __ mov(r23, _array->as_pointer_register()); >>>>>>>>> stub_id = Runtime1::throw_range_check_failed_id; >>>>>>>>> } >>>>>>>>> __ far_call(RuntimeAddress(Runtime1::entry_for(stub_id)), >> NULL, >>>>>>> rscratch2); >>>>>>>>> diff -r 874f2b999ff6 >>>> src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp >>>>>>>>> --- a/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Mon >> Apr >>>> 16 >>>>>>> 15:17:20 2018 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Thu >> Apr >>>> 26 >>>>>>> 15:55:18 2018 +0200 >>>>>>>>> @@ -327,7 +327,7 @@ >>>>>>>>> >>>>>>>>> >>>>>>>>> // target: the entry point of the method that creates and posts >> the >>>>>>> exception oop >>>>>>>>> -// has_argument: true if the exception needs an argument >> (passed >>>> in >>>>>>> rscratch1) >>>>>>>>> +// has_argument: true if the exception needs arguments (passed >> in >>>>>> r22 >>>>>>> and r23) >>>>>>>>> >>>>>>>>> OopMapSet* >>>> Runtime1::generate_exception_throw(StubAssembler* >>>>>>> sasm, address target, bool has_argument) { >>>>>>>>> // make a frame and preserve the caller's caller-save registers >>>>>>>>> @@ -336,7 +336,7 @@ >>>>>>>>> if (!has_argument) { >>>>>>>>> call_offset = __ call_RT(noreg, noreg, target); >>>>>>>>> } else { >>>>>>>>> - call_offset = __ call_RT(noreg, noreg, target, rscratch1, >> rscratch2); >>>>>>>>> + call_offset = __ call_RT(noreg, noreg, target, r22, r23); >>>>>>>>> } >>>>>>>>> OopMapSet* oop_maps = new OopMapSet(); >>>>>>>>> oop_maps->add_gc_map(call_offset, oop_map); >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> Goetz. >>>>>>>>> >>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Stuart Monteith [mailto:stuart.monteith at linaro.org] >>>>>>>>>> Sent: Donnerstag, 26. April 2018 12:52 >>>>>>>>>> To: Andrew Haley >>>>>>>>>> Cc: Lindenmaier, Goetz ; hotspot- >>>>>>> compiler- >>>>>>>>>> dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; >>>> hotspot- >>>>>>>>>> runtime-dev at openjdk.java.net; aarch32-port- >>>> dev at openjdk.java.net >>>>>>>>>> Subject: Re: RFR(M): 8201593: Print array length in >>>>>>>>>> ArrayIndexOutOfBoundsException. >>>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> Using c_rarg1 and c_rarg2 instead of rscratch1 and overwriting >>>>>>>>>> rscratch2 causes a SIGSEGV. >>>>>>>>>> Using r22 and r23 instead, the test ran successfully. >>>>>>>>>> >>>>>>>>>> In c1_CodeStubs_aarch64.cpp >>>>>>>>>> : >>>>>>>>>> 77 if (_index->is_cpu_register()) { >>>>>>>>>> 78 __ mov(r22, _index->as_register()); >>>>>>>>>> 79 } else { >>>>>>>>>> 80 __ mov(r22, _index->as_jint()); >>>>>>>>>> 81 } >>>>>>>>>> 82 Runtime1::StubID stub_id; >>>>>>>>>> 83 if (_throw_index_out_of_bounds_exception) { >>>>>>>>>> 84 stub_id = Runtime1::throw_index_exception_id; >>>>>>>>>> 85 } else { >>>>>>>>>> 86 assert(_array != NULL, "sanity"); >>>>>>>>>> 87 __ mov(r23, _array->as_pointer_register()); >>>>>>>>>> 88 stub_id = Runtime1::throw_range_check_failed_id; >>>>>>>>>> 89 } >>>>>>>>>> >>>>>>>>>> in c1_Runtime_aarch64.cpp: >>>>>>>>>> >>>>>>>>>> 336 if (!has_argument) { >>>>>>>>>> 337 call_offset = __ call_RT(noreg, noreg, target); >>>>>>>>>> 338 } else { >>>>>>>>>> 339 call_offset = __ call_RT(noreg, noreg, target, r22, r23); >>>>>>>>>> 340 } >>>>>>>>>> >>>>>>>>>> There is the possibility of overwriting live values though, aren't >>>>>>>>>> there? The registers are saved by call_RT. Should I be concerned >>>> about >>>>>>>>>> deopt and debugging going wrong? Furthermore, won't there be >>>>>> issues >>>>>>> in >>>>>>>>>> exception handlers? >>>>>>>>>> >>>>>>>>>> BR, >>>>>>>>>> Stuart >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 25 April 2018 at 16:49, Stuart Monteith >>>>>> >>>>>>>>>> wrote: >>>>>>>>>>> Indeed - and that is what I am seeing. Usually no parameters are >>>>>> being >>>>>>>>>>> called with this pattern, or rscratch1, with the temporary variable >>>>>>>>>>> being changed to use rscratch2 in such circumstances. >>>>>>>>>>> I'll try c_rarg1 and c_rarg2 - they should pass straight through,if I >>>>>>>>>>> interpret the code correcting. >>>>>>>>>>> >>>>>>>>>>> On 25 April 2018 at 16:26, Andrew Haley >> wrote: >>>>>>>>>>>> On 04/25/2018 04:00 PM, Stuart Monteith wrote: >>>>>>>>>>>>> I'm not quite sure to solve this yet - we'll need to use the >> stack in >>>>>>>>>>>>> some safe way. >>>>>>>>>>>> >>>>>>>>>>>> It's not a great idea to pass arguments in rscratch1 or rscratch2. >>>>>> These >>>>>>>>>>>> registers are for use in macros and should be treated as >> volatile. >>>>>>> Given >>>>>>>>>>>> that you're throwing an exception, registers will be clobbered >>>>>>> anyway. >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Andrew Haley >>>>>>>>>>>> Java Platform Lead Engineer >>>>>>>>>>>> Red Hat UK Ltd. >>>>>>>>>>>> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From coleen.phillimore at oracle.com Mon May 7 21:14:13 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 7 May 2018 17:14:13 -0400 Subject: RFR (trivial) 8202606: Deprecate AllowNonVirtualCalls option In-Reply-To: <24525a32-b3d7-459f-8eec-3b37d53d6a6d@oracle.com> References: <3dc8508f-b924-7df7-3d52-1a57fda2081b@oracle.com> <24525a32-b3d7-459f-8eec-3b37d53d6a6d@oracle.com> Message-ID: <64acfa3c-2cd4-d54b-f85d-775b0eef360d@oracle.com> On 5/7/18 4:37 PM, David Holmes wrote: > Hi Coleen, > > On 8/05/2018 4:09 AM, coleen.phillimore at oracle.com wrote: >> The CSR request is approved, see bug for details.? Tested with java >> -XX:+AllowNonVirtualCalls -version >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8202606.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8202606 > > Please also add it to the runtime/CommandLine/VMDeprecatedOptions.java > test. Oh, ok. thanks, Coleen > > Thanks, > David > >> Thanks, >> Coleen From goetz.lindenmaier at sap.com Mon May 7 21:21:53 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 7 May 2018 21:21:53 +0000 Subject: RFR(M): 8201593: Print array length in ArrayIndexOutOfBoundsException. In-Reply-To: References: <504bdd8469bb45cc8c5cf790d1f4b3fc@sap.com> <12e8ad2367484b0b88bcee4bcb77ac47@sap.com> <033adcb3-1ab0-094e-78cf-4062a8868d3e@redhat.com> <7767af56c6744e8aa5479a5dd9626936@sap.com> <9d2753f6975c4477912420a2fe459ef3@sap.com> <14107212c2664aee9965e9f9da8006ad@sap.com> <54e1796c-46e8-18be-2773-69636c4b4c0d@oracle.com> <7887adc187914fef9b4a77de28b1745f@sap.com> <243f21b7-ebb6-a372-1536-7d55c76acf01@oracle.com> Message-ID: <5eb6fbbb319d4ce0bd6f4a5e4f2979ba@sap.com> Hi David, Thanks a lot! I'll push it tomorrow once our tests have passed. Best regards, Goetz. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Monday, May 7, 2018 11:03 PM > To: Lindenmaier, Goetz ; Stuart Monteith > > Cc: hotspot-compiler-dev at openjdk.java.net; aarch64-port- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; aarch32- > port-dev at openjdk.java.net > Subject: Re: RFR(M): 8201593: Print array length in > ArrayIndexOutOfBoundsException. > > Nothing further from me. > > Thanks, > David > > On 7/05/2018 10:20 PM, Lindenmaier, Goetz wrote: > > Hi, > > > > Webrev withoug %i: > > http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/08/ > > it passed the jdk/submit testing. > > > > Best regards, > > Goetz. > > > >> -----Original Message----- > >> From: David Holmes [mailto:david.holmes at oracle.com] > >> Sent: Montag, 7. Mai 2018 11:00 > >> To: Lindenmaier, Goetz ; Stuart Monteith > >> > >> Cc: hotspot-compiler-dev at openjdk.java.net; aarch64-port- > >> dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; > aarch32- > >> port-dev at openjdk.java.net > >> Subject: Re: RFR(M): 8201593: Print array length in > >> ArrayIndexOutOfBoundsException. > >> > >> Hi Goetz, > >> > >> On 7/05/2018 6:16 PM, Lindenmaier, Goetz wrote: > >>> Hi David, > >>> > >>> New webrev with the punctuation changed: > >>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/07/ > >>> For the punctuation see also my mail reply to Roger's mail. > >> > >> Okay. That seems okay. > >> > >> Only further oddity I noticed is the use of %i rather than %d. Use of %i > >> is very rare in hotspot. (I had to go and lookup what it means :) ). > >> > >> Thanks, > >> David > >> > >>>> -----Original Message----- > >>>> From: David Holmes [mailto:david.holmes at oracle.com] > >>>> Sent: Montag, 7. Mai 2018 09:30 > >>>> > >>>> Hi Goetz, > >>>> > >>>> On 4/05/2018 7:22 PM, Lindenmaier, Goetz wrote: > >>>>> Hi, > >>>>> > >>>>> thanks to Aleksey and Boris this is now also tested on arm. > >>>>> This final webrev contains some fixes needed in the arm files: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06- > arm/ > >>>>> > >>>>> David, can I consider this as finally reviewed? > >>>> > >>>> This follows on top of my +1 comments to Roger about the message > >>>> consistency and punctuation etc. > >>>> > >>>> Aside: Took me a while to realize the > >>>> throw_index_out_of_bounds_exception field was not a throw/don't- > >> throw > >>>> flag, but a throw AIOOBE or IOOBE flag! > >>> Yes, this is not very intuitive ... > >>> > >>>> Again note I can't comment on the detailed CPU specific code. > >>> The CPU code was reviewed by Martin Doerr, Stuart Monteith, Aleksey > >> Shipilev and Boris Ulasevich. > >>> > >>>> One further nit: > >>>> src/hotspot/cpu/aarch64/templateTable_aarch64.cpp > >>>> I don't like the > >>>> // ??? convention > >>>> comments. It suggests the code is not understood. I don't expect you > to > >>>> fix existing ones but adding new ones doesn't seem good. > >>> I don't like that either, nor the design of using a convention here. > >>> The reviewers had trouble with that, too. > >>> But as the two values are handled similarly, I would to like to > >>> document it similarly, following the existing format. Stuart > >>> was fine with that, anyways. > >>> > >>> An improvement of the design how these values are handled > >>> would require changes on all platforms (it's similarly bad everywhere) > >>> and I don't like to do that in this scope. > >>> > >>> Best regards, > >>> Goetz. > >>> > >>> > >>>> > >>>> > >>>> Thanks, > >>>> David > >>>> > >>>>> Best regards, > >>>>> Goetz > >>>>> > >>>>> > >>>>> > >>>>>> -----Original Message----- > >>>>>> From: aarch64-port-dev [mailto:aarch64-port-dev- > >>>>>> bounces at openjdk.java.net] On Behalf Of Lindenmaier, Goetz > >>>>>> Sent: Mittwoch, 2. Mai 2018 17:57 > >>>>>> To: Stuart Monteith > >>>>>> Cc: hotspot-compiler-dev at openjdk.java.net; hotspot-runtime- > >>>>>> dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; > aarch32- > >>>> port- > >>>>>> dev at openjdk.java.net > >>>>>> Subject: [CAUTION] Re: [aarch64-port-dev ] RFR(M): 8201593: Print > >> array > >>>>>> length in ArrayIndexOutOfBoundsException. > >>>>>> > >>>>>> Hi, > >>>>>> > >>>>>> I needed to move the edit from c1_LIRGenerator_.cpp to > >>>>>> the shared file after "8201543: Modularize C1 GC barriers" > >>>>>> New webrev: > >>>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593-lenInAIOOB/06/ > >>>>>> > >>>>>> @Stuart > >>>>>> Thanks for testing! > >>>>>>> so as to accommodate the array pointer you are pushing onto the > >> stack? > >>>>>> Yes, what you are pointing out seems to be wrong, I changed it to '2'. > >>>>>> > >>>>>> Best regards, > >>>>>> Goetz. > >>>>>> > >>>>>> > >>>>>>> -----Original Message----- > >>>>>>> From: Stuart Monteith [mailto:stuart.monteith at linaro.org] > >>>>>>> Sent: Freitag, 27. April 2018 16:37 > >>>>>>> To: Lindenmaier, Goetz > >>>>>>> Cc: hotspot-compiler-dev at openjdk.java.net; aarch64-port- > >>>>>>> dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; > >>>> aarch32- > >>>>>>> port-dev at openjdk.java.net > >>>>>>> Subject: Re: RFR(M): 8201593: Print array length in > >>>>>>> ArrayIndexOutOfBoundsException. > >>>>>>> > >>>>>>> Hi, > >>>>>>> JTregs hasn't flagged any issues, so it should be ok. > >>>>>>> > >>>>>>> Regarding the 32-bit arm code, in "void > >>>>>>> RangeCheckStub::emit_code(LIR_Assembler* ce)" should: > >>>>>>> ce->verify_reserved_argument_area_size(1); > >>>>>>> be > >>>>>>> ce->verify_reserved_argument_area_size(2); > >>>>>>> > >>>>>>> so as to accommodate the array pointer you are pushing onto the > >> stack? > >>>>>>> > >>>>>>> I've not tested 32-bit arm. > >>>>>>> > >>>>>>> > >>>>>>> BR, > >>>>>>> Stuart > >>>>>>> > >>>>>>> On 26 April 2018 at 15:31, Stuart Monteith > >> > >>>>>>> wrote: > >>>>>>>> Thanks, I'm happy with that. > >>>>>>>> > >>>>>>>> The registers have a clean path to call_RT - r22 and r23 aren't used > >>>>>>>> inbetween. They are an arbitrary choice - c_rarg0 and c_rarg1 > were > >>>>>>>> always going to cause problems. If _array->as_pointer_register() > >>>>>>>> and/or _index->as_register() or _index->as_jint() were the > registers > >>>>>>>> we were using as parameters there would be trouble. However, > >> with > >>>>>>>> pd_last_allocatable_cpu_reg = 16, that shouldn't happen with > >> r22/23, > >>>>>>>> or indeed anything else in the range r17 to r28. > >>>>>>>> > >>>>>>>> I'm going to run all of JTRegs and seem what that produces now. > >>>>>>>> > >>>>>>>> BR, > >>>>>>>> Stuart > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On 26 April 2018 at 15:14, Lindenmaier, Goetz > >>>>>>> wrote: > >>>>>>>>> Hi Stuart, > >>>>>>>>> > >>>>>>>>> thanks for fixing this! Webrev with your changes: > >>>>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8201593- > lenInAIOOB/05/ > >>>>>>>>> > >>>>>>>>>> There is the possibility of overwriting live values though, aren't > >>>>>>>>>> there? The registers are saved by call_RT. Should I be > concerned > >>>> about > >>>>>>>>>> deopt and debugging going wrong? Furthermore, won't there > be > >>>>>> issues > >>>>>>> in > >>>>>>>>>> exception handlers? > >>>>>>>>> As I understand, this just has to survive the far_call. > >>>>>>>>> The call_RT in c1_Runtime then moves it into the > >>>>>>>>> proper argument registers. This is just the handling of an > >>>>>>>>> exception, and in these few instructions no java code is > >>>>>>>>> executed, no safepoint is passed, so this should be fine. > >>>>>>>>> > >>>>>>>>> incremental diff: > >>>>>>>>> iff -r 874f2b999ff6 > >>>> src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp > >>>>>>>>> --- a/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp Mon > >> Apr > >>>> 16 > >>>>>>> 15:17:20 2018 +0200 > >>>>>>>>> +++ b/src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp > Thu > >> Apr > >>>> 26 > >>>>>>> 15:55:18 2018 +0200 > >>>>>>>>> @@ -75,16 +75,16 @@ > >>>>>>>>> } > >>>>>>>>> > >>>>>>>>> if (_index->is_cpu_register()) { > >>>>>>>>> - __ mov(rscratch1, _index->as_register()); > >>>>>>>>> + __ mov(r22, _index->as_register()); > >>>>>>>>> } else { > >>>>>>>>> - __ mov(rscratch1, _index->as_jint()); > >>>>>>>>> + __ mov(r22, _index->as_jint()); > >>>>>>>>> } > >>>>>>>>> Runtime1::StubID stub_id; > >>>>>>>>> if (_throw_index_out_of_bounds_exception) { > >>>>>>>>> stub_id = Runtime1::throw_index_exception_id; > >>>>>>>>> } else { > >>>>>>>>> assert(_array != NULL, "sanity"); > >>>>>>>>> - __ mov(rscratch2, _array->as_pointer_register()); > >>>>>>>>> + __ mov(r23, _array->as_pointer_register()); > >>>>>>>>> stub_id = Runtime1::throw_range_check_failed_id; > >>>>>>>>> } > >>>>>>>>> __ far_call(RuntimeAddress(Runtime1::entry_for(stub_id)), > >> NULL, > >>>>>>> rscratch2); > >>>>>>>>> diff -r 874f2b999ff6 > >>>> src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp > >>>>>>>>> --- a/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Mon > >> Apr > >>>> 16 > >>>>>>> 15:17:20 2018 +0200 > >>>>>>>>> +++ b/src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp Thu > >> Apr > >>>> 26 > >>>>>>> 15:55:18 2018 +0200 > >>>>>>>>> @@ -327,7 +327,7 @@ > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> // target: the entry point of the method that creates and posts > >> the > >>>>>>> exception oop > >>>>>>>>> -// has_argument: true if the exception needs an argument > >> (passed > >>>> in > >>>>>>> rscratch1) > >>>>>>>>> +// has_argument: true if the exception needs arguments > (passed > >> in > >>>>>> r22 > >>>>>>> and r23) > >>>>>>>>> > >>>>>>>>> OopMapSet* > >>>> Runtime1::generate_exception_throw(StubAssembler* > >>>>>>> sasm, address target, bool has_argument) { > >>>>>>>>> // make a frame and preserve the caller's caller-save registers > >>>>>>>>> @@ -336,7 +336,7 @@ > >>>>>>>>> if (!has_argument) { > >>>>>>>>> call_offset = __ call_RT(noreg, noreg, target); > >>>>>>>>> } else { > >>>>>>>>> - call_offset = __ call_RT(noreg, noreg, target, rscratch1, > >> rscratch2); > >>>>>>>>> + call_offset = __ call_RT(noreg, noreg, target, r22, r23); > >>>>>>>>> } > >>>>>>>>> OopMapSet* oop_maps = new OopMapSet(); > >>>>>>>>> oop_maps->add_gc_map(call_offset, oop_map); > >>>>>>>>> > >>>>>>>>> Best regards, > >>>>>>>>> Goetz. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> -----Original Message----- > >>>>>>>>>> From: Stuart Monteith [mailto:stuart.monteith at linaro.org] > >>>>>>>>>> Sent: Donnerstag, 26. April 2018 12:52 > >>>>>>>>>> To: Andrew Haley > >>>>>>>>>> Cc: Lindenmaier, Goetz ; > hotspot- > >>>>>>> compiler- > >>>>>>>>>> dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; > >>>> hotspot- > >>>>>>>>>> runtime-dev at openjdk.java.net; aarch32-port- > >>>> dev at openjdk.java.net > >>>>>>>>>> Subject: Re: RFR(M): 8201593: Print array length in > >>>>>>>>>> ArrayIndexOutOfBoundsException. > >>>>>>>>>> > >>>>>>>>>> Hi, > >>>>>>>>>> Using c_rarg1 and c_rarg2 instead of rscratch1 and > overwriting > >>>>>>>>>> rscratch2 causes a SIGSEGV. > >>>>>>>>>> Using r22 and r23 instead, the test ran successfully. > >>>>>>>>>> > >>>>>>>>>> In c1_CodeStubs_aarch64.cpp > >>>>>>>>>> : > >>>>>>>>>> 77 if (_index->is_cpu_register()) { > >>>>>>>>>> 78 __ mov(r22, _index->as_register()); > >>>>>>>>>> 79 } else { > >>>>>>>>>> 80 __ mov(r22, _index->as_jint()); > >>>>>>>>>> 81 } > >>>>>>>>>> 82 Runtime1::StubID stub_id; > >>>>>>>>>> 83 if (_throw_index_out_of_bounds_exception) { > >>>>>>>>>> 84 stub_id = Runtime1::throw_index_exception_id; > >>>>>>>>>> 85 } else { > >>>>>>>>>> 86 assert(_array != NULL, "sanity"); > >>>>>>>>>> 87 __ mov(r23, _array->as_pointer_register()); > >>>>>>>>>> 88 stub_id = Runtime1::throw_range_check_failed_id; > >>>>>>>>>> 89 } > >>>>>>>>>> > >>>>>>>>>> in c1_Runtime_aarch64.cpp: > >>>>>>>>>> > >>>>>>>>>> 336 if (!has_argument) { > >>>>>>>>>> 337 call_offset = __ call_RT(noreg, noreg, target); > >>>>>>>>>> 338 } else { > >>>>>>>>>> 339 call_offset = __ call_RT(noreg, noreg, target, r22, r23); > >>>>>>>>>> 340 } > >>>>>>>>>> > >>>>>>>>>> There is the possibility of overwriting live values though, aren't > >>>>>>>>>> there? The registers are saved by call_RT. Should I be > concerned > >>>> about > >>>>>>>>>> deopt and debugging going wrong? Furthermore, won't there > be > >>>>>> issues > >>>>>>> in > >>>>>>>>>> exception handlers? > >>>>>>>>>> > >>>>>>>>>> BR, > >>>>>>>>>> Stuart > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On 25 April 2018 at 16:49, Stuart Monteith > >>>>>> > >>>>>>>>>> wrote: > >>>>>>>>>>> Indeed - and that is what I am seeing. Usually no parameters > are > >>>>>> being > >>>>>>>>>>> called with this pattern, or rscratch1, with the temporary > variable > >>>>>>>>>>> being changed to use rscratch2 in such circumstances. > >>>>>>>>>>> I'll try c_rarg1 and c_rarg2 - they should pass straight > through,if I > >>>>>>>>>>> interpret the code correcting. > >>>>>>>>>>> > >>>>>>>>>>> On 25 April 2018 at 16:26, Andrew Haley > >> wrote: > >>>>>>>>>>>> On 04/25/2018 04:00 PM, Stuart Monteith wrote: > >>>>>>>>>>>>> I'm not quite sure to solve this yet - we'll need to use the > >> stack in > >>>>>>>>>>>>> some safe way. > >>>>>>>>>>>> > >>>>>>>>>>>> It's not a great idea to pass arguments in rscratch1 or > rscratch2. > >>>>>> These > >>>>>>>>>>>> registers are for use in macros and should be treated as > >> volatile. > >>>>>>> Given > >>>>>>>>>>>> that you're throwing an exception, registers will be > clobbered > >>>>>>> anyway. > >>>>>>>>>>>> > >>>>>>>>>>>> -- > >>>>>>>>>>>> Andrew Haley > >>>>>>>>>>>> Java Platform Lead Engineer > >>>>>>>>>>>> Red Hat UK Ltd. > >>>>>>>>>>>> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From coleen.phillimore at oracle.com Mon May 7 21:22:11 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 7 May 2018 17:22:11 -0400 Subject: RFR (trivial) 8202606: Deprecate AllowNonVirtualCalls option In-Reply-To: <24525a32-b3d7-459f-8eec-3b37d53d6a6d@oracle.com> References: <3dc8508f-b924-7df7-3d52-1a57fda2081b@oracle.com> <24525a32-b3d7-459f-8eec-3b37d53d6a6d@oracle.com> Message-ID: On 5/7/18 4:37 PM, David Holmes wrote: > Hi Coleen, > > On 8/05/2018 4:09 AM, coleen.phillimore at oracle.com wrote: >> The CSR request is approved, see bug for details.? Tested with java >> -XX:+AllowNonVirtualCalls -version >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8202606.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8202606 > > Please also add it to the runtime/CommandLine/VMDeprecatedOptions.java > test. http://cr.openjdk.java.net/~coleenp/8202606.02/webrev/test/hotspot/jtreg/runtime/CommandLine/VMDeprecatedOptions.java.udiff.html Thanks, Coleen > > Thanks, > David > >> Thanks, >> Coleen From goetz.lindenmaier at sap.com Mon May 7 21:26:28 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 7 May 2018 21:26:28 +0000 Subject: RFR(M): 8202745: Remove hyphens from "out-of-bounds". Message-ID: Hi, as a follow up to "8201593: Print array length in ArrayIndexOutOfBoundsException." I promised to check all occurances of out-of-bounds. This change fixes them wherever the term is not used as an adjective: http://cr.openjdk.java.net/~goetz/wr18/8202745-out-of-bounds/01/ Please review. Best regards, Goetz. From david.holmes at oracle.com Mon May 7 21:26:36 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 8 May 2018 07:26:36 +1000 Subject: RFR (trivial) 8202606: Deprecate AllowNonVirtualCalls option In-Reply-To: References: <3dc8508f-b924-7df7-3d52-1a57fda2081b@oracle.com> <24525a32-b3d7-459f-8eec-3b37d53d6a6d@oracle.com> Message-ID: Thanks Coleen. Reviewed. Trivial. Push at will :) David On 8/05/2018 7:22 AM, coleen.phillimore at oracle.com wrote: > > > On 5/7/18 4:37 PM, David Holmes wrote: >> Hi Coleen, >> >> On 8/05/2018 4:09 AM, coleen.phillimore at oracle.com wrote: >>> The CSR request is approved, see bug for details.? Tested with java >>> -XX:+AllowNonVirtualCalls -version >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8202606.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8202606 >> >> Please also add it to the runtime/CommandLine/VMDeprecatedOptions.java >> test. > > http://cr.openjdk.java.net/~coleenp/8202606.02/webrev/test/hotspot/jtreg/runtime/CommandLine/VMDeprecatedOptions.java.udiff.html > > > Thanks, > Coleen > >> >> Thanks, >> David >> >>> Thanks, >>> Coleen > From coleen.phillimore at oracle.com Mon May 7 21:28:38 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 7 May 2018 17:28:38 -0400 Subject: RFR (trivial) 8202606: Deprecate AllowNonVirtualCalls option In-Reply-To: References: <3dc8508f-b924-7df7-3d52-1a57fda2081b@oracle.com> <24525a32-b3d7-459f-8eec-3b37d53d6a6d@oracle.com> Message-ID: On 5/7/18 5:26 PM, David Holmes wrote: > Thanks Coleen. > > Reviewed. Trivial. Push at will :) Thanks David! Also Harold. Coleen > > David > > On 8/05/2018 7:22 AM, coleen.phillimore at oracle.com wrote: >> >> >> On 5/7/18 4:37 PM, David Holmes wrote: >>> Hi Coleen, >>> >>> On 8/05/2018 4:09 AM, coleen.phillimore at oracle.com wrote: >>>> The CSR request is approved, see bug for details.? Tested with java >>>> -XX:+AllowNonVirtualCalls -version >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8202606.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8202606 >>> >>> Please also add it to the >>> runtime/CommandLine/VMDeprecatedOptions.java test. >> >> http://cr.openjdk.java.net/~coleenp/8202606.02/webrev/test/hotspot/jtreg/runtime/CommandLine/VMDeprecatedOptions.java.udiff.html >> >> >> Thanks, >> Coleen >> >>> >>> Thanks, >>> David >>> >>>> Thanks, >>>> Coleen >> From david.holmes at oracle.com Mon May 7 21:43:19 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 8 May 2018 07:43:19 +1000 Subject: RFR(M): 8202745: Remove hyphens from "out-of-bounds". In-Reply-To: References: Message-ID: <46607c38-d850-a9e0-9300-e680a8c20ce7@oracle.com> Hi Goetz, The grammar can be a bit subtle here. IIUC we would say: "Index %d out of bounds for length %d" but if we turn it around we'd say: "out-of-bounds index %d for length %d" Anyway changes seem fine to me. Couple of comments below. On 8/05/2018 7:26 AM, Lindenmaier, Goetz wrote: > Hi, > > as a follow up to "8201593: Print array length in ArrayIndexOutOfBoundsException." I promised > to check all occurances of out-of-bounds. > This change fixes them wherever the term is not used as an adjective: > http://cr.openjdk.java.net/~goetz/wr18/8202745-out-of-bounds/01/ > Please review. src/hotspot/cpu/ppc/ppc.ad I'd consider that an adjective form. It should say "// Throw out-of-bounds exception if ..." src/java.base/share/classes/java/util/Objects.java I'll just note that this grammatical change does not require a CSR request. Thanks, David > Best regards, > Goetz. > > From jiangli.zhou at oracle.com Mon May 7 21:58:35 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Mon, 7 May 2018 14:58:35 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time In-Reply-To: <5AF09BE7.30208@oracle.com> References: <5AEB8C04.4040501@oracle.com> <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> <5AEB98B7.9030209@oracle.com> <56AA6E39-131E-4D5B-BBEF-9FA8C4A60F5A@oracle.com> <5AF07AE9.9000700@oracle.com> <5AF09BE7.30208@oracle.com> Message-ID: <25B748EB-27EC-4FAB-956F-F864323531EC@oracle.com> > On May 7, 2018, at 11:33 AM, Calvin Cheung wrote: > > > > On 5/7/18, 10:28 AM, Jiangli Zhou wrote: >> >> Hi Calvin, >> >> -classLoader.cpp >> >> The get_canonical_path() call for ?path? should be moved out of the ?for' loop. >> >> 1569 get_canonical_path(path, canonical_src_path, JVM_MAXPATHLEN)) { > Fixed. >> >> To make the code more readable, please rename ?canonical_path? to ?canonical_path_table_entry? (or ?canonical_path_table_entry_name?) > I chose the shorter one. >> and ?canonical_src_path? to ?canonical_class_src_path?. > Renamed. >> >> BTW, when we create the path entries, the paths are already converted to canonical forms. I think there is no need to call get_canonical_path() on ?ent->name()?. Can you please double check? > The call to get_canonical_path() is only for the non-directory case (please refer to ClassLoader::create_class_path_entry() for details). > I did try get_canonical_path() for the directory case but it didn't work during runtime; it failed in SystemDictionaryShared::is_shared_class_visible_for_classloader() in the following check: > > (strcmp(ent->name(), ClassLoader::skip_uri_protocol(mod_entry->location()->as_C_string())) == 0)) { > > The location() in the mod_entry isn't in canonical form. If we convert it to canonical form during runtime, there maybe performance impact. > I think the proper way of fixing it is storing the location() in canonical form when a module entry is being defined. I'm not sure if it is possible for the core lib to pass the location in canonical form to jvm when a module entry is being defined. > I can file a follow-up RFE to address that. Please file a RFE. There is no need to check ?res? in very iteration since it doesn?t change after being initialized by get_canonical_path(). Also, I think we are not handling get_canonical_path() (both calls) failure properly. However, I question why get_canonical_path() would fail when given a valid path in this particular case. So I think we should either assert get_canonical_path() returns true, or report a fatal error for dumping. >> >> - os_windows.cpp >> >> Please fix indentation at line 4390 & 4391. > That was done on purpose because the condition in line 4390 is '||' with line 4391. > I'm further indenting line 4391 to make it clearer. > > 4389 if (search_path[1] == ':' && > 4390 (search_path[2] == '\0' || > 4391 (search_path[2] == '\\' && search_path[3] == '\0'))) { Looks better. > Updated webrev: > > http://cr.openjdk.java.net/~ccheung/8202289/webrev.02/ Thanks, Jiangli > > thanks, > Calvin > >> >> Thanks, >> Jiangli >> >> >>> On May 7, 2018, at 9:12 AM, Calvin Cheung wrote: >>> >>> Hi Jiangli, >>> >>> On 5/3/18, 5:33 PM, Jiangli Zhou wrote: >>>> Hi Calvin, >>>> >>>> >>>>> On May 3, 2018, at 4:18 PM, Calvin Cheung wrote: >>>>> >>>>> >>>>> >>>>> On 5/3/18, 3:42 PM, Jiangli Zhou wrote: >>>>>> Hi Calvin, >>>>>> >>>>>> I think we don?t need to check the module path entries if only boot classes are archived. The ?end? is set to app_class_paths_start_index when there are only boot classes are archived. If there are app module classes loaded from jars/directories at dump time, has_platform_or_app_classes() should return true and ?end? is set to the end of the shared path table, which includes all module path entries. The following loop is not needed. The existing loop in the code covers all cases with different ?end? value based on the type of classes loaded in the archive. >>>>>> 366 // may need to check module path entries. >>>>>> 367 if ((end == ClassLoaderExt::app_class_paths_start_index())&& (ClassLoader::num_module_path_entries()> 0)) { >>>>>> 368 for (int i = ClassLoaderExt::app_module_paths_start_index(); i< _shared_path_table_size; i++) { >>>>>> 369 SharedClassPathEntry* e = shared_path(i); >>>>>> 370 has_nonempty_dir = check_nonempty_dir(e); >>>>>> >>>>>> 371 } >>>>>> 372 } >>>>>> >>>>> The has_platform_or_app_classes() will return false if ClassLoaderExt::record_result() hasn't been called while a class is being loaded during dumping. >>>>> This could happen before this proposed change if one specifies a --module-path with an non-empty directory containing a module. While the class will be loaded during dumping but its classpath_index will still be -1. Therefore, the has_platform_or_app_classes() will return false and the module path entries won't be checked without the above block of code. >>>> With the module directory path being added to the path table now, it should record the correct path table index instead of -1 for module classes loaded from directory at dump time. >>>> >>>> I applied your patch and stepped through ClassLoader::record_result() with your test case. The following path comparison fails due the the extra ?/? in ?path?. That why it fails to find the correct entry in the path table, which causes the class type cannot be correctly identified. >>>> >>>> if (strcmp(canonical_path, os::native_path((char*)path)) == 0) {os::dir_is_empty >>>> >>>> (gdb) p canonical_path >>>> $74 = 0x7ffff0aea0f0 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple" >>>> (gdb) p os::native_path((char*)path) >>>> $75 = 0x7ffff7fd4a65 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple/" >>>> >>>> The trailing ?/? should be excluded for the path comparison. JDK-8202519 is a related issue in the same area. It might be better to fix it together with this one. >>> Thanks for debugging the issue. >>> I've fixed it by calling get_canonical_path() on the "path" before doing strcmp(). It should also fix the bug in JDK-8202519 since a buffer is being allocated for get_canonical_path() to write into. The os::native_path() on linux does nothing and just returns the same string. >>> >>> I also discovered a problem in os::dir_is_empty() on windows. When a directory path is passed into the function, it may have the following form with forward slashes: C:/somedir >>> The fix is to call os::native_path() which essentially converts the forward slashes to back slashes so that subsequent windows api call returns the correct result. >>> >>>> Once the above issue is fixed, the class type should be recorded correctly for a module classes loaded from directory at dump time. Then ClassLoaderExt::record_result() can set ?has_app_classes? flag properly. And, no change should be needed in FileMapInfo::check_nonempty_dir_in_shared_path_table(). >>> I don't need to touch filemap.cpp in the following updated webrev: >>> >>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.01/ >>> >>> thanks, >>> Calvin >>>>>> We also need a unit test for non-empty directory in module path. Please add. >>>>> It is already in my webrev: >>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/MainModuleOnly.java.sdiff.html >>>> Ok. I was expecting a separate unit test. I?m fine with an added test case in MainModuleOnly.java. >>>> >>>> Thanks, >>>> Jiangli >>>> >>>>> thanks, >>>>> Calvin >>>>>> Thanks, >>>>>> Jiangli >>>>>> >>>>>>> On May 3, 2018, at 3:24 PM, Calvin Cheung wrote: >>>>>>> >>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8202289 >>>>>>> webrev: http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ >>>>>>> >>>>>>> In ClassLoaderExt::process_module_table(), it adds all entries from the ModuleEntryTable with the "file:" protocol to the _module_path_entries and eventually to the _shared_path_table. >>>>>>> The FileMapInfo::check_nonempty_dir_in_shared_path_table() will check for non-empty directory in the module path. >>>>>>> >>>>>>> Ran all CDS and AppCDS tests locally on linux-x64. >>>>>>> Will run hs-tier{1,2,3} tests. >>>>>>> >>>>>>> thanks, >>>>>>> Calvin From john.r.rose at oracle.com Mon May 7 22:08:26 2018 From: john.r.rose at oracle.com (John Rose) Date: Mon, 7 May 2018 15:08:26 -0700 Subject: RFR(M): 8202745: Remove hyphens from "out-of-bounds". In-Reply-To: <46607c38-d850-a9e0-9300-e680a8c20ce7@oracle.com> References: <46607c38-d850-a9e0-9300-e680a8c20ce7@oracle.com> Message-ID: <8514FA0D-8EB2-4879-8A24-91947B822B74@oracle.com> On May 7, 2018, at 2:43 PM, David Holmes wrote: > > The grammar can be a bit subtle here. IIUC we would say: > > "Index %d out of bounds for length %d" > > but if we turn it around we'd say: > > "out-of-bounds index %d for length %d" +1 And the hyphens could be dropped altogether, which I think reads better, usually. (Grammarian hat donned.) You _may_ use "out-of-bounds" when it modifies a noun. The form "out-of-bounds" is a compound adjective. It is the compound-adjective form of "out of bounds". See Rule 1 of this page: https://www.grammarbook.com/punctuation/hyphens.asp Rule 10 on the same page says compound adjectives that are well known do not need hyphens. A compound adjective usage can omit hyphens if it will be well understood. A compound-adjective usage can include hyphens if it might not be well-understood. Thus, hyphenating the compound adjective is sometimes a matter of taste. Here's bad grammar: "Hyphenating the compound-adjective is sometimes a matter-of-taste." Hyphenating nouns is only correct if the noun specifically requires it, like the numbers between twenty-one and ninety-nine, as the noted grammarian Salman Al-Azami might observe. (Grammarian hat doffed.) From david.holmes at oracle.com Tue May 8 00:32:06 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 8 May 2018 10:32:06 +1000 Subject: RFR (trivial): 8202744: Expired flag removal for JDK 11 Message-ID: <10c90892-766a-ed3a-c4ab-436b164692b3@oracle.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8202744 Webrev: http://cr.openjdk.java.net/~dholmes/8202744/webrev/ Three flags are expired as of JDK 11 and should be removed from the flag table. In addition the ObsoleteFlagErrorMessage.java test had to be updated to use a new flag. Thanks, David From coleen.phillimore at oracle.com Tue May 8 00:54:23 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 7 May 2018 20:54:23 -0400 Subject: RFR (trivial): 8202744: Expired flag removal for JDK 11 In-Reply-To: <10c90892-766a-ed3a-c4ab-436b164692b3@oracle.com> References: <10c90892-766a-ed3a-c4ab-436b164692b3@oracle.com> Message-ID: <1aca6f17-e81b-f3a9-6f20-68d68d57a81d@oracle.com> Looks good to me. Coleen On 5/7/18 8:32 PM, David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8202744 > Webrev: http://cr.openjdk.java.net/~dholmes/8202744/webrev/ > > Three flags are expired as of JDK 11 and should be removed from the flag > table. > > In addition the ObsoleteFlagErrorMessage.java test had to be updated > to use a new flag. > > Thanks, > David From jiangli.zhou at oracle.com Tue May 8 01:05:18 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Mon, 7 May 2018 18:05:18 -0700 Subject: RFR (trivial): 8202744: Expired flag removal for JDK 11 In-Reply-To: <1aca6f17-e81b-f3a9-6f20-68d68d57a81d@oracle.com> References: <10c90892-766a-ed3a-c4ab-436b164692b3@oracle.com> <1aca6f17-e81b-f3a9-6f20-68d68d57a81d@oracle.com> Message-ID: +1 Thanks, Jiangli > On May 7, 2018, at 5:54 PM, coleen.phillimore at oracle.com wrote: > > Looks good to me. > Coleen > > On 5/7/18 8:32 PM, David Holmes wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8202744 >> Webrev: http://cr.openjdk.java.net/~dholmes/8202744/webrev/ >> >> Three flags are expired as of JDK 11 and should be removed from the flag >> table. >> >> In addition the ObsoleteFlagErrorMessage.java test had to be updated to use a new flag. >> >> Thanks, >> David > From david.holmes at oracle.com Tue May 8 01:47:02 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 8 May 2018 11:47:02 +1000 Subject: RFR (trivial): 8202744: Expired flag removal for JDK 11 In-Reply-To: References: <10c90892-766a-ed3a-c4ab-436b164692b3@oracle.com> <1aca6f17-e81b-f3a9-6f20-68d68d57a81d@oracle.com> Message-ID: <12c88144-be14-21ca-b446-aa8adeceaa31@oracle.com> Thanks Coleen and Jiangli! David On 8/05/2018 11:05 AM, Jiangli Zhou wrote: > +1 > > Thanks, > Jiangli > >> On May 7, 2018, at 5:54 PM, coleen.phillimore at oracle.com wrote: >> >> Looks good to me. >> Coleen >> >> On 5/7/18 8:32 PM, David Holmes wrote: >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202744 >>> Webrev: http://cr.openjdk.java.net/~dholmes/8202744/webrev/ >>> >>> Three flags are expired as of JDK 11 and should be removed from the flag >>> table. >>> >>> In addition the ObsoleteFlagErrorMessage.java test had to be updated to use a new flag. >>> >>> Thanks, >>> David >> > From jini.george at oracle.com Tue May 8 03:38:45 2018 From: jini.george at oracle.com (Jini George) Date: Tue, 8 May 2018 09:08:45 +0530 Subject: RFR: (S): SA: clhsdb 'where -a' throws Assertion Failure with illegal code 236 when CDS is used In-Reply-To: References: <82c69a0d-02f3-5db8-fedd-d52b00571a55@oracle.com> Message-ID: <41204e1d-11e1-41a9-af90-7cabd12098aa@oracle.com> Thank you very much, Ioi, for the review and for the clarifications and help provided offline. I have added the checks for _nofast_getfield and _nofast_putfield. SA has a bug due to which for iload, only the base bytecode (iload) gets displayed -- fast_iload and nofast_iload do not get displayed. JDK-8202693 (SA: clhsdb printall only displays the base bytecode for iload) has been filed for this. I would add the test for nofast_iload along with the fix for JDK-8202693. The modified webrev is at: http://cr.openjdk.java.net/~jgeorge/8174995/webrev.01/ Thanks, Jini. On 4/27/2018 1:54 AM, Ioi Lam wrote: > HI Jini, > > [1] "_nofast_aload" should be "_nofast_aload_0": aload and aload_0 are > two different bytecodes. > > [2] Only the _nofast_aload_0 bytecode is tested. For completeness, do > you think it makes sense to add test cases for these other 3 bytecodes? > > ??? _nofast_getfield > ??? _nofast_putfield > ??? _nofast_iload > > > Thanks > - Ioi > > On 4/26/18 11:15 AM, Jini George wrote: >> Hello all, >> >> Please review the following proposed fix for the issue: >> >> https://bugs.openjdk.java.net/browse/JDK-8174995 >> >> Webrev: http://cr.openjdk.java.net/~jgeorge/8174995/webrev.00/ >> >> Issue: Clhsdb commands like 'where -a', 'printall' would throw an >> illegal code assertion failure when CDS is used. >> >> Root cause and proposed fix: SA has been unaware of the new bytecodes >> introduced for rewriting at CDS dump time (_nofast* bytecodes). The >> fix is to make SA aware of these new _nofast* bytecodes. >> >> Tests Run and Passed: SA tests on Mach5 (including the tests modified >> to test this fix). >> >> Thank you, >> Jini. > From thomas.stuefe at gmail.com Tue May 8 04:47:31 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 8 May 2018 06:47:31 +0200 Subject: RFR: 8202634: Metaspace: simplify SpaceManager lists In-Reply-To: References: Message-ID: (jdk-submit tests ok. SAP tests did not show anything attributable to this patch.) On Mon, May 7, 2018 at 5:29 PM, Thomas St?fe wrote: > Hi all, > > may I please have reviews for this small simplification. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202634 > webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8202634-simplify-spacemanager/webrev.00/webrev/ > > This replaces the four separate in-use-chunk-lists in SpaceManager > with a single one. The reasoning behind this is that we do not need to > keep the chunks sorted by chunk size, a single list containing all > chunks of all sizes in use by this SpaceManager is enough, and it > simplifies the coding. > > There is only one place where we really did use the former > by-chunktype lists - that is to count how many chunks per chunk type > there are. > > However, arguably that is better done with running counters, so this > is what I did. The added benefit is that in > SpaceManager::calc_chunk_size() we do not need to walk the lists > anymore to sum up chunks. > > Changes in detail: > > - removed the "ChunkIndex" argument in > ChunkManager::return_single_chunk() and > Chunkmanager::return_chunk_list() - that argument was unnecessary, > since the Metachunk we return knows its index. > > - Unfolded SpaceManager::initialize into Spacemanager constructor (I > prefer initializing members in ctor init lists) > > - replaces SpaceManager::sum_count_chunks_in_use() with running counters. > > - I dumbed down the logging inside ChunkManager::return_chunk_list() > somewhat. Should anyone miss the former details, I can reinstate it. > > Thank you, > > Thomas From aph at redhat.com Tue May 8 08:04:45 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 8 May 2018 09:04:45 +0100 Subject: RFR(M): 8202745: Remove hyphens from "out-of-bounds". In-Reply-To: <8514FA0D-8EB2-4879-8A24-91947B822B74@oracle.com> References: <46607c38-d850-a9e0-9300-e680a8c20ce7@oracle.com> <8514FA0D-8EB2-4879-8A24-91947B822B74@oracle.com> Message-ID: <16957584-e2e6-2d68-793e-51d61ce1b812@redhat.com> On 07/05/18 23:08, John Rose wrote: > You_may_ use "out-of-bounds" when it modifies a noun. > The form "out-of-bounds" is a compound adjective. > It is the compound-adjective form of "out of bounds". > > See Rule 1 of this page: > https://www.grammarbook.com/punctuation/hyphens.asp We should be internally consistent. The JLS hyphenates compound adjectives and appears to do so consistently. For example: "An attempt access an array component with a long index value results in a compile-time error." -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From goetz.lindenmaier at sap.com Tue May 8 08:16:42 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 8 May 2018 08:16:42 +0000 Subject: RFR(M): 8202745: Remove hyphens from "out-of-bounds". In-Reply-To: <46607c38-d850-a9e0-9300-e680a8c20ce7@oracle.com> References: <46607c38-d850-a9e0-9300-e680a8c20ce7@oracle.com> Message-ID: Hi David, Thanks for looking at this. > The grammar can be a bit subtle here. IIUC we would say: > "Index %d out of bounds for length %d" > but if we turn it around we'd say: > "out-of-bounds index %d for length %d" Yes, this is the assumption of my edits. > src/hotspot/cpu/ppc/ppc.ad > I'd consider that an adjective form. It should say "// Throw > out-of-bounds exception if ..." I changed it to // Runs out of bounds and asserts if Proj not found. > src/java.base/share/classes/java/util/Objects.java > I'll just note that this grammatical change does not require a CSR request. Great ?? Best regards, Goetz. > > > Thanks, > David > > > Best regards, > > Goetz. > > > > From zgu at redhat.com Tue May 8 12:51:53 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 8 May 2018 08:51:53 -0400 Subject: RFR: 8202634: Metaspace: simplify SpaceManager lists In-Reply-To: References: Message-ID: Hi Thomas, Looks good to me. Nits: metaspace.cpp L3049 assertion is no longer needed. Thanks. -Zhengyu On 05/08/2018 12:47 AM, Thomas St?fe wrote: > (jdk-submit tests ok. SAP tests did not show anything attributable to > this patch.) > > On Mon, May 7, 2018 at 5:29 PM, Thomas St?fe wrote: >> Hi all, >> >> may I please have reviews for this small simplification. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8202634 >> webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8202634-simplify-spacemanager/webrev.00/webrev/ >> >> This replaces the four separate in-use-chunk-lists in SpaceManager >> with a single one. The reasoning behind this is that we do not need to >> keep the chunks sorted by chunk size, a single list containing all >> chunks of all sizes in use by this SpaceManager is enough, and it >> simplifies the coding. >> >> There is only one place where we really did use the former >> by-chunktype lists - that is to count how many chunks per chunk type >> there are. >> >> However, arguably that is better done with running counters, so this >> is what I did. The added benefit is that in >> SpaceManager::calc_chunk_size() we do not need to walk the lists >> anymore to sum up chunks. >> >> Changes in detail: >> >> - removed the "ChunkIndex" argument in >> ChunkManager::return_single_chunk() and >> Chunkmanager::return_chunk_list() - that argument was unnecessary, >> since the Metachunk we return knows its index. >> >> - Unfolded SpaceManager::initialize into Spacemanager constructor (I >> prefer initializing members in ctor init lists) >> >> - replaces SpaceManager::sum_count_chunks_in_use() with running counters. >> >> - I dumbed down the logging inside ChunkManager::return_chunk_list() >> somewhat. Should anyone miss the former details, I can reinstate it. >> >> Thank you, >> >> Thomas From ioi.lam at oracle.com Tue May 8 15:25:26 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 8 May 2018 08:25:26 -0700 Subject: RFR: (S): SA: clhsdb 'where -a' throws Assertion Failure with illegal code 236 when CDS is used In-Reply-To: <41204e1d-11e1-41a9-af90-7cabd12098aa@oracle.com> References: <82c69a0d-02f3-5db8-fedd-d52b00571a55@oracle.com> <41204e1d-11e1-41a9-af90-7cabd12098aa@oracle.com> Message-ID: Looks good. Thanks! - Ioi On 5/7/18 8:38 PM, Jini George wrote: > Thank you very much, Ioi, for the review and for the clarifications > and help provided offline. I have added the checks for > _nofast_getfield and _nofast_putfield. SA has a bug due to which for > iload, only the base bytecode (iload) gets displayed -- fast_iload and > nofast_iload do not get displayed. JDK-8202693 (SA: clhsdb printall > only displays the base bytecode for iload) has been filed for this. I > would add the test for nofast_iload along with the fix for JDK-8202693. > > The modified webrev is at: > > http://cr.openjdk.java.net/~jgeorge/8174995/webrev.01/ > > Thanks, > Jini. > > On 4/27/2018 1:54 AM, Ioi Lam wrote: >> HI Jini, >> >> [1] "_nofast_aload" should be "_nofast_aload_0": aload and aload_0 >> are two different bytecodes. >> >> [2] Only the _nofast_aload_0 bytecode is tested. For completeness, do >> you think it makes sense to add test cases for these other 3 bytecodes? >> >> ???? _nofast_getfield >> ???? _nofast_putfield >> ???? _nofast_iload >> >> >> Thanks >> - Ioi >> >> On 4/26/18 11:15 AM, Jini George wrote: >>> Hello all, >>> >>> Please review the following proposed fix for the issue: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8174995 >>> >>> Webrev: http://cr.openjdk.java.net/~jgeorge/8174995/webrev.00/ >>> >>> Issue: Clhsdb commands like 'where -a', 'printall' would throw an >>> illegal code assertion failure when CDS is used. >>> >>> Root cause and proposed fix: SA has been unaware of the new >>> bytecodes introduced for rewriting at CDS dump time (_nofast* >>> bytecodes). The fix is to make SA aware of these new _nofast* >>> bytecodes. >>> >>> Tests Run and Passed: SA tests on Mach5 (including the tests >>> modified to test this fix). >>> >>> Thank you, >>> Jini. >> From jini.george at oracle.com Tue May 8 15:53:20 2018 From: jini.george at oracle.com (Jini George) Date: Tue, 8 May 2018 21:23:20 +0530 Subject: RFR: (S): SA: clhsdb 'where -a' throws Assertion Failure with illegal code 236 when CDS is used In-Reply-To: References: <82c69a0d-02f3-5db8-fedd-d52b00571a55@oracle.com> <41204e1d-11e1-41a9-af90-7cabd12098aa@oracle.com> Message-ID: Thanks, Ioi. Could I get one more reviewer to take a look at this ? Thanks, Jini. On 5/8/2018 8:55 PM, Ioi Lam wrote: > Looks good. Thanks! > > - Ioi > > > On 5/7/18 8:38 PM, Jini George wrote: >> Thank you very much, Ioi, for the review and for the clarifications >> and help provided offline. I have added the checks for >> _nofast_getfield and _nofast_putfield. SA has a bug due to which for >> iload, only the base bytecode (iload) gets displayed -- fast_iload and >> nofast_iload do not get displayed. JDK-8202693 (SA: clhsdb printall >> only displays the base bytecode for iload) has been filed for this. I >> would add the test for nofast_iload along with the fix for JDK-8202693. >> >> The modified webrev is at: >> >> http://cr.openjdk.java.net/~jgeorge/8174995/webrev.01/ >> >> Thanks, >> Jini. >> >> On 4/27/2018 1:54 AM, Ioi Lam wrote: >>> HI Jini, >>> >>> [1] "_nofast_aload" should be "_nofast_aload_0": aload and aload_0 >>> are two different bytecodes. >>> >>> [2] Only the _nofast_aload_0 bytecode is tested. For completeness, do >>> you think it makes sense to add test cases for these other 3 bytecodes? >>> >>> ???? _nofast_getfield >>> ???? _nofast_putfield >>> ???? _nofast_iload >>> >>> >>> Thanks >>> - Ioi >>> >>> On 4/26/18 11:15 AM, Jini George wrote: >>>> Hello all, >>>> >>>> Please review the following proposed fix for the issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8174995 >>>> >>>> Webrev: http://cr.openjdk.java.net/~jgeorge/8174995/webrev.00/ >>>> >>>> Issue: Clhsdb commands like 'where -a', 'printall' would throw an >>>> illegal code assertion failure when CDS is used. >>>> >>>> Root cause and proposed fix: SA has been unaware of the new >>>> bytecodes introduced for rewriting at CDS dump time (_nofast* >>>> bytecodes). The fix is to make SA aware of these new _nofast* >>>> bytecodes. >>>> >>>> Tests Run and Passed: SA tests on Mach5 (including the tests >>>> modified to test this fix). >>>> >>>> Thank you, >>>> Jini. >>> > From coleen.phillimore at oracle.com Tue May 8 15:53:44 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 8 May 2018 11:53:44 -0400 Subject: RFR: 8202634: Metaspace: simplify SpaceManager lists In-Reply-To: References: Message-ID: <0418aef9-d4ef-14f4-4e84-b6321dfa00b9@oracle.com> This change looks great.? Reviewed. On 5/7/18 11:29 AM, Thomas St?fe wrote: > Hi all, > > may I please have reviews for this small simplification. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202634 > webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8202634-simplify-spacemanager/webrev.00/webrev/ > > This replaces the four separate in-use-chunk-lists in SpaceManager > with a single one. The reasoning behind this is that we do not need to > keep the chunks sorted by chunk size, a single list containing all > chunks of all sizes in use by this SpaceManager is enough, and it > simplifies the coding. > > There is only one place where we really did use the former > by-chunktype lists - that is to count how many chunks per chunk type > there are. > > However, arguably that is better done with running counters, so this > is what I did. The added benefit is that in > SpaceManager::calc_chunk_size() we do not need to walk the lists > anymore to sum up chunks. > > Changes in detail: > > - removed the "ChunkIndex" argument in > ChunkManager::return_single_chunk() and > Chunkmanager::return_chunk_list() - that argument was unnecessary, > since the Metachunk we return knows its index. good. > > - Unfolded SpaceManager::initialize into Spacemanager constructor (I > prefer initializing members in ctor init lists) I agree! > > - replaces SpaceManager::sum_count_chunks_in_use() with running counters. > > - I dumbed down the logging inside ChunkManager::return_chunk_list() > somewhat. Should anyone miss the former details, I can reinstate it. I don't think we'll miss the implementation details in the logging. Knowing how much space is returned seems most useful. thanks, Coleen > > Thank you, > > Thomas From mikhailo.seledtsov at oracle.com Tue May 8 17:38:49 2018 From: mikhailo.seledtsov at oracle.com (mikhailo) Date: Tue, 8 May 2018 10:38:49 -0700 Subject: RFR(M): 8199257: [TESTBUG] Open source VM testbase metaspace tests Message-ID: http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html Please review this change open sourcing VM metaspace tests. These tests have been used internally for a while, and are now being open sourced. Since this is not an creation of new tests, we would like to keep the changes during this review to an absolute required minimum. If you have any feedback on improvements of these tests, please file RFE(s) that will be addressed later in order of priority. Here is what was done for this change: ? 1. Moved the tests to OpenJDK repository to the specified directory location and structure. ? 3. Updated Copyright statements accordingly. ? 4. Updated "@library" statements accordingly. ? 5. Updated TEST.groups and ProblemList.txt ? JBS:??? https://bugs.openjdk.java.net/browse/JDK-8199257 ? Webrev: http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html ? Testing: ????? 1. Ran the following tests on open-only repository and build, using "make run-test" (Linux-x64) ???????? vmTestbase_vm_metaspace, vmTestbase_largepages ???????? All PASS ????? 2. Automated multip-platform test system (usual 4 platforms): ???????? - vmTestbase_vm_metaspace, vmTestbase_largepages ???????? - hs-tier{1,2,3} ???????? All PASS Thank you, Misha From calvin.cheung at oracle.com Tue May 8 17:41:43 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Tue, 08 May 2018 10:41:43 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time In-Reply-To: <25B748EB-27EC-4FAB-956F-F864323531EC@oracle.com> References: <5AEB8C04.4040501@oracle.com> <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> <5AEB98B7.9030209@oracle.com> <56AA6E39-131E-4D5B-BBEF-9FA8C4A60F5A@oracle.com> <5AF07AE9.9000700@oracle.com> <5AF09BE7.30208@oracle.com> <25B748EB-27EC-4FAB-956F-F864323531EC@oracle.com> Message-ID: <5AF1E157.9050905@oracle.com> On 5/7/18, 2:58 PM, Jiangli Zhou wrote: > >> On May 7, 2018, at 11:33 AM, Calvin Cheung > > wrote: >> >> >> >> On 5/7/18, 10:28 AM, Jiangli Zhou wrote: >>> Hi Calvin, >>> >>> -classLoader.cpp >>> >>> The get_canonical_path() call for ?path? should be moved out of the ?for' loop. >>> >>> 1569 get_canonical_path(path, canonical_src_path, JVM_MAXPATHLEN)) { >> Fixed. >>> To make the code more readable, please rename ?canonical_path? to ?canonical_path_table_entry? (or ?canonical_path_table_entry_name?) >> I chose the shorter one. >>> and ?canonical_src_path? to ?canonical_class_src_path?. >> Renamed. >>> BTW, when we create the path entries, the paths are already converted to canonical forms. I think there is no need to call get_canonical_path() on ?ent->name()?. Can you please double check? >> The call to get_canonical_path() is only for the non-directory case >> (please refer to ClassLoader::create_class_path_entry() for details). >> I did try get_canonical_path() for the directory case but it didn't >> work during runtime; it failed in >> SystemDictionaryShared::is_shared_class_visible_for_classloader() in >> the following check: >> >> (strcmp(ent->name(), >> ClassLoader::skip_uri_protocol(mod_entry->location()->as_C_string())) >> == 0)) { >> >> The location() in the mod_entry isn't in canonical form. If we >> convert it to canonical form during runtime, there maybe performance >> impact. >> I think the proper way of fixing it is storing the location() in >> canonical form when a module entry is being defined. I'm not sure if >> it is possible for the core lib to pass the location in canonical >> form to jvm when a module entry is being defined. >> I can file a follow-up RFE to address that. > > Please file a RFE. I've filed JDK-8202750 > > There is no need to check ?res? in very iteration since it doesn?t > change after being initialized by get_canonical_path(). Also, I think > we are not handling get_canonical_path() (both calls) failure > properly. However, I question why get_canonical_path() would fail when > given a valid path in this particular case. So I think we should > either assert get_canonical_path() returns true, or report a fatal > error for dumping. I've made the change and added assert after get_canonical_path() at both calls. Updated webrev: http://cr.openjdk.java.net/~ccheung/8202289/webrev.03/ thanks, Calvin >>> - os_windows.cpp >>> >>> Please fix indentation at line 4390& 4391. >> That was done on purpose because the condition in line 4390 is '||' >> with line 4391. >> I'm further indenting line 4391 to make it clearer. >> >> 4389 if (search_path[1] == ':'&& >> 4390 (search_path[2] == '\0' || >> 4391 (search_path[2] == '\\' && search_path[3] == '\0'))) { > > > Looks better. > >> Updated webrev: >> >> http://cr.openjdk.java.net/~ccheung/8202289/webrev.02/ > > > Thanks, > Jiangli > >> >> thanks, >> Calvin >> >>> Thanks, >>> Jiangli >>> >>> >>>> On May 7, 2018, at 9:12 AM, Calvin Cheung wrote: >>>> >>>> Hi Jiangli, >>>> >>>> On 5/3/18, 5:33 PM, Jiangli Zhou wrote: >>>>> Hi Calvin, >>>>> >>>>> >>>>>> On May 3, 2018, at 4:18 PM, Calvin Cheung wrote: >>>>>> >>>>>> >>>>>> >>>>>> On 5/3/18, 3:42 PM, Jiangli Zhou wrote: >>>>>>> Hi Calvin, >>>>>>> >>>>>>> I think we don?t need to check the module path entries if only boot classes are archived. The ?end? is set to app_class_paths_start_index when there are only boot classes are archived. If there are app module classes loaded from jars/directories at dump time, has_platform_or_app_classes() should return true and ?end? is set to the end of the shared path table, which includes all module path entries. The following loop is not needed. The existing loop in the code covers all cases with different ?end? value based on the type of classes loaded in the archive. >>>>>>> 366 // may need to check module path entries. >>>>>>> 367 if ((end == ClassLoaderExt::app_class_paths_start_index())&& (ClassLoader::num_module_path_entries()> 0)) { >>>>>>> 368 for (int i = ClassLoaderExt::app_module_paths_start_index(); i< _shared_path_table_size; i++) { >>>>>>> 369 SharedClassPathEntry* e = shared_path(i); >>>>>>> 370 has_nonempty_dir = check_nonempty_dir(e); >>>>>>> >>>>>>> 371 } >>>>>>> 372 } >>>>>>> >>>>>> The has_platform_or_app_classes() will return false if ClassLoaderExt::record_result() hasn't been called while a class is being loaded during dumping. >>>>>> This could happen before this proposed change if one specifies a --module-path with an non-empty directory containing a module. While the class will be loaded during dumping but its classpath_index will still be -1. Therefore, the has_platform_or_app_classes() will return false and the module path entries won't be checked without the above block of code. >>>>> With the module directory path being added to the path table now, it should record the correct path table index instead of -1 for module classes loaded from directory at dump time. >>>>> >>>>> I applied your patch and stepped through ClassLoader::record_result() with your test case. The following path comparison fails due the the extra ?/? in ?path?. That why it fails to find the correct entry in the path table, which causes the class type cannot be correctly identified. >>>>> >>>>> if (strcmp(canonical_path, os::native_path((char*)path)) == 0) {os::dir_is_empty >>>>> >>>>> (gdb) p canonical_path >>>>> $74 = 0x7ffff0aea0f0 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple" >>>>> (gdb) p os::native_path((char*)path) >>>>> $75 = 0x7ffff7fd4a65 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple/" >>>>> >>>>> The trailing ?/? should be excluded for the path comparison. JDK-8202519 is a related issue in the same area. It might be better to fix it together with this one. >>>> Thanks for debugging the issue. >>>> I've fixed it by calling get_canonical_path() on the "path" before doing strcmp(). It should also fix the bug in JDK-8202519 since a buffer is being allocated for get_canonical_path() to write into. The os::native_path() on linux does nothing and just returns the same string. >>>> >>>> I also discovered a problem in os::dir_is_empty() on windows. When a directory path is passed into the function, it may have the following form with forward slashes: C:/somedir >>>> The fix is to call os::native_path() which essentially converts the forward slashes to back slashes so that subsequent windows api call returns the correct result. >>>> >>>>> Once the above issue is fixed, the class type should be recorded correctly for a module classes loaded from directory at dump time. Then ClassLoaderExt::record_result() can set ?has_app_classes? flag properly. And, no change should be needed in FileMapInfo::check_nonempty_dir_in_shared_path_table(). >>>> I don't need to touch filemap.cpp in the following updated webrev: >>>> >>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.01/ >>>> >>>> thanks, >>>> Calvin >>>>>>> We also need a unit test for non-empty directory in module path. Please add. >>>>>> It is already in my webrev: >>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/MainModuleOnly.java.sdiff.html >>>>> Ok. I was expecting a separate unit test. I?m fine with an added test case in MainModuleOnly.java. >>>>> >>>>> Thanks, >>>>> Jiangli >>>>> >>>>>> thanks, >>>>>> Calvin >>>>>>> Thanks, >>>>>>> Jiangli >>>>>>> >>>>>>>> On May 3, 2018, at 3:24 PM, Calvin Cheung wrote: >>>>>>>> >>>>>>>> bug:https://bugs.openjdk.java.net/browse/JDK-8202289 >>>>>>>> webrev:http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ >>>>>>>> >>>>>>>> In ClassLoaderExt::process_module_table(), it adds all entries from the ModuleEntryTable with the "file:" protocol to the _module_path_entries and eventually to the _shared_path_table. >>>>>>>> The FileMapInfo::check_nonempty_dir_in_shared_path_table() will check for non-empty directory in the module path. >>>>>>>> >>>>>>>> Ran all CDS and AppCDS tests locally on linux-x64. >>>>>>>> Will run hs-tier{1,2,3} tests. >>>>>>>> >>>>>>>> thanks, >>>>>>>> Calvin > From chris.plummer at oracle.com Tue May 8 18:13:45 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Tue, 8 May 2018 11:13:45 -0700 Subject: RFR: (S): SA: clhsdb 'where -a' throws Assertion Failure with illegal code 236 when CDS is used In-Reply-To: References: <82c69a0d-02f3-5db8-fedd-d52b00571a55@oracle.com> <41204e1d-11e1-41a9-af90-7cabd12098aa@oracle.com> Message-ID: Hi Jini, Why are _nofast_aload_0 and _nofast_iload using return type BasicType.getTIllegal() when the return type is known? The test changes look good. thanks, Chris On 5/8/18 8:53 AM, Jini George wrote: > Thanks, Ioi. Could I get one more reviewer to take a look at this ? > > Thanks, > Jini. > > On 5/8/2018 8:55 PM, Ioi Lam wrote: >> Looks good. Thanks! >> >> - Ioi >> >> >> On 5/7/18 8:38 PM, Jini George wrote: >>> Thank you very much, Ioi, for the review and for the clarifications >>> and help provided offline. I have added the checks for >>> _nofast_getfield and _nofast_putfield. SA has a bug due to which for >>> iload, only the base bytecode (iload) gets displayed -- fast_iload >>> and nofast_iload do not get displayed. JDK-8202693 (SA: clhsdb >>> printall only displays the base bytecode for iload) has been filed >>> for this. I would add the test for nofast_iload along with the fix >>> for JDK-8202693. >>> >>> The modified webrev is at: >>> >>> http://cr.openjdk.java.net/~jgeorge/8174995/webrev.01/ >>> >>> Thanks, >>> Jini. >>> >>> On 4/27/2018 1:54 AM, Ioi Lam wrote: >>>> HI Jini, >>>> >>>> [1] "_nofast_aload" should be "_nofast_aload_0": aload and aload_0 >>>> are two different bytecodes. >>>> >>>> [2] Only the _nofast_aload_0 bytecode is tested. For completeness, >>>> do you think it makes sense to add test cases for these other 3 >>>> bytecodes? >>>> >>>> ???? _nofast_getfield >>>> ???? _nofast_putfield >>>> ???? _nofast_iload >>>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>> On 4/26/18 11:15 AM, Jini George wrote: >>>>> Hello all, >>>>> >>>>> Please review the following proposed fix for the issue: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8174995 >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~jgeorge/8174995/webrev.00/ >>>>> >>>>> Issue: Clhsdb commands like 'where -a', 'printall' would throw an >>>>> illegal code assertion failure when CDS is used. >>>>> >>>>> Root cause and proposed fix: SA has been unaware of the new >>>>> bytecodes introduced for rewriting at CDS dump time (_nofast* >>>>> bytecodes). The fix is to make SA aware of these new _nofast* >>>>> bytecodes. >>>>> >>>>> Tests Run and Passed: SA tests on Mach5 (including the tests >>>>> modified to test this fix). >>>>> >>>>> Thank you, >>>>> Jini. >>>> >> From jiangli.zhou at oracle.com Tue May 8 21:16:06 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Tue, 8 May 2018 14:16:06 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time In-Reply-To: <5AF1E157.9050905@oracle.com> References: <5AEB8C04.4040501@oracle.com> <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> <5AEB98B7.9030209@oracle.com> <56AA6E39-131E-4D5B-BBEF-9FA8C4A60F5A@oracle.com> <5AF07AE9.9000700@oracle.com> <5AF09BE7.30208@oracle.com> <25B748EB-27EC-4FAB-956F-F864323531EC@oracle.com> <5AF1E157.9050905@oracle.com> Message-ID: <350D2006-F382-429D-9C13-3AABF7AF6954@oracle.com> > On May 8, 2018, at 10:41 AM, Calvin Cheung wrote: > > > > On 5/7/18, 2:58 PM, Jiangli Zhou wrote: >> >> >>> On May 7, 2018, at 11:33 AM, Calvin Cheung > wrote: >>> >>> >>> >>> On 5/7/18, 10:28 AM, Jiangli Zhou wrote: >>>> >>>> Hi Calvin, >>>> >>>> -classLoader.cpp >>>> >>>> The get_canonical_path() call for ?path? should be moved out of the ?for' loop. >>>> >>>> 1569 get_canonical_path(path, canonical_src_path, JVM_MAXPATHLEN)) { >>> Fixed. >>>> To make the code more readable, please rename ?canonical_path? to ?canonical_path_table_entry? (or ?canonical_path_table_entry_name?) >>> I chose the shorter one. >>>> and ?canonical_src_path? to ?canonical_class_src_path?. >>> Renamed. >>>> BTW, when we create the path entries, the paths are already converted to canonical forms. I think there is no need to call get_canonical_path() on ?ent->name()?. Can you please double check? >>> The call to get_canonical_path() is only for the non-directory case (please refer to ClassLoader::create_class_path_entry() for details). >>> I did try get_canonical_path() for the directory case but it didn't work during runtime; it failed in SystemDictionaryShared::is_shared_class_visible_for_classloader() in the following check: >>> >>> (strcmp(ent->name(), ClassLoader::skip_uri_protocol(mod_entry->location()->as_C_string())) == 0)) { >>> >>> The location() in the mod_entry isn't in canonical form. If we convert it to canonical form during runtime, there maybe performance impact. >>> I think the proper way of fixing it is storing the location() in canonical form when a module entry is being defined. I'm not sure if it is possible for the core lib to pass the location in canonical form to jvm when a module entry is being defined. >>> I can file a follow-up RFE to address that. >> >> >> Please file a RFE. > I've filed JDK-8202750 Thanks. >> >> There is no need to check ?res? in very iteration since it doesn?t change after being initialized by get_canonical_path(). Also, I think we are not handling get_canonical_path() (both calls) failure properly. However, I question why get_canonical_path() would fail when given a valid path in this particular case. So I think we should either assert get_canonical_path() returns true, or report a fatal error for dumping. > I've made the change and added assert after get_canonical_path() at both calls. > > Updated webrev: > > http://cr.openjdk.java.net/~ccheung/8202289/webrev.03/ Looks good. Thanks, Jiangli > > thanks, > Calvin >>>> - os_windows.cpp >>>> >>>> Please fix indentation at line 4390 & 4391. >>> That was done on purpose because the condition in line 4390 is '||' with line 4391. >>> I'm further indenting line 4391 to make it clearer. >>> >>> 4389 if (search_path[1] == ':' && >>> 4390 (search_path[2] == '\0' || >>> 4391 (search_path[2] == '\\' && search_path[3] == '\0'))) { >> >> >> Looks better. >> >>> Updated webrev: >>> >>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.02/ >> >> >> Thanks, >> Jiangli >> >>> >>> thanks, >>> Calvin >>> >>>> Thanks, >>>> Jiangli >>>> >>>> >>>>> On May 7, 2018, at 9:12 AM, Calvin Cheung wrote: >>>>> >>>>> Hi Jiangli, >>>>> >>>>> On 5/3/18, 5:33 PM, Jiangli Zhou wrote: >>>>>> Hi Calvin, >>>>>> >>>>>> >>>>>>> On May 3, 2018, at 4:18 PM, Calvin Cheung wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 5/3/18, 3:42 PM, Jiangli Zhou wrote: >>>>>>>> Hi Calvin, >>>>>>>> >>>>>>>> I think we don?t need to check the module path entries if only boot classes are archived. The ?end? is set to app_class_paths_start_index when there are only boot classes are archived. If there are app module classes loaded from jars/directories at dump time, has_platform_or_app_classes() should return true and ?end? is set to the end of the shared path table, which includes all module path entries. The following loop is not needed. The existing loop in the code covers all cases with different ?end? value based on the type of classes loaded in the archive. >>>>>>>> 366 // may need to check module path entries. >>>>>>>> 367 if ((end == ClassLoaderExt::app_class_paths_start_index())&& (ClassLoader::num_module_path_entries()> 0)) { >>>>>>>> 368 for (int i = ClassLoaderExt::app_module_paths_start_index(); i< _shared_path_table_size; i++) { >>>>>>>> 369 SharedClassPathEntry* e = shared_path(i); >>>>>>>> 370 has_nonempty_dir = check_nonempty_dir(e); >>>>>>>> >>>>>>>> 371 } >>>>>>>> 372 } >>>>>>>> >>>>>>> The has_platform_or_app_classes() will return false if ClassLoaderExt::record_result() hasn't been called while a class is being loaded during dumping. >>>>>>> This could happen before this proposed change if one specifies a --module-path with an non-empty directory containing a module. While the class will be loaded during dumping but its classpath_index will still be -1. Therefore, the has_platform_or_app_classes() will return false and the module path entries won't be checked without the above block of code. >>>>>> With the module directory path being added to the path table now, it should record the correct path table index instead of -1 for module classes loaded from directory at dump time. >>>>>> >>>>>> I applied your patch and stepped through ClassLoader::record_result() with your test case. The following path comparison fails due the the extra ?/? in ?path?. That why it fails to find the correct entry in the path table, which causes the class type cannot be correctly identified. >>>>>> >>>>>> if (strcmp(canonical_path, os::native_path((char*)path)) == 0) {os::dir_is_empty >>>>>> >>>>>> (gdb) p canonical_path >>>>>> $74 = 0x7ffff0aea0f0 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple" >>>>>> (gdb) p os::native_path((char*)path) >>>>>> $75 = 0x7ffff7fd4a65 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple/" >>>>>> >>>>>> The trailing ?/? should be excluded for the path comparison. JDK-8202519 is a related issue in the same area. It might be better to fix it together with this one. >>>>> Thanks for debugging the issue. >>>>> I've fixed it by calling get_canonical_path() on the "path" before doing strcmp(). It should also fix the bug in JDK-8202519 since a buffer is being allocated for get_canonical_path() to write into. The os::native_path() on linux does nothing and just returns the same string. >>>>> >>>>> I also discovered a problem in os::dir_is_empty() on windows. When a directory path is passed into the function, it may have the following form with forward slashes: C:/somedir >>>>> The fix is to call os::native_path() which essentially converts the forward slashes to back slashes so that subsequent windows api call returns the correct result. >>>>> >>>>>> Once the above issue is fixed, the class type should be recorded correctly for a module classes loaded from directory at dump time. Then ClassLoaderExt::record_result() can set ?has_app_classes? flag properly. And, no change should be needed in FileMapInfo::check_nonempty_dir_in_shared_path_table(). >>>>> I don't need to touch filemap.cpp in the following updated webrev: >>>>> >>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.01/ >>>>> >>>>> thanks, >>>>> Calvin >>>>>>>> We also need a unit test for non-empty directory in module path. Please add. >>>>>>> It is already in my webrev: >>>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/MainModuleOnly.java.sdiff.html >>>>>> Ok. I was expecting a separate unit test. I?m fine with an added test case in MainModuleOnly.java. >>>>>> >>>>>> Thanks, >>>>>> Jiangli >>>>>> >>>>>>> thanks, >>>>>>> Calvin >>>>>>>> Thanks, >>>>>>>> Jiangli >>>>>>>> >>>>>>>>> On May 3, 2018, at 3:24 PM, Calvin Cheung wrote: >>>>>>>>> >>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8202289 >>>>>>>>> webrev: http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ >>>>>>>>> >>>>>>>>> In ClassLoaderExt::process_module_table(), it adds all entries from the ModuleEntryTable with the "file:" protocol to the _module_path_entries and eventually to the _shared_path_table. >>>>>>>>> The FileMapInfo::check_nonempty_dir_in_shared_path_table() will check for non-empty directory in the module path. >>>>>>>>> >>>>>>>>> Ran all CDS and AppCDS tests locally on linux-x64. >>>>>>>>> Will run hs-tier{1,2,3} tests. >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> Calvin >> From ioi.lam at oracle.com Tue May 8 21:24:51 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 8 May 2018 14:24:51 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time In-Reply-To: <5AF1E157.9050905@oracle.com> References: <5AEB8C04.4040501@oracle.com> <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> <5AEB98B7.9030209@oracle.com> <56AA6E39-131E-4D5B-BBEF-9FA8C4A60F5A@oracle.com> <5AF07AE9.9000700@oracle.com> <5AF09BE7.30208@oracle.com> <25B748EB-27EC-4FAB-956F-F864323531EC@oracle.com> <5AF1E157.9050905@oracle.com> Message-ID: I think we do need to check the results of get_canonical_path. On Linux, the canonical path can be longer than the source path, which means ENAMETOOLONG may be returned. http://hg.openjdk.java.net/jdk/hs/file/tip/src/java.base/unix/native/libjava/canonicalize_md.c#l253 So instead of assert, it's best to check for failure and exit the dumping process. The rest of the changes look OK to me. Thanks - Ioi On 5/8/18 10:41 AM, Calvin Cheung wrote: > > > On 5/7/18, 2:58 PM, Jiangli Zhou wrote: >> >>> On May 7, 2018, at 11:33 AM, Calvin Cheung >> > wrote: >>> >>> >>> >>> On 5/7/18, 10:28 AM, Jiangli Zhou wrote: >>>> Hi Calvin, >>>> >>>> -classLoader.cpp >>>> >>>> The get_canonical_path() call for ?path? should be moved out of the >>>> ?for' loop. >>>> >>>> 1569?????????? get_canonical_path(path, canonical_src_path, >>>> JVM_MAXPATHLEN)) { >>> Fixed. >>>> To make the code more readable, please rename ?canonical_path? to >>>> ?canonical_path_table_entry? (or ?canonical_path_table_entry_name?) >>> I chose the shorter one. >>>> ? and ?canonical_src_path? to ?canonical_class_src_path?. >>> Renamed. >>>> BTW, when we create the path entries, the paths are already >>>> converted to canonical forms. I think there is no need to call >>>> get_canonical_path() on ?ent->name()?. Can you please double check? >>> The call to get_canonical_path() is only for the non-directory case >>> (please refer to ClassLoader::create_class_path_entry() for details). >>> I did try get_canonical_path() for the directory case but it didn't >>> work during runtime; it failed in >>> SystemDictionaryShared::is_shared_class_visible_for_classloader() in >>> the following check: >>> >>> (strcmp(ent->name(), >>> ClassLoader::skip_uri_protocol(mod_entry->location()->as_C_string())) >>> == 0)) { >>> >>> The location() in the mod_entry isn't in canonical form. If we >>> convert it to canonical form during runtime, there maybe performance >>> impact. >>> I think the proper way of fixing it is storing the location() in >>> canonical form when a module entry is being defined. I'm not sure if >>> it is possible for the core lib to pass the location in canonical >>> form to jvm when a module entry is being defined. >>> I can file a follow-up RFE to address that. >> >> Please file a RFE. > I've filed JDK-8202750 >> >> There is no need to check ?res? in very iteration since it doesn?t >> change after being initialized by get_canonical_path(). Also, I think >> we are not handling get_canonical_path() (both calls) failure >> properly. However, I question why get_canonical_path() would fail >> when given a valid path in this particular case. So I think we should >> either assert get_canonical_path() returns true, or report a fatal >> error for dumping. > I've made the change and added assert after get_canonical_path() at > both calls. > > Updated webrev: > > ??? http://cr.openjdk.java.net/~ccheung/8202289/webrev.03/ > > thanks, > Calvin >>>> - os_windows.cpp >>>> >>>> Please fix indentation at line 4390&? 4391. >>> That was done on purpose because the condition in line 4390 is '||' >>> with line 4391. >>> I'm further indenting line 4391 to make it clearer. >>> >>> 4389?? if (search_path[1] == ':'&& >>> 4390??????? (search_path[2] == '\0' || >>> 4391????????? (search_path[2] == '\\'? &&? >>> search_path[3] == '\0'))) { >> >> >> Looks better. >> >>> Updated webrev: >>> >>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.02/ >> >> >> Thanks, >> Jiangli >> >>> >>> thanks, >>> Calvin >>> >>>> Thanks, >>>> Jiangli >>>> >>>> >>>>> On May 7, 2018, at 9:12 AM, Calvin >>>>> Cheung? wrote: >>>>> >>>>> Hi Jiangli, >>>>> >>>>> On 5/3/18, 5:33 PM, Jiangli Zhou wrote: >>>>>> Hi Calvin, >>>>>> >>>>>> >>>>>>> On May 3, 2018, at 4:18 PM, Calvin >>>>>>> Cheung?? wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 5/3/18, 3:42 PM, Jiangli Zhou wrote: >>>>>>>> Hi Calvin, >>>>>>>> >>>>>>>> I think we don?t need to check the module path entries if only >>>>>>>> boot classes are archived. The ?end? is set to >>>>>>>> app_class_paths_start_index when there are only boot classes >>>>>>>> are archived. If there are app module classes loaded from >>>>>>>> jars/directories at dump time, has_platform_or_app_classes() >>>>>>>> should return true and ?end? is set to the end of the shared >>>>>>>> path table, which includes all module path entries. The >>>>>>>> following loop is not needed. The existing loop in the code >>>>>>>> covers all cases with different ?end? value based on the type >>>>>>>> of classes loaded in the archive. >>>>>>>> ? 366?? // may need to check module path entries. >>>>>>>> ? 367?? if ((end == >>>>>>>> ClassLoaderExt::app_class_paths_start_index())&& >>>>>>>> (ClassLoader::num_module_path_entries()>?? 0)) { >>>>>>>> ? 368???? for (int i = >>>>>>>> ClassLoaderExt::app_module_paths_start_index(); i>>>>>>> _shared_path_table_size; i++) { >>>>>>>> ? 369?????? SharedClassPathEntry* e = shared_path(i); >>>>>>>> ? 370?????? has_nonempty_dir = check_nonempty_dir(e); >>>>>>>> >>>>>>>> ? 371???? } >>>>>>>> ? 372?? } >>>>>>>> >>>>>>> The has_platform_or_app_classes() will return false if >>>>>>> ClassLoaderExt::record_result() hasn't been called while a class >>>>>>> is being loaded during dumping. >>>>>>> This could happen before this proposed change if one specifies a >>>>>>> --module-path with an non-empty directory containing a module. >>>>>>> While the class will be loaded during dumping but its >>>>>>> classpath_index will still be -1. Therefore, the >>>>>>> has_platform_or_app_classes() will return false and the module >>>>>>> path entries won't be checked without the above block of code. >>>>>> With the module directory path being added to the path table now, >>>>>> it should record the correct path table index instead of -1 for >>>>>> module classes loaded from directory at dump time. >>>>>> >>>>>> I applied your patch and stepped through >>>>>> ClassLoader::record_result() with your test case. The following >>>>>> path comparison fails due the the extra ?/? in ?path?. That why >>>>>> it fails to find the correct entry in the path table, which >>>>>> causes the class type cannot be correctly identified. >>>>>> >>>>>> ???????? if (strcmp(canonical_path, os::native_path((char*)path)) >>>>>> == 0) {os::dir_is_empty >>>>>> >>>>>> (gdb) p canonical_path >>>>>> $74 = 0x7ffff0aea0f0 >>>>>> ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple" >>>>>> (gdb) p os::native_path((char*)path) >>>>>> $75 = 0x7ffff7fd4a65 >>>>>> ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple/" >>>>>> >>>>>> The trailing ?/? should be excluded for the path comparison. >>>>>> JDK-8202519 is a related issue in the same area. It might be >>>>>> better to fix it together with this one. >>>>> Thanks for debugging the issue. >>>>> I've fixed it by calling get_canonical_path() on the "path" before >>>>> doing strcmp(). It should also fix the bug in JDK-8202519 since a >>>>> buffer is being allocated for get_canonical_path() to write into. >>>>> The os::native_path() on linux does nothing and just returns the >>>>> same string. >>>>> >>>>> I also discovered a problem in os::dir_is_empty() on windows. When >>>>> a directory path is passed into the function, it may have the >>>>> following form with forward slashes: C:/somedir >>>>> The fix is to call os::native_path() which essentially converts >>>>> the forward slashes to back slashes so that subsequent windows api >>>>> call returns the correct result. >>>>> >>>>>> Once the above issue is fixed, the class type should be recorded >>>>>> correctly for a module classes loaded from directory at dump >>>>>> time. Then ClassLoaderExt::record_result() can set >>>>>> ?has_app_classes? flag properly. And, no change should be needed >>>>>> in FileMapInfo::check_nonempty_dir_in_shared_path_table(). >>>>> I don't need to touch filemap.cpp in the following updated webrev: >>>>> >>>>> ??? http://cr.openjdk.java.net/~ccheung/8202289/webrev.01/ >>>>> >>>>> thanks, >>>>> Calvin >>>>>>>> We also need a unit test for non-empty directory in module >>>>>>>> path. Please add. >>>>>>> It is already in my webrev: >>>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/MainModuleOnly.java.sdiff.html >>>>>>> >>>>>> Ok. I was expecting a separate unit test. I?m fine with an added >>>>>> test case in MainModuleOnly.java. >>>>>> >>>>>> Thanks, >>>>>> Jiangli >>>>>> >>>>>>> thanks, >>>>>>> Calvin >>>>>>>> Thanks, >>>>>>>> Jiangli >>>>>>>> >>>>>>>>> On May 3, 2018, at 3:24 PM, Calvin >>>>>>>>> Cheung wrote: >>>>>>>>> >>>>>>>>> bug:https://bugs.openjdk.java.net/browse/JDK-8202289 >>>>>>>>> webrev:http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ >>>>>>>>> >>>>>>>>> In ClassLoaderExt::process_module_table(), it adds all entries >>>>>>>>> from the ModuleEntryTable with the "file:" protocol to the >>>>>>>>> _module_path_entries and eventually to the _shared_path_table. >>>>>>>>> The FileMapInfo::check_nonempty_dir_in_shared_path_table() >>>>>>>>> will check for non-empty directory in the module path. >>>>>>>>> >>>>>>>>> Ran all CDS and AppCDS tests locally on linux-x64. >>>>>>>>> Will run hs-tier{1,2,3} tests. >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> Calvin >> From jiangli.zhou at oracle.com Tue May 8 21:37:45 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Tue, 8 May 2018 14:37:45 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time In-Reply-To: References: <5AEB8C04.4040501@oracle.com> <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> <5AEB98B7.9030209@oracle.com> <56AA6E39-131E-4D5B-BBEF-9FA8C4A60F5A@oracle.com> <5AF07AE9.9000700@oracle.com> <5AF09BE7.30208@oracle.com> <25B748EB-27EC-4FAB-956F-F864323531EC@oracle.com> <5AF1E157.9050905@oracle.com> Message-ID: <18CD7B50-2063-47A3-867B-EF6150E2260E@oracle.com> > On May 8, 2018, at 2:24 PM, Ioi Lam wrote: > > I think we do need to check the results of get_canonical_path. On Linux, the canonical path can be longer than the source path, which means ENAMETOOLONG may be returned. > > http://hg.openjdk.java.net/jdk/hs/file/tip/src/java.base/unix/native/libjava/canonicalize_md.c#l253 We need a test case for long path if this could happen by the time we are searching the CDS path table. I was thinking to file a new test bug earlier, but it seems this should be done together with Calvin?s current bug fix. Thanks, Jiangli > > So instead of assert, it's best to check for failure and exit the dumping process. > > The rest of the changes look OK to me. > > Thanks > > - Ioi > > > > On 5/8/18 10:41 AM, Calvin Cheung wrote: >> >> >> On 5/7/18, 2:58 PM, Jiangli Zhou wrote: >>> >>>> On May 7, 2018, at 11:33 AM, Calvin Cheung > wrote: >>>> >>>> >>>> >>>> On 5/7/18, 10:28 AM, Jiangli Zhou wrote: >>>>> Hi Calvin, >>>>> >>>>> -classLoader.cpp >>>>> >>>>> The get_canonical_path() call for ?path? should be moved out of the ?for' loop. >>>>> >>>>> 1569 get_canonical_path(path, canonical_src_path, JVM_MAXPATHLEN)) { >>>> Fixed. >>>>> To make the code more readable, please rename ?canonical_path? to ?canonical_path_table_entry? (or ?canonical_path_table_entry_name?) >>>> I chose the shorter one. >>>>> and ?canonical_src_path? to ?canonical_class_src_path?. >>>> Renamed. >>>>> BTW, when we create the path entries, the paths are already converted to canonical forms. I think there is no need to call get_canonical_path() on ?ent->name()?. Can you please double check? >>>> The call to get_canonical_path() is only for the non-directory case (please refer to ClassLoader::create_class_path_entry() for details). >>>> I did try get_canonical_path() for the directory case but it didn't work during runtime; it failed in SystemDictionaryShared::is_shared_class_visible_for_classloader() in the following check: >>>> >>>> (strcmp(ent->name(), ClassLoader::skip_uri_protocol(mod_entry->location()->as_C_string())) == 0)) { >>>> >>>> The location() in the mod_entry isn't in canonical form. If we convert it to canonical form during runtime, there maybe performance impact. >>>> I think the proper way of fixing it is storing the location() in canonical form when a module entry is being defined. I'm not sure if it is possible for the core lib to pass the location in canonical form to jvm when a module entry is being defined. >>>> I can file a follow-up RFE to address that. >>> >>> Please file a RFE. >> I've filed JDK-8202750 >>> >>> There is no need to check ?res? in very iteration since it doesn?t change after being initialized by get_canonical_path(). Also, I think we are not handling get_canonical_path() (both calls) failure properly. However, I question why get_canonical_path() would fail when given a valid path in this particular case. So I think we should either assert get_canonical_path() returns true, or report a fatal error for dumping. >> I've made the change and added assert after get_canonical_path() at both calls. >> >> Updated webrev: >> >> http://cr.openjdk.java.net/~ccheung/8202289/webrev.03/ >> >> thanks, >> Calvin >>>>> - os_windows.cpp >>>>> >>>>> Please fix indentation at line 4390& 4391. >>>> That was done on purpose because the condition in line 4390 is '||' with line 4391. >>>> I'm further indenting line 4391 to make it clearer. >>>> >>>> 4389 if (search_path[1] == ':'&& >>>> 4390 (search_path[2] == '\0' || >>>> 4391 (search_path[2] == '\\' && search_path[3] == '\0'))) { >>> >>> >>> Looks better. >>> >>>> Updated webrev: >>>> >>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.02/ >>> >>> >>> Thanks, >>> Jiangli >>> >>>> >>>> thanks, >>>> Calvin >>>> >>>>> Thanks, >>>>> Jiangli >>>>> >>>>> >>>>>> On May 7, 2018, at 9:12 AM, Calvin Cheung wrote: >>>>>> >>>>>> Hi Jiangli, >>>>>> >>>>>> On 5/3/18, 5:33 PM, Jiangli Zhou wrote: >>>>>>> Hi Calvin, >>>>>>> >>>>>>> >>>>>>>> On May 3, 2018, at 4:18 PM, Calvin Cheung wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 5/3/18, 3:42 PM, Jiangli Zhou wrote: >>>>>>>>> Hi Calvin, >>>>>>>>> >>>>>>>>> I think we don?t need to check the module path entries if only boot classes are archived. The ?end? is set to app_class_paths_start_index when there are only boot classes are archived. If there are app module classes loaded from jars/directories at dump time, has_platform_or_app_classes() should return true and ?end? is set to the end of the shared path table, which includes all module path entries. The following loop is not needed. The existing loop in the code covers all cases with different ?end? value based on the type of classes loaded in the archive. >>>>>>>>> 366 // may need to check module path entries. >>>>>>>>> 367 if ((end == ClassLoaderExt::app_class_paths_start_index())&& (ClassLoader::num_module_path_entries()> 0)) { >>>>>>>>> 368 for (int i = ClassLoaderExt::app_module_paths_start_index(); i< _shared_path_table_size; i++) { >>>>>>>>> 369 SharedClassPathEntry* e = shared_path(i); >>>>>>>>> 370 has_nonempty_dir = check_nonempty_dir(e); >>>>>>>>> >>>>>>>>> 371 } >>>>>>>>> 372 } >>>>>>>>> >>>>>>>> The has_platform_or_app_classes() will return false if ClassLoaderExt::record_result() hasn't been called while a class is being loaded during dumping. >>>>>>>> This could happen before this proposed change if one specifies a --module-path with an non-empty directory containing a module. While the class will be loaded during dumping but its classpath_index will still be -1. Therefore, the has_platform_or_app_classes() will return false and the module path entries won't be checked without the above block of code. >>>>>>> With the module directory path being added to the path table now, it should record the correct path table index instead of -1 for module classes loaded from directory at dump time. >>>>>>> >>>>>>> I applied your patch and stepped through ClassLoader::record_result() with your test case. The following path comparison fails due the the extra ?/? in ?path?. That why it fails to find the correct entry in the path table, which causes the class type cannot be correctly identified. >>>>>>> >>>>>>> if (strcmp(canonical_path, os::native_path((char*)path)) == 0) {os::dir_is_empty >>>>>>> >>>>>>> (gdb) p canonical_path >>>>>>> $74 = 0x7ffff0aea0f0 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple" >>>>>>> (gdb) p os::native_path((char*)path) >>>>>>> $75 = 0x7ffff7fd4a65 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple/" >>>>>>> >>>>>>> The trailing ?/? should be excluded for the path comparison. JDK-8202519 is a related issue in the same area. It might be better to fix it together with this one. >>>>>> Thanks for debugging the issue. >>>>>> I've fixed it by calling get_canonical_path() on the "path" before doing strcmp(). It should also fix the bug in JDK-8202519 since a buffer is being allocated for get_canonical_path() to write into. The os::native_path() on linux does nothing and just returns the same string. >>>>>> >>>>>> I also discovered a problem in os::dir_is_empty() on windows. When a directory path is passed into the function, it may have the following form with forward slashes: C:/somedir >>>>>> The fix is to call os::native_path() which essentially converts the forward slashes to back slashes so that subsequent windows api call returns the correct result. >>>>>> >>>>>>> Once the above issue is fixed, the class type should be recorded correctly for a module classes loaded from directory at dump time. Then ClassLoaderExt::record_result() can set ?has_app_classes? flag properly. And, no change should be needed in FileMapInfo::check_nonempty_dir_in_shared_path_table(). >>>>>> I don't need to touch filemap.cpp in the following updated webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.01/ >>>>>> >>>>>> thanks, >>>>>> Calvin >>>>>>>>> We also need a unit test for non-empty directory in module path. Please add. >>>>>>>> It is already in my webrev: >>>>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/MainModuleOnly.java.sdiff.html >>>>>>> Ok. I was expecting a separate unit test. I?m fine with an added test case in MainModuleOnly.java. >>>>>>> >>>>>>> Thanks, >>>>>>> Jiangli >>>>>>> >>>>>>>> thanks, >>>>>>>> Calvin >>>>>>>>> Thanks, >>>>>>>>> Jiangli >>>>>>>>> >>>>>>>>>> On May 3, 2018, at 3:24 PM, Calvin Cheung wrote: >>>>>>>>>> >>>>>>>>>> bug:https://bugs.openjdk.java.net/browse/JDK-8202289 >>>>>>>>>> webrev:http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ >>>>>>>>>> >>>>>>>>>> In ClassLoaderExt::process_module_table(), it adds all entries from the ModuleEntryTable with the "file:" protocol to the _module_path_entries and eventually to the _shared_path_table. >>>>>>>>>> The FileMapInfo::check_nonempty_dir_in_shared_path_table() will check for non-empty directory in the module path. >>>>>>>>>> >>>>>>>>>> Ran all CDS and AppCDS tests locally on linux-x64. >>>>>>>>>> Will run hs-tier{1,2,3} tests. >>>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> Calvin >>> > From thomas.stuefe at gmail.com Wed May 9 04:48:48 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 9 May 2018 06:48:48 +0200 Subject: RFR: 8202634: Metaspace: simplify SpaceManager lists In-Reply-To: References: Message-ID: HI Zhengyu, On Tue, May 8, 2018 at 2:51 PM, Zhengyu Gu wrote: > Hi Thomas, > > Looks good to me. > Thanks! > Nits: > > metaspace.cpp L3049 > assertion is no longer needed. > You are right, I'll remove it before I push. > > Thanks. > > -Zhengyu > ..Thomas > > > On 05/08/2018 12:47 AM, Thomas St?fe wrote: >> >> (jdk-submit tests ok. SAP tests did not show anything attributable to >> this patch.) >> >> On Mon, May 7, 2018 at 5:29 PM, Thomas St?fe >> wrote: >>> >>> Hi all, >>> >>> may I please have reviews for this small simplification. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202634 >>> webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8202634-simplify-spacemanager/webrev.00/webrev/ >>> >>> This replaces the four separate in-use-chunk-lists in SpaceManager >>> with a single one. The reasoning behind this is that we do not need to >>> keep the chunks sorted by chunk size, a single list containing all >>> chunks of all sizes in use by this SpaceManager is enough, and it >>> simplifies the coding. >>> >>> There is only one place where we really did use the former >>> by-chunktype lists - that is to count how many chunks per chunk type >>> there are. >>> >>> However, arguably that is better done with running counters, so this >>> is what I did. The added benefit is that in >>> SpaceManager::calc_chunk_size() we do not need to walk the lists >>> anymore to sum up chunks. >>> >>> Changes in detail: >>> >>> - removed the "ChunkIndex" argument in >>> ChunkManager::return_single_chunk() and >>> Chunkmanager::return_chunk_list() - that argument was unnecessary, >>> since the Metachunk we return knows its index. >>> >>> - Unfolded SpaceManager::initialize into Spacemanager constructor (I >>> prefer initializing members in ctor init lists) >>> >>> - replaces SpaceManager::sum_count_chunks_in_use() with running counters. >>> >>> - I dumbed down the logging inside ChunkManager::return_chunk_list() >>> somewhat. Should anyone miss the former details, I can reinstate it. >>> >>> Thank you, >>> >>> Thomas From thomas.stuefe at gmail.com Wed May 9 04:49:12 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 9 May 2018 06:49:12 +0200 Subject: RFR: 8202634: Metaspace: simplify SpaceManager lists In-Reply-To: <0418aef9-d4ef-14f4-4e84-b6321dfa00b9@oracle.com> References: <0418aef9-d4ef-14f4-4e84-b6321dfa00b9@oracle.com> Message-ID: Hi Coleen, thanks for the review! ..Thomas On Tue, May 8, 2018 at 5:53 PM, wrote: > > This change looks great. Reviewed. > > On 5/7/18 11:29 AM, Thomas St?fe wrote: >> >> Hi all, >> >> may I please have reviews for this small simplification. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8202634 >> webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/8202634-simplify-spacemanager/webrev.00/webrev/ >> >> This replaces the four separate in-use-chunk-lists in SpaceManager >> with a single one. The reasoning behind this is that we do not need to >> keep the chunks sorted by chunk size, a single list containing all >> chunks of all sizes in use by this SpaceManager is enough, and it >> simplifies the coding. >> >> There is only one place where we really did use the former >> by-chunktype lists - that is to count how many chunks per chunk type >> there are. >> >> However, arguably that is better done with running counters, so this >> is what I did. The added benefit is that in >> SpaceManager::calc_chunk_size() we do not need to walk the lists >> anymore to sum up chunks. >> >> Changes in detail: >> >> - removed the "ChunkIndex" argument in >> ChunkManager::return_single_chunk() and >> Chunkmanager::return_chunk_list() - that argument was unnecessary, >> since the Metachunk we return knows its index. > > > good. >> >> >> - Unfolded SpaceManager::initialize into Spacemanager constructor (I >> prefer initializing members in ctor init lists) > > > I agree! >> >> >> - replaces SpaceManager::sum_count_chunks_in_use() with running counters. >> >> - I dumbed down the logging inside ChunkManager::return_chunk_list() >> somewhat. Should anyone miss the former details, I can reinstate it. > > > I don't think we'll miss the implementation details in the logging. Knowing > how much space is returned seems most useful. > > thanks, > Coleen >> >> >> Thank you, >> >> Thomas > > From thomas.stuefe at gmail.com Wed May 9 15:08:03 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 9 May 2018 17:08:03 +0200 Subject: RFR(L): 8176808: Split up metaspace.cpp Message-ID: Hi all, This was a long time coming. Lets cut up this beast :) Bug: https://bugs.openjdk.java.net/browse/JDK-8176808 Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace-cpp/webrev.00/webrev/ This change splits up metaspace.cpp into multiple parts. Note that I did not change much code (exceptions see below in details) - so this is mostly moving code around. This change follows the scheme tentatively outlined in JDK-8201572: - metaspace internal classes go into the sub directory share/memory/metaspace. Ideally, those should never be accessed from outside, apart from other metaspace classes and metaspace tests. They also go into the namespace ::metaspace::internals. - metaspace external classes (MetaspaceUtils, ClassLoaderMetaspace, etc) remain inside share/memory and, for now, remain in the global namespace. Note: the original idea was to move metaspace external classes to namespace ::metaspace. But I shied away from that, since that would mean fixing up all usages of these classes either with metaspace:: scope or with using declarations. I had to make a cut at some point. So here is something to decide: - should we then get rid of the ::metaspace::internals namespace, move metaspace-internal classes to the enclosing ::metaspace namespace and leave external classes in the global namespace ? - or should we follow through (maybe in a future patch): internal classes in ::metaspace::internal, and move external classes to ::metaspace ? Some more details: - the following classes moved out of metaspace.cpp into namespace "metaspace::internal" and the metaspace sub directory: MetaDebug, ChunkManager, SpaceManager, BlockFreeList and SmallBlocks, OccupancyMap, VirtualSpaceNode and VirtualSpaceList, the PrintCLDMetaspaceInfoClosure. - I also moved metachunk.hpp to internals, since class Metachunk is not used externally. What is used externally is Metablock though, so I split up metachunk.hpp into metabase.hpp, metablock.hpp and metachunk.hpp, holding Metabase, Metablock and Metachunk, respectively. - Now we see some ugly seams: metaspace.hpp is supposed to be the outside API contract, however, it contains implementation details which should not be there and which manifest now by having to use the metaspace::internals namespace here and there. If we care, we could solve this with a bit redesigning. - The ChunkManager gtest and the SpaceManager gtest had been implemented inside metaspace.cpp, since it was not possible to access the internal classes those tests needed in any other way. Since that is now possible, I moved the coding out to gtest-land and made them real gtests (exchanging the asserts for real gtest ASSERTS, among other things). Note that there are some tests left in metaspace.cpp (TestMetaspaceUtilsTest, TestVirtualSpaceNodeTest) - those were no gtests-in-disguise but were part of -XX:+ExecuteInternalVMTests. I guess these tests precede the gtest framework? I leave it up to others to decide what to do with them and to potentially move them out of metaspace.cpp. Side note: -XX:+ExecuteInternalVMTests seems to have bitrotted, see https://bugs.openjdk.java.net/browse/JDK-8202848. Does this get tested regularly? - metaspace.cpp is quite a bit smaller now - from ~5000 loc down to ~1700 loc. Arguably, one could split out more and clean up more, but I think this is a good start. --- I built locally on linux (release, fastdebug with and without pch, zero) and windows (64, 32bit). jdk-submit tests ran through with a build error on solaris sparc - I currently try to reproduce that build error with our very slow solaris machine. Thanks, Thomas From thomas.stuefe at gmail.com Wed May 9 16:01:22 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 9 May 2018 18:01:22 +0200 Subject: RFR(M): 8199257: [TESTBUG] Open source VM testbase metaspace tests In-Reply-To: References: Message-ID: Not a review, but I ran these tests on my local machine (with my current metaspace changes) and they all passed. Will these tests be somehow merged with the existing metaspace tests at runtime/Metaspace? I think the general question is what is vmTestBase, and what sets the tests in there apart from the existing tests outside vmTestBase? Thanks, Thomas On Tue, May 8, 2018 at 7:38 PM, mikhailo wrote: > http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html > > Please review this change open sourcing VM metaspace tests. These tests have > been used internally for a while, and are now being open sourced. Since this > is not an creation of new tests, we would like to keep the changes during > this review to an absolute required minimum. If you have any feedback on > improvements of these tests, please file RFE(s) that will be addressed later > in order of priority. > > > Here is what was done for this change: > 1. Moved the tests to OpenJDK repository to the specified directory > location and structure. > 3. Updated Copyright statements accordingly. > 4. Updated "@library" statements accordingly. > 5. Updated TEST.groups and ProblemList.txt > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199257 > Webrev: http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html > > Testing: > 1. Ran the following tests on open-only repository and build, using > "make run-test" (Linux-x64) > vmTestbase_vm_metaspace, vmTestbase_largepages > All PASS > > 2. Automated multip-platform test system (usual 4 platforms): > - vmTestbase_vm_metaspace, vmTestbase_largepages > - hs-tier{1,2,3} > All PASS > > Thank you, > Misha > From yumin.qi at gmail.com Wed May 9 17:59:44 2018 From: yumin.qi at gmail.com (yumin qi) Date: Wed, 9 May 2018 10:59:44 -0700 Subject: AppCDS only used in G1GC? Message-ID: MetaspaceShared::is_heap_object_archiving_allowed static bool is_heap_object_archiving_allowed() { CDS_JAVA_HEAP_ONLY(return (UseG1GC && UseCompressedOops && UseCompressedClassPointers);) NOT_CDS_JAVA_HEAP(return false;) } Thanks Yumin From jiangli.zhou at oracle.com Wed May 9 18:06:03 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Wed, 9 May 2018 11:06:03 -0700 Subject: AppCDS only used in G1GC? In-Reply-To: References: Message-ID: <23885C42-8CFD-49A6-9E9D-492DB166158A@oracle.com> Hi Yumin, Currently java heap object archiving is only support for G1 GC. If other GC algorithm is used at either CDS dump time or runtime, the java heap object archiving is disable without affecting the class metadata archiving. Thanks, Jiangli > On May 9, 2018, at 10:59 AM, yumin qi wrote: > > MetaspaceShared::is_heap_object_archiving_allowed > > static bool is_heap_object_archiving_allowed() { > CDS_JAVA_HEAP_ONLY(return (UseG1GC && UseCompressedOops && > UseCompressedClassPointers);) > NOT_CDS_JAVA_HEAP(return false;) > } > > > Thanks > Yumin From yumin.qi at gmail.com Wed May 9 18:52:18 2018 From: yumin.qi at gmail.com (yumin qi) Date: Wed, 9 May 2018 11:52:18 -0700 Subject: AppCDS only used in G1GC? In-Reply-To: <23885C42-8CFD-49A6-9E9D-492DB166158A@oracle.com> References: <23885C42-8CFD-49A6-9E9D-492DB166158A@oracle.com> Message-ID: Jiangli, Thanks for the explanation. Yumin On Wed, May 9, 2018 at 11:06 AM, Jiangli Zhou wrote: > Hi Yumin, > > Currently java heap object archiving is only support for G1 GC. If other > GC algorithm is used at either CDS dump time or runtime, the java heap > object archiving is disable without affecting the class metadata archiving. > > Thanks, > Jiangli > > > On May 9, 2018, at 10:59 AM, yumin qi wrote: > > > > MetaspaceShared::is_heap_object_archiving_allowed > > > > static bool is_heap_object_archiving_allowed() { > > CDS_JAVA_HEAP_ONLY(return (UseG1GC && UseCompressedOops && > > UseCompressedClassPointers);) > > NOT_CDS_JAVA_HEAP(return false;) > > } > > > > > > Thanks > > Yumin > > From calvin.cheung at oracle.com Thu May 10 00:37:13 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 09 May 2018 17:37:13 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time In-Reply-To: References: <5AEB8C04.4040501@oracle.com> <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> <5AEB98B7.9030209@oracle.com> <56AA6E39-131E-4D5B-BBEF-9FA8C4A60F5A@oracle.com> <5AF07AE9.9000700@oracle.com> <5AF09BE7.30208@oracle.com> <25B748EB-27EC-4FAB-956F-F864323531EC@oracle.com> <5AF1E157.9050905@oracle.com> Message-ID: <5AF39439.8060403@oracle.com> On 5/8/18, 2:24 PM, Ioi Lam wrote: > I think we do need to check the results of get_canonical_path. On > Linux, the canonical path can be longer than the source path, which > means ENAMETOOLONG may be returned. > > http://hg.openjdk.java.net/jdk/hs/file/tip/src/java.base/unix/native/libjava/canonicalize_md.c#l253 > > > So instead of assert, it's best to check for failure and exit the > dumping process. I've made the above change. Updated webrev: http://cr.openjdk.java.net/~ccheung/8202289/webrev.04/ thanks, Calvin > > The rest of the changes look OK to me. > > Thanks > > - Ioi > > > > On 5/8/18 10:41 AM, Calvin Cheung wrote: >> >> >> On 5/7/18, 2:58 PM, Jiangli Zhou wrote: >>> >>>> On May 7, 2018, at 11:33 AM, Calvin Cheung >>>> > wrote: >>>> >>>> >>>> >>>> On 5/7/18, 10:28 AM, Jiangli Zhou wrote: >>>>> Hi Calvin, >>>>> >>>>> -classLoader.cpp >>>>> >>>>> The get_canonical_path() call for ?path? should be moved out of >>>>> the ?for' loop. >>>>> >>>>> 1569 get_canonical_path(path, canonical_src_path, >>>>> JVM_MAXPATHLEN)) { >>>> Fixed. >>>>> To make the code more readable, please rename ?canonical_path? to >>>>> ?canonical_path_table_entry? (or ?canonical_path_table_entry_name?) >>>> I chose the shorter one. >>>>> and ?canonical_src_path? to ?canonical_class_src_path?. >>>> Renamed. >>>>> BTW, when we create the path entries, the paths are already >>>>> converted to canonical forms. I think there is no need to call >>>>> get_canonical_path() on ?ent->name()?. Can you please double check? >>>> The call to get_canonical_path() is only for the non-directory case >>>> (please refer to ClassLoader::create_class_path_entry() for details). >>>> I did try get_canonical_path() for the directory case but it didn't >>>> work during runtime; it failed in >>>> SystemDictionaryShared::is_shared_class_visible_for_classloader() >>>> in the following check: >>>> >>>> (strcmp(ent->name(), >>>> ClassLoader::skip_uri_protocol(mod_entry->location()->as_C_string())) >>>> == 0)) { >>>> >>>> The location() in the mod_entry isn't in canonical form. If we >>>> convert it to canonical form during runtime, there maybe >>>> performance impact. >>>> I think the proper way of fixing it is storing the location() in >>>> canonical form when a module entry is being defined. I'm not sure >>>> if it is possible for the core lib to pass the location in >>>> canonical form to jvm when a module entry is being defined. >>>> I can file a follow-up RFE to address that. >>> >>> Please file a RFE. >> I've filed JDK-8202750 >> >>> >>> There is no need to check ?res? in very iteration since it doesn?t >>> change after being initialized by get_canonical_path(). Also, I >>> think we are not handling get_canonical_path() (both calls) failure >>> properly. However, I question why get_canonical_path() would fail >>> when given a valid path in this particular case. So I think we >>> should either assert get_canonical_path() returns true, or report a >>> fatal error for dumping. >> I've made the change and added assert after get_canonical_path() at >> both calls. >> >> Updated webrev: >> >> http://cr.openjdk.java.net/~ccheung/8202289/webrev.03/ >> >> thanks, >> Calvin >>>>> - os_windows.cpp >>>>> >>>>> Please fix indentation at line 4390& 4391. >>>> That was done on purpose because the condition in line 4390 is '||' >>>> with line 4391. >>>> I'm further indenting line 4391 to make it clearer. >>>> >>>> 4389 if (search_path[1] == ':'&& >>>> 4390 (search_path[2] == '\0' || >>>> 4391 (search_path[2] == '\\' && >>>> search_path[3] == '\0'))) { >>> >>> >>> Looks better. >>> >>>> Updated webrev: >>>> >>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.02/ >>> >>> >>> Thanks, >>> Jiangli >>> >>>> >>>> thanks, >>>> Calvin >>>> >>>>> Thanks, >>>>> Jiangli >>>>> >>>>> >>>>>> On May 7, 2018, at 9:12 AM, Calvin >>>>>> Cheung wrote: >>>>>> >>>>>> Hi Jiangli, >>>>>> >>>>>> On 5/3/18, 5:33 PM, Jiangli Zhou wrote: >>>>>>> Hi Calvin, >>>>>>> >>>>>>> >>>>>>>> On May 3, 2018, at 4:18 PM, Calvin >>>>>>>> Cheung wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 5/3/18, 3:42 PM, Jiangli Zhou wrote: >>>>>>>>> Hi Calvin, >>>>>>>>> >>>>>>>>> I think we don?t need to check the module path entries if only >>>>>>>>> boot classes are archived. The ?end? is set to >>>>>>>>> app_class_paths_start_index when there are only boot classes >>>>>>>>> are archived. If there are app module classes loaded from >>>>>>>>> jars/directories at dump time, has_platform_or_app_classes() >>>>>>>>> should return true and ?end? is set to the end of the shared >>>>>>>>> path table, which includes all module path entries. The >>>>>>>>> following loop is not needed. The existing loop in the code >>>>>>>>> covers all cases with different ?end? value based on the type >>>>>>>>> of classes loaded in the archive. >>>>>>>>> 366 // may need to check module path entries. >>>>>>>>> 367 if ((end == >>>>>>>>> ClassLoaderExt::app_class_paths_start_index())&& >>>>>>>>> (ClassLoader::num_module_path_entries()> 0)) { >>>>>>>>> 368 for (int i = >>>>>>>>> ClassLoaderExt::app_module_paths_start_index(); i< >>>>>>>>> _shared_path_table_size; i++) { >>>>>>>>> 369 SharedClassPathEntry* e = shared_path(i); >>>>>>>>> 370 has_nonempty_dir = check_nonempty_dir(e); >>>>>>>>> >>>>>>>>> 371 } >>>>>>>>> 372 } >>>>>>>>> >>>>>>>> The has_platform_or_app_classes() will return false if >>>>>>>> ClassLoaderExt::record_result() hasn't been called while a >>>>>>>> class is being loaded during dumping. >>>>>>>> This could happen before this proposed change if one specifies >>>>>>>> a --module-path with an non-empty directory containing a >>>>>>>> module. While the class will be loaded during dumping but its >>>>>>>> classpath_index will still be -1. Therefore, the >>>>>>>> has_platform_or_app_classes() will return false and the module >>>>>>>> path entries won't be checked without the above block of code. >>>>>>> With the module directory path being added to the path table >>>>>>> now, it should record the correct path table index instead of -1 >>>>>>> for module classes loaded from directory at dump time. >>>>>>> >>>>>>> I applied your patch and stepped through >>>>>>> ClassLoader::record_result() with your test case. The following >>>>>>> path comparison fails due the the extra ?/? in ?path?. That why >>>>>>> it fails to find the correct entry in the path table, which >>>>>>> causes the class type cannot be correctly identified. >>>>>>> >>>>>>> if (strcmp(canonical_path, >>>>>>> os::native_path((char*)path)) == 0) {os::dir_is_empty >>>>>>> >>>>>>> (gdb) p canonical_path >>>>>>> $74 = 0x7ffff0aea0f0 >>>>>>> ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple" >>>>>>> (gdb) p os::native_path((char*)path) >>>>>>> $75 = 0x7ffff7fd4a65 >>>>>>> ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple/" >>>>>>> >>>>>>> The trailing ?/? should be excluded for the path comparison. >>>>>>> JDK-8202519 is a related issue in the same area. It might be >>>>>>> better to fix it together with this one. >>>>>> Thanks for debugging the issue. >>>>>> I've fixed it by calling get_canonical_path() on the "path" >>>>>> before doing strcmp(). It should also fix the bug in JDK-8202519 >>>>>> since a buffer is being allocated for get_canonical_path() to >>>>>> write into. The os::native_path() on linux does nothing and just >>>>>> returns the same string. >>>>>> >>>>>> I also discovered a problem in os::dir_is_empty() on windows. >>>>>> When a directory path is passed into the function, it may have >>>>>> the following form with forward slashes: C:/somedir >>>>>> The fix is to call os::native_path() which essentially converts >>>>>> the forward slashes to back slashes so that subsequent windows >>>>>> api call returns the correct result. >>>>>> >>>>>>> Once the above issue is fixed, the class type should be recorded >>>>>>> correctly for a module classes loaded from directory at dump >>>>>>> time. Then ClassLoaderExt::record_result() can set >>>>>>> ?has_app_classes? flag properly. And, no change should be needed >>>>>>> in FileMapInfo::check_nonempty_dir_in_shared_path_table(). >>>>>> I don't need to touch filemap.cpp in the following updated webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.01/ >>>>>> >>>>>> thanks, >>>>>> Calvin >>>>>>>>> We also need a unit test for non-empty directory in module >>>>>>>>> path. Please add. >>>>>>>> It is already in my webrev: >>>>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/MainModuleOnly.java.sdiff.html >>>>>>>> >>>>>>> Ok. I was expecting a separate unit test. I?m fine with an added >>>>>>> test case in MainModuleOnly.java. >>>>>>> >>>>>>> Thanks, >>>>>>> Jiangli >>>>>>> >>>>>>>> thanks, >>>>>>>> Calvin >>>>>>>>> Thanks, >>>>>>>>> Jiangli >>>>>>>>> >>>>>>>>>> On May 3, 2018, at 3:24 PM, Calvin >>>>>>>>>> Cheung wrote: >>>>>>>>>> >>>>>>>>>> bug:https://bugs.openjdk.java.net/browse/JDK-8202289 >>>>>>>>>> webrev:http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ >>>>>>>>>> >>>>>>>>>> In ClassLoaderExt::process_module_table(), it adds all >>>>>>>>>> entries from the ModuleEntryTable with the "file:" protocol >>>>>>>>>> to the _module_path_entries and eventually to the >>>>>>>>>> _shared_path_table. >>>>>>>>>> The FileMapInfo::check_nonempty_dir_in_shared_path_table() >>>>>>>>>> will check for non-empty directory in the module path. >>>>>>>>>> >>>>>>>>>> Ran all CDS and AppCDS tests locally on linux-x64. >>>>>>>>>> Will run hs-tier{1,2,3} tests. >>>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> Calvin >>> > From calvin.cheung at oracle.com Thu May 10 00:41:15 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 09 May 2018 17:41:15 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time In-Reply-To: <18CD7B50-2063-47A3-867B-EF6150E2260E@oracle.com> References: <5AEB8C04.4040501@oracle.com> <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> <5AEB98B7.9030209@oracle.com> <56AA6E39-131E-4D5B-BBEF-9FA8C4A60F5A@oracle.com> <5AF07AE9.9000700@oracle.com> <5AF09BE7.30208@oracle.com> <25B748EB-27EC-4FAB-956F-F864323531EC@oracle.com> <5AF1E157.9050905@oracle.com> <18CD7B50-2063-47A3-867B-EF6150E2260E@oracle.com> Message-ID: <5AF3952B.9070801@oracle.com> On 5/8/18, 2:37 PM, Jiangli Zhou wrote: >> On May 8, 2018, at 2:24 PM, Ioi Lam wrote: >> >> I think we do need to check the results of get_canonical_path. On Linux, the canonical path can be longer than the source path, which means ENAMETOOLONG may be returned. >> >> http://hg.openjdk.java.net/jdk/hs/file/tip/src/java.base/unix/native/libjava/canonicalize_md.c#l253 > We need a test case for long path if this could happen by the time we are searching the CDS path table. I was thinking to file a new test bug earlier, but it seems this should be done together with Calvin?s current bug fix. I've updated the bug report on why it is hard to write a test case with a long path to test the failure of get_canonical_path() during CDS dumping. thanks, Calvin > > Thanks, > Jiangli > >> So instead of assert, it's best to check for failure and exit the dumping process. >> >> The rest of the changes look OK to me. >> >> Thanks >> >> - Ioi >> >> >> >> On 5/8/18 10:41 AM, Calvin Cheung wrote: >>> >>> On 5/7/18, 2:58 PM, Jiangli Zhou wrote: >>>>> On May 7, 2018, at 11:33 AM, Calvin Cheung> wrote: >>>>> >>>>> >>>>> >>>>> On 5/7/18, 10:28 AM, Jiangli Zhou wrote: >>>>>> Hi Calvin, >>>>>> >>>>>> -classLoader.cpp >>>>>> >>>>>> The get_canonical_path() call for ?path? should be moved out of the ?for' loop. >>>>>> >>>>>> 1569 get_canonical_path(path, canonical_src_path, JVM_MAXPATHLEN)) { >>>>> Fixed. >>>>>> To make the code more readable, please rename ?canonical_path? to ?canonical_path_table_entry? (or ?canonical_path_table_entry_name?) >>>>> I chose the shorter one. >>>>>> and ?canonical_src_path? to ?canonical_class_src_path?. >>>>> Renamed. >>>>>> BTW, when we create the path entries, the paths are already converted to canonical forms. I think there is no need to call get_canonical_path() on ?ent->name()?. Can you please double check? >>>>> The call to get_canonical_path() is only for the non-directory case (please refer to ClassLoader::create_class_path_entry() for details). >>>>> I did try get_canonical_path() for the directory case but it didn't work during runtime; it failed in SystemDictionaryShared::is_shared_class_visible_for_classloader() in the following check: >>>>> >>>>> (strcmp(ent->name(), ClassLoader::skip_uri_protocol(mod_entry->location()->as_C_string())) == 0)) { >>>>> >>>>> The location() in the mod_entry isn't in canonical form. If we convert it to canonical form during runtime, there maybe performance impact. >>>>> I think the proper way of fixing it is storing the location() in canonical form when a module entry is being defined. I'm not sure if it is possible for the core lib to pass the location in canonical form to jvm when a module entry is being defined. >>>>> I can file a follow-up RFE to address that. >>>> Please file a RFE. >>> I've filed JDK-8202750 >>>> There is no need to check ?res? in very iteration since it doesn?t change after being initialized by get_canonical_path(). Also, I think we are not handling get_canonical_path() (both calls) failure properly. However, I question why get_canonical_path() would fail when given a valid path in this particular case. So I think we should either assert get_canonical_path() returns true, or report a fatal error for dumping. >>> I've made the change and added assert after get_canonical_path() at both calls. >>> >>> Updated webrev: >>> >>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.03/ >>> >>> thanks, >>> Calvin >>>>>> - os_windows.cpp >>>>>> >>>>>> Please fix indentation at line 4390& 4391. >>>>> That was done on purpose because the condition in line 4390 is '||' with line 4391. >>>>> I'm further indenting line 4391 to make it clearer. >>>>> >>>>> 4389 if (search_path[1] == ':'&& >>>>> 4390 (search_path[2] == '\0' || >>>>> 4391 (search_path[2] == '\\' && search_path[3] == '\0'))) { >>>> >>>> Looks better. >>>> >>>>> Updated webrev: >>>>> >>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.02/ >>>> >>>> Thanks, >>>> Jiangli >>>> >>>>> thanks, >>>>> Calvin >>>>> >>>>>> Thanks, >>>>>> Jiangli >>>>>> >>>>>> >>>>>>> On May 7, 2018, at 9:12 AM, Calvin Cheung wrote: >>>>>>> >>>>>>> Hi Jiangli, >>>>>>> >>>>>>> On 5/3/18, 5:33 PM, Jiangli Zhou wrote: >>>>>>>> Hi Calvin, >>>>>>>> >>>>>>>> >>>>>>>>> On May 3, 2018, at 4:18 PM, Calvin Cheung wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 5/3/18, 3:42 PM, Jiangli Zhou wrote: >>>>>>>>>> Hi Calvin, >>>>>>>>>> >>>>>>>>>> I think we don?t need to check the module path entries if only boot classes are archived. The ?end? is set to app_class_paths_start_index when there are only boot classes are archived. If there are app module classes loaded from jars/directories at dump time, has_platform_or_app_classes() should return true and ?end? is set to the end of the shared path table, which includes all module path entries. The following loop is not needed. The existing loop in the code covers all cases with different ?end? value based on the type of classes loaded in the archive. >>>>>>>>>> 366 // may need to check module path entries. >>>>>>>>>> 367 if ((end == ClassLoaderExt::app_class_paths_start_index())&& (ClassLoader::num_module_path_entries()> 0)) { >>>>>>>>>> 368 for (int i = ClassLoaderExt::app_module_paths_start_index(); i< _shared_path_table_size; i++) { >>>>>>>>>> 369 SharedClassPathEntry* e = shared_path(i); >>>>>>>>>> 370 has_nonempty_dir = check_nonempty_dir(e); >>>>>>>>>> >>>>>>>>>> 371 } >>>>>>>>>> 372 } >>>>>>>>>> >>>>>>>>> The has_platform_or_app_classes() will return false if ClassLoaderExt::record_result() hasn't been called while a class is being loaded during dumping. >>>>>>>>> This could happen before this proposed change if one specifies a --module-path with an non-empty directory containing a module. While the class will be loaded during dumping but its classpath_index will still be -1. Therefore, the has_platform_or_app_classes() will return false and the module path entries won't be checked without the above block of code. >>>>>>>> With the module directory path being added to the path table now, it should record the correct path table index instead of -1 for module classes loaded from directory at dump time. >>>>>>>> >>>>>>>> I applied your patch and stepped through ClassLoader::record_result() with your test case. The following path comparison fails due the the extra ?/? in ?path?. That why it fails to find the correct entry in the path table, which causes the class type cannot be correctly identified. >>>>>>>> >>>>>>>> if (strcmp(canonical_path, os::native_path((char*)path)) == 0) {os::dir_is_empty >>>>>>>> >>>>>>>> (gdb) p canonical_path >>>>>>>> $74 = 0x7ffff0aea0f0 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple" >>>>>>>> (gdb) p os::native_path((char*)path) >>>>>>>> $75 = 0x7ffff7fd4a65 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple/" >>>>>>>> >>>>>>>> The trailing ?/? should be excluded for the path comparison. JDK-8202519 is a related issue in the same area. It might be better to fix it together with this one. >>>>>>> Thanks for debugging the issue. >>>>>>> I've fixed it by calling get_canonical_path() on the "path" before doing strcmp(). It should also fix the bug in JDK-8202519 since a buffer is being allocated for get_canonical_path() to write into. The os::native_path() on linux does nothing and just returns the same string. >>>>>>> >>>>>>> I also discovered a problem in os::dir_is_empty() on windows. When a directory path is passed into the function, it may have the following form with forward slashes: C:/somedir >>>>>>> The fix is to call os::native_path() which essentially converts the forward slashes to back slashes so that subsequent windows api call returns the correct result. >>>>>>> >>>>>>>> Once the above issue is fixed, the class type should be recorded correctly for a module classes loaded from directory at dump time. Then ClassLoaderExt::record_result() can set ?has_app_classes? flag properly. And, no change should be needed in FileMapInfo::check_nonempty_dir_in_shared_path_table(). >>>>>>> I don't need to touch filemap.cpp in the following updated webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.01/ >>>>>>> >>>>>>> thanks, >>>>>>> Calvin >>>>>>>>>> We also need a unit test for non-empty directory in module path. Please add. >>>>>>>>> It is already in my webrev: >>>>>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/MainModuleOnly.java.sdiff.html >>>>>>>> Ok. I was expecting a separate unit test. I?m fine with an added test case in MainModuleOnly.java. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jiangli >>>>>>>> >>>>>>>>> thanks, >>>>>>>>> Calvin >>>>>>>>>> Thanks, >>>>>>>>>> Jiangli >>>>>>>>>> >>>>>>>>>>> On May 3, 2018, at 3:24 PM, Calvin Cheung wrote: >>>>>>>>>>> >>>>>>>>>>> bug:https://bugs.openjdk.java.net/browse/JDK-8202289 >>>>>>>>>>> webrev:http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> In ClassLoaderExt::process_module_table(), it adds all entries from the ModuleEntryTable with the "file:" protocol to the _module_path_entries and eventually to the _shared_path_table. >>>>>>>>>>> The FileMapInfo::check_nonempty_dir_in_shared_path_table() will check for non-empty directory in the module path. >>>>>>>>>>> >>>>>>>>>>> Ran all CDS and AppCDS tests locally on linux-x64. >>>>>>>>>>> Will run hs-tier{1,2,3} tests. >>>>>>>>>>> >>>>>>>>>>> thanks, >>>>>>>>>>> Calvin From ioi.lam at oracle.com Thu May 10 03:51:15 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 9 May 2018 20:51:15 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time In-Reply-To: <5AF39439.8060403@oracle.com> References: <5AEB8C04.4040501@oracle.com> <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> <5AEB98B7.9030209@oracle.com> <56AA6E39-131E-4D5B-BBEF-9FA8C4A60F5A@oracle.com> <5AF07AE9.9000700@oracle.com> <5AF09BE7.30208@oracle.com> <25B748EB-27EC-4FAB-956F-F864323531EC@oracle.com> <5AF1E157.9050905@oracle.com> <5AF39439.8060403@oracle.com> Message-ID: Looks good. Thanks! - Ioi On 5/9/18 5:37 PM, Calvin Cheung wrote: > > > On 5/8/18, 2:24 PM, Ioi Lam wrote: >> I think we do need to check the results of get_canonical_path. On >> Linux, the canonical path can be longer than the source path, which >> means ENAMETOOLONG may be returned. >> >> http://hg.openjdk.java.net/jdk/hs/file/tip/src/java.base/unix/native/libjava/canonicalize_md.c#l253 >> >> >> So instead of assert, it's best to check for failure and exit the >> dumping process. > I've made the above change. > Updated webrev: > > ??? http://cr.openjdk.java.net/~ccheung/8202289/webrev.04/ > > thanks, > Calvin >> >> The rest of the changes look OK to me. >> >> Thanks >> >> - Ioi >> >> >> >> On 5/8/18 10:41 AM, Calvin Cheung wrote: >>> >>> >>> On 5/7/18, 2:58 PM, Jiangli Zhou wrote: >>>> >>>>> On May 7, 2018, at 11:33 AM, Calvin Cheung >>>>> > wrote: >>>>> >>>>> >>>>> >>>>> On 5/7/18, 10:28 AM, Jiangli Zhou wrote: >>>>>> Hi Calvin, >>>>>> >>>>>> -classLoader.cpp >>>>>> >>>>>> The get_canonical_path() call for ?path? should be moved out of >>>>>> the ?for' loop. >>>>>> >>>>>> 1569?????????? get_canonical_path(path, canonical_src_path, >>>>>> JVM_MAXPATHLEN)) { >>>>> Fixed. >>>>>> To make the code more readable, please rename ?canonical_path? to >>>>>> ?canonical_path_table_entry? (or ?canonical_path_table_entry_name?) >>>>> I chose the shorter one. >>>>>> ? and ?canonical_src_path? to ?canonical_class_src_path?. >>>>> Renamed. >>>>>> BTW, when we create the path entries, the paths are already >>>>>> converted to canonical forms. I think there is no need to call >>>>>> get_canonical_path() on ?ent->name()?. Can you please double check? >>>>> The call to get_canonical_path() is only for the non-directory >>>>> case (please refer to ClassLoader::create_class_path_entry() for >>>>> details). >>>>> I did try get_canonical_path() for the directory case but it >>>>> didn't work during runtime; it failed in >>>>> SystemDictionaryShared::is_shared_class_visible_for_classloader() >>>>> in the following check: >>>>> >>>>> (strcmp(ent->name(), >>>>> ClassLoader::skip_uri_protocol(mod_entry->location()->as_C_string())) >>>>> == 0)) { >>>>> >>>>> The location() in the mod_entry isn't in canonical form. If we >>>>> convert it to canonical form during runtime, there maybe >>>>> performance impact. >>>>> I think the proper way of fixing it is storing the location() in >>>>> canonical form when a module entry is being defined. I'm not sure >>>>> if it is possible for the core lib to pass the location in >>>>> canonical form to jvm when a module entry is being defined. >>>>> I can file a follow-up RFE to address that. >>>> >>>> Please file a RFE. >>> I've filed JDK-8202750 >>> >>>> >>>> There is no need to check ?res? in very iteration since it doesn?t >>>> change after being initialized by get_canonical_path(). Also, I >>>> think we are not handling get_canonical_path() (both calls) failure >>>> properly. However, I question why get_canonical_path() would fail >>>> when given a valid path in this particular case. So I think we >>>> should either assert get_canonical_path() returns true, or report a >>>> fatal error for dumping. >>> I've made the change and added assert after get_canonical_path() at >>> both calls. >>> >>> Updated webrev: >>> >>> ??? http://cr.openjdk.java.net/~ccheung/8202289/webrev.03/ >>> >>> thanks, >>> Calvin >>>>>> - os_windows.cpp >>>>>> >>>>>> Please fix indentation at line 4390&? 4391. >>>>> That was done on purpose because the condition in line 4390 is >>>>> '||' with line 4391. >>>>> I'm further indenting line 4391 to make it clearer. >>>>> >>>>> 4389?? if (search_path[1] == ':'&& >>>>> 4390??????? (search_path[2] == '\0' || >>>>> 4391????????? (search_path[2] == '\\' &&? >>>>> search_path[3] == '\0'))) { >>>> >>>> >>>> Looks better. >>>> >>>>> Updated webrev: >>>>> >>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.02/ >>>> >>>> >>>> Thanks, >>>> Jiangli >>>> >>>>> >>>>> thanks, >>>>> Calvin >>>>> >>>>>> Thanks, >>>>>> Jiangli >>>>>> >>>>>> >>>>>>> On May 7, 2018, at 9:12 AM, Calvin >>>>>>> Cheung? wrote: >>>>>>> >>>>>>> Hi Jiangli, >>>>>>> >>>>>>> On 5/3/18, 5:33 PM, Jiangli Zhou wrote: >>>>>>>> Hi Calvin, >>>>>>>> >>>>>>>> >>>>>>>>> On May 3, 2018, at 4:18 PM, Calvin >>>>>>>>> Cheung wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 5/3/18, 3:42 PM, Jiangli Zhou wrote: >>>>>>>>>> Hi Calvin, >>>>>>>>>> >>>>>>>>>> I think we don?t need to check the module path entries if >>>>>>>>>> only boot classes are archived. The ?end? is set to >>>>>>>>>> app_class_paths_start_index when there are only boot classes >>>>>>>>>> are archived. If there are app module classes loaded from >>>>>>>>>> jars/directories at dump time, has_platform_or_app_classes() >>>>>>>>>> should return true and ?end? is set to the end of the shared >>>>>>>>>> path table, which includes all module path entries. The >>>>>>>>>> following loop is not needed. The existing loop in the code >>>>>>>>>> covers all cases with different ?end? value based on the type >>>>>>>>>> of classes loaded in the archive. >>>>>>>>>> ? 366?? // may need to check module path entries. >>>>>>>>>> ? 367?? if ((end == >>>>>>>>>> ClassLoaderExt::app_class_paths_start_index())&& >>>>>>>>>> (ClassLoader::num_module_path_entries()> 0)) { >>>>>>>>>> ? 368???? for (int i = >>>>>>>>>> ClassLoaderExt::app_module_paths_start_index(); i>>>>>>>>> _shared_path_table_size; i++) { >>>>>>>>>> ? 369?????? SharedClassPathEntry* e = shared_path(i); >>>>>>>>>> ? 370?????? has_nonempty_dir = check_nonempty_dir(e); >>>>>>>>>> >>>>>>>>>> ? 371???? } >>>>>>>>>> ? 372?? } >>>>>>>>>> >>>>>>>>> The has_platform_or_app_classes() will return false if >>>>>>>>> ClassLoaderExt::record_result() hasn't been called while a >>>>>>>>> class is being loaded during dumping. >>>>>>>>> This could happen before this proposed change if one specifies >>>>>>>>> a --module-path with an non-empty directory containing a >>>>>>>>> module. While the class will be loaded during dumping but its >>>>>>>>> classpath_index will still be -1. Therefore, the >>>>>>>>> has_platform_or_app_classes() will return false and the module >>>>>>>>> path entries won't be checked without the above block of code. >>>>>>>> With the module directory path being added to the path table >>>>>>>> now, it should record the correct path table index instead of >>>>>>>> -1 for module classes loaded from directory at dump time. >>>>>>>> >>>>>>>> I applied your patch and stepped through >>>>>>>> ClassLoader::record_result() with your test case. The following >>>>>>>> path comparison fails due the the extra ?/? in ?path?. That why >>>>>>>> it fails to find the correct entry in the path table, which >>>>>>>> causes the class type cannot be correctly identified. >>>>>>>> >>>>>>>> ???????? if (strcmp(canonical_path, >>>>>>>> os::native_path((char*)path)) == 0) {os::dir_is_empty >>>>>>>> >>>>>>>> (gdb) p canonical_path >>>>>>>> $74 = 0x7ffff0aea0f0 >>>>>>>> ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple" >>>>>>>> (gdb) p os::native_path((char*)path) >>>>>>>> $75 = 0x7ffff7fd4a65 >>>>>>>> ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple/" >>>>>>>> >>>>>>>> The trailing ?/? should be excluded for the path comparison. >>>>>>>> JDK-8202519 is a related issue in the same area. It might be >>>>>>>> better to fix it together with this one. >>>>>>> Thanks for debugging the issue. >>>>>>> I've fixed it by calling get_canonical_path() on the "path" >>>>>>> before doing strcmp(). It should also fix the bug in JDK-8202519 >>>>>>> since a buffer is being allocated for get_canonical_path() to >>>>>>> write into. The os::native_path() on linux does nothing and just >>>>>>> returns the same string. >>>>>>> >>>>>>> I also discovered a problem in os::dir_is_empty() on windows. >>>>>>> When a directory path is passed into the function, it may have >>>>>>> the following form with forward slashes: C:/somedir >>>>>>> The fix is to call os::native_path() which essentially converts >>>>>>> the forward slashes to back slashes so that subsequent windows >>>>>>> api call returns the correct result. >>>>>>> >>>>>>>> Once the above issue is fixed, the class type should be >>>>>>>> recorded correctly for a module classes loaded from directory >>>>>>>> at dump time. Then ClassLoaderExt::record_result() can set >>>>>>>> ?has_app_classes? flag properly. And, no change should be >>>>>>>> needed in FileMapInfo::check_nonempty_dir_in_shared_path_table(). >>>>>>> I don't need to touch filemap.cpp in the following updated webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.01/ >>>>>>> >>>>>>> thanks, >>>>>>> Calvin >>>>>>>>>> We also need a unit test for non-empty directory in module >>>>>>>>>> path. Please add. >>>>>>>>> It is already in my webrev: >>>>>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/MainModuleOnly.java.sdiff.html >>>>>>>>> >>>>>>>> Ok. I was expecting a separate unit test. I?m fine with an >>>>>>>> added test case in MainModuleOnly.java. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jiangli >>>>>>>> >>>>>>>>> thanks, >>>>>>>>> Calvin >>>>>>>>>> Thanks, >>>>>>>>>> Jiangli >>>>>>>>>> >>>>>>>>>>> On May 3, 2018, at 3:24 PM, Calvin >>>>>>>>>>> Cheung wrote: >>>>>>>>>>> >>>>>>>>>>> bug:https://bugs.openjdk.java.net/browse/JDK-8202289 >>>>>>>>>>> webrev:http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> In ClassLoaderExt::process_module_table(), it adds all >>>>>>>>>>> entries from the ModuleEntryTable with the "file:" protocol >>>>>>>>>>> to the _module_path_entries and eventually to the >>>>>>>>>>> _shared_path_table. >>>>>>>>>>> The FileMapInfo::check_nonempty_dir_in_shared_path_table() >>>>>>>>>>> will check for non-empty directory in the module path. >>>>>>>>>>> >>>>>>>>>>> Ran all CDS and AppCDS tests locally on linux-x64. >>>>>>>>>>> Will run hs-tier{1,2,3} tests. >>>>>>>>>>> >>>>>>>>>>> thanks, >>>>>>>>>>> Calvin >>>> >> From yasuenag at gmail.com Thu May 10 04:13:38 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 10 May 2018 13:13:38 +0900 Subject: RFR: 8202889: Remove trailing LF from perf log Message-ID: Hi all, Could you review this change? JBS: https://bugs.openjdk.java.net/browse/JDK-8202889 webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8202889/webrev.00/ JDK-8168122 introduced new UL tags for perfMemory. However they trails unnecessary LF in each log entries. Examples are here: http://cr.openjdk.java.net/~ysuenaga/openjdk-fc28/trace-all.log This change removes unnecessary LFs. Thanks, Yasumasa From thomas.stuefe at gmail.com Thu May 10 05:19:28 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 10 May 2018 07:19:28 +0200 Subject: RFR: 8202889: Remove trailing LF from perf log In-Reply-To: References: Message-ID: Hi Yasumasa, seems fine and trivial. Please update copyright years (in case of aix specific files, please both SAP and Oracle). I do not need a new webrev. Thanks, Thomas On Thu, May 10, 2018 at 6:13 AM, Yasumasa Suenaga wrote: > Hi all, > > Could you review this change? > > JBS: https://bugs.openjdk.java.net/browse/JDK-8202889 > webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8202889/webrev.00/ > > > JDK-8168122 introduced new UL tags for perfMemory. However they trails > unnecessary LF in each log entries. > > Examples are here: > http://cr.openjdk.java.net/~ysuenaga/openjdk-fc28/trace-all.log > > This change removes unnecessary LFs. > > > Thanks, > > Yasumasa From yasuenag at gmail.com Thu May 10 05:42:58 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 10 May 2018 14:42:58 +0900 Subject: RFR: 8202889: Remove trailing LF from perf log In-Reply-To: References: Message-ID: Hi Thomas, Thank you for your review! I will update copyright year in all files. I'm waiting for second reviewer. Thanks, Yasumasa 2018-05-10 14:19 GMT+09:00 Thomas St?fe : > Hi Yasumasa, > > seems fine and trivial. Please update copyright years (in case of aix > specific files, please both SAP and Oracle). I do not need a new > webrev. > > Thanks, Thomas > > On Thu, May 10, 2018 at 6:13 AM, Yasumasa Suenaga wrote: >> Hi all, >> >> Could you review this change? >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8202889 >> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8202889/webrev.00/ >> >> >> JDK-8168122 introduced new UL tags for perfMemory. However they trails >> unnecessary LF in each log entries. >> >> Examples are here: >> http://cr.openjdk.java.net/~ysuenaga/openjdk-fc28/trace-all.log >> >> This change removes unnecessary LFs. >> >> >> Thanks, >> >> Yasumasa From daniel.daugherty at oracle.com Thu May 10 13:52:36 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 10 May 2018 09:52:36 -0400 Subject: RFR(XS): 8132287: obsolete the "InlineNotify" flag option Message-ID: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> Greetings, This review has been sent to two aliases so please reply to both. I'm obsoleting the '-XX:+InlineNotify' option and moving its functionality to the experimental '-XX:SyncKnobs' option. This is a leftover cleanup from the "Contended Locking fast notify bucket". Since this is a diagnostic option, a CSR is not required: https://wiki.openjdk.java.net/display/csr/CSR+FAQs > Interfaces that are experimental or for diagnostic purposes do not need > to go through CSR process, but the CSR process may be employed if feedback > from the CSR reviewers is desired. Webrev URL: http://cr.openjdk.java.net/~dcubed/8132287-webrev/0_for_jdk_jdk/ See the bug for the results of test invocations of the '-XX:+InlineNotify' option and the new '-XX:SyncKnobs' suboption. Thanks, in advance, for any feedback. Dan From vladimir.kozlov at oracle.com Thu May 10 16:43:40 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 10 May 2018 09:43:40 -0700 Subject: RFR(XS): 8132287: obsolete the "InlineNotify" flag option In-Reply-To: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> References: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> Message-ID: <1c5581ca-cc33-2587-9751-b66c23e30769@oracle.com> Looks good. Only concern is that library_call.cpp does not include objectMonitor.hpp. You need to verify build without pre-compiled header file. Thanks, Vladimir On 5/10/18 6:52 AM, Daniel D. Daugherty wrote: > Greetings, > > This review has been sent to two aliases so please reply to both. > > I'm obsoleting the '-XX:+InlineNotify' option and moving its functionality > to the experimental '-XX:SyncKnobs' option. This is a leftover cleanup from > the "Contended Locking fast notify bucket". > > Since this is a diagnostic option, a CSR is not required: > > https://wiki.openjdk.java.net/display/csr/CSR+FAQs > > > Interfaces that are experimental or for diagnostic purposes do not need > > to go through CSR process, but the CSR process may be employed if > feedback > > from the CSR reviewers is desired. > > Webrev URL: > http://cr.openjdk.java.net/~dcubed/8132287-webrev/0_for_jdk_jdk/ > > See the bug for the results of test invocations of the '-XX:+InlineNotify' > option and the new '-XX:SyncKnobs' suboption. > > Thanks, in advance, for any feedback. > > Dan From jiangli.zhou at oracle.com Thu May 10 16:58:16 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Thu, 10 May 2018 09:58:16 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time In-Reply-To: <5AF3952B.9070801@oracle.com> References: <5AEB8C04.4040501@oracle.com> <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> <5AEB98B7.9030209@oracle.com> <56AA6E39-131E-4D5B-BBEF-9FA8C4A60F5A@oracle.com> <5AF07AE9.9000700@oracle.com> <5AF09BE7.30208@oracle.com> <25B748EB-27EC-4FAB-956F-F864323531EC@oracle.com> <5AF1E157.9050905@oracle.com> <18CD7B50-2063-47A3-867B-EF6150E2260E@oracle.com> <5AF3952B.9070801@oracle.com> Message-ID: <251E8B68-F9BE-4B73-8080-F9D954F03CAA@oracle.com> Hi Calvin, Thank you for looking into the test case. That?s helpful. As I also stated in the bug report, although errors caused by extra-long path would be detected much earlier and should not reach ClassLoader::record_result(), it still seem to be a safe choice to report fatal error if get_canonical_path() fails in our case. I have a style comment for your latest webrev for the following two tty->print_cr calls. The indentation at line 1563 and 1570 are off. There is no need to break each tty->print_cr into two lines since they are short. Could you please change each tty->print_cr into one line? No need for a new webrev with the line fix. 1562 tty->print_cr("Bad pathname %s. CDS dump aborted.", 1563 path); 1569 tty->print_cr("Bad pathname %s. CDS dump aborted.", 1570 ent->name()); Thanks, Jiangli > On May 9, 2018, at 5:41 PM, Calvin Cheung wrote: > > > > On 5/8/18, 2:37 PM, Jiangli Zhou wrote: >>> On May 8, 2018, at 2:24 PM, Ioi Lam wrote: >>> >>> I think we do need to check the results of get_canonical_path. On Linux, the canonical path can be longer than the source path, which means ENAMETOOLONG may be returned. >>> >>> http://hg.openjdk.java.net/jdk/hs/file/tip/src/java.base/unix/native/libjava/canonicalize_md.c#l253 >> We need a test case for long path if this could happen by the time we are searching the CDS path table. I was thinking to file a new test bug earlier, but it seems this should be done together with Calvin?s current bug fix. > I've updated the bug report on why it is hard to write a test case with a long path to test the failure of get_canonical_path() during CDS dumping. > > thanks, > Calvin >> >> Thanks, >> Jiangli >> >>> So instead of assert, it's best to check for failure and exit the dumping process. >>> >>> The rest of the changes look OK to me. >>> >>> Thanks >>> >>> - Ioi >>> >>> >>> >>> On 5/8/18 10:41 AM, Calvin Cheung wrote: >>>> >>>> On 5/7/18, 2:58 PM, Jiangli Zhou wrote: >>>>>> On May 7, 2018, at 11:33 AM, Calvin Cheung> wrote: >>>>>> >>>>>> >>>>>> >>>>>> On 5/7/18, 10:28 AM, Jiangli Zhou wrote: >>>>>>> Hi Calvin, >>>>>>> >>>>>>> -classLoader.cpp >>>>>>> >>>>>>> The get_canonical_path() call for ?path? should be moved out of the ?for' loop. >>>>>>> >>>>>>> 1569 get_canonical_path(path, canonical_src_path, JVM_MAXPATHLEN)) { >>>>>> Fixed. >>>>>>> To make the code more readable, please rename ?canonical_path? to ?canonical_path_table_entry? (or ?canonical_path_table_entry_name?) >>>>>> I chose the shorter one. >>>>>>> and ?canonical_src_path? to ?canonical_class_src_path?. >>>>>> Renamed. >>>>>>> BTW, when we create the path entries, the paths are already converted to canonical forms. I think there is no need to call get_canonical_path() on ?ent->name()?. Can you please double check? >>>>>> The call to get_canonical_path() is only for the non-directory case (please refer to ClassLoader::create_class_path_entry() for details). >>>>>> I did try get_canonical_path() for the directory case but it didn't work during runtime; it failed in SystemDictionaryShared::is_shared_class_visible_for_classloader() in the following check: >>>>>> >>>>>> (strcmp(ent->name(), ClassLoader::skip_uri_protocol(mod_entry->location()->as_C_string())) == 0)) { >>>>>> >>>>>> The location() in the mod_entry isn't in canonical form. If we convert it to canonical form during runtime, there maybe performance impact. >>>>>> I think the proper way of fixing it is storing the location() in canonical form when a module entry is being defined. I'm not sure if it is possible for the core lib to pass the location in canonical form to jvm when a module entry is being defined. >>>>>> I can file a follow-up RFE to address that. >>>>> Please file a RFE. >>>> I've filed JDK-8202750 >>>>> There is no need to check ?res? in very iteration since it doesn?t change after being initialized by get_canonical_path(). Also, I think we are not handling get_canonical_path() (both calls) failure properly. However, I question why get_canonical_path() would fail when given a valid path in this particular case. So I think we should either assert get_canonical_path() returns true, or report a fatal error for dumping. >>>> I've made the change and added assert after get_canonical_path() at both calls. >>>> >>>> Updated webrev: >>>> >>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.03/ >>>> >>>> thanks, >>>> Calvin >>>>>>> - os_windows.cpp >>>>>>> >>>>>>> Please fix indentation at line 4390& 4391. >>>>>> That was done on purpose because the condition in line 4390 is '||' with line 4391. >>>>>> I'm further indenting line 4391 to make it clearer. >>>>>> >>>>>> 4389 if (search_path[1] == ':'&& >>>>>> 4390 (search_path[2] == '\0' || >>>>>> 4391 (search_path[2] == '\\' && search_path[3] == '\0'))) { >>>>> >>>>> Looks better. >>>>> >>>>>> Updated webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.02/ >>>>> >>>>> Thanks, >>>>> Jiangli >>>>> >>>>>> thanks, >>>>>> Calvin >>>>>> >>>>>>> Thanks, >>>>>>> Jiangli >>>>>>> >>>>>>> >>>>>>>> On May 7, 2018, at 9:12 AM, Calvin Cheung wrote: >>>>>>>> >>>>>>>> Hi Jiangli, >>>>>>>> >>>>>>>> On 5/3/18, 5:33 PM, Jiangli Zhou wrote: >>>>>>>>> Hi Calvin, >>>>>>>>> >>>>>>>>> >>>>>>>>>> On May 3, 2018, at 4:18 PM, Calvin Cheung wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 5/3/18, 3:42 PM, Jiangli Zhou wrote: >>>>>>>>>>> Hi Calvin, >>>>>>>>>>> >>>>>>>>>>> I think we don?t need to check the module path entries if only boot classes are archived. The ?end? is set to app_class_paths_start_index when there are only boot classes are archived. If there are app module classes loaded from jars/directories at dump time, has_platform_or_app_classes() should return true and ?end? is set to the end of the shared path table, which includes all module path entries. The following loop is not needed. The existing loop in the code covers all cases with different ?end? value based on the type of classes loaded in the archive. >>>>>>>>>>> 366 // may need to check module path entries. >>>>>>>>>>> 367 if ((end == ClassLoaderExt::app_class_paths_start_index())&& (ClassLoader::num_module_path_entries()> 0)) { >>>>>>>>>>> 368 for (int i = ClassLoaderExt::app_module_paths_start_index(); i< _shared_path_table_size; i++) { >>>>>>>>>>> 369 SharedClassPathEntry* e = shared_path(i); >>>>>>>>>>> 370 has_nonempty_dir = check_nonempty_dir(e); >>>>>>>>>>> >>>>>>>>>>> 371 } >>>>>>>>>>> 372 } >>>>>>>>>>> >>>>>>>>>> The has_platform_or_app_classes() will return false if ClassLoaderExt::record_result() hasn't been called while a class is being loaded during dumping. >>>>>>>>>> This could happen before this proposed change if one specifies a --module-path with an non-empty directory containing a module. While the class will be loaded during dumping but its classpath_index will still be -1. Therefore, the has_platform_or_app_classes() will return false and the module path entries won't be checked without the above block of code. >>>>>>>>> With the module directory path being added to the path table now, it should record the correct path table index instead of -1 for module classes loaded from directory at dump time. >>>>>>>>> >>>>>>>>> I applied your patch and stepped through ClassLoader::record_result() with your test case. The following path comparison fails due the the extra ?/? in ?path?. That why it fails to find the correct entry in the path table, which causes the class type cannot be correctly identified. >>>>>>>>> >>>>>>>>> if (strcmp(canonical_path, os::native_path((char*)path)) == 0) {os::dir_is_empty >>>>>>>>> >>>>>>>>> (gdb) p canonical_path >>>>>>>>> $74 = 0x7ffff0aea0f0 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple" >>>>>>>>> (gdb) p os::native_path((char*)path) >>>>>>>>> $75 = 0x7ffff7fd4a65 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple/" >>>>>>>>> >>>>>>>>> The trailing ?/? should be excluded for the path comparison. JDK-8202519 is a related issue in the same area. It might be better to fix it together with this one. >>>>>>>> Thanks for debugging the issue. >>>>>>>> I've fixed it by calling get_canonical_path() on the "path" before doing strcmp(). It should also fix the bug in JDK-8202519 since a buffer is being allocated for get_canonical_path() to write into. The os::native_path() on linux does nothing and just returns the same string. >>>>>>>> >>>>>>>> I also discovered a problem in os::dir_is_empty() on windows. When a directory path is passed into the function, it may have the following form with forward slashes: C:/somedir >>>>>>>> The fix is to call os::native_path() which essentially converts the forward slashes to back slashes so that subsequent windows api call returns the correct result. >>>>>>>> >>>>>>>>> Once the above issue is fixed, the class type should be recorded correctly for a module classes loaded from directory at dump time. Then ClassLoaderExt::record_result() can set ?has_app_classes? flag properly. And, no change should be needed in FileMapInfo::check_nonempty_dir_in_shared_path_table(). >>>>>>>> I don't need to touch filemap.cpp in the following updated webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.01/ >>>>>>>> >>>>>>>> thanks, >>>>>>>> Calvin >>>>>>>>>>> We also need a unit test for non-empty directory in module path. Please add. >>>>>>>>>> It is already in my webrev: >>>>>>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/MainModuleOnly.java.sdiff.html >>>>>>>>> Ok. I was expecting a separate unit test. I?m fine with an added test case in MainModuleOnly.java. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Jiangli >>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> Calvin >>>>>>>>>>> Thanks, >>>>>>>>>>> Jiangli >>>>>>>>>>> >>>>>>>>>>>> On May 3, 2018, at 3:24 PM, Calvin Cheung wrote: >>>>>>>>>>>> >>>>>>>>>>>> bug:https://bugs.openjdk.java.net/browse/JDK-8202289 >>>>>>>>>>>> webrev:http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> In ClassLoaderExt::process_module_table(), it adds all entries from the ModuleEntryTable with the "file:" protocol to the _module_path_entries and eventually to the _shared_path_table. >>>>>>>>>>>> The FileMapInfo::check_nonempty_dir_in_shared_path_table() will check for non-empty directory in the module path. >>>>>>>>>>>> >>>>>>>>>>>> Ran all CDS and AppCDS tests locally on linux-x64. >>>>>>>>>>>> Will run hs-tier{1,2,3} tests. >>>>>>>>>>>> >>>>>>>>>>>> thanks, >>>>>>>>>>>> Calvin From yumin.qi at gmail.com Thu May 10 17:09:51 2018 From: yumin.qi at gmail.com (yumin qi) Date: Thu, 10 May 2018 10:09:51 -0700 Subject: Improving AppCDS for Custom Loaders In-Reply-To: <9abbe65c-93e9-3931-45ce-1688541c29c4@oracle.com> References: <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> <732ce957-b07c-bff2-277b-ca335c348753@oracle.com> <660A1644-3A08-455D-931D-D95069C39853@oracle.com> <9abbe65c-93e9-3931-45ce-1688541c29c4@oracle.com> Message-ID: HI, I joined the discussion too late --- so nice to see your valuable discussion and an action to further improve CDS (AppCDS is removed, CDS is for all now) in progress. We plan to run every application in docker --- dockerize every application and multiple deployment of different applications in single host. This is a trend for industrial applications save machine resources and better memory management. A base archive can save footprint for the image, CDS for base classes should be default. Agree not including objects in java heap at now --- thanks Jiangli for answering my question in other thread. I think for #1, it does not conflict with the two layer archive solution. We can skip the classes from base CDS archive, only dump the non-base loaded classes into second archive, this gives one more option for user to choose. Also it will be good after dump archive to let the application continue to run. Can we do a Full GC before dump at exit? it may unload not referenced classes, or it is not necessary since CDS wants to resolve the startup performance for all loaded classes? * If performance measurement does show regression, John Rose has a very interesting idea that could be used to reduce overhead. Really want to know what is the interesting idea. Thanks Yumin On Mon, May 7, 2018 at 1:39 PM, Ioi Lam wrote: > > > On 5/5/18 1:22 PM, Volker Simonis wrote: > >> Hi Jiangli, >> >> thanks a lot explaining your plans more precisely. I think the approach >> makes much more sense now. >> >> While I?m all for it in general, I still have some questions and remarks >> :) >> >> 1. Will it be possible to use the two archives independantly ? I.e. will >> it be possible to choose just one of the two archives as well as a >> combination of both? >> >> 2. We should make sure to not introduce any performance regressions if we >> have two archives (and potentially three SymbolTables and three >> ?Metaspaces?) >> >> 3. In general I think we?re moving away from scenarios where we have one >> single, central Java installation on a system which is used by the >> ?tens/hundreds different applications? you mentioned in your mail towards a >> world where every of these ?tens/hundreds different applications? comes >> with its own, bundled JDK/JRE version (and as far as I understand Oracle >> is propagating this new distribution model after the deprecation of >> Applets/WebStart). In such a world it would be not easy to have a static, >> common system layer archive. We could try to keep the format of the >> archived classes as compatible between Java versions as possible, although >> I understand that this will be quite complicated. Do you already have any >> thoughts on this specific problem? >> >> > I think the 2 level archive will work very well with containerized app > deployment. For example, with docker, you can create a "base docker image" > with the JDK and the "base" archive that contains the frequently used JDK > and library classes. Then, you create an "app docker image" for each > application, which contains the modules of the app, as well as the CDS > archive of the classes used by this app. > > So imagine you have created 2 different app images from the same base > image. When you deploy these 2 app images on the same hardware, they can > share the files for the JDK and the base CDS archive between them. This > could be quite significant (tens or 100s of MB). > > > 4. Finally we should design the new model in a way which allows it to >> easily integrate AOT compiled code into the same one or two archives in the >> future. Because with AOT we face the exactly same problems and having to >> maintain four different archives will be not very user friendly. OpenJ9 >> already uses a single archive for CDT/AOT/ProfilingData. I don?t say we >> have to unify the AOT and CDT archives in HotSpot now, I just want to >> suggest that if we?re redesigning the CDS archive anyway to perhaps think >> twice before we come to a new model such that we at least don?t exclude the >> possibility of a unification with the AOT archive in the future. >> >> > There are 2 reasons we have not unified the AOT and CDS storage: > > 1. If we store AOT code inside the CDS archive, we would need to implement > our own native code loader (similar to ld.so). This could be quite > involving since we have many supported platforms. > > 2. If we store CDS data into AOT (e.g., add .text/.data sections to the > .so files generated by AOT), we have very large overhead -- CDS has a lot > of direct data pointers. E.g., InstanceKlass::_super, etc. Last time I > checked, ELF uses 2x8 bytes of relocation data per pointer. All of these > pointers are relocated when the .so is loaded, which means all the "read > only" CDS data is effectively copied-on-write. > > So, I think we need a good justification to see why the unification is > needed, before we try to resolve the technical issues. > > Thanks > - Ioi > >> Thanks, >> Volker >> >> >> >> >> Jiangli Zhou > >> schrieb am Fr. 4. Mai 2018 um 19:42: >> >> Hi Volker, >> >> Thank you so much for the feedbacks! >> >> You comments made me realize that some clarifications are need for >> the top-layer dynamic archiving. The top-layer would include all >> application classes, middleware classes, system library classes >> that are not in the bottom layer. Apologizing for not stating that >> clearly in the proposal. With the top layer including all >> necessary classes (excluding the classes in bottom layer) for a >> specific application, multiple instances running the same >> application can achieve maximum memory sharing. Using your >> example, the extra AWT/Swing classes are archived along with the >> application classes in the top layer. >> >> With the dynamic archiving ability, it?s natural to think there is >> no need for the two-layer scheme. In fact, I originally had the >> thoughts. However, after exploring different possible use cases, I >> agree two-layer archiving do bring benefits that are lacking in >> the single archive design. One example of such use case is >> tens/hundreds different applications running on the same machine. >> Without a common system layer archive, it is difficult to achieve >> memory sharing for common system classes, String objects, etc >> between different applications. >> >> Another reason why we should go with a two-layer & static/dynamic >> archiving is some runtime information may be sensitive and should >> not be archived. Strings can be a good example in this case. >> Currently, we archive interned String objects without running >> application. The Strings include resolved CONSTANT_strings, class >> names, method names, etc, which are all static information from >> class files. During execution of an application, interned Strings >> might contain sensitive user information, which is not safe to be >> included in the archive. It?s difficult to distinguish between the >> static information and runtime information if we go with the >> single layer dynamic-only archiving. >> >> We had many discussions internally over long period of time on how >> to improve usability with different approaches. In the end, >> dynamic-only or static-only two-layer archiving all have their own >> disadvantages and fail to meet requirements in some use cases. The >> hybrid archiving combines benefits/advantages of different >> approaches and seem to be flexible enough to fit most usages. >> >> Further comments and feedbacks are much appreciated. >> >> Best regards, >> >> Jiangli >> >> > On May 4, 2018, at 3:01 AM, Volker Simonis >> > wrote: >> > >> > Hi Jiangli, >> > >> > thanks for sharing the hybrid archiving proposal. I think it is very >> > interesting and useful! >> > >> > One issue I see is that the "system library classes" which should go >> > into the "static" archive are inherently application specific >> (i.e. if >> > my application uses Swing, it will use most of the AWT/Swing >> classes). >> > So how will you cope with this problem. Either you put ALL the >> system >> > classes into the static archive which will be a waste for most >> > applications. Or you just put a small commonly used subset of >> classes >> > into the static archive which may miss many classes for specific >> > applications. >> > >> > If we would add the possibility to create a dynamic archive at >> runtime >> > / program end (which I think would be great from a usability >> > perspective) I don't see a big need for two different archives. Two >> > archives will further complicate (and slow down) things like Symbol >> > lookup (we already have two SymbolTable now and we'd probably need a >> > third one if we would have two archives). >> > >> > I don't think that running a few different Java applications on one >> > host is the most important use case for CDS. In such a scenario the >> > current, build time generated archive is probably the best we can do >> > anyway. Instead, I think the most important use case is if we have >> > many instances of the same Java application running on the same >> host. >> > And this is becoming more common with the raise of containerization. >> > For the latter use case, a dynamically generated, application >> specific >> > archive would be the optimal solution. >> > >> > Thank you and best regards, >> > Volker >> > >> > >> > >> > On Fri, May 4, 2018 at 3:42 AM, Jiangli Zhou >> > wrote: >> >> Hi Volker, >> >> >> >> Here are some details about the hybrid archiving. The goal is >> to harvest the benefit of archiving by default and improve its >> usability. The hybrid approach combines a two-layer archiving >> (proposed by Ioi internally) and static & dynamic archiving >> techniques: >> >> >> >> - Statically archive system library classes from a provided >> classlist using the existing method. The archiving includes class >> metadata, interned string objects, constant pool >> resolved_references arrays, class mirror objects, etc. Static >> archiving can be done at the JDK image build time and shipped >> together with JDK binary. Following need to be addressed: >> >> *Relaxing the runtime CDS/AppCDS boot path check, so the >> packaged archive can be used after the JDK binary is installed on >> the target device. JDK-8199807 was created to address this issue >> and is targeted for JDK 11. >> >> *Add the static archiving generation in JDK build steps >> and package the generated archive with JDK image. The archive can >> only be generated for the same target (both OS can CPU >> architecture) as the build platform. I will create a RFE. >> >> >> >> - Dynamic archiving can done for application classes at the >> first execution of a specific application >> >> * The archive is created on top of the default system >> archive shipped with the JDK image. A separate top-layer archive >> file is generated for each different application. >> >> * Archiving is done at the end of the application >> execution before VM exists by relocating the class metadata to the >> archive spaces. Cleanup also needs to be done for copied class >> meta data to remove any runtime information. Most of the required >> functionality already exists today. For example, class metadata >> relocation was implemented by Ioi in JDK 10. >> >> * Only archive class metadata for application in the top >> layer initially. Archiving java heap objects in the top-layer >> requires more investigations. >> >> >> >> Benefits of the hybrid archiving: >> >> * The system archive can be shared by different applications >> and provides memory saving. >> >> * Archive for application is created and used transparently. No >> more profiling step and class list are required! >> >> * Separating the system archiving from application archiving >> reduces the cost of archiving at application execution time. The >> overhead added to the first execution time is reduced. >> >> >> >> Thanks, >> >> >> >> Jiangli >> >> >> >> >> >>> On May 3, 2018, at 10:34 AM, Jiangli Zhou >> wrote: >> >>> >> >>> >> >>>> On May 3, 2018, at 2:14 AM, Volker Simonis >> > wrote: >> >>>> >> >>>> On Thu, May 3, 2018 at 11:01 AM, David Holmes >> > wrote: >> >>>>> On 3/05/2018 5:16 PM, Volker Simonis wrote: >> >>>>>> >> >>>>>> On Thu, May 3, 2018 at 8:55 AM, David Holmes >> > >> >> >>>>>> wrote: >> >>>>>>> >> >>>>>>> Just lurking here but ... >> >>>>>>> >> >>>>>>>> But is this really y relevant use case? Why would I like >> to create ONE >> >>>>>>>> archive for several apps? This would actually increase >> the footprint >> >>>>>>>> of a single instance which uses this archive. If I have >> several apps I >> >>>>>>>> would expect that users create a specific archive for >> each app to get >> >>>>>>>> the best out of CDS. >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> One app instance may get increased footprint but you >> presumably use CDS >> >>>>>>> because you have multiple apps running (whether the same >> or not). These >> >>>>>>> apps >> >>>>>>> all share the core JDK classes from the archive so the >> overall footprint >> >>>>>>> per >> >>>>>>> instance is less. >> >>>>>>> >> >>>>>> >> >>>>>> If we just want to share the core JDK classes that's easy. >> For that we >> >>>>>> could mostly use the default class list (or a slightly >> extended one) >> >>>>>> which is generated at JDK build time (at >> JAVA_HOME/lib/classlist). >> >>>>> >> >>>>> >> >>>>> The point is that you are presumably running multiple >> instances of multiple >> >>>>> apps, hence you want to share one set of core classes across >> all, and share >> >>>>> the app classes across each app instance. >> >>>>> >> >>>> >> >>>> But that would require two archives: a general one with the core >> >>>> classes and an application specific one for each application. >> >>>> Combining the core classes and the application of various >> applications >> >>>> will not be optimal because the application classes will be >> all mixed >> >>>> in the same archive. The archive is being mapped page-wise >> into the >> >>>> java process so you'll probably end up mapping the whole >> archive into >> >>>> each process although you'll only use a fraction of the >> classes in the >> >>>> archive. >> >>>> >> >>>>>> If we want to use ONE archive for several applications and >> we can >> >>>>>> accept to have a bigger footprint if running a single (or >> just a few) >> >>>>>> applications in parallel I suppose the overhead of simply >> dumping all >> >>>>>> the classes from the classpathes of the various >> applications compared >> >>>>>> to an accurate solution where we only dump the actually >> used classes >> >>>>>> of all applications would be not that big. >> >>>>> >> >>>>> >> >>>>> But those "accurate" solutions duplicate the core classes >> and that's a waste >> >>>>> of footprint. >> >>>>> >> >>>> >> >>>> By "accurate" I meant one "fat" archive which contains all >> the classes >> >>>> USED by several applications plus the core classes. My >> argument was >> >>>> that such an "accurate" "fat" archive won't be much smaller >> compared >> >>>> to a "fat" archive which simply contains all the core classes >> plus all >> >>>> the application classes (i.e. from the application class >> pathes, no >> >>>> matter if they are ever used or not). But the latter would be >> much >> >>>> simpler to implement. >> >>> >> >>> The above discussion and an internal proposal for hybrid >> archiving seem to converge on a few points. If there is no >> objection to the hybrid archiving proposal internally, maybe we >> can shared the details of the proposal on openjdk soon. >> >>> >> >>> Thanks, >> >>> >> >>> Jiangli >> >>> >> >>> >> >>>> >> >>>>> David >> >>>>> ----- >> >>>>> >> >>>>> >> >>>>>>> David >> >>>>>>> ----- >> >>>>>>> >> >>>>>>> >> >>>>>>> On 3/05/2018 4:48 PM, Volker Simonis wrote: >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> On Thu, May 3, 2018 at 6:52 AM, Ioi Lam >> > wrote: >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> On 5/2/18 10:00 AM, Volker Simonis wrote: >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> On Tue, May 1, 2018 at 8:32 PM, Ioi Lam >> > wrote: >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> PROBLEM: >> >>>>>>>>>>> >> >>>>>>>>>>> As discussed with Volker and Yumin in previous >> e-mails, AppCDS has >> >>>>>>>>>>> some >> >>>>>>>>>>> experimental support for custom class loaders. >> However, it's not very >> >>>>>>>>>>> easy >> >>>>>>>>>>> to use. >> >>>>>>>>>>> >> >>>>>>>>>>> For example, you can write a classlist like this: >> >>>>>>>>>>> >> >>>>>>>>>>> java/lang/Object id: 1 >> >>>>>>>>>>> CustomLoadee id: 2 super: 1 source: /tmp/foo.jar >> >>>>>>>>>>> >> >>>>>>>>>>> The CustomLoadee class will be stored in the shared >> archive with a >> >>>>>>>>>>> CRC >> >>>>>>>>>>> code. >> >>>>>>>>>>> During runtime, if a customed loader wants to load a >> class of the >> >>>>>>>>>>> same >> >>>>>>>>>>> name, >> >>>>>>>>>>> and its classfile has the same size and CRC as the >> archived class, >> >>>>>>>>>>> the >> >>>>>>>>>>> archived version will be loaded. This speeds up class >> loading by >> >>>>>>>>>>> avoiding >> >>>>>>>>>>> parsing the class file, and saves space by sharing the >> mmap'ed class >> >>>>>>>>>>> metadata across processes. >> >>>>>>>>>>> >> >>>>>>>>>>> You can see an example test at: >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hot >> spot/jtreg/runtime/appcds/customLoader/HelloCustom.java >> >>>>>>>>>>> >> >>>>>>>>>>> However, the current scheme requires you to specify >> all the super >> >>>>>>>>>>> classes >> >>>>>>>>>>> and interfaces. There's no support provided by the >> >>>>>>>>>>> -XX:DumpLoadedClassList >> >>>>>>>>>>> option. It can be helped somewhat with Volker's tool: >> >>>>>>>>>>> https://github.com/simonis/cl4cds >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> POSSIBLE SOLUTIONS: >> >>>>>>>>>>> >> >>>>>>>>>>> 1. "Dump-as-you-go". As suggested by Yumin, we can >> provide a jcmd to >> >>>>>>>>>>> ask >> >>>>>>>>>>> a >> >>>>>>>>>>> running JVM process to dump all of its loaded classes, >> including >> >>>>>>>>>>> those >> >>>>>>>>>>> loaded by custom loaders, into an archive. An >> alternative is to dump >> >>>>>>>>>>> the >> >>>>>>>>>>> archive at JVM exit time (or when you press Ctrl-C, etc. >> >>>>>>>>>>> >> >>>>>>>>>>> 2. Add information about the custom classes for >> >>>>>>>>>>> -XX:DumpLoadedClassList. >> >>>>>>>>>>> The >> >>>>>>>>>>> trouble is some class loaders don't specify a code >> source that can be >> >>>>>>>>>>> understood by the built-in class loaders. For example, >> the "Fat Jars" >> >>>>>>>>>>> would >> >>>>>>>>>>> have a code source like >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/valida >> tion-api-2.0.1.Final.jar!/ >> >>>>>>>>>>> >> >>>>>>>>>>> also, many custom loaders would pre-process the >> classfile data before >> >>>>>>>>>>> defining the class, so we can't simply archive the >> version of the >> >>>>>>>>>>> class >> >>>>>>>>>>> on >> >>>>>>>>>>> disk. >> >>>>>>>>>>> >> >>>>>>>>>>> One possible solution for #2 is to include the class >> file data in the >> >>>>>>>>>>> -XX:DumpLoadedClassList output: >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> java/lang/Object id: 1 >> >>>>>>>>>>> CustomLoadee id: 2 super: 1 source: base64 >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xh >> c3NlcwEABjxpbml0 >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAK >> U291cmNlRmlsZQEA >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2Jq >> ZWN0AQAISGVsbG8k >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUq >> twABsQAAAAEACwAA >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYA >> AQAAADEAAgAOAAAA >> >>>>>>>>>>> AgAPAAcAAAAKAAEABQADAAYACA== >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> Of the 2 solutions: >> >>>>>>>>>>> >> >>>>>>>>>>> #1 seems easier to use, but may require more invasive >> modifications >> >>>>>>>>>>> in >> >>>>>>>>>>> the >> >>>>>>>>>>> VM, especially if you want to be able to continue >> execution after >> >>>>>>>>>>> dumping. >> >>>>>>>>>>> >> >>>>>>>>>> Not sure what #1 really proposes: dumping the complete >> .jsa archive at >> >>>>>>>>>> runtime or dumping just the loaded classes. >> >>>>>>>>>> >> >>>>>>>>>> If it's just about dumping the loaded class without >> generating the >> >>>>>>>>>> .jsa archive there's the problem that by default the VM >> doesn't store >> >>>>>>>>>> the exact bytes of a class after the class was loaded >> (except when >> >>>>>>>>>> class transformers are registered). So the class files >> would have to >> >>>>>>>>>> be re-assembled from the internal VM structures (in the >> same way this >> >>>>>>>>>> is done for class redefinition) and the resulting >> class-file may be >> >>>>>>>>>> different from the original bytes (i.e. some attributes >> may be >> >>>>>>>>>> missing). >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> #1 is for creating the JSA file, not just dumping the >> class files. >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> If #1 is about creating the whole .jsa archive at >> runtime (or at VM >> >>>>>>>>>> exit) I think that would be the most attractive >> solution from a >> >>>>>>>>>> usability point of view although I understand that #2 >> will be easier >> >>>>>>>>>> to implement in the short term. Regarding the argument >> that #1 will >> >>>>>>>>>> produce a "binary blob" that's true, but that's already >> true now when >> >>>>>>>>>> we use "Xshare:dump". I think it should be not to hard >> to implement a >> >>>>>>>>>> tool based an SA which could introspect a .jsa archive. >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> The argument about the binary blob is to compare it >> against the text >> >>>>>>>>> file >> >>>>>>>>> produced by -XX:DumpLoadedClassList. >> >>>>>>>>> >> >>>>>>>>> One use case to consider is when you have a JAR file >> that contains >> >>>>>>>>> several >> >>>>>>>>> apps that each load a unique set of classes. Today, >> (assuming that >> >>>>>>>>> custom >> >>>>>>>>> class loaders are not used), you can run each app once with >> >>>>>>>>> -XX:DumpLoadedClassList, and then do an >> >>>>>>>>> >> >>>>>>>>> cat *.classlist | sort | uniq > combined.classlist >> >>>>>>>>> >> >>>>>>>>> and then create an archive that would work for all these >> apps. >> >>>>>>>>> >> >>>>>>>> >> >>>>>>>> But is this really y relevant use case? Why would I like >> to create ONE >> >>>>>>>> archive for several apps? This would actually increase >> the footprint >> >>>>>>>> of a single instance which uses this archive. If I have >> several apps I >> >>>>>>>> would expect that users create a specific archive for >> each app to get >> >>>>>>>> the best out of CDS. >> >>>>>>>> >> >>>>>>>>> With the binary blob, there's no easy way of doing this. >> It will be >> >>>>>>>>> very >> >>>>>>>>> difficult to write a tool to decipher each blob and then >> somehow >> >>>>>>>>> combine >> >>>>>>>>> them into a single one. >> >>>>>>>>> >> >>>>>>>> >> >>>>>>>> But if users really wants such a "fat" archive, there's a >> much easier >> >>>>>>>> way: just dump ALL the classes from the .jar file into >> the archive. A >> >>>>>>>> class list for this could easily be assembled either with >> an external >> >>>>>>>> tool like cl4cds (or even a simple shell scripts which >> converts the >> >>>>>>>> output of `unzip -l ` into the correct format). >> Or, even >> >>>>>>>> simpler, by adding a new option to the VM similar to >> >>>>>>>> -XX:DumpLoadedClassList which dumps all the classes it >> can find on the >> >>>>>>>> class path (and potentially other, configurable locations). >> >>>>>>>> >> >>>>>>>>> Thanks >> >>>>>>>>> - Ioi >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>>>> #2 would be easier to implement, but the classlist >> might be huge. >> >>>>>>>>>>> >> >>>>>>>>>>> Also, #2 would allow post-processing tools to remove >> unneeded >> >>>>>>>>>>> classes, >> >>>>>>>>>>> or >> >>>>>>>>>>> merge two runs into a single list. The output of #1 is >> essentially a >> >>>>>>>>>>> binary >> >>>>>>>>>>> blob that's impossible for off-line >> analysis/optimizations. >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> Any comments, or suggestions for alternatives? >> >>>>>>>>>>> >> >>>>>>>>>>> Thanks >> >>>>>>>>>>> - Ioi >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>> >> >>>>>>> >> >>>>> >> >>> >> >> >> >> > From daniel.daugherty at Oracle.com Thu May 10 17:07:04 2018 From: daniel.daugherty at Oracle.com (Daniel D. Daugherty) Date: Thu, 10 May 2018 13:07:04 -0400 Subject: RFR(XS): 8132287: obsolete the "InlineNotify" flag option In-Reply-To: <1c5581ca-cc33-2587-9751-b66c23e30769@oracle.com> References: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> <1c5581ca-cc33-2587-9751-b66c23e30769@oracle.com> Message-ID: On 5/10/18 12:43 PM, Vladimir Kozlov wrote: > Looks good. Thanks for the quick review! > Only concern is that library_call.cpp does not include > objectMonitor.hpp. You need to verify build without pre-compiled > header file. I should add an include of objectMonitor.hpp. Do you foresee any problem with that? Dan > > Thanks, > Vladimir > > On 5/10/18 6:52 AM, Daniel D. Daugherty wrote: >> Greetings, >> >> This review has been sent to two aliases so please reply to both. >> >> I'm obsoleting the '-XX:+InlineNotify' option and moving its >> functionality >> to the experimental '-XX:SyncKnobs' option. This is a leftover >> cleanup from >> the "Contended Locking fast notify bucket". >> >> Since this is a diagnostic option, a CSR is not required: >> >> https://wiki.openjdk.java.net/display/csr/CSR+FAQs >> >> ?> Interfaces that are experimental or for diagnostic purposes do not >> need >> ?> to go through CSR process, but the CSR process may be employed if >> feedback >> ?> from the CSR reviewers is desired. >> >> Webrev URL: >> http://cr.openjdk.java.net/~dcubed/8132287-webrev/0_for_jdk_jdk/ >> >> See the bug for the results of test invocations of the >> '-XX:+InlineNotify' >> option and the new '-XX:SyncKnobs' suboption. >> >> Thanks, in advance, for any feedback. >> >> Dan From vladimir.kozlov at oracle.com Thu May 10 17:46:06 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 10 May 2018 10:46:06 -0700 Subject: RFR(XS): 8132287: obsolete the "InlineNotify" flag option In-Reply-To: References: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> <1c5581ca-cc33-2587-9751-b66c23e30769@oracle.com> Message-ID: On 5/10/18 10:07 AM, Daniel D. Daugherty wrote: > On 5/10/18 12:43 PM, Vladimir Kozlov wrote: >> Looks good. > > Thanks for the quick review! > > >> Only concern is that library_call.cpp does not include >> objectMonitor.hpp. You need to verify build without pre-compiled >> header file. > > I should add an include of objectMonitor.hpp. Do you foresee any problem > with that? No problem with that. Thanks, Vladimir > > Dan > > >> >> Thanks, >> Vladimir >> >> On 5/10/18 6:52 AM, Daniel D. Daugherty wrote: >>> Greetings, >>> >>> This review has been sent to two aliases so please reply to both. >>> >>> I'm obsoleting the '-XX:+InlineNotify' option and moving its >>> functionality >>> to the experimental '-XX:SyncKnobs' option. This is a leftover >>> cleanup from >>> the "Contended Locking fast notify bucket". >>> >>> Since this is a diagnostic option, a CSR is not required: >>> >>> https://wiki.openjdk.java.net/display/csr/CSR+FAQs >>> >>> ?> Interfaces that are experimental or for diagnostic purposes do not >>> need >>> ?> to go through CSR process, but the CSR process may be employed if >>> feedback >>> ?> from the CSR reviewers is desired. >>> >>> Webrev URL: >>> http://cr.openjdk.java.net/~dcubed/8132287-webrev/0_for_jdk_jdk/ >>> >>> See the bug for the results of test invocations of the >>> '-XX:+InlineNotify' >>> option and the new '-XX:SyncKnobs' suboption. >>> >>> Thanks, in advance, for any feedback. >>> >>> Dan > From daniel.daugherty at oracle.com Thu May 10 17:50:46 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 10 May 2018 13:50:46 -0400 Subject: RFR(XS): 8132287: obsolete the "InlineNotify" flag option In-Reply-To: References: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> <1c5581ca-cc33-2587-9751-b66c23e30769@oracle.com> Message-ID: On 5/10/18 1:46 PM, Vladimir Kozlov wrote: > On 5/10/18 10:07 AM, Daniel D. Daugherty wrote: >> On 5/10/18 12:43 PM, Vladimir Kozlov wrote: >>> Looks good. >> >> Thanks for the quick review! >> >> >>> Only concern is that library_call.cpp does not include >>> objectMonitor.hpp. You need to verify build without pre-compiled >>> header file. >> >> I should add an include of objectMonitor.hpp. Do you foresee any problem >> with that? > > No problem with that. Thanks! I kicked off a "builds-tier1,hs-tier1,hs-tier2" Mach5 job just to cover my build bases... Now we just need someone from Runtime to chime in on this review... :-) Dan > > Thanks, > Vladimir > >> >> Dan >> >> >>> >>> Thanks, >>> Vladimir >>> >>> On 5/10/18 6:52 AM, Daniel D. Daugherty wrote: >>>> Greetings, >>>> >>>> This review has been sent to two aliases so please reply to both. >>>> >>>> I'm obsoleting the '-XX:+InlineNotify' option and moving its >>>> functionality >>>> to the experimental '-XX:SyncKnobs' option. This is a leftover >>>> cleanup from >>>> the "Contended Locking fast notify bucket". >>>> >>>> Since this is a diagnostic option, a CSR is not required: >>>> >>>> https://wiki.openjdk.java.net/display/csr/CSR+FAQs >>>> >>>> ?> Interfaces that are experimental or for diagnostic purposes do >>>> not need >>>> ?> to go through CSR process, but the CSR process may be employed >>>> if feedback >>>> ?> from the CSR reviewers is desired. >>>> >>>> Webrev URL: >>>> http://cr.openjdk.java.net/~dcubed/8132287-webrev/0_for_jdk_jdk/ >>>> >>>> See the bug for the results of test invocations of the >>>> '-XX:+InlineNotify' >>>> option and the new '-XX:SyncKnobs' suboption. >>>> >>>> Thanks, in advance, for any feedback. >>>> >>>> Dan >> From calvin.cheung at oracle.com Thu May 10 17:56:40 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Thu, 10 May 2018 10:56:40 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time In-Reply-To: <251E8B68-F9BE-4B73-8080-F9D954F03CAA@oracle.com> References: <5AEB8C04.4040501@oracle.com> <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> <5AEB98B7.9030209@oracle.com> <56AA6E39-131E-4D5B-BBEF-9FA8C4A60F5A@oracle.com> <5AF07AE9.9000700@oracle.com> <5AF09BE7.30208@oracle.com> <25B748EB-27EC-4FAB-956F-F864323531EC@oracle.com> <5AF1E157.9050905@oracle.com> <18CD7B50-2063-47A3-867B-EF6150E2260E@oracle.com> <5AF3952B.9070801@oracle.com> <251E8B68-F9BE-4B73-8080-F9D954F03CAA@oracle.com> Message-ID: <5AF487D8.6040706@oracle.com> On 5/10/18, 9:58 AM, Jiangli Zhou wrote: > Hi Calvin, > > Thank you for looking into the test case. That?s helpful. As I also stated in the bug report, although errors caused by extra-long path would be detected much earlier and should not reach ClassLoader::record_result(), it still seem to be a safe choice to report fatal error if get_canonical_path() fails in our case. > > I have a style comment for your latest webrev for the following two tty->print_cr calls. The indentation at line 1563 and 1570 are off. There is no need to break each tty->print_cr into two lines since they are short. Could you please change each tty->print_cr into one line? No need for a new webrev with the line fix. > > 1562 tty->print_cr("Bad pathname %s. CDS dump aborted.", > 1563 path); > > 1569 tty->print_cr("Bad pathname %s. CDS dump aborted.", > 1570 ent->name()); I had a longer string than "Bad pathname" originally. That's why the tty->print_cr line was broken into 2 lines. Yes, I will join the 2 lines into one in both places. Thanks for taking another look. Calvin > > Thanks, > Jiangli > >> On May 9, 2018, at 5:41 PM, Calvin Cheung wrote: >> >> >> >> On 5/8/18, 2:37 PM, Jiangli Zhou wrote: >>>> On May 8, 2018, at 2:24 PM, Ioi Lam wrote: >>>> >>>> I think we do need to check the results of get_canonical_path. On Linux, the canonical path can be longer than the source path, which means ENAMETOOLONG may be returned. >>>> >>>> http://hg.openjdk.java.net/jdk/hs/file/tip/src/java.base/unix/native/libjava/canonicalize_md.c#l253 >>> We need a test case for long path if this could happen by the time we are searching the CDS path table. I was thinking to file a new test bug earlier, but it seems this should be done together with Calvin?s current bug fix. >> I've updated the bug report on why it is hard to write a test case with a long path to test the failure of get_canonical_path() during CDS dumping. >> >> thanks, >> Calvin >>> Thanks, >>> Jiangli >>> >>>> So instead of assert, it's best to check for failure and exit the dumping process. >>>> >>>> The rest of the changes look OK to me. >>>> >>>> Thanks >>>> >>>> - Ioi >>>> >>>> >>>> >>>> On 5/8/18 10:41 AM, Calvin Cheung wrote: >>>>> On 5/7/18, 2:58 PM, Jiangli Zhou wrote: >>>>>>> On May 7, 2018, at 11:33 AM, Calvin Cheung> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 5/7/18, 10:28 AM, Jiangli Zhou wrote: >>>>>>>> Hi Calvin, >>>>>>>> >>>>>>>> -classLoader.cpp >>>>>>>> >>>>>>>> The get_canonical_path() call for ?path? should be moved out of the ?for' loop. >>>>>>>> >>>>>>>> 1569 get_canonical_path(path, canonical_src_path, JVM_MAXPATHLEN)) { >>>>>>> Fixed. >>>>>>>> To make the code more readable, please rename ?canonical_path? to ?canonical_path_table_entry? (or ?canonical_path_table_entry_name?) >>>>>>> I chose the shorter one. >>>>>>>> and ?canonical_src_path? to ?canonical_class_src_path?. >>>>>>> Renamed. >>>>>>>> BTW, when we create the path entries, the paths are already converted to canonical forms. I think there is no need to call get_canonical_path() on ?ent->name()?. Can you please double check? >>>>>>> The call to get_canonical_path() is only for the non-directory case (please refer to ClassLoader::create_class_path_entry() for details). >>>>>>> I did try get_canonical_path() for the directory case but it didn't work during runtime; it failed in SystemDictionaryShared::is_shared_class_visible_for_classloader() in the following check: >>>>>>> >>>>>>> (strcmp(ent->name(), ClassLoader::skip_uri_protocol(mod_entry->location()->as_C_string())) == 0)) { >>>>>>> >>>>>>> The location() in the mod_entry isn't in canonical form. If we convert it to canonical form during runtime, there maybe performance impact. >>>>>>> I think the proper way of fixing it is storing the location() in canonical form when a module entry is being defined. I'm not sure if it is possible for the core lib to pass the location in canonical form to jvm when a module entry is being defined. >>>>>>> I can file a follow-up RFE to address that. >>>>>> Please file a RFE. >>>>> I've filed JDK-8202750 >>>>>> There is no need to check ?res? in very iteration since it doesn?t change after being initialized by get_canonical_path(). Also, I think we are not handling get_canonical_path() (both calls) failure properly. However, I question why get_canonical_path() would fail when given a valid path in this particular case. So I think we should either assert get_canonical_path() returns true, or report a fatal error for dumping. >>>>> I've made the change and added assert after get_canonical_path() at both calls. >>>>> >>>>> Updated webrev: >>>>> >>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.03/ >>>>> >>>>> thanks, >>>>> Calvin >>>>>>>> - os_windows.cpp >>>>>>>> >>>>>>>> Please fix indentation at line 4390& 4391. >>>>>>> That was done on purpose because the condition in line 4390 is '||' with line 4391. >>>>>>> I'm further indenting line 4391 to make it clearer. >>>>>>> >>>>>>> 4389 if (search_path[1] == ':'&& >>>>>>> 4390 (search_path[2] == '\0' || >>>>>>> 4391 (search_path[2] == '\\' && search_path[3] == '\0'))) { >>>>>> Looks better. >>>>>> >>>>>>> Updated webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.02/ >>>>>> Thanks, >>>>>> Jiangli >>>>>> >>>>>>> thanks, >>>>>>> Calvin >>>>>>> >>>>>>>> Thanks, >>>>>>>> Jiangli >>>>>>>> >>>>>>>> >>>>>>>>> On May 7, 2018, at 9:12 AM, Calvin Cheung wrote: >>>>>>>>> >>>>>>>>> Hi Jiangli, >>>>>>>>> >>>>>>>>> On 5/3/18, 5:33 PM, Jiangli Zhou wrote: >>>>>>>>>> Hi Calvin, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On May 3, 2018, at 4:18 PM, Calvin Cheung wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 5/3/18, 3:42 PM, Jiangli Zhou wrote: >>>>>>>>>>>> Hi Calvin, >>>>>>>>>>>> >>>>>>>>>>>> I think we don?t need to check the module path entries if only boot classes are archived. The ?end? is set to app_class_paths_start_index when there are only boot classes are archived. If there are app module classes loaded from jars/directories at dump time, has_platform_or_app_classes() should return true and ?end? is set to the end of the shared path table, which includes all module path entries. The following loop is not needed. The existing loop in the code covers all cases with different ?end? value based on the type of classes loaded in the archive. >>>>>>>>>>>> 366 // may need to check module path entries. >>>>>>>>>>>> 367 if ((end == ClassLoaderExt::app_class_paths_start_index())&& (ClassLoader::num_module_path_entries()> 0)) { >>>>>>>>>>>> 368 for (int i = ClassLoaderExt::app_module_paths_start_index(); i< _shared_path_table_size; i++) { >>>>>>>>>>>> 369 SharedClassPathEntry* e = shared_path(i); >>>>>>>>>>>> 370 has_nonempty_dir = check_nonempty_dir(e); >>>>>>>>>>>> >>>>>>>>>>>> 371 } >>>>>>>>>>>> 372 } >>>>>>>>>>>> >>>>>>>>>>> The has_platform_or_app_classes() will return false if ClassLoaderExt::record_result() hasn't been called while a class is being loaded during dumping. >>>>>>>>>>> This could happen before this proposed change if one specifies a --module-path with an non-empty directory containing a module. While the class will be loaded during dumping but its classpath_index will still be -1. Therefore, the has_platform_or_app_classes() will return false and the module path entries won't be checked without the above block of code. >>>>>>>>>> With the module directory path being added to the path table now, it should record the correct path table index instead of -1 for module classes loaded from directory at dump time. >>>>>>>>>> >>>>>>>>>> I applied your patch and stepped through ClassLoader::record_result() with your test case. The following path comparison fails due the the extra ?/? in ?path?. That why it fails to find the correct entry in the path table, which causes the class type cannot be correctly identified. >>>>>>>>>> >>>>>>>>>> if (strcmp(canonical_path, os::native_path((char*)path)) == 0) {os::dir_is_empty >>>>>>>>>> >>>>>>>>>> (gdb) p canonical_path >>>>>>>>>> $74 = 0x7ffff0aea0f0 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple" >>>>>>>>>> (gdb) p os::native_path((char*)path) >>>>>>>>>> $75 = 0x7ffff7fd4a65 ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple/" >>>>>>>>>> >>>>>>>>>> The trailing ?/? should be excluded for the path comparison. JDK-8202519 is a related issue in the same area. It might be better to fix it together with this one. >>>>>>>>> Thanks for debugging the issue. >>>>>>>>> I've fixed it by calling get_canonical_path() on the "path" before doing strcmp(). It should also fix the bug in JDK-8202519 since a buffer is being allocated for get_canonical_path() to write into. The os::native_path() on linux does nothing and just returns the same string. >>>>>>>>> >>>>>>>>> I also discovered a problem in os::dir_is_empty() on windows. When a directory path is passed into the function, it may have the following form with forward slashes: C:/somedir >>>>>>>>> The fix is to call os::native_path() which essentially converts the forward slashes to back slashes so that subsequent windows api call returns the correct result. >>>>>>>>> >>>>>>>>>> Once the above issue is fixed, the class type should be recorded correctly for a module classes loaded from directory at dump time. Then ClassLoaderExt::record_result() can set ?has_app_classes? flag properly. And, no change should be needed in FileMapInfo::check_nonempty_dir_in_shared_path_table(). >>>>>>>>> I don't need to touch filemap.cpp in the following updated webrev: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.01/ >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> Calvin >>>>>>>>>>>> We also need a unit test for non-empty directory in module path. Please add. >>>>>>>>>>> It is already in my webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/MainModuleOnly.java.sdiff.html >>>>>>>>>> Ok. I was expecting a separate unit test. I?m fine with an added test case in MainModuleOnly.java. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Jiangli >>>>>>>>>> >>>>>>>>>>> thanks, >>>>>>>>>>> Calvin >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Jiangli >>>>>>>>>>>> >>>>>>>>>>>>> On May 3, 2018, at 3:24 PM, Calvin Cheung wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> bug:https://bugs.openjdk.java.net/browse/JDK-8202289 >>>>>>>>>>>>> webrev:http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> In ClassLoaderExt::process_module_table(), it adds all entries from the ModuleEntryTable with the "file:" protocol to the _module_path_entries and eventually to the _shared_path_table. >>>>>>>>>>>>> The FileMapInfo::check_nonempty_dir_in_shared_path_table() will check for non-empty directory in the module path. >>>>>>>>>>>>> >>>>>>>>>>>>> Ran all CDS and AppCDS tests locally on linux-x64. >>>>>>>>>>>>> Will run hs-tier{1,2,3} tests. >>>>>>>>>>>>> >>>>>>>>>>>>> thanks, >>>>>>>>>>>>> Calvin From calvin.cheung at oracle.com Thu May 10 18:00:01 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Thu, 10 May 2018 11:00:01 -0700 Subject: RFR(S): 8202289: Non-empty directory in module path is not handled properly at CDS/AppCDS dump time In-Reply-To: References: <5AEB8C04.4040501@oracle.com> <42264FA8-5D3C-4611-A98B-BD56C8E4EEA6@oracle.com> <5AEB98B7.9030209@oracle.com> <56AA6E39-131E-4D5B-BBEF-9FA8C4A60F5A@oracle.com> <5AF07AE9.9000700@oracle.com> <5AF09BE7.30208@oracle.com> <25B748EB-27EC-4FAB-956F-F864323531EC@oracle.com> <5AF1E157.9050905@oracle.com> <5AF39439.8060403@oracle.com> Message-ID: <5AF488A1.4020803@oracle.com> Thanks, Ioi. Calvin On 5/9/18, 8:51 PM, Ioi Lam wrote: > Looks good. Thanks! > > - Ioi > > > On 5/9/18 5:37 PM, Calvin Cheung wrote: >> >> >> On 5/8/18, 2:24 PM, Ioi Lam wrote: >>> I think we do need to check the results of get_canonical_path. On >>> Linux, the canonical path can be longer than the source path, which >>> means ENAMETOOLONG may be returned. >>> >>> http://hg.openjdk.java.net/jdk/hs/file/tip/src/java.base/unix/native/libjava/canonicalize_md.c#l253 >>> >>> >>> So instead of assert, it's best to check for failure and exit the >>> dumping process. >> I've made the above change. >> Updated webrev: >> >> http://cr.openjdk.java.net/~ccheung/8202289/webrev.04/ >> >> thanks, >> Calvin >>> >>> The rest of the changes look OK to me. >>> >>> Thanks >>> >>> - Ioi >>> >>> >>> >>> On 5/8/18 10:41 AM, Calvin Cheung wrote: >>>> >>>> >>>> On 5/7/18, 2:58 PM, Jiangli Zhou wrote: >>>>> >>>>>> On May 7, 2018, at 11:33 AM, Calvin Cheung >>>>>> > wrote: >>>>>> >>>>>> >>>>>> >>>>>> On 5/7/18, 10:28 AM, Jiangli Zhou wrote: >>>>>>> Hi Calvin, >>>>>>> >>>>>>> -classLoader.cpp >>>>>>> >>>>>>> The get_canonical_path() call for ?path? should be moved out of >>>>>>> the ?for' loop. >>>>>>> >>>>>>> 1569 get_canonical_path(path, canonical_src_path, >>>>>>> JVM_MAXPATHLEN)) { >>>>>> Fixed. >>>>>>> To make the code more readable, please rename ?canonical_path? >>>>>>> to ?canonical_path_table_entry? (or >>>>>>> ?canonical_path_table_entry_name?) >>>>>> I chose the shorter one. >>>>>>> and ?canonical_src_path? to ?canonical_class_src_path?. >>>>>> Renamed. >>>>>>> BTW, when we create the path entries, the paths are already >>>>>>> converted to canonical forms. I think there is no need to call >>>>>>> get_canonical_path() on ?ent->name()?. Can you please double check? >>>>>> The call to get_canonical_path() is only for the non-directory >>>>>> case (please refer to ClassLoader::create_class_path_entry() for >>>>>> details). >>>>>> I did try get_canonical_path() for the directory case but it >>>>>> didn't work during runtime; it failed in >>>>>> SystemDictionaryShared::is_shared_class_visible_for_classloader() >>>>>> in the following check: >>>>>> >>>>>> (strcmp(ent->name(), >>>>>> ClassLoader::skip_uri_protocol(mod_entry->location()->as_C_string())) >>>>>> == 0)) { >>>>>> >>>>>> The location() in the mod_entry isn't in canonical form. If we >>>>>> convert it to canonical form during runtime, there maybe >>>>>> performance impact. >>>>>> I think the proper way of fixing it is storing the location() in >>>>>> canonical form when a module entry is being defined. I'm not sure >>>>>> if it is possible for the core lib to pass the location in >>>>>> canonical form to jvm when a module entry is being defined. >>>>>> I can file a follow-up RFE to address that. >>>>> >>>>> Please file a RFE. >>>> I've filed JDK-8202750 >>>> >>>>> >>>>> There is no need to check ?res? in very iteration since it doesn?t >>>>> change after being initialized by get_canonical_path(). Also, I >>>>> think we are not handling get_canonical_path() (both calls) >>>>> failure properly. However, I question why get_canonical_path() >>>>> would fail when given a valid path in this particular case. So I >>>>> think we should either assert get_canonical_path() returns true, >>>>> or report a fatal error for dumping. >>>> I've made the change and added assert after get_canonical_path() at >>>> both calls. >>>> >>>> Updated webrev: >>>> >>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.03/ >>>> >>>> thanks, >>>> Calvin >>>>>>> - os_windows.cpp >>>>>>> >>>>>>> Please fix indentation at line 4390& 4391. >>>>>> That was done on purpose because the condition in line 4390 is >>>>>> '||' with line 4391. >>>>>> I'm further indenting line 4391 to make it clearer. >>>>>> >>>>>> 4389 if (search_path[1] == ':'&& >>>>>> 4390 (search_path[2] == '\0' || >>>>>> 4391 (search_path[2] == '\\' && >>>>>> search_path[3] == '\0'))) { >>>>> >>>>> >>>>> Looks better. >>>>> >>>>>> Updated webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.02/ >>>>> >>>>> >>>>> Thanks, >>>>> Jiangli >>>>> >>>>>> >>>>>> thanks, >>>>>> Calvin >>>>>> >>>>>>> Thanks, >>>>>>> Jiangli >>>>>>> >>>>>>> >>>>>>>> On May 7, 2018, at 9:12 AM, Calvin >>>>>>>> Cheung wrote: >>>>>>>> >>>>>>>> Hi Jiangli, >>>>>>>> >>>>>>>> On 5/3/18, 5:33 PM, Jiangli Zhou wrote: >>>>>>>>> Hi Calvin, >>>>>>>>> >>>>>>>>> >>>>>>>>>> On May 3, 2018, at 4:18 PM, Calvin >>>>>>>>>> Cheung wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 5/3/18, 3:42 PM, Jiangli Zhou wrote: >>>>>>>>>>> Hi Calvin, >>>>>>>>>>> >>>>>>>>>>> I think we don?t need to check the module path entries if >>>>>>>>>>> only boot classes are archived. The ?end? is set to >>>>>>>>>>> app_class_paths_start_index when there are only boot classes >>>>>>>>>>> are archived. If there are app module classes loaded from >>>>>>>>>>> jars/directories at dump time, has_platform_or_app_classes() >>>>>>>>>>> should return true and ?end? is set to the end of the shared >>>>>>>>>>> path table, which includes all module path entries. The >>>>>>>>>>> following loop is not needed. The existing loop in the code >>>>>>>>>>> covers all cases with different ?end? value based on the >>>>>>>>>>> type of classes loaded in the archive. >>>>>>>>>>> 366 // may need to check module path entries. >>>>>>>>>>> 367 if ((end == >>>>>>>>>>> ClassLoaderExt::app_class_paths_start_index())&& >>>>>>>>>>> (ClassLoader::num_module_path_entries()> 0)) { >>>>>>>>>>> 368 for (int i = >>>>>>>>>>> ClassLoaderExt::app_module_paths_start_index(); i< >>>>>>>>>>> _shared_path_table_size; i++) { >>>>>>>>>>> 369 SharedClassPathEntry* e = shared_path(i); >>>>>>>>>>> 370 has_nonempty_dir = check_nonempty_dir(e); >>>>>>>>>>> >>>>>>>>>>> 371 } >>>>>>>>>>> 372 } >>>>>>>>>>> >>>>>>>>>> The has_platform_or_app_classes() will return false if >>>>>>>>>> ClassLoaderExt::record_result() hasn't been called while a >>>>>>>>>> class is being loaded during dumping. >>>>>>>>>> This could happen before this proposed change if one >>>>>>>>>> specifies a --module-path with an non-empty directory >>>>>>>>>> containing a module. While the class will be loaded during >>>>>>>>>> dumping but its classpath_index will still be -1. Therefore, >>>>>>>>>> the has_platform_or_app_classes() will return false and the >>>>>>>>>> module path entries won't be checked without the above block >>>>>>>>>> of code. >>>>>>>>> With the module directory path being added to the path table >>>>>>>>> now, it should record the correct path table index instead of >>>>>>>>> -1 for module classes loaded from directory at dump time. >>>>>>>>> >>>>>>>>> I applied your patch and stepped through >>>>>>>>> ClassLoader::record_result() with your test case. The >>>>>>>>> following path comparison fails due the the extra ?/? in >>>>>>>>> ?path?. That why it fails to find the correct entry in the >>>>>>>>> path table, which causes the class type cannot be correctly >>>>>>>>> identified. >>>>>>>>> >>>>>>>>> if (strcmp(canonical_path, >>>>>>>>> os::native_path((char*)path)) == 0) {os::dir_is_empty >>>>>>>>> >>>>>>>>> (gdb) p canonical_path >>>>>>>>> $74 = 0x7ffff0aea0f0 >>>>>>>>> ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple" >>>>>>>>> (gdb) p os::native_path((char*)path) >>>>>>>>> $75 = 0x7ffff7fd4a65 >>>>>>>>> ?/jdk/open/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/JTwork/scratch/mods/com.simple/" >>>>>>>>> >>>>>>>>> The trailing ?/? should be excluded for the path comparison. >>>>>>>>> JDK-8202519 is a related issue in the same area. It might be >>>>>>>>> better to fix it together with this one. >>>>>>>> Thanks for debugging the issue. >>>>>>>> I've fixed it by calling get_canonical_path() on the "path" >>>>>>>> before doing strcmp(). It should also fix the bug in >>>>>>>> JDK-8202519 since a buffer is being allocated for >>>>>>>> get_canonical_path() to write into. The os::native_path() on >>>>>>>> linux does nothing and just returns the same string. >>>>>>>> >>>>>>>> I also discovered a problem in os::dir_is_empty() on windows. >>>>>>>> When a directory path is passed into the function, it may have >>>>>>>> the following form with forward slashes: C:/somedir >>>>>>>> The fix is to call os::native_path() which essentially converts >>>>>>>> the forward slashes to back slashes so that subsequent windows >>>>>>>> api call returns the correct result. >>>>>>>> >>>>>>>>> Once the above issue is fixed, the class type should be >>>>>>>>> recorded correctly for a module classes loaded from directory >>>>>>>>> at dump time. Then ClassLoaderExt::record_result() can set >>>>>>>>> ?has_app_classes? flag properly. And, no change should be >>>>>>>>> needed in FileMapInfo::check_nonempty_dir_in_shared_path_table(). >>>>>>>> I don't need to touch filemap.cpp in the following updated webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.01/ >>>>>>>> >>>>>>>> thanks, >>>>>>>> Calvin >>>>>>>>>>> We also need a unit test for non-empty directory in module >>>>>>>>>>> path. Please add. >>>>>>>>>> It is already in my webrev: >>>>>>>>>> http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/test/hotspot/jtreg/runtime/appcds/jigsaw/modulepath/MainModuleOnly.java.sdiff.html >>>>>>>>>> >>>>>>>>> Ok. I was expecting a separate unit test. I?m fine with an >>>>>>>>> added test case in MainModuleOnly.java. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Jiangli >>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> Calvin >>>>>>>>>>> Thanks, >>>>>>>>>>> Jiangli >>>>>>>>>>> >>>>>>>>>>>> On May 3, 2018, at 3:24 PM, Calvin >>>>>>>>>>>> Cheung wrote: >>>>>>>>>>>> >>>>>>>>>>>> bug:https://bugs.openjdk.java.net/browse/JDK-8202289 >>>>>>>>>>>> webrev:http://cr.openjdk.java.net/~ccheung/8202289/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> In ClassLoaderExt::process_module_table(), it adds all >>>>>>>>>>>> entries from the ModuleEntryTable with the "file:" protocol >>>>>>>>>>>> to the _module_path_entries and eventually to the >>>>>>>>>>>> _shared_path_table. >>>>>>>>>>>> The FileMapInfo::check_nonempty_dir_in_shared_path_table() >>>>>>>>>>>> will check for non-empty directory in the module path. >>>>>>>>>>>> >>>>>>>>>>>> Ran all CDS and AppCDS tests locally on linux-x64. >>>>>>>>>>>> Will run hs-tier{1,2,3} tests. >>>>>>>>>>>> >>>>>>>>>>>> thanks, >>>>>>>>>>>> Calvin >>>>> >>> > From jiangli.zhou at Oracle.COM Thu May 10 18:39:14 2018 From: jiangli.zhou at Oracle.COM (Jiangli Zhou) Date: Thu, 10 May 2018 11:39:14 -0700 Subject: Improving AppCDS for Custom Loaders In-Reply-To: References: <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> <732ce957-b07c-bff2-277b-ca335c348753@oracle.com> <660A1644-3A08-455D-931D-D95069C39853@oracle.com> <9abbe! 65c-93e9-3931-45ce-1688541c29c4@oracle.com> Message-ID: <16902EC1-7A5F-4DBC-A815-855BA04509BA@oracle.com> Hi Yumin and Volker, I?ve create an umbrella RFE JDK-8202854 to keep track this effort. New sub-RFEs will be created and targeted for specific JDK releases when each sub-item matures. Please refer to JDK-8202854 for future progress. Yumin, please see below for comments/questions. > On May 10, 2018, at 10:09 AM, yumin qi wrote: > > HI, > > I joined the discussion too late --- so nice to see your valuable discussion and an action to further improve CDS (AppCDS is removed, CDS is for all now) in progress. > We plan to run every application in docker --- dockerize every application and multiple deployment of different applications in single host. This is a trend for industrial applications save machine resources and better memory management. A base archive can save footprint for the image, CDS for base classes should be default. Thank you for providing information on your use case. That?s very helpful. > Agree not including objects in java heap at now --- thanks Jiangli for answering my question in other thread. I conclude from above that you are not using java heap object archiving due to different GC algorithm being used(?). I would like to understand more about your GC choices. Could you please provide more information on that? G1 GC is the default GC in JDK. Thanks to our GC team, the performance of G1 has been consistently improving over the recent JDK releases, especially in JDK 9 and 10. I strongly encourage you to do performance measurement/comparison and reevaluate your GC choice if G1 is not used. > I think for #1, it does not conflict with the two layer archive solution. We can skip the classes from base CDS archive, only dump the non-base loaded classes into second archive, this gives one more option for user to choose. Also it will be good after dump archive to let the application continue to run. > > Can we do a Full GC before dump at exit? it may unload not referenced classes, or it is not necessary since CDS wants to resolve the startup performance for all loaded classes? For classes loaded by the builtin loaders, they can be kept alive when archiving phase start. For classes loaded by user defined class loaders, we also need to prevent them from being unloaded before archiving. I have a preliminary idea for how to do that. I can share more information on that when it matures. Thanks! Jiangli > > * If performance measurement does show regression, John Rose has a very interesting idea that could be used to reduce overhead. > Really want to know what is the interesting idea. > > Thanks > Yumin > > On Mon, May 7, 2018 at 1:39 PM, Ioi Lam wrote: > > > On 5/5/18 1:22 PM, Volker Simonis wrote: > Hi Jiangli, > > thanks a lot explaining your plans more precisely. I think the approach makes much more sense now. > > While I?m all for it in general, I still have some questions and remarks :) > > 1. Will it be possible to use the two archives independantly ? I.e. will it be possible to choose just one of the two archives as well as a combination of both? > > 2. We should make sure to not introduce any performance regressions if we have two archives (and potentially three SymbolTables and three ?Metaspaces?) > > 3. In general I think we?re moving away from scenarios where we have one single, central Java installation on a system which is used by the ?tens/hundreds different applications? you mentioned in your mail towards a world where every of these ?tens/hundreds different applications? comes with its own, bundled JDK/JRE version (and as far as I understand Oracle is propagating this new distribution model after the deprecation of Applets/WebStart). In such a world it would be not easy to have a static, common system layer archive. We could try to keep the format of the archived classes as compatible between Java versions as possible, although I understand that this will be quite complicated. Do you already have any thoughts on this specific problem? > > > I think the 2 level archive will work very well with containerized app deployment. For example, with docker, you can create a "base docker image" with the JDK and the "base" archive that contains the frequently used JDK and library classes. Then, you create an "app docker image" for each application, which contains the modules of the app, as well as the CDS archive of the classes used by this app. > > So imagine you have created 2 different app images from the same base image. When you deploy these 2 app images on the same hardware, they can share the files for the JDK and the base CDS archive between them. This could be quite significant (tens or 100s of MB). > > > 4. Finally we should design the new model in a way which allows it to easily integrate AOT compiled code into the same one or two archives in the future. Because with AOT we face the exactly same problems and having to maintain four different archives will be not very user friendly. OpenJ9 already uses a single archive for CDT/AOT/ProfilingData. I don?t say we have to unify the AOT and CDT archives in HotSpot now, I just want to suggest that if we?re redesigning the CDS archive anyway to perhaps think twice before we come to a new model such that we at least don?t exclude the possibility of a unification with the AOT archive in the future. > > > There are 2 reasons we have not unified the AOT and CDS storage: > > 1. If we store AOT code inside the CDS archive, we would need to implement our own native code loader (similar to ld.so). This could be quite involving since we have many supported platforms. > > 2. If we store CDS data into AOT (e.g., add .text/.data sections to the .so files generated by AOT), we have very large overhead -- CDS has a lot of direct data pointers. E.g., InstanceKlass::_super, etc. Last time I checked, ELF uses 2x8 bytes of relocation data per pointer. All of these pointers are relocated when the .so is loaded, which means all the "read only" CDS data is effectively copied-on-write. > > So, I think we need a good justification to see why the unification is needed, before we try to resolve the technical issues. > > Thanks > - Ioi > Thanks, > Volker > > > > > Jiangli Zhou > schrieb am Fr. 4. Mai 2018 um 19:42: > > Hi Volker, > > Thank you so much for the feedbacks! > > You comments made me realize that some clarifications are need for > the top-layer dynamic archiving. The top-layer would include all > application classes, middleware classes, system library classes > that are not in the bottom layer. Apologizing for not stating that > clearly in the proposal. With the top layer including all > necessary classes (excluding the classes in bottom layer) for a > specific application, multiple instances running the same > application can achieve maximum memory sharing. Using your > example, the extra AWT/Swing classes are archived along with the > application classes in the top layer. > > With the dynamic archiving ability, it?s natural to think there is > no need for the two-layer scheme. In fact, I originally had the > thoughts. However, after exploring different possible use cases, I > agree two-layer archiving do bring benefits that are lacking in > the single archive design. One example of such use case is > tens/hundreds different applications running on the same machine. > Without a common system layer archive, it is difficult to achieve > memory sharing for common system classes, String objects, etc > between different applications. > > Another reason why we should go with a two-layer & static/dynamic > archiving is some runtime information may be sensitive and should > not be archived. Strings can be a good example in this case. > Currently, we archive interned String objects without running > application. The Strings include resolved CONSTANT_strings, class > names, method names, etc, which are all static information from > class files. During execution of an application, interned Strings > might contain sensitive user information, which is not safe to be > included in the archive. It?s difficult to distinguish between the > static information and runtime information if we go with the > single layer dynamic-only archiving. > > We had many discussions internally over long period of time on how > to improve usability with different approaches. In the end, > dynamic-only or static-only two-layer archiving all have their own > disadvantages and fail to meet requirements in some use cases. The > hybrid archiving combines benefits/advantages of different > approaches and seem to be flexible enough to fit most usages. > > Further comments and feedbacks are much appreciated. > > Best regards, > > Jiangli > > > On May 4, 2018, at 3:01 AM, Volker Simonis > > wrote: > > > > Hi Jiangli, > > > > thanks for sharing the hybrid archiving proposal. I think it is very > > interesting and useful! > > > > One issue I see is that the "system library classes" which should go > > into the "static" archive are inherently application specific > (i.e. if > > my application uses Swing, it will use most of the AWT/Swing > classes). > > So how will you cope with this problem. Either you put ALL the > system > > classes into the static archive which will be a waste for most > > applications. Or you just put a small commonly used subset of > classes > > into the static archive which may miss many classes for specific > > applications. > > > > If we would add the possibility to create a dynamic archive at > runtime > > / program end (which I think would be great from a usability > > perspective) I don't see a big need for two different archives. Two > > archives will further complicate (and slow down) things like Symbol > > lookup (we already have two SymbolTable now and we'd probably need a > > third one if we would have two archives). > > > > I don't think that running a few different Java applications on one > > host is the most important use case for CDS. In such a scenario the > > current, build time generated archive is probably the best we can do > > anyway. Instead, I think the most important use case is if we have > > many instances of the same Java application running on the same > host. > > And this is becoming more common with the raise of containerization. > > For the latter use case, a dynamically generated, application > specific > > archive would be the optimal solution. > > > > Thank you and best regards, > > Volker > > > > > > > > On Fri, May 4, 2018 at 3:42 AM, Jiangli Zhou > > wrote: > >> Hi Volker, > >> > >> Here are some details about the hybrid archiving. The goal is > to harvest the benefit of archiving by default and improve its > usability. The hybrid approach combines a two-layer archiving > (proposed by Ioi internally) and static & dynamic archiving > techniques: > >> > >> - Statically archive system library classes from a provided > classlist using the existing method. The archiving includes class > metadata, interned string objects, constant pool > resolved_references arrays, class mirror objects, etc. Static > archiving can be done at the JDK image build time and shipped > together with JDK binary. Following need to be addressed: > >> *Relaxing the runtime CDS/AppCDS boot path check, so the > packaged archive can be used after the JDK binary is installed on > the target device. JDK-8199807 was created to address this issue > and is targeted for JDK 11. > >> *Add the static archiving generation in JDK build steps > and package the generated archive with JDK image. The archive can > only be generated for the same target (both OS can CPU > architecture) as the build platform. I will create a RFE. > >> > >> - Dynamic archiving can done for application classes at the > first execution of a specific application > >> * The archive is created on top of the default system > archive shipped with the JDK image. A separate top-layer archive > file is generated for each different application. > >> * Archiving is done at the end of the application > execution before VM exists by relocating the class metadata to the > archive spaces. Cleanup also needs to be done for copied class > meta data to remove any runtime information. Most of the required > functionality already exists today. For example, class metadata > relocation was implemented by Ioi in JDK 10. > >> * Only archive class metadata for application in the top > layer initially. Archiving java heap objects in the top-layer > requires more investigations. > >> > >> Benefits of the hybrid archiving: > >> * The system archive can be shared by different applications > and provides memory saving. > >> * Archive for application is created and used transparently. No > more profiling step and class list are required! > >> * Separating the system archiving from application archiving > reduces the cost of archiving at application execution time. The > overhead added to the first execution time is reduced. > >> > >> Thanks, > >> > >> Jiangli > >> > >> > >>> On May 3, 2018, at 10:34 AM, Jiangli Zhou > wrote: > >>> > >>> > >>>> On May 3, 2018, at 2:14 AM, Volker Simonis > > wrote: > >>>> > >>>> On Thu, May 3, 2018 at 11:01 AM, David Holmes > > wrote: > >>>>> On 3/05/2018 5:16 PM, Volker Simonis wrote: > >>>>>> > >>>>>> On Thu, May 3, 2018 at 8:55 AM, David Holmes > > > > >>>>>> wrote: > >>>>>>> > >>>>>>> Just lurking here but ... > >>>>>>> > >>>>>>>> But is this really y relevant use case? Why would I like > to create ONE > >>>>>>>> archive for several apps? This would actually increase > the footprint > >>>>>>>> of a single instance which uses this archive. If I have > several apps I > >>>>>>>> would expect that users create a specific archive for > each app to get > >>>>>>>> the best out of CDS. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> One app instance may get increased footprint but you > presumably use CDS > >>>>>>> because you have multiple apps running (whether the same > or not). These > >>>>>>> apps > >>>>>>> all share the core JDK classes from the archive so the > overall footprint > >>>>>>> per > >>>>>>> instance is less. > >>>>>>> > >>>>>> > >>>>>> If we just want to share the core JDK classes that's easy. > For that we > >>>>>> could mostly use the default class list (or a slightly > extended one) > >>>>>> which is generated at JDK build time (at > JAVA_HOME/lib/classlist). > >>>>> > >>>>> > >>>>> The point is that you are presumably running multiple > instances of multiple > >>>>> apps, hence you want to share one set of core classes across > all, and share > >>>>> the app classes across each app instance. > >>>>> > >>>> > >>>> But that would require two archives: a general one with the core > >>>> classes and an application specific one for each application. > >>>> Combining the core classes and the application of various > applications > >>>> will not be optimal because the application classes will be > all mixed > >>>> in the same archive. The archive is being mapped page-wise > into the > >>>> java process so you'll probably end up mapping the whole > archive into > >>>> each process although you'll only use a fraction of the > classes in the > >>>> archive. > >>>> > >>>>>> If we want to use ONE archive for several applications and > we can > >>>>>> accept to have a bigger footprint if running a single (or > just a few) > >>>>>> applications in parallel I suppose the overhead of simply > dumping all > >>>>>> the classes from the classpathes of the various > applications compared > >>>>>> to an accurate solution where we only dump the actually > used classes > >>>>>> of all applications would be not that big. > >>>>> > >>>>> > >>>>> But those "accurate" solutions duplicate the core classes > and that's a waste > >>>>> of footprint. > >>>>> > >>>> > >>>> By "accurate" I meant one "fat" archive which contains all > the classes > >>>> USED by several applications plus the core classes. My > argument was > >>>> that such an "accurate" "fat" archive won't be much smaller > compared > >>>> to a "fat" archive which simply contains all the core classes > plus all > >>>> the application classes (i.e. from the application class > pathes, no > >>>> matter if they are ever used or not). But the latter would be > much > >>>> simpler to implement. > >>> > >>> The above discussion and an internal proposal for hybrid > archiving seem to converge on a few points. If there is no > objection to the hybrid archiving proposal internally, maybe we > can shared the details of the proposal on openjdk soon. > >>> > >>> Thanks, > >>> > >>> Jiangli > >>> > >>> > >>>> > >>>>> David > >>>>> ----- > >>>>> > >>>>> > >>>>>>> David > >>>>>>> ----- > >>>>>>> > >>>>>>> > >>>>>>> On 3/05/2018 4:48 PM, Volker Simonis wrote: > >>>>>>>> > >>>>>>>> > >>>>>>>> On Thu, May 3, 2018 at 6:52 AM, Ioi Lam > > wrote: > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On 5/2/18 10:00 AM, Volker Simonis wrote: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On Tue, May 1, 2018 at 8:32 PM, Ioi Lam > > wrote: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> PROBLEM: > >>>>>>>>>>> > >>>>>>>>>>> As discussed with Volker and Yumin in previous > e-mails, AppCDS has > >>>>>>>>>>> some > >>>>>>>>>>> experimental support for custom class loaders. > However, it's not very > >>>>>>>>>>> easy > >>>>>>>>>>> to use. > >>>>>>>>>>> > >>>>>>>>>>> For example, you can write a classlist like this: > >>>>>>>>>>> > >>>>>>>>>>> java/lang/Object id: 1 > >>>>>>>>>>> CustomLoadee id: 2 super: 1 source: /tmp/foo.jar > >>>>>>>>>>> > >>>>>>>>>>> The CustomLoadee class will be stored in the shared > archive with a > >>>>>>>>>>> CRC > >>>>>>>>>>> code. > >>>>>>>>>>> During runtime, if a customed loader wants to load a > class of the > >>>>>>>>>>> same > >>>>>>>>>>> name, > >>>>>>>>>>> and its classfile has the same size and CRC as the > archived class, > >>>>>>>>>>> the > >>>>>>>>>>> archived version will be loaded. This speeds up class > loading by > >>>>>>>>>>> avoiding > >>>>>>>>>>> parsing the class file, and saves space by sharing the > mmap'ed class > >>>>>>>>>>> metadata across processes. > >>>>>>>>>>> > >>>>>>>>>>> You can see an example test at: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > http://hg.openjdk.java.net/jdk/hs/file/46dc568d6804/test/hotspot/jtreg/runtime/appcds/customLoader/HelloCustom.java > >>>>>>>>>>> > >>>>>>>>>>> However, the current scheme requires you to specify > all the super > >>>>>>>>>>> classes > >>>>>>>>>>> and interfaces. There's no support provided by the > >>>>>>>>>>> -XX:DumpLoadedClassList > >>>>>>>>>>> option. It can be helped somewhat with Volker's tool: > >>>>>>>>>>> https://github.com/simonis/cl4cds > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> POSSIBLE SOLUTIONS: > >>>>>>>>>>> > >>>>>>>>>>> 1. "Dump-as-you-go". As suggested by Yumin, we can > provide a jcmd to > >>>>>>>>>>> ask > >>>>>>>>>>> a > >>>>>>>>>>> running JVM process to dump all of its loaded classes, > including > >>>>>>>>>>> those > >>>>>>>>>>> loaded by custom loaders, into an archive. An > alternative is to dump > >>>>>>>>>>> the > >>>>>>>>>>> archive at JVM exit time (or when you press Ctrl-C, etc. > >>>>>>>>>>> > >>>>>>>>>>> 2. Add information about the custom classes for > >>>>>>>>>>> -XX:DumpLoadedClassList. > >>>>>>>>>>> The > >>>>>>>>>>> trouble is some class loaders don't specify a code > source that can be > >>>>>>>>>>> understood by the built-in class loaders. For example, > the "Fat Jars" > >>>>>>>>>>> would > >>>>>>>>>>> have a code source like > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > jar:file:/jdk/tmp/test-1.0-SNAPSHOT.jar!/BOOT-INF/lib/validation-api-2.0.1.Final.jar!/ > >>>>>>>>>>> > >>>>>>>>>>> also, many custom loaders would pre-process the > classfile data before > >>>>>>>>>>> defining the class, so we can't simply archive the > version of the > >>>>>>>>>>> class > >>>>>>>>>>> on > >>>>>>>>>>> disk. > >>>>>>>>>>> > >>>>>>>>>>> One possible solution for #2 is to include the class > file data in the > >>>>>>>>>>> -XX:DumpLoadedClassList output: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> java/lang/Object id: 1 > >>>>>>>>>>> CustomLoadee id: 2 super: 1 source: base64 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > yv66vgAAADQAFwoABAAQCQAFABEHABIHABMHABQBAAJIaQEADElubmVyQ2xhc3NlcwEABjxpbml0 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > PgEAAygpVgEABENvZGUBAA9MaW5lTnVtYmVyVGFibGUBAAFmAQADKClJAQAKU291cmNlRmlsZQEA > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > CkhlbGxvLmphdmEMAAgACQwAFQAWAQAFSGVsbG8BABBqYXZhL2xhbmcvT2JqZWN0AQAISGVsbG8k > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > SGkBAAFYAQABSQAhAAMABAAAAAAAAgABAAgACQABAAoAAAAdAAEAAQAAAAUqtwABsQAAAAEACwAA > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > AAYAAQAAAC8ACAAMAA0AAQAKAAAAHAABAAAAAAAEsgACrAAAAAEACwAAAAYAAQAAADEAAgAOAAAA > >>>>>>>>>>> AgAPAAcAAAAKAAEABQADAAYACA== > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Of the 2 solutions: > >>>>>>>>>>> > >>>>>>>>>>> #1 seems easier to use, but may require more invasive > modifications > >>>>>>>>>>> in > >>>>>>>>>>> the > >>>>>>>>>>> VM, especially if you want to be able to continue > execution after > >>>>>>>>>>> dumping. > >>>>>>>>>>> > >>>>>>>>>> Not sure what #1 really proposes: dumping the complete > .jsa archive at > >>>>>>>>>> runtime or dumping just the loaded classes. > >>>>>>>>>> > >>>>>>>>>> If it's just about dumping the loaded class without > generating the > >>>>>>>>>> .jsa archive there's the problem that by default the VM > doesn't store > >>>>>>>>>> the exact bytes of a class after the class was loaded > (except when > >>>>>>>>>> class transformers are registered). So the class files > would have to > >>>>>>>>>> be re-assembled from the internal VM structures (in the > same way this > >>>>>>>>>> is done for class redefinition) and the resulting > class-file may be > >>>>>>>>>> different from the original bytes (i.e. some attributes > may be > >>>>>>>>>> missing). > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> #1 is for creating the JSA file, not just dumping the > class files. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> If #1 is about creating the whole .jsa archive at > runtime (or at VM > >>>>>>>>>> exit) I think that would be the most attractive > solution from a > >>>>>>>>>> usability point of view although I understand that #2 > will be easier > >>>>>>>>>> to implement in the short term. Regarding the argument > that #1 will > >>>>>>>>>> produce a "binary blob" that's true, but that's already > true now when > >>>>>>>>>> we use "Xshare:dump". I think it should be not to hard > to implement a > >>>>>>>>>> tool based an SA which could introspect a .jsa archive. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> The argument about the binary blob is to compare it > against the text > >>>>>>>>> file > >>>>>>>>> produced by -XX:DumpLoadedClassList. > >>>>>>>>> > >>>>>>>>> One use case to consider is when you have a JAR file > that contains > >>>>>>>>> several > >>>>>>>>> apps that each load a unique set of classes. Today, > (assuming that > >>>>>>>>> custom > >>>>>>>>> class loaders are not used), you can run each app once with > >>>>>>>>> -XX:DumpLoadedClassList, and then do an > >>>>>>>>> > >>>>>>>>> cat *.classlist | sort | uniq > combined.classlist > >>>>>>>>> > >>>>>>>>> and then create an archive that would work for all these > apps. > >>>>>>>>> > >>>>>>>> > >>>>>>>> But is this really y relevant use case? Why would I like > to create ONE > >>>>>>>> archive for several apps? This would actually increase > the footprint > >>>>>>>> of a single instance which uses this archive. If I have > several apps I > >>>>>>>> would expect that users create a specific archive for > each app to get > >>>>>>>> the best out of CDS. > >>>>>>>> > >>>>>>>>> With the binary blob, there's no easy way of doing this. > It will be > >>>>>>>>> very > >>>>>>>>> difficult to write a tool to decipher each blob and then > somehow > >>>>>>>>> combine > >>>>>>>>> them into a single one. > >>>>>>>>> > >>>>>>>> > >>>>>>>> But if users really wants such a "fat" archive, there's a > much easier > >>>>>>>> way: just dump ALL the classes from the .jar file into > the archive. A > >>>>>>>> class list for this could easily be assembled either with > an external > >>>>>>>> tool like cl4cds (or even a simple shell scripts which > converts the > >>>>>>>> output of `unzip -l ` into the correct format). > Or, even > >>>>>>>> simpler, by adding a new option to the VM similar to > >>>>>>>> -XX:DumpLoadedClassList which dumps all the classes it > can find on the > >>>>>>>> class path (and potentially other, configurable locations). > >>>>>>>> > >>>>>>>>> Thanks > >>>>>>>>> - Ioi > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>>> #2 would be easier to implement, but the classlist > might be huge. > >>>>>>>>>>> > >>>>>>>>>>> Also, #2 would allow post-processing tools to remove > unneeded > >>>>>>>>>>> classes, > >>>>>>>>>>> or > >>>>>>>>>>> merge two runs into a single list. The output of #1 is > essentially a > >>>>>>>>>>> binary > >>>>>>>>>>> blob that's impossible for off-line > analysis/optimizations. > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Any comments, or suggestions for alternatives? > >>>>>>>>>>> > >>>>>>>>>>> Thanks > >>>>>>>>>>> - Ioi > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>> > >>> > >> > > > From dean.long at oracle.com Thu May 10 18:44:13 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 10 May 2018 11:44:13 -0700 Subject: RFR(XS): 8132287: obsolete the "InlineNotify" flag option In-Reply-To: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> References: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> Message-ID: I believe we'll need a fix for Graal so it doesn't break. http://hg.openjdk.java.net/jdk/jdk/file/tip/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java#l47 dl On 5/10/18 6:52 AM, Daniel D. Daugherty wrote: > Greetings, > > This review has been sent to two aliases so please reply to both. > > I'm obsoleting the '-XX:+InlineNotify' option and moving its > functionality > to the experimental '-XX:SyncKnobs' option. This is a leftover cleanup > from > the "Contended Locking fast notify bucket". > > Since this is a diagnostic option, a CSR is not required: > > https://wiki.openjdk.java.net/display/csr/CSR+FAQs > > > Interfaces that are experimental or for diagnostic purposes do not need > > to go through CSR process, but the CSR process may be employed if > feedback > > from the CSR reviewers is desired. > > Webrev URL: > http://cr.openjdk.java.net/~dcubed/8132287-webrev/0_for_jdk_jdk/ > > See the bug for the results of test invocations of the > '-XX:+InlineNotify' > option and the new '-XX:SyncKnobs' suboption. > > Thanks, in advance, for any feedback. > > Dan From daniel.daugherty at oracle.com Thu May 10 18:52:11 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 10 May 2018 14:52:11 -0400 Subject: RFR(XS): 8132287: obsolete the "InlineNotify" flag option In-Reply-To: References: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> Message-ID: <0492b6f5-5afc-fef3-49d4-b6e287d12a5e@oracle.com> Hi Dean! Thanks for reviewing. This is a flag that is default true and I'm not aware of anyone outside of me or Dave Dice that uses it. The flag is not removed (yet), it is simply ignored (with a warning) so what will break in Graal in this situation? I believe the normal Graal protocol is to make changes upstream and they land in jdk/jdk when Vladimir does a sync-down. Not knowing Graal at all, I have no idea if Graal knows about '-XX:SyncKnobs' or even if it should... Dan On 5/10/18 2:44 PM, dean.long at oracle.com wrote: > I believe we'll need a fix for Graal so it doesn't break. > > http://hg.openjdk.java.net/jdk/jdk/file/tip/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java#l47 > > > dl > > On 5/10/18 6:52 AM, Daniel D. Daugherty wrote: >> Greetings, >> >> This review has been sent to two aliases so please reply to both. >> >> I'm obsoleting the '-XX:+InlineNotify' option and moving its >> functionality >> to the experimental '-XX:SyncKnobs' option. This is a leftover >> cleanup from >> the "Contended Locking fast notify bucket". >> >> Since this is a diagnostic option, a CSR is not required: >> >> https://wiki.openjdk.java.net/display/csr/CSR+FAQs >> >> > Interfaces that are experimental or for diagnostic purposes do not >> need >> > to go through CSR process, but the CSR process may be employed if >> feedback >> > from the CSR reviewers is desired. >> >> Webrev URL: >> http://cr.openjdk.java.net/~dcubed/8132287-webrev/0_for_jdk_jdk/ >> >> See the bug for the results of test invocations of the >> '-XX:+InlineNotify' >> option and the new '-XX:SyncKnobs' suboption. >> >> Thanks, in advance, for any feedback. >> >> Dan > From dean.long at oracle.com Thu May 10 19:15:59 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 10 May 2018 12:15:59 -0700 Subject: RFR(XS): 8132287: obsolete the "InlineNotify" flag option In-Reply-To: <0492b6f5-5afc-fef3-49d4-b6e287d12a5e@oracle.com> References: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> <0492b6f5-5afc-fef3-49d4-b6e287d12a5e@oracle.com> Message-ID: <49458504-51e6-e18c-a413-455abf956822@oracle.com> Hi Dan, On 5/10/18 11:52 AM, Daniel D. Daugherty wrote: > Hi Dean! > > Thanks for reviewing. > > This is a flag that is default true and I'm not aware of anyone outside > of me or Dave Dice that uses it. The flag is not removed (yet), it is > simply ignored (with a warning) so what will break in Graal in this > situation? > Where is it set to true?? It looks like it is being removed from globals.hpp. > I believe the normal Graal protocol is to make changes upstream and they > land in jdk/jdk when Vladimir does a sync-down. Not knowing Graal at all, > I have no idea if Graal knows about '-XX:SyncKnobs' or even if it > should... > To prevent breakage we can push the change to jdk/jdk first.? Let me test your change out with Graal and get back to you. dl > Dan > > > On 5/10/18 2:44 PM, dean.long at oracle.com wrote: >> I believe we'll need a fix for Graal so it doesn't break. >> >> http://hg.openjdk.java.net/jdk/jdk/file/tip/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java#l47 >> >> >> dl >> >> On 5/10/18 6:52 AM, Daniel D. Daugherty wrote: >>> Greetings, >>> >>> This review has been sent to two aliases so please reply to both. >>> >>> I'm obsoleting the '-XX:+InlineNotify' option and moving its >>> functionality >>> to the experimental '-XX:SyncKnobs' option. This is a leftover >>> cleanup from >>> the "Contended Locking fast notify bucket". >>> >>> Since this is a diagnostic option, a CSR is not required: >>> >>> https://wiki.openjdk.java.net/display/csr/CSR+FAQs >>> >>> > Interfaces that are experimental or for diagnostic purposes do not >>> need >>> > to go through CSR process, but the CSR process may be employed if >>> feedback >>> > from the CSR reviewers is desired. >>> >>> Webrev URL: >>> http://cr.openjdk.java.net/~dcubed/8132287-webrev/0_for_jdk_jdk/ >>> >>> See the bug for the results of test invocations of the >>> '-XX:+InlineNotify' >>> option and the new '-XX:SyncKnobs' suboption. >>> >>> Thanks, in advance, for any feedback. >>> >>> Dan >> > From daniel.daugherty at oracle.com Thu May 10 19:16:10 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 10 May 2018 15:16:10 -0400 Subject: RFR(XS): 8132287: obsolete the "InlineNotify" flag option In-Reply-To: <0492b6f5-5afc-fef3-49d4-b6e287d12a5e@oracle.com> References: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> <0492b6f5-5afc-fef3-49d4-b6e287d12a5e@oracle.com> Message-ID: <34c8178b-57cc-63c9-54e4-1e2cca87d757@oracle.com> Answering my own question: Exception in thread "main" jdk.vm.ci.common.JVMCIError: expected VM flag not found: InlineNotify ??? at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotVMConfigAccess.getFlag(HotSpotVMConfigAccess.java:288) ??? at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotVMConfigAccess.getFlag(HotSpotVMConfigAccess.java:264) ??? at jdk.internal.vm.compiler at 11-internal/org.graalvm.compiler.hotspot.GraalHotSpotVMConfigVersioned.(GraalHotSpotVMConfigVersioned.java:47) ??? at jdk.internal.vm.compiler at 11-internal/org.graalvm.compiler.hotspot.GraalHotSpotVMConfigBase.(GraalHotSpotVMConfigBase.java:40) ??? at jdk.internal.vm.compiler at 11-internal/org.graalvm.compiler.hotspot.GraalHotSpotVMConfig.(GraalHotSpotVMConfig.java:40) ??? at jdk.internal.vm.compiler at 11-internal/org.graalvm.compiler.hotspot.AOTGraalHotSpotVMConfig.(AOTGraalHotSpotVMConfig.java:34) ??? at jdk.internal.vm.compiler at 11-internal/org.graalvm.compiler.hotspot.HotSpotGraalRuntime.(HotSpotGraalRuntime.java:138) ??? at jdk.internal.vm.compiler at 11-internal/org.graalvm.compiler.hotspot.HotSpotGraalCompilerFactory.createCompiler(HotSpotGraalCompilerFactory.java:143) ??? at jdk.aot/jdk.tools.jaotc.Main.run(Main.java:144) ??? at jdk.aot/jdk.tools.jaotc.Main.run(Main.java:101) ??? at jdk.aot/jdk.tools.jaotc.Main.main(Main.java:80) Tier1 test failures! Vladimir or Dean, I have no idea how to fix this (yet). Dan On 5/10/18 2:52 PM, Daniel D. Daugherty wrote: > Hi Dean! > > Thanks for reviewing. > > This is a flag that is default true and I'm not aware of anyone outside > of me or Dave Dice that uses it. The flag is not removed (yet), it is > simply ignored (with a warning) so what will break in Graal in this > situation? > > I believe the normal Graal protocol is to make changes upstream and they > land in jdk/jdk when Vladimir does a sync-down. Not knowing Graal at all, > I have no idea if Graal knows about '-XX:SyncKnobs' or even if it > should... > > Dan > > > On 5/10/18 2:44 PM, dean.long at oracle.com wrote: >> I believe we'll need a fix for Graal so it doesn't break. >> >> http://hg.openjdk.java.net/jdk/jdk/file/tip/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java#l47 >> >> >> dl >> >> On 5/10/18 6:52 AM, Daniel D. Daugherty wrote: >>> Greetings, >>> >>> This review has been sent to two aliases so please reply to both. >>> >>> I'm obsoleting the '-XX:+InlineNotify' option and moving its >>> functionality >>> to the experimental '-XX:SyncKnobs' option. This is a leftover >>> cleanup from >>> the "Contended Locking fast notify bucket". >>> >>> Since this is a diagnostic option, a CSR is not required: >>> >>> https://wiki.openjdk.java.net/display/csr/CSR+FAQs >>> >>> > Interfaces that are experimental or for diagnostic purposes do not >>> need >>> > to go through CSR process, but the CSR process may be employed if >>> feedback >>> > from the CSR reviewers is desired. >>> >>> Webrev URL: >>> http://cr.openjdk.java.net/~dcubed/8132287-webrev/0_for_jdk_jdk/ >>> >>> See the bug for the results of test invocations of the >>> '-XX:+InlineNotify' >>> option and the new '-XX:SyncKnobs' suboption. >>> >>> Thanks, in advance, for any feedback. >>> >>> Dan >> > > From daniel.daugherty at oracle.com Thu May 10 19:19:39 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 10 May 2018 15:19:39 -0400 Subject: RFR(XS): 8132287: obsolete the "InlineNotify" flag option In-Reply-To: <49458504-51e6-e18c-a413-455abf956822@oracle.com> References: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> <0492b6f5-5afc-fef3-49d4-b6e287d12a5e@oracle.com> <49458504-51e6-e18c-a413-455abf956822@oracle.com> Message-ID: <61158762-8ac4-a7f8-2d72-a90c30bba371@oracle.com> On 5/10/18 3:15 PM, dean.long at oracle.com wrote: > Hi Dan, > > On 5/10/18 11:52 AM, Daniel D. Daugherty wrote: >> Hi Dean! >> >> Thanks for reviewing. >> >> This is a flag that is default true and I'm not aware of anyone outside >> of me or Dave Dice that uses it. The flag is not removed (yet), it is >> simply ignored (with a warning) so what will break in Graal in this >> situation? >> > > Where is it set to true?? It looks like it is being removed from > globals.hpp. I should have been more clear. The flag was default true and, as far as I knew, only Dave Dice and I used it. Now it has been obsoleted and is ignored when specified on the command line. The flag itself no longer exists (which is what is making those AOT tests unhappy in my reply a few minutes ago). > >> I believe the normal Graal protocol is to make changes upstream and they >> land in jdk/jdk when Vladimir does a sync-down. Not knowing Graal at >> all, >> I have no idea if Graal knows about '-XX:SyncKnobs' or even if it >> should... >> > > To prevent breakage we can push the change to jdk/jdk first.? Let me > test your change out with Graal and get back to you. I can include the Graal fix in my push if that works for you... Dan > > dl > >> Dan >> >> >> On 5/10/18 2:44 PM, dean.long at oracle.com wrote: >>> I believe we'll need a fix for Graal so it doesn't break. >>> >>> http://hg.openjdk.java.net/jdk/jdk/file/tip/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java#l47 >>> >>> >>> dl >>> >>> On 5/10/18 6:52 AM, Daniel D. Daugherty wrote: >>>> Greetings, >>>> >>>> This review has been sent to two aliases so please reply to both. >>>> >>>> I'm obsoleting the '-XX:+InlineNotify' option and moving its >>>> functionality >>>> to the experimental '-XX:SyncKnobs' option. This is a leftover >>>> cleanup from >>>> the "Contended Locking fast notify bucket". >>>> >>>> Since this is a diagnostic option, a CSR is not required: >>>> >>>> https://wiki.openjdk.java.net/display/csr/CSR+FAQs >>>> >>>> > Interfaces that are experimental or for diagnostic purposes do >>>> not need >>>> > to go through CSR process, but the CSR process may be employed if >>>> feedback >>>> > from the CSR reviewers is desired. >>>> >>>> Webrev URL: >>>> http://cr.openjdk.java.net/~dcubed/8132287-webrev/0_for_jdk_jdk/ >>>> >>>> See the bug for the results of test invocations of the >>>> '-XX:+InlineNotify' >>>> option and the new '-XX:SyncKnobs' suboption. >>>> >>>> Thanks, in advance, for any feedback. >>>> >>>> Dan >>> >> > From dean.long at oracle.com Thu May 10 19:52:14 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 10 May 2018 12:52:14 -0700 Subject: RFR(XS): 8132287: obsolete the "InlineNotify" flag option In-Reply-To: <61158762-8ac4-a7f8-2d72-a90c30bba371@oracle.com> References: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> <0492b6f5-5afc-fef3-49d4-b6e287d12a5e@oracle.com> <49458504-51e6-e18c-a413-455abf956822@oracle.com> <61158762-8ac4-a7f8-2d72-a90c30bba371@oracle.com> Message-ID: On 5/10/18 12:19 PM, Daniel D. Daugherty wrote: > I can include the Graal fix in my push if that works for you... Yes, please do.? This should do it: diff -r 0509b9be3bda src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java --- a/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java Thu May 10 11:45:40 2018 -0700 +++ b/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java Thu May 10 12:50:48 2018 -0700 @@ -44,7 +44,7 @@ ???? final boolean useCRC32CIntrinsics = getFlag("UseCRC32CIntrinsics", Boolean.class); ???? // JDK-8075171 -??? final boolean inlineNotify = getFlag("InlineNotify", Boolean.class); +??? final boolean inlineNotify = getFlag("InlineNotify", Boolean.class, true); ???? // JDK-8046936 ???? final int javaThreadReservedStackActivationOffset = getFieldOffset("JavaThread::_reserved_stack_activation", Integer.class, "address"); dl From daniel.daugherty at oracle.com Thu May 10 19:55:05 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 10 May 2018 15:55:05 -0400 Subject: RFR(XS): 8132287: obsolete the "InlineNotify" flag option In-Reply-To: References: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> <0492b6f5-5afc-fef3-49d4-b6e287d12a5e@oracle.com> <49458504-51e6-e18c-a413-455abf956822@oracle.com> <61158762-8ac4-a7f8-2d72-a90c30bba371@oracle.com> Message-ID: <1ffa819b-fdf4-5710-e7a0-c4299cbf1680@oracle.com> On 5/10/18 3:52 PM, dean.long at oracle.com wrote: > On 5/10/18 12:19 PM, Daniel D. Daugherty wrote: >> I can include the Graal fix in my push if that works for you... > > Yes, please do.? This should do it: Thanks! I'll take that change for a Mach5 spin in a few minutes... Dan > > diff -r 0509b9be3bda > src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java > --- > a/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java > Thu May 10 11:45:40 2018 -0700 > +++ > b/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java > Thu May 10 12:50:48 2018 -0700 > @@ -44,7 +44,7 @@ > ???? final boolean useCRC32CIntrinsics = > getFlag("UseCRC32CIntrinsics", Boolean.class); > > ???? // JDK-8075171 > -??? final boolean inlineNotify = getFlag("InlineNotify", Boolean.class); > +??? final boolean inlineNotify = getFlag("InlineNotify", > Boolean.class, true); > > ???? // JDK-8046936 > ???? final int javaThreadReservedStackActivationOffset = > getFieldOffset("JavaThread::_reserved_stack_activation", > Integer.class, "address"); > > > dl From yumin.qi at gmail.com Thu May 10 20:20:21 2018 From: yumin.qi at gmail.com (yumin qi) Date: Thu, 10 May 2018 13:20:21 -0700 Subject: Improving AppCDS for Custom Loaders In-Reply-To: <16902EC1-7A5F-4DBC-A815-855BA04509BA@oracle.com> References: <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> <732ce957-b07c-bff2-277b-ca335c348753@oracle.com> <660A1644-3A08-455D-931D-D95069C39853@oracle.com> <16902EC1-7A5F-4DBC-A815-855BA04509BA@oracle.com> Message-ID: Hi, Jiangli Thanks for the info. On Thu, May 10, 2018 at 11:39 AM, Jiangli Zhou wrote: > Hi Yumin and Volker, > > I?ve create an umbrella RFE JDK-8202854 to keep track this effort. New > sub-RFEs will be created and targeted for specific JDK releases when each > sub-item matures. Please refer to JDK-8202854 for future progress. > > Yumin, please see below for comments/questions. > > > Agree not including objects in java heap at now --- thanks Jiangli for > answering my question in other thread. > I conclude from above that you are not using java heap object archiving due > to different GC algorithm being used(?). I would like to understand more > about your GC choices. Could you please provide more information on that? > G1 GC is the default GC in JDK. Thanks to our GC team, the performance of > G1 has been consistently improving over the recent JDK releases, especially > in JDK 9 and 10. I strongly encourage you to do performance > measurement/comparison and reevaluate your GC choice if G1 is not used. > Most of our applications still using CMS, we are pushing to switch to G1GC at this time but still most of them still with CMS. I know CMS will be removed so it will not be your focus. > > > I think for #1, it does not conflict with the two layer archive > solution. We can skip the classes from base CDS archive, only dump the > non-base loaded classes into second archive, this gives one more option for > user to choose. Also it will be good after dump archive to let the > application continue to run. > > > > Can we do a Full GC before dump at exit? it may unload not referenced > classes, or it is not necessary since CDS wants to resolve the startup > performance for all loaded classes? > > For classes loaded by the builtin loaders, they can be kept alive when > archiving phase start. For classes loaded by user defined class loaders, we > also need to prevent them from being unloaded before archiving. I have a > preliminary idea for how to do that. I can share more information on that > when it matures. > > Yes, if classes with custom class loader cleaned, the class loader itself will be purged. Full GC before exit may not be a good idea, but we can skip unloading classes --- in case dump objects in java heap. Thanks Yumin > > From christoph.langer at sap.com Thu May 10 21:04:36 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 10 May 2018 21:04:36 +0000 Subject: RFR: 8202889: Remove trailing LF from perf log In-Reply-To: References: Message-ID: <12c07097e9a24449ab551b2c8c35b6f2@sap.com> +1 > -----Original Message----- > From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- > bounces at openjdk.java.net] On Behalf Of Yasumasa Suenaga > Sent: Donnerstag, 10. Mai 2018 07:43 > To: Thomas St?fe > Cc: Hotspot dev runtime > Subject: Re: RFR: 8202889: Remove trailing LF from perf log > > Hi Thomas, > > Thank you for your review! > I will update copyright year in all files. > > I'm waiting for second reviewer. > > > Thanks, > > Yasumasa > > > > 2018-05-10 14:19 GMT+09:00 Thomas St?fe : > > Hi Yasumasa, > > > > seems fine and trivial. Please update copyright years (in case of aix > > specific files, please both SAP and Oracle). I do not need a new > > webrev. > > > > Thanks, Thomas > > > > On Thu, May 10, 2018 at 6:13 AM, Yasumasa Suenaga > wrote: > >> Hi all, > >> > >> Could you review this change? > >> > >> JBS: https://bugs.openjdk.java.net/browse/JDK-8202889 > >> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8202889/webrev.00/ > >> > >> > >> JDK-8168122 introduced new UL tags for perfMemory. However they > trails > >> unnecessary LF in each log entries. > >> > >> Examples are here: > >> http://cr.openjdk.java.net/~ysuenaga/openjdk-fc28/trace-all.log > >> > >> This change removes unnecessary LFs. > >> > >> > >> Thanks, > >> > >> Yasumasa From daniel.daugherty at oracle.com Thu May 10 21:24:59 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 10 May 2018 17:24:59 -0400 Subject: RFR(XS): 8132287: obsolete the "InlineNotify" flag option In-Reply-To: <1ffa819b-fdf4-5710-e7a0-c4299cbf1680@oracle.com> References: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> <0492b6f5-5afc-fef3-49d4-b6e287d12a5e@oracle.com> <49458504-51e6-e18c-a413-455abf956822@oracle.com> <61158762-8ac4-a7f8-2d72-a90c30bba371@oracle.com> <1ffa819b-fdf4-5710-e7a0-c4299cbf1680@oracle.com> Message-ID: <933425f4-1f72-c532-5095-31a5411309ea@oracle.com> A Mach5 builds-tier1,hs-tier1,hs-tier2 job with the following additional changes (thanks Vladimir and Dean!) passes... Still waiting for Runtime team review... $ hg diff -r 8132287.cr0 diff -r cbeb21e5a2c6 src/hotspot/share/opto/library_call.cpp --- a/src/hotspot/share/opto/library_call.cpp??? Thu May 10 09:37:27 2018 -0400 +++ b/src/hotspot/share/opto/library_call.cpp??? Thu May 10 17:23:08 2018 -0400 @@ -52,6 +52,7 @@ ?#include "opto/subnode.hpp" ?#include "prims/nativeLookup.hpp" ?#include "prims/unsafe.hpp" +#include "runtime/objectMonitor.hpp" ?#include "runtime/sharedRuntime.hpp" ?#ifdef TRACE_HAVE_INTRINSICS ?#include "trace/traceMacros.hpp" diff -r cbeb21e5a2c6 src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java --- a/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java Thu May 10 09:37:27 2018 -0400 +++ b/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java Thu May 10 17:23:08 2018 -0400 @@ -44,7 +44,7 @@ ???? final boolean useCRC32CIntrinsics = getFlag("UseCRC32CIntrinsics", Boolean.class); ???? // JDK-8075171 -??? final boolean inlineNotify = getFlag("InlineNotify", Boolean.class); +??? final boolean inlineNotify = getFlag("InlineNotify", Boolean.class, true); ???? // JDK-8046936 ???? final int javaThreadReservedStackActivationOffset = getFieldOffset("JavaThread::_reserved_stack_activation", Integer.class, "address"); On 5/10/18 3:55 PM, Daniel D. Daugherty wrote: > On 5/10/18 3:52 PM, dean.long at oracle.com wrote: >> On 5/10/18 12:19 PM, Daniel D. Daugherty wrote: >>> I can include the Graal fix in my push if that works for you... >> >> Yes, please do.? This should do it: > > Thanks! I'll take that change for a Mach5 spin in a few minutes... > > Dan > > >> >> diff -r 0509b9be3bda >> src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >> --- >> a/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >> Thu May 10 11:45:40 2018 -0700 >> +++ >> b/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >> Thu May 10 12:50:48 2018 -0700 >> @@ -44,7 +44,7 @@ >> ???? final boolean useCRC32CIntrinsics = >> getFlag("UseCRC32CIntrinsics", Boolean.class); >> >> ???? // JDK-8075171 >> -??? final boolean inlineNotify = getFlag("InlineNotify", >> Boolean.class); >> +??? final boolean inlineNotify = getFlag("InlineNotify", >> Boolean.class, true); >> >> ???? // JDK-8046936 >> ???? final int javaThreadReservedStackActivationOffset = >> getFieldOffset("JavaThread::_reserved_stack_activation", >> Integer.class, "address"); >> >> >> dl > > From mikhailo.seledtsov at oracle.com Thu May 10 21:39:53 2018 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Thu, 10 May 2018 14:39:53 -0700 Subject: RFR(M): 8199257: [TESTBUG] Open source VM testbase metaspace tests In-Reply-To: References: Message-ID: <5AF4BC29.6080806@oracle.com> Hi Thomas, On 5/9/18, 9:01 AM, Thomas St?fe wrote: > Not a review, but I ran these tests on my local machine (with my > current metaspace changes) and they all passed. Thank you. > > Will these tests be somehow merged with the existing metaspace tests > at runtime/Metaspace? We can consider this in the future. We can either collect them into the same test group, or consider migrating the tests into a different location/directory. > > I think the general question is what is vmTestBase, and what sets the > tests in there apart from the existing tests outside vmTestBase? VM Testbase is a collection of tests used for testing HotSpot. There are several things that set them apart from other JTReg HotSpot tests, such as certain common conventions (coding conventions, keywords) and common test libraries. There is a bit more info in this README file: http://hg.openjdk.java.net/jdk/jdk/file/tip/test/hotspot/jtreg/vmTestbase/README.md I hope I have answered your questions, Thank you, Misha > > Thanks, Thomas > > > On Tue, May 8, 2018 at 7:38 PM, mikhailo wrote: >> http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html >> >> Please review this change open sourcing VM metaspace tests. These tests have >> been used internally for a while, and are now being open sourced. Since this >> is not an creation of new tests, we would like to keep the changes during >> this review to an absolute required minimum. If you have any feedback on >> improvements of these tests, please file RFE(s) that will be addressed later >> in order of priority. >> >> >> Here is what was done for this change: >> 1. Moved the tests to OpenJDK repository to the specified directory >> location and structure. >> 3. Updated Copyright statements accordingly. >> 4. Updated "@library" statements accordingly. >> 5. Updated TEST.groups and ProblemList.txt >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8199257 >> Webrev: http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html >> >> Testing: >> 1. Ran the following tests on open-only repository and build, using >> "make run-test" (Linux-x64) >> vmTestbase_vm_metaspace, vmTestbase_largepages >> All PASS >> >> 2. Automated multip-platform test system (usual 4 platforms): >> - vmTestbase_vm_metaspace, vmTestbase_largepages >> - hs-tier{1,2,3} >> All PASS >> >> Thank you, >> Misha >> From david.holmes at oracle.com Thu May 10 21:48:33 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 11 May 2018 07:48:33 +1000 Subject: RFR(XS): 8132287: obsolete the "InlineNotify" flag option In-Reply-To: <933425f4-1f72-c532-5095-31a5411309ea@oracle.com> References: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> <0492b6f5-5afc-fef3-49d4-b6e287d12a5e@oracle.com> <49458504-51e6-e18c-a413-455abf956822@oracle.com> <61158762-8ac4-a7f8-2d72-a90c30bba371@oracle.com> <1ffa819b-fdf4-5710-e7a0-c4299cbf1680@oracle.com> <933425f4-1f72-c532-5095-31a5411309ea@oracle.com> Message-ID: Hi Dan, On 11/05/2018 7:24 AM, Daniel D. Daugherty wrote: > A Mach5 builds-tier1,hs-tier1,hs-tier2 job with the following > additional changes (thanks Vladimir and Dean!) passes... > > Still waiting for Runtime team review... Runtime Reviewed. > $ hg diff -r 8132287.cr0 > diff -r cbeb21e5a2c6 src/hotspot/share/opto/library_call.cpp > --- a/src/hotspot/share/opto/library_call.cpp??? Thu May 10 09:37:27 > 2018 -0400 > +++ b/src/hotspot/share/opto/library_call.cpp??? Thu May 10 17:23:08 > 2018 -0400 > @@ -52,6 +52,7 @@ > ?#include "opto/subnode.hpp" > ?#include "prims/nativeLookup.hpp" > ?#include "prims/unsafe.hpp" > +#include "runtime/objectMonitor.hpp" > ?#include "runtime/sharedRuntime.hpp" > ?#ifdef TRACE_HAVE_INTRINSICS > ?#include "trace/traceMacros.hpp" > diff -r cbeb21e5a2c6 > src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java > > --- > a/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java > Thu May 10 09:37:27 2018 -0400 > +++ > b/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java > Thu May 10 17:23:08 2018 -0400 > @@ -44,7 +44,7 @@ > ???? final boolean useCRC32CIntrinsics = getFlag("UseCRC32CIntrinsics", > Boolean.class); > > ???? // JDK-8075171 > -??? final boolean inlineNotify = getFlag("InlineNotify", Boolean.class); > +??? final boolean inlineNotify = getFlag("InlineNotify", Boolean.class, > true); I hope a follow up bug has been filed so that this usage is handled correctly now that the flag no longer exists. Thanks, David ------ > ???? // JDK-8046936 > ???? final int javaThreadReservedStackActivationOffset = > getFieldOffset("JavaThread::_reserved_stack_activation", Integer.class, > "address"); > > On 5/10/18 3:55 PM, Daniel D. Daugherty wrote: >> On 5/10/18 3:52 PM, dean.long at oracle.com wrote: >>> On 5/10/18 12:19 PM, Daniel D. Daugherty wrote: >>>> I can include the Graal fix in my push if that works for you... >>> >>> Yes, please do.? This should do it: >> >> Thanks! I'll take that change for a Mach5 spin in a few minutes... >> >> Dan >> >> >>> >>> diff -r 0509b9be3bda >>> src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>> >>> --- >>> a/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>> Thu May 10 11:45:40 2018 -0700 >>> +++ >>> b/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>> Thu May 10 12:50:48 2018 -0700 >>> @@ -44,7 +44,7 @@ >>> ???? final boolean useCRC32CIntrinsics = >>> getFlag("UseCRC32CIntrinsics", Boolean.class); >>> >>> ???? // JDK-8075171 >>> -??? final boolean inlineNotify = getFlag("InlineNotify", >>> Boolean.class); >>> +??? final boolean inlineNotify = getFlag("InlineNotify", >>> Boolean.class, true); >>> >>> ???? // JDK-8046936 >>> ???? final int javaThreadReservedStackActivationOffset = >>> getFieldOffset("JavaThread::_reserved_stack_activation", >>> Integer.class, "address"); >>> >>> >>> dl >> >> > From volker.simonis at gmail.com Thu May 10 22:34:37 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 10 May 2018 22:34:37 +0000 Subject: Improving AppCDS for Custom Loaders In-Reply-To: References: <2839906b-3147-e8ee-7773-f421997bf192@oracle.com> <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> <732ce957-b07c-bff2-277b-ca335c348753@oracle.com> <660A1644-3A08-455D-931D-D95069C39853@oracle.com> <16902EC1-7A5F-4DBC-A815-855BA04509BA@oracle.com> Message-ID: yumin qi schrieb am Do. 10. Mai 2018 um 22:20: > Hi, Jiangli > > Thanks for the info. > > On Thu, May 10, 2018 at 11:39 AM, Jiangli Zhou > wrote: > >> Hi Yumin and Volker, >> >> I?ve create an umbrella RFE JDK-8202854 to keep track this effort. New >> sub-RFEs will be created and targeted for specific JDK releases when each >> sub-item matures. Please refer to JDK-8202854 for future progress. >> >> Yumin, please see below for comments/questions. >> >> > Agree not including objects in java heap at now --- thanks Jiangli for >> answering my question in other thread. >> > I conclude from above that you are not using java heap object archiving >> due to different GC algorithm being used(?). I would like to understand >> more about your GC choices. Could you please provide more information on >> that? G1 GC is the default GC in JDK. Thanks to our GC team, the >> performance of G1 has been consistently improving over the recent JDK >> releases, especially in JDK 9 and 10. I strongly encourage you to do >> performance measurement/comparison and reevaluate your GC choice if G1 is >> not used. >> > > Most of our applications still using CMS, we are pushing to switch to > G1GC at this time but still most of them still with CMS. I know CMS will be > removed so it will not be your focus. > >> >> > I think for #1, it does not conflict with the two layer archive >> solution. We can skip the classes from base CDS archive, only dump the >> non-base loaded classes into second archive, this gives one more option for >> user to choose. Also it will be good after dump archive to let the >> application continue to run. >> > >> > Can we do a Full GC before dump at exit? it may unload not referenced >> classes, or it is not necessary since CDS wants to resolve the startup >> performance for all loaded classes? >> >> For classes loaded by the builtin loaders, they can be kept alive when >> archiving phase start. For classes loaded by user defined class loaders, we >> also need to prevent them from being unloaded before archiving. I have a >> preliminary idea for how to do that. I can share more information on that >> when it matures. >> >> > Yes, if classes with custom class loader cleaned, the class loader itself > will be purged. Full GC before exit may not be a good idea, but we can skip > unloading classes --- in case dump objects in java heap. > > Why not dump the classes iteratively, while they are loaded. Class loading time would be the most natural time for dumping a class, and it?s the time when we know the most about a class (i.e. full class bytes). > Thanks > Yumin > >> >> From yasuenag at gmail.com Thu May 10 22:47:22 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 10 May 2018 22:47:22 +0000 Subject: RFR: 8202889: Remove trailing LF from perf log In-Reply-To: <12c07097e9a24449ab551b2c8c35b6f2@sap.com> References: <12c07097e9a24449ab551b2c8c35b6f2@sap.com> Message-ID: Thanks! 2018?5?11?(?) 6:04 Langer, Christoph : > +1 > > > -----Original Message----- > > From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- > > bounces at openjdk.java.net] On Behalf Of Yasumasa Suenaga > > Sent: Donnerstag, 10. Mai 2018 07:43 > > To: Thomas St?fe > > Cc: Hotspot dev runtime > > Subject: Re: RFR: 8202889: Remove trailing LF from perf log > > > > Hi Thomas, > > > > Thank you for your review! > > I will update copyright year in all files. > > > > I'm waiting for second reviewer. > > > > > > Thanks, > > > > Yasumasa > > > > > > > > 2018-05-10 14:19 GMT+09:00 Thomas St?fe : > > > Hi Yasumasa, > > > > > > seems fine and trivial. Please update copyright years (in case of aix > > > specific files, please both SAP and Oracle). I do not need a new > > > webrev. > > > > > > Thanks, Thomas > > > > > > On Thu, May 10, 2018 at 6:13 AM, Yasumasa Suenaga > > wrote: > > >> Hi all, > > >> > > >> Could you review this change? > > >> > > >> JBS: https://bugs.openjdk.java.net/browse/JDK-8202889 > > >> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8202889/webrev.00/ > > >> > > >> > > >> JDK-8168122 introduced new UL tags for perfMemory. However they > > trails > > >> unnecessary LF in each log entries. > > >> > > >> Examples are here: > > >> http://cr.openjdk.java.net/~ysuenaga/openjdk-fc28/trace-all.log > > >> > > >> This change removes unnecessary LFs. > > >> > > >> > > >> Thanks, > > >> > > >> Yasumasa > From ioi.lam at oracle.com Thu May 10 23:05:18 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 10 May 2018 16:05:18 -0700 Subject: Improving AppCDS for Custom Loaders In-Reply-To: References: <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> <732ce957-b07c-bff2-277b-ca335c348753@oracle.com> <660A1644-3A08-455D-931D-D95069C39853@oracle.com> <16902EC1-7A5F-4DBC-A815-855BA04509BA@oracle.com> Message-ID: <32beda94-2187-7b80-a933-ca42dddf63ca@oracle.com> On 5/10/18 3:34 PM, Volker Simonis wrote: > > yumin qi > schrieb am > Do. 10. Mai 2018 um 22:20: > > Hi, Jiangli > > ? Thanks for the info. > > On Thu, May 10, 2018 at 11:39 AM, Jiangli Zhou > > wrote: > > Hi Yumin and Volker, > > I?ve create an umbrella RFE JDK-8202854 to keep track this > effort. New sub-RFEs will be created and targeted for specific > JDK releases when each sub-item matures. Please refer to > JDK-8202854 for future progress. > > Yumin, please see below for comments/questions. > > > Agree not including objects in java heap at now --- thanks Jiangli for answering my question > in other thread. > > I conclude from above that you are not using java heap object > archiving due to different GC algorithm being used(?).? I > would like to understand more about your GC choices. Could you > please provide more information on that? G1 GC is the default > GC in JDK. Thanks to our GC team, the performance of G1 has > been consistently improving over the recent JDK releases, > especially in JDK 9 and 10. I strongly encourage you to do > performance measurement/comparison and reevaluate your GC > choice if G1 is not used. > > > Most of our applications still using CMS, we are pushing to switch > to G1GC at this time but still most of them still with CMS. I know > CMS will be removed so it will not be your focus. > > > >? ?I think for #1, it does not conflict with the two layer > archive solution. We can skip the classes from base CDS > archive, only dump the non-base loaded classes into second > archive, this gives one more option for user to choose. Also > it will be good after dump archive to let the application > continue to run. > > > >? ?Can we do a Full GC before dump at exit? it may unload not > referenced classes, or it is not necessary since CDS wants to > resolve the startup performance for all loaded classes? > > For classes loaded by the builtin loaders, they can be kept > alive when archiving phase start. For classes loaded by user > defined class loaders, we also need to prevent them from being > unloaded before archiving. I have a preliminary idea for how > to do that. I can share more information on that when it matures. > > > Yes, if classes with custom class loader cleaned, the class loader > itself will be purged. Full GC before exit may not be a good idea, > but we can skip unloading classes --- in case dump objects in java > heap. > > > Why not dump the classes iteratively, while they are loaded. Class > loading time would be the most natural time for dumping a class, and > it?s the time when we know the most about a class (i.e. full class bytes). > The archive is divided into RO/RW sections. Mutable metadata such as InstanceKlasses are stored in RW, while non-mutable ones such as ConstMethod and Symbol are stored into the RO section. In order to have a single section each for RW and RO, currently we load all the classes first, and then calculate the sizes of the 2 sections. > Thanks > Yumin > > From jiangli.zhou at Oracle.COM Thu May 10 23:11:36 2018 From: jiangli.zhou at Oracle.COM (Jiangli Zhou) Date: Thu, 10 May 2018 16:11:36 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check Message-ID: Hi, Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation & discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 The webrev includes the following three main parts: 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. Thanks, Jiangli From daniel.daugherty at oracle.com Thu May 10 23:21:10 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 10 May 2018 19:21:10 -0400 Subject: RFR(XS): 8132287: obsolete the "InlineNotify" flag option In-Reply-To: References: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> <0492b6f5-5afc-fef3-49d4-b6e287d12a5e@oracle.com> <49458504-51e6-e18c-a413-455abf956822@oracle.com> <61158762-8ac4-a7f8-2d72-a90c30bba371@oracle.com> <1ffa819b-fdf4-5710-e7a0-c4299cbf1680@oracle.com> <933425f4-1f72-c532-5095-31a5411309ea@oracle.com> Message-ID: <1a5bb68b-b447-9e0f-d009-e5420f89a339@oracle.com> On 5/10/18 5:48 PM, David Holmes wrote: > Hi Dan, > > On 11/05/2018 7:24 AM, Daniel D. Daugherty wrote: >> A Mach5 builds-tier1,hs-tier1,hs-tier2 job with the following >> additional changes (thanks Vladimir and Dean!) passes... >> >> Still waiting for Runtime team review... > > Runtime Reviewed. Thanks for the review. > >> $ hg diff -r 8132287.cr0 >> diff -r cbeb21e5a2c6 src/hotspot/share/opto/library_call.cpp >> --- a/src/hotspot/share/opto/library_call.cpp??? Thu May 10 09:37:27 >> 2018 -0400 >> +++ b/src/hotspot/share/opto/library_call.cpp??? Thu May 10 17:23:08 >> 2018 -0400 >> @@ -52,6 +52,7 @@ >> ??#include "opto/subnode.hpp" >> ??#include "prims/nativeLookup.hpp" >> ??#include "prims/unsafe.hpp" >> +#include "runtime/objectMonitor.hpp" >> ??#include "runtime/sharedRuntime.hpp" >> ??#ifdef TRACE_HAVE_INTRINSICS >> ??#include "trace/traceMacros.hpp" >> diff -r cbeb21e5a2c6 >> src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >> >> --- >> a/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >> Thu May 10 09:37:27 2018 -0400 >> +++ >> b/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >> Thu May 10 17:23:08 2018 -0400 >> @@ -44,7 +44,7 @@ >> ????? final boolean useCRC32CIntrinsics = >> getFlag("UseCRC32CIntrinsics", Boolean.class); >> >> ????? // JDK-8075171 >> -??? final boolean inlineNotify = getFlag("InlineNotify", >> Boolean.class); >> +??? final boolean inlineNotify = getFlag("InlineNotify", >> Boolean.class, true); > > I hope a follow up bug has been filed so that this usage is handled > correctly now that the flag no longer exists. I'll figure that out with Dean. Dan > > Thanks, > David > ------ > >> ????? // JDK-8046936 >> ????? final int javaThreadReservedStackActivationOffset = >> getFieldOffset("JavaThread::_reserved_stack_activation", >> Integer.class, "address"); >> >> On 5/10/18 3:55 PM, Daniel D. Daugherty wrote: >>> On 5/10/18 3:52 PM, dean.long at oracle.com wrote: >>>> On 5/10/18 12:19 PM, Daniel D. Daugherty wrote: >>>>> I can include the Graal fix in my push if that works for you... >>>> >>>> Yes, please do.? This should do it: >>> >>> Thanks! I'll take that change for a Mach5 spin in a few minutes... >>> >>> Dan >>> >>> >>>> >>>> diff -r 0509b9be3bda >>>> src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>>> >>>> --- >>>> a/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>>> Thu May 10 11:45:40 2018 -0700 >>>> +++ >>>> b/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>>> Thu May 10 12:50:48 2018 -0700 >>>> @@ -44,7 +44,7 @@ >>>> ???? final boolean useCRC32CIntrinsics = >>>> getFlag("UseCRC32CIntrinsics", Boolean.class); >>>> >>>> ???? // JDK-8075171 >>>> -??? final boolean inlineNotify = getFlag("InlineNotify", >>>> Boolean.class); >>>> +??? final boolean inlineNotify = getFlag("InlineNotify", >>>> Boolean.class, true); >>>> >>>> ???? // JDK-8046936 >>>> ???? final int javaThreadReservedStackActivationOffset = >>>> getFieldOffset("JavaThread::_reserved_stack_activation", >>>> Integer.class, "address"); >>>> >>>> >>>> dl >>> >>> >> From thomas.stuefe at gmail.com Fri May 11 05:46:57 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 11 May 2018 07:46:57 +0200 Subject: RFR(M): 8199257: [TESTBUG] Open source VM testbase metaspace tests In-Reply-To: <5AF4BC29.6080806@oracle.com> References: <5AF4BC29.6080806@oracle.com> Message-ID: Hi Mikhailo, On Thu, May 10, 2018 at 11:39 PM, Mikhailo Seledtsov wrote: > Hi Thomas, > > On 5/9/18, 9:01 AM, Thomas St?fe wrote: >> >> Not a review, but I ran these tests on my local machine (with my >> current metaspace changes) and they all passed. > > Thank you. >> >> >> Will these tests be somehow merged with the existing metaspace tests >> at runtime/Metaspace? > > We can consider this in the future. We can either collect them into the same > test group, or consider migrating the tests into a different > location/directory. >> >> >> I think the general question is what is vmTestBase, and what sets the >> tests in there apart from the existing tests outside vmTestBase? > > VM Testbase is a collection of tests used for testing HotSpot. There are > several things that set them apart from other JTReg HotSpot tests, such as > certain common conventions (coding conventions, keywords) and common test > libraries. There is a bit more info in this README file: > http://hg.openjdk.java.net/jdk/jdk/file/tip/test/hotspot/jtreg/vmTestbase/README.md > > > I hope I have answered your questions, > Yes they did. Thanks for answering. Good to have the tests as open source. Best, Thomas > Thank you, > Misha > >> >> Thanks, Thomas >> >> >> On Tue, May 8, 2018 at 7:38 PM, mikhailo >> wrote: >>> >>> http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html >>> >>> Please review this change open sourcing VM metaspace tests. These tests >>> have >>> been used internally for a while, and are now being open sourced. Since >>> this >>> is not an creation of new tests, we would like to keep the changes during >>> this review to an absolute required minimum. If you have any feedback on >>> improvements of these tests, please file RFE(s) that will be addressed >>> later >>> in order of priority. >>> >>> >>> Here is what was done for this change: >>> 1. Moved the tests to OpenJDK repository to the specified directory >>> location and structure. >>> 3. Updated Copyright statements accordingly. >>> 4. Updated "@library" statements accordingly. >>> 5. Updated TEST.groups and ProblemList.txt >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8199257 >>> Webrev: >>> http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html >>> >>> Testing: >>> 1. Ran the following tests on open-only repository and build, >>> using >>> "make run-test" (Linux-x64) >>> vmTestbase_vm_metaspace, vmTestbase_largepages >>> All PASS >>> >>> 2. Automated multip-platform test system (usual 4 platforms): >>> - vmTestbase_vm_metaspace, vmTestbase_largepages >>> - hs-tier{1,2,3} >>> All PASS >>> >>> Thank you, >>> Misha >>> > From thomas.stuefe at gmail.com Fri May 11 06:31:40 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 11 May 2018 08:31:40 +0200 Subject: RFR(L): 8176808: Split up metaspace.cpp In-Reply-To: References: Message-ID: All builds are fixed, jdk-submit tests ran through sucessfully. Latest webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace-cpp/webrev.01/webrev/ Only difference to the first webrev is the placement of a single semicolon in occupancyMap.hpp, to satisfy the Solaris compiler. Thanks, Thomas On Wed, May 9, 2018 at 5:08 PM, Thomas St?fe wrote: > Hi all, > > This was a long time coming. Lets cut up this beast :) > > Bug: https://bugs.openjdk.java.net/browse/JDK-8176808 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace-cpp/webrev.00/webrev/ > > This change splits up metaspace.cpp into multiple parts. Note that I > did not change much code (exceptions see below in details) - so this > is mostly moving code around. > > This change follows the scheme tentatively outlined in JDK-8201572: > > - metaspace internal classes go into the sub directory > share/memory/metaspace. Ideally, those should never be accessed from > outside, apart from other metaspace classes and metaspace tests. They > also go into the namespace ::metaspace::internals. > > - metaspace external classes (MetaspaceUtils, ClassLoaderMetaspace, > etc) remain inside share/memory and, for now, remain in the global > namespace. > > Note: the original idea was to move metaspace external classes to > namespace ::metaspace. But I shied away from that, since that would > mean fixing up all usages of these classes either with metaspace:: > scope or with using declarations. I had to make a cut at some point. > > So here is something to decide: > - should we then get rid of the ::metaspace::internals namespace, move > metaspace-internal classes to the enclosing ::metaspace namespace and > leave external classes in the global namespace ? > - or should we follow through (maybe in a future patch): internal > classes in ::metaspace::internal, and move external classes to > ::metaspace ? > > Some more details: > > - the following classes moved out of metaspace.cpp into namespace > "metaspace::internal" and the metaspace sub directory: > MetaDebug, ChunkManager, SpaceManager, BlockFreeList and SmallBlocks, > OccupancyMap, VirtualSpaceNode and VirtualSpaceList, the > PrintCLDMetaspaceInfoClosure. > > - I also moved metachunk.hpp to internals, since class Metachunk is > not used externally. What is used externally is Metablock though, so I > split up metachunk.hpp into metabase.hpp, metablock.hpp and > metachunk.hpp, holding Metabase, Metablock and Metachunk, > respectively. > > - Now we see some ugly seams: metaspace.hpp is supposed to be the > outside API contract, however, it contains implementation details > which should not be there and which manifest now by having to use the > metaspace::internals namespace here and there. If we care, we could > solve this with a bit redesigning. > > - The ChunkManager gtest and the SpaceManager gtest had been > implemented inside metaspace.cpp, since it was not possible to access > the internal classes those tests needed in any other way. Since that > is now possible, I moved the coding out to gtest-land and made them > real gtests (exchanging the asserts for real gtest ASSERTS, among > other things). > Note that there are some tests left in metaspace.cpp > (TestMetaspaceUtilsTest, TestVirtualSpaceNodeTest) - those were no > gtests-in-disguise but were part of -XX:+ExecuteInternalVMTests. I > guess these tests precede the gtest framework? I leave it up to others > to decide what to do with them and to potentially move them out of > metaspace.cpp. > Side note: -XX:+ExecuteInternalVMTests seems to have bitrotted, see > https://bugs.openjdk.java.net/browse/JDK-8202848. Does this get tested > regularly? > > - metaspace.cpp is quite a bit smaller now - from ~5000 loc down to > ~1700 loc. Arguably, one could split out more and clean up more, but I > think this is a good start. > > --- > > I built locally on linux (release, fastdebug with and without pch, > zero) and windows (64, 32bit). jdk-submit tests ran through with a > build error on solaris sparc - I currently try to reproduce that build > error with our very slow solaris machine. > > Thanks, Thomas From jini.george at oracle.com Fri May 11 07:29:08 2018 From: jini.george at oracle.com (Jini George) Date: Fri, 11 May 2018 12:59:08 +0530 Subject: RFR: (S): SA: clhsdb 'where -a' throws Assertion Failure with illegal code 236 when CDS is used In-Reply-To: References: <82c69a0d-02f3-5db8-fedd-d52b00571a55@oracle.com> <41204e1d-11e1-41a9-af90-7cabd12098aa@oracle.com> Message-ID: <466139cc-b2c4-9abd-5f41-d7664291b2f6@oracle.com> Thank you very much, Chris, for taking a look. The bytecodes table initialization in SA's Bytecodes.java mimics the bytecodes table initialization in hotspot's Bytecodes::initialize() from bytecodes.cpp. In bytecodes.cpp, we have the initialization of the _nofast* bytecodes thus: def(_nofast_aload_0 , "nofast_aload_0" , "b" , NULL , T_ILLEGAL, 1, true , _aload_0 ); def(_nofast_iload , "nofast_iload" , "bi" , NULL , T_ILLEGAL, 1, false, _iload ); So looks like the result type in hotspot's bytecodes.cpp also needs to be changed ? Thanks, Jini. On 5/8/2018 11:43 PM, Chris Plummer wrote: > Hi Jini, > > Why are _nofast_aload_0 and _nofast_iload using return type > BasicType.getTIllegal() when the return type is known? > > The test changes look good. > > thanks, > > Chris > > On 5/8/18 8:53 AM, Jini George wrote: >> Thanks, Ioi. Could I get one more reviewer to take a look at this >> ? >> >> Thanks, Jini. >> >> On 5/8/2018 8:55 PM, Ioi Lam wrote: >>> Looks good. Thanks! >>> >>> - Ioi >>> >>> >>> On 5/7/18 8:38 PM, Jini George wrote: >>>> Thank you very much, Ioi, for the review and for the >>>> clarifications and help provided offline. I have added the >>>> checks for _nofast_getfield and _nofast_putfield. SA has a bug >>>> due to which for iload, only the base bytecode (iload) gets >>>> displayed -- fast_iload and nofast_iload do not get displayed. >>>> JDK-8202693 (SA: clhsdb printall only displays the base >>>> bytecode for iload) has been filed for this. I would add the >>>> test for nofast_iload along with the fix for JDK-8202693. >>>> >>>> The modified webrev is at: >>>> >>>> http://cr.openjdk.java.net/~jgeorge/8174995/webrev.01/ >>>> >>>> Thanks, Jini. >>>> >>>> On 4/27/2018 1:54 AM, Ioi Lam wrote: >>>>> HI Jini, >>>>> >>>>> [1] "_nofast_aload" should be "_nofast_aload_0": aload and >>>>> aload_0 are two different bytecodes. >>>>> >>>>> [2] Only the _nofast_aload_0 bytecode is tested. For >>>>> completeness, do you think it makes sense to add test cases >>>>> for these other 3 bytecodes? >>>>> >>>>> _nofast_getfield _nofast_putfield _nofast_iload >>>>> >>>>> >>>>> Thanks - Ioi >>>>> >>>>> On 4/26/18 11:15 AM, Jini George wrote: >>>>>> Hello all, >>>>>> >>>>>> Please review the following proposed fix for the issue: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8174995 >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~jgeorge/8174995/webrev.00/ >>>>>> >>>>>> Issue: Clhsdb commands like 'where -a', 'printall' would >>>>>> throw an illegal code assertion failure when CDS is used. >>>>>> >>>>>> Root cause and proposed fix: SA has been unaware of the new >>>>>> bytecodes introduced for rewriting at CDS dump time >>>>>> (_nofast* bytecodes). The fix is to make SA aware of these >>>>>> new _nofast* bytecodes. >>>>>> >>>>>> Tests Run and Passed: SA tests on Mach5 (including the >>>>>> tests modified to test this fix). >>>>>> >>>>>> Thank you, Jini. >>>>> >>> > > From chris.plummer at oracle.com Fri May 11 07:45:07 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 11 May 2018 00:45:07 -0700 Subject: RFR: (S): SA: clhsdb 'where -a' throws Assertion Failure with illegal code 236 when CDS is used In-Reply-To: <466139cc-b2c4-9abd-5f41-d7664291b2f6@oracle.com> References: <82c69a0d-02f3-5db8-fedd-d52b00571a55@oracle.com> <41204e1d-11e1-41a9-af90-7cabd12098aa@oracle.com> <466139cc-b2c4-9abd-5f41-d7664291b2f6@oracle.com> Message-ID: <3cfea08f-2dca-4777-f6ff-fa643e9a5f01@oracle.com> I'm not too sure how that field ends up getting used. I'd say what's likely in this case is that it's wrong but doesn't matter. Hopefully someone with a better understanding of the use of this field will chime in. Chris On 5/11/18 12:29 AM, Jini George wrote: > Thank you very much, Chris, for taking a look. > > The bytecodes table initialization in SA's Bytecodes.java mimics the > bytecodes table initialization in hotspot's Bytecodes::initialize() from > bytecodes.cpp. In bytecodes.cpp, we have the initialization of the > _nofast* bytecodes thus: > > > ? def(_nofast_aload_0????? , "nofast_aload_0"????? , "b"??? , NULL??? > , T_ILLEGAL,? 1, true , _aload_0??????? ); > ? def(_nofast_iload??????? , "nofast_iload"??????? , "bi"?? , NULL??? > , T_ILLEGAL,? 1, false, _iload????????? ); > > So looks like the result type in hotspot's bytecodes.cpp also needs to > be changed ? > > Thanks, > Jini. > > > On 5/8/2018 11:43 PM, Chris Plummer wrote: >> Hi Jini, >> >> Why are _nofast_aload_0 and _nofast_iload using return type >> BasicType.getTIllegal() when the return type is known? >> >> The test changes look good. >> >> thanks, >> >> Chris >> >> On 5/8/18 8:53 AM, Jini George wrote: >>> Thanks, Ioi. Could I get one more reviewer to take a look at this >>> ? >>> >>> Thanks, Jini. >>> >>> On 5/8/2018 8:55 PM, Ioi Lam wrote: >>>> Looks good. Thanks! >>>> >>>> - Ioi >>>> >>>> >>>> On 5/7/18 8:38 PM, Jini George wrote: >>>>> Thank you very much, Ioi, for the review and for the >>>>> clarifications and help provided offline. I have added the >>>>> checks for _nofast_getfield and _nofast_putfield. SA has a bug >>>>> due to which for iload, only the base bytecode (iload) gets >>>>> displayed -- fast_iload and nofast_iload do not get displayed. >>>>> JDK-8202693 (SA: clhsdb printall only displays the base >>>>> bytecode for iload) has been filed for this. I would add the >>>>> test for nofast_iload along with the fix for JDK-8202693. >>>>> >>>>> The modified webrev is at: >>>>> >>>>> http://cr.openjdk.java.net/~jgeorge/8174995/webrev.01/ >>>>> >>>>> Thanks, Jini. >>>>> >>>>> On 4/27/2018 1:54 AM, Ioi Lam wrote: >>>>>> HI Jini, >>>>>> >>>>>> [1] "_nofast_aload" should be "_nofast_aload_0": aload and >>>>>> aload_0 are two different bytecodes. >>>>>> >>>>>> [2] Only the _nofast_aload_0 bytecode is tested. For >>>>>> completeness, do you think it makes sense to add test cases >>>>>> for these other 3 bytecodes? >>>>>> >>>>>> _nofast_getfield _nofast_putfield _nofast_iload >>>>>> >>>>>> >>>>>> Thanks - Ioi >>>>>> >>>>>> On 4/26/18 11:15 AM, Jini George wrote: >>>>>>> Hello all, >>>>>>> >>>>>>> Please review the following proposed fix for the issue: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8174995 >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~jgeorge/8174995/webrev.00/ >>>>>>> >>>>>>> Issue: Clhsdb commands like 'where -a', 'printall' would >>>>>>> throw an illegal code assertion failure when CDS is used. >>>>>>> >>>>>>> Root cause and proposed fix: SA has been unaware of the new >>>>>>> ?bytecodes introduced for rewriting at CDS dump time >>>>>>> (_nofast* bytecodes). The fix is to make SA aware of these >>>>>>> new _nofast* bytecodes. >>>>>>> >>>>>>> Tests Run and Passed: SA tests on Mach5 (including the >>>>>>> tests modified to test this fix). >>>>>>> >>>>>>> Thank you, Jini. >>>>>> >>>> >> >> From per.liden at oracle.com Fri May 11 08:31:45 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 11 May 2018 10:31:45 +0200 Subject: RFR: 8202978: Incorrect tmp register passed to MacroAssembler::load_mirror() Message-ID: On x86, MacroAssembler::load_mirror() defaults to using rscratch2 as tmp register, unless something else is specified. In TemplateInterpreterGenerator::generate_native_entry() we call load_mirror(), but the rscratch2 register is already in use in this context, which obviously leads to problems. This is not a performance critical path, so we should just pass noreg as the tmp register. Bug: https://bugs.openjdk.java.net/browse/JDK-8202978 Webrev: http://cr.openjdk.java.net/~pliden/8202978/webrev.0 Testing: passes hs-tier{1,2} /Per From martin.doerr at sap.com Fri May 11 10:43:29 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 11 May 2018 10:43:29 +0000 Subject: RFR(M): 8202713: Create a MacroAssembler::access_load/store_at wrapper for S390 and PPC Message-ID: Hi Erik, thanks for creating the bug. It makes sense to use load/store_heap_oop like on the other platforms. It will especially be useful when we add support for a GC with load barriers. Here's the webrev: http://cr.openjdk.java.net/~mdoerr/8202713_ppc64_s390_masm_access/webrev.00/ Please review. Best regards, Martin From vladimir.kozlov at oracle.com Fri May 11 12:32:03 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 11 May 2018 05:32:03 -0700 Subject: RFR: 8202978: Incorrect tmp register passed to MacroAssembler::load_mirror() In-Reply-To: References: Message-ID: <70f41abc-9f3f-98fa-a66d-76f931640d02@oracle.com> Looks good. Is this new code? Should we backport it otherwise? Thanks, Vladimir K On 5/11/18 1:31 AM, Per Liden wrote: > On x86, MacroAssembler::load_mirror() defaults to using rscratch2 as tmp register, unless something else is specified. > In TemplateInterpreterGenerator::generate_native_entry() we call load_mirror(), but the rscratch2 register is already in > use in this context, which obviously leads to problems. This is not a performance critical path, so we should just pass > noreg as the tmp register. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202978 > Webrev: http://cr.openjdk.java.net/~pliden/8202978/webrev.0 > > Testing: passes hs-tier{1,2} > > /Per From per.liden at oracle.com Fri May 11 12:48:51 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 11 May 2018 14:48:51 +0200 Subject: RFR: 8202978: Incorrect tmp register passed to MacroAssembler::load_mirror() In-Reply-To: <70f41abc-9f3f-98fa-a66d-76f931640d02@oracle.com> References: <70f41abc-9f3f-98fa-a66d-76f931640d02@oracle.com> Message-ID: <506c8615-5d79-be76-5e09-9331ec50465e@oracle.com> On 05/11/2018 02:32 PM, Vladimir Kozlov wrote: > Looks good. Thanks Vladimir! > Is this new code? Should we backport it otherwise? Yes, this is new code, i.e. it's not in JDK 10 or older. Also, it just so happens that this bug doesn't affect the existing GC, only GCs which do load barriers (like ZGC), which is why it hasn't been caught before. /Per > > Thanks, > Vladimir K > > On 5/11/18 1:31 AM, Per Liden wrote: >> On x86, MacroAssembler::load_mirror() defaults to using rscratch2 as >> tmp register, unless something else is specified. In >> TemplateInterpreterGenerator::generate_native_entry() we call >> load_mirror(), but the rscratch2 register is already in use in this >> context, which obviously leads to problems. This is not a performance >> critical path, so we should just pass noreg as the tmp register. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8202978 >> Webrev: http://cr.openjdk.java.net/~pliden/8202978/webrev.0 >> >> Testing: passes hs-tier{1,2} >> >> /Per From david.holmes at oracle.com Fri May 11 13:03:42 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 11 May 2018 23:03:42 +1000 Subject: RFR(XS): 8132287: obsolete the "InlineNotify" flag option In-Reply-To: <1a5bb68b-b447-9e0f-d009-e5420f89a339@oracle.com> References: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> <0492b6f5-5afc-fef3-49d4-b6e287d12a5e@oracle.com> <49458504-51e6-e18c-a413-455abf956822@oracle.com> <61158762-8ac4-a7f8-2d72-a90c30bba371@oracle.com> <1ffa819b-fdf4-5710-e7a0-c4299cbf1680@oracle.com> <933425f4-1f72-c532-5095-31a5411309ea@oracle.com> <1a5bb68b-b447-9e0f-d009-e5420f89a339@oracle.com> Message-ID: <61f13d18-51c7-ea9b-e2c6-1a8bc0110ae7@oracle.com> I see Dean already took care of the follow up Graal bug: JDK-8202942 Cheers, David On 11/05/2018 9:21 AM, Daniel D. Daugherty wrote: > On 5/10/18 5:48 PM, David Holmes wrote: >> Hi Dan, >> >> On 11/05/2018 7:24 AM, Daniel D. Daugherty wrote: >>> A Mach5 builds-tier1,hs-tier1,hs-tier2 job with the following >>> additional changes (thanks Vladimir and Dean!) passes... >>> >>> Still waiting for Runtime team review... >> >> Runtime Reviewed. > > Thanks for the review. > > >> >>> $ hg diff -r 8132287.cr0 >>> diff -r cbeb21e5a2c6 src/hotspot/share/opto/library_call.cpp >>> --- a/src/hotspot/share/opto/library_call.cpp??? Thu May 10 09:37:27 >>> 2018 -0400 >>> +++ b/src/hotspot/share/opto/library_call.cpp??? Thu May 10 17:23:08 >>> 2018 -0400 >>> @@ -52,6 +52,7 @@ >>> ??#include "opto/subnode.hpp" >>> ??#include "prims/nativeLookup.hpp" >>> ??#include "prims/unsafe.hpp" >>> +#include "runtime/objectMonitor.hpp" >>> ??#include "runtime/sharedRuntime.hpp" >>> ??#ifdef TRACE_HAVE_INTRINSICS >>> ??#include "trace/traceMacros.hpp" >>> diff -r cbeb21e5a2c6 >>> src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>> >>> --- >>> a/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>> Thu May 10 09:37:27 2018 -0400 >>> +++ >>> b/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>> Thu May 10 17:23:08 2018 -0400 >>> @@ -44,7 +44,7 @@ >>> ????? final boolean useCRC32CIntrinsics = >>> getFlag("UseCRC32CIntrinsics", Boolean.class); >>> >>> ????? // JDK-8075171 >>> -??? final boolean inlineNotify = getFlag("InlineNotify", >>> Boolean.class); >>> +??? final boolean inlineNotify = getFlag("InlineNotify", >>> Boolean.class, true); >> >> I hope a follow up bug has been filed so that this usage is handled >> correctly now that the flag no longer exists. > > I'll figure that out with Dean. > > Dan > > >> >> Thanks, >> David >> ------ >> >>> ????? // JDK-8046936 >>> ????? final int javaThreadReservedStackActivationOffset = >>> getFieldOffset("JavaThread::_reserved_stack_activation", >>> Integer.class, "address"); >>> >>> On 5/10/18 3:55 PM, Daniel D. Daugherty wrote: >>>> On 5/10/18 3:52 PM, dean.long at oracle.com wrote: >>>>> On 5/10/18 12:19 PM, Daniel D. Daugherty wrote: >>>>>> I can include the Graal fix in my push if that works for you... >>>>> >>>>> Yes, please do.? This should do it: >>>> >>>> Thanks! I'll take that change for a Mach5 spin in a few minutes... >>>> >>>> Dan >>>> >>>> >>>>> >>>>> diff -r 0509b9be3bda >>>>> src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>>>> >>>>> --- >>>>> a/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>>>> Thu May 10 11:45:40 2018 -0700 >>>>> +++ >>>>> b/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>>>> Thu May 10 12:50:48 2018 -0700 >>>>> @@ -44,7 +44,7 @@ >>>>> ???? final boolean useCRC32CIntrinsics = >>>>> getFlag("UseCRC32CIntrinsics", Boolean.class); >>>>> >>>>> ???? // JDK-8075171 >>>>> -??? final boolean inlineNotify = getFlag("InlineNotify", >>>>> Boolean.class); >>>>> +??? final boolean inlineNotify = getFlag("InlineNotify", >>>>> Boolean.class, true); >>>>> >>>>> ???? // JDK-8046936 >>>>> ???? final int javaThreadReservedStackActivationOffset = >>>>> getFieldOffset("JavaThread::_reserved_stack_activation", >>>>> Integer.class, "address"); >>>>> >>>>> >>>>> dl >>>> >>>> >>> > From daniel.daugherty at oracle.com Fri May 11 13:36:18 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 11 May 2018 09:36:18 -0400 Subject: RFR(XS): 8132287: obsolete the "InlineNotify" flag option In-Reply-To: <61f13d18-51c7-ea9b-e2c6-1a8bc0110ae7@oracle.com> References: <9ba34b7b-96bd-1e02-7690-03688f75cf59@oracle.com> <0492b6f5-5afc-fef3-49d4-b6e287d12a5e@oracle.com> <49458504-51e6-e18c-a413-455abf956822@oracle.com> <61158762-8ac4-a7f8-2d72-a90c30bba371@oracle.com> <1ffa819b-fdf4-5710-e7a0-c4299cbf1680@oracle.com> <933425f4-1f72-c532-5095-31a5411309ea@oracle.com> <1a5bb68b-b447-9e0f-d009-e5420f89a339@oracle.com> <61f13d18-51c7-ea9b-e2c6-1a8bc0110ae7@oracle.com> Message-ID: <1bcc71fe-0bef-64e3-c854-75840a9cfbf4@oracle.com> David H., thanks for the pointer to the new bug. Dean, thanks for filing the new bug. Dan On 5/11/18 9:03 AM, David Holmes wrote: > I see Dean already took care of the follow up Graal bug: JDK-8202942 > > Cheers, > David > > On 11/05/2018 9:21 AM, Daniel D. Daugherty wrote: >> On 5/10/18 5:48 PM, David Holmes wrote: >>> Hi Dan, >>> >>> On 11/05/2018 7:24 AM, Daniel D. Daugherty wrote: >>>> A Mach5 builds-tier1,hs-tier1,hs-tier2 job with the following >>>> additional changes (thanks Vladimir and Dean!) passes... >>>> >>>> Still waiting for Runtime team review... >>> >>> Runtime Reviewed. >> >> Thanks for the review. >> >> >>> >>>> $ hg diff -r 8132287.cr0 >>>> diff -r cbeb21e5a2c6 src/hotspot/share/opto/library_call.cpp >>>> --- a/src/hotspot/share/opto/library_call.cpp??? Thu May 10 >>>> 09:37:27 2018 -0400 >>>> +++ b/src/hotspot/share/opto/library_call.cpp??? Thu May 10 >>>> 17:23:08 2018 -0400 >>>> @@ -52,6 +52,7 @@ >>>> ??#include "opto/subnode.hpp" >>>> ??#include "prims/nativeLookup.hpp" >>>> ??#include "prims/unsafe.hpp" >>>> +#include "runtime/objectMonitor.hpp" >>>> ??#include "runtime/sharedRuntime.hpp" >>>> ??#ifdef TRACE_HAVE_INTRINSICS >>>> ??#include "trace/traceMacros.hpp" >>>> diff -r cbeb21e5a2c6 >>>> src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>>> >>>> --- >>>> a/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>>> Thu May 10 09:37:27 2018 -0400 >>>> +++ >>>> b/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>>> Thu May 10 17:23:08 2018 -0400 >>>> @@ -44,7 +44,7 @@ >>>> ????? final boolean useCRC32CIntrinsics = >>>> getFlag("UseCRC32CIntrinsics", Boolean.class); >>>> >>>> ????? // JDK-8075171 >>>> -??? final boolean inlineNotify = getFlag("InlineNotify", >>>> Boolean.class); >>>> +??? final boolean inlineNotify = getFlag("InlineNotify", >>>> Boolean.class, true); >>> >>> I hope a follow up bug has been filed so that this usage is handled >>> correctly now that the flag no longer exists. >> >> I'll figure that out with Dean. >> >> Dan >> >> >>> >>> Thanks, >>> David >>> ------ >>> >>>> ????? // JDK-8046936 >>>> ????? final int javaThreadReservedStackActivationOffset = >>>> getFieldOffset("JavaThread::_reserved_stack_activation", >>>> Integer.class, "address"); >>>> >>>> On 5/10/18 3:55 PM, Daniel D. Daugherty wrote: >>>>> On 5/10/18 3:52 PM, dean.long at oracle.com wrote: >>>>>> On 5/10/18 12:19 PM, Daniel D. Daugherty wrote: >>>>>>> I can include the Graal fix in my push if that works for you... >>>>>> >>>>>> Yes, please do.? This should do it: >>>>> >>>>> Thanks! I'll take that change for a Mach5 spin in a few minutes... >>>>> >>>>> Dan >>>>> >>>>> >>>>>> >>>>>> diff -r 0509b9be3bda >>>>>> src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>>>>> >>>>>> --- >>>>>> a/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>>>>> Thu May 10 11:45:40 2018 -0700 >>>>>> +++ >>>>>> b/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot/src/org/graalvm/compiler/hotspot/GraalHotSpotVMConfigVersioned.java >>>>>> Thu May 10 12:50:48 2018 -0700 >>>>>> @@ -44,7 +44,7 @@ >>>>>> ???? final boolean useCRC32CIntrinsics = >>>>>> getFlag("UseCRC32CIntrinsics", Boolean.class); >>>>>> >>>>>> ???? // JDK-8075171 >>>>>> -??? final boolean inlineNotify = getFlag("InlineNotify", >>>>>> Boolean.class); >>>>>> +??? final boolean inlineNotify = getFlag("InlineNotify", >>>>>> Boolean.class, true); >>>>>> >>>>>> ???? // JDK-8046936 >>>>>> ???? final int javaThreadReservedStackActivationOffset = >>>>>> getFieldOffset("JavaThread::_reserved_stack_activation", >>>>>> Integer.class, "address"); >>>>>> >>>>>> >>>>>> dl >>>>> >>>>> >>>> >> From ioi.lam at oracle.com Fri May 11 16:34:59 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 11 May 2018 09:34:59 -0700 Subject: RFR: (S): SA: clhsdb 'where -a' throws Assertion Failure with illegal code 236 when CDS is used In-Reply-To: <3cfea08f-2dca-4777-f6ff-fa643e9a5f01@oracle.com> References: <82c69a0d-02f3-5db8-fedd-d52b00571a55@oracle.com> <41204e1d-11e1-41a9-af90-7cabd12098aa@oracle.com> <466139cc-b2c4-9abd-5f41-d7664291b2f6@oracle.com> <3cfea08f-2dca-4777-f6ff-fa643e9a5f01@oracle.com> Message-ID: <08b8fe9b-b662-f395-7bc4-b021795219b7@oracle.com> I think all code that looks at the top of stack (e.g., users of BytecodeStream) operate with the original bytecode (_aload_0), not the quicken one (_fast_aload_0 or _nofast_aload_0), so this bug is never discovered. But I think this is still a bug and should be fixed. I filed https://bugs.openjdk.java.net/browse/JDK-8203005 Thanks - Ioi On 5/11/18 12:45 AM, Chris Plummer wrote: > I'm not too sure how that field ends up getting used. I'd say what's > likely in this case is that it's wrong but doesn't matter. Hopefully > someone with a better understanding of the use of this field will > chime in. > > Chris > > On 5/11/18 12:29 AM, Jini George wrote: >> Thank you very much, Chris, for taking a look. >> >> The bytecodes table initialization in SA's Bytecodes.java mimics the >> bytecodes table initialization in hotspot's Bytecodes::initialize() from >> bytecodes.cpp. In bytecodes.cpp, we have the initialization of the >> _nofast* bytecodes thus: >> >> >> ? def(_nofast_aload_0????? , "nofast_aload_0"????? , "b"??? , NULL??? >> , T_ILLEGAL,? 1, true , _aload_0??????? ); >> ? def(_nofast_iload??????? , "nofast_iload"??????? , "bi"?? , NULL??? >> , T_ILLEGAL,? 1, false, _iload????????? ); >> >> So looks like the result type in hotspot's bytecodes.cpp also needs >> to be changed ? >> >> Thanks, >> Jini. >> >> >> On 5/8/2018 11:43 PM, Chris Plummer wrote: >>> Hi Jini, >>> >>> Why are _nofast_aload_0 and _nofast_iload using return type >>> BasicType.getTIllegal() when the return type is known? >>> >>> The test changes look good. >>> >>> thanks, >>> >>> Chris >>> >>> On 5/8/18 8:53 AM, Jini George wrote: >>>> Thanks, Ioi. Could I get one more reviewer to take a look at this >>>> ? >>>> >>>> Thanks, Jini. >>>> >>>> On 5/8/2018 8:55 PM, Ioi Lam wrote: >>>>> Looks good. Thanks! >>>>> >>>>> - Ioi >>>>> >>>>> >>>>> On 5/7/18 8:38 PM, Jini George wrote: >>>>>> Thank you very much, Ioi, for the review and for the >>>>>> clarifications and help provided offline. I have added the >>>>>> checks for _nofast_getfield and _nofast_putfield. SA has a bug >>>>>> due to which for iload, only the base bytecode (iload) gets >>>>>> displayed -- fast_iload and nofast_iload do not get displayed. >>>>>> JDK-8202693 (SA: clhsdb printall only displays the base >>>>>> bytecode for iload) has been filed for this. I would add the >>>>>> test for nofast_iload along with the fix for JDK-8202693. >>>>>> >>>>>> The modified webrev is at: >>>>>> >>>>>> http://cr.openjdk.java.net/~jgeorge/8174995/webrev.01/ >>>>>> >>>>>> Thanks, Jini. >>>>>> >>>>>> On 4/27/2018 1:54 AM, Ioi Lam wrote: >>>>>>> HI Jini, >>>>>>> >>>>>>> [1] "_nofast_aload" should be "_nofast_aload_0": aload and >>>>>>> aload_0 are two different bytecodes. >>>>>>> >>>>>>> [2] Only the _nofast_aload_0 bytecode is tested. For >>>>>>> completeness, do you think it makes sense to add test cases >>>>>>> for these other 3 bytecodes? >>>>>>> >>>>>>> _nofast_getfield _nofast_putfield _nofast_iload >>>>>>> >>>>>>> >>>>>>> Thanks - Ioi >>>>>>> >>>>>>> On 4/26/18 11:15 AM, Jini George wrote: >>>>>>>> Hello all, >>>>>>>> >>>>>>>> Please review the following proposed fix for the issue: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8174995 >>>>>>>> >>>>>>>> Webrev: >>>>>>>> http://cr.openjdk.java.net/~jgeorge/8174995/webrev.00/ >>>>>>>> >>>>>>>> Issue: Clhsdb commands like 'where -a', 'printall' would >>>>>>>> throw an illegal code assertion failure when CDS is used. >>>>>>>> >>>>>>>> Root cause and proposed fix: SA has been unaware of the new >>>>>>>> ?bytecodes introduced for rewriting at CDS dump time >>>>>>>> (_nofast* bytecodes). The fix is to make SA aware of these >>>>>>>> new _nofast* bytecodes. >>>>>>>> >>>>>>>> Tests Run and Passed: SA tests on Mach5 (including the >>>>>>>> tests modified to test this fix). >>>>>>>> >>>>>>>> Thank you, Jini. >>>>>>> >>>>> >>> >>> > > From chris.plummer at oracle.com Fri May 11 17:56:37 2018 From: chris.plummer at oracle.com (Chris Plummer) Date: Fri, 11 May 2018 10:56:37 -0700 Subject: RFR: (S): SA: clhsdb 'where -a' throws Assertion Failure with illegal code 236 when CDS is used In-Reply-To: <08b8fe9b-b662-f395-7bc4-b021795219b7@oracle.com> References: <82c69a0d-02f3-5db8-fedd-d52b00571a55@oracle.com> <41204e1d-11e1-41a9-af90-7cabd12098aa@oracle.com> <466139cc-b2c4-9abd-5f41-d7664291b2f6@oracle.com> <3cfea08f-2dca-4777-f6ff-fa643e9a5f01@oracle.com> <08b8fe9b-b662-f395-7bc4-b021795219b7@oracle.com> Message-ID: <6227ef6c-45bd-f44a-c34b-37e84d243e0a@oracle.com> Thanks Ioi. Jini, you can count me as a reviewer for your changes. thanks, Chris On 5/11/18 9:34 AM, Ioi Lam wrote: > I think all code that looks at the top of stack (e.g., users of > BytecodeStream) operate with the original bytecode (_aload_0), not the > quicken one (_fast_aload_0 or _nofast_aload_0), so this bug is never > discovered. But I think this is still a bug and should be fixed. > > I filed https://bugs.openjdk.java.net/browse/JDK-8203005 > > Thanks > > - Ioi > > > > > On 5/11/18 12:45 AM, Chris Plummer wrote: >> I'm not too sure how that field ends up getting used. I'd say what's >> likely in this case is that it's wrong but doesn't matter. Hopefully >> someone with a better understanding of the use of this field will >> chime in. >> >> Chris >> >> On 5/11/18 12:29 AM, Jini George wrote: >>> Thank you very much, Chris, for taking a look. >>> >>> The bytecodes table initialization in SA's Bytecodes.java mimics the >>> bytecodes table initialization in hotspot's Bytecodes::initialize() >>> from >>> bytecodes.cpp. In bytecodes.cpp, we have the initialization of the >>> _nofast* bytecodes thus: >>> >>> >>> ? def(_nofast_aload_0????? , "nofast_aload_0"????? , "b"??? , >>> NULL??? , T_ILLEGAL,? 1, true , _aload_0??????? ); >>> ? def(_nofast_iload??????? , "nofast_iload"??????? , "bi"?? , >>> NULL??? , T_ILLEGAL,? 1, false, _iload????????? ); >>> >>> So looks like the result type in hotspot's bytecodes.cpp also needs >>> to be changed ? >>> >>> Thanks, >>> Jini. >>> >>> >>> On 5/8/2018 11:43 PM, Chris Plummer wrote: >>>> Hi Jini, >>>> >>>> Why are _nofast_aload_0 and _nofast_iload using return type >>>> BasicType.getTIllegal() when the return type is known? >>>> >>>> The test changes look good. >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>> On 5/8/18 8:53 AM, Jini George wrote: >>>>> Thanks, Ioi. Could I get one more reviewer to take a look at this >>>>> ? >>>>> >>>>> Thanks, Jini. >>>>> >>>>> On 5/8/2018 8:55 PM, Ioi Lam wrote: >>>>>> Looks good. Thanks! >>>>>> >>>>>> - Ioi >>>>>> >>>>>> >>>>>> On 5/7/18 8:38 PM, Jini George wrote: >>>>>>> Thank you very much, Ioi, for the review and for the >>>>>>> clarifications and help provided offline. I have added the >>>>>>> checks for _nofast_getfield and _nofast_putfield. SA has a bug >>>>>>> due to which for iload, only the base bytecode (iload) gets >>>>>>> displayed -- fast_iload and nofast_iload do not get displayed. >>>>>>> JDK-8202693 (SA: clhsdb printall only displays the base >>>>>>> bytecode for iload) has been filed for this. I would add the >>>>>>> test for nofast_iload along with the fix for JDK-8202693. >>>>>>> >>>>>>> The modified webrev is at: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~jgeorge/8174995/webrev.01/ >>>>>>> >>>>>>> Thanks, Jini. >>>>>>> >>>>>>> On 4/27/2018 1:54 AM, Ioi Lam wrote: >>>>>>>> HI Jini, >>>>>>>> >>>>>>>> [1] "_nofast_aload" should be "_nofast_aload_0": aload and >>>>>>>> aload_0 are two different bytecodes. >>>>>>>> >>>>>>>> [2] Only the _nofast_aload_0 bytecode is tested. For >>>>>>>> completeness, do you think it makes sense to add test cases >>>>>>>> for these other 3 bytecodes? >>>>>>>> >>>>>>>> _nofast_getfield _nofast_putfield _nofast_iload >>>>>>>> >>>>>>>> >>>>>>>> Thanks - Ioi >>>>>>>> >>>>>>>> On 4/26/18 11:15 AM, Jini George wrote: >>>>>>>>> Hello all, >>>>>>>>> >>>>>>>>> Please review the following proposed fix for the issue: >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8174995 >>>>>>>>> >>>>>>>>> Webrev: >>>>>>>>> http://cr.openjdk.java.net/~jgeorge/8174995/webrev.00/ >>>>>>>>> >>>>>>>>> Issue: Clhsdb commands like 'where -a', 'printall' would >>>>>>>>> throw an illegal code assertion failure when CDS is used. >>>>>>>>> >>>>>>>>> Root cause and proposed fix: SA has been unaware of the new >>>>>>>>> ?bytecodes introduced for rewriting at CDS dump time >>>>>>>>> (_nofast* bytecodes). The fix is to make SA aware of these >>>>>>>>> new _nofast* bytecodes. >>>>>>>>> >>>>>>>>> Tests Run and Passed: SA tests on Mach5 (including the >>>>>>>>> tests modified to test this fix). >>>>>>>>> >>>>>>>>> Thank you, Jini. >>>>>>>> >>>>>> >>>> >>>> >> >> > From mikhailo.seledtsov at oracle.com Fri May 11 21:10:07 2018 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Fri, 11 May 2018 14:10:07 -0700 Subject: RFR(M): 8199252: [TESTBUG] Open source VM testbase system dictionary tests Message-ID: <5AF606AF.3020404@oracle.com> Please review this change open sourcing VM system dictionary tests. These tests have been used internally for a while, and are now being open sourced. Since this is not an creation of new tests, simply open sourcing existing tests, we would like to keep the changes during this review to an absolute required minimum. If you have any feedback on improvements of these tests, please file RFE(s) that will be addressed later in order of priority. Here is what was done for this change: 1. Moved the tests to OpenJDK repository preserving relative directory location and structure. 3. Updated Copyright statements accordingly. 4. Updated "@library" statements accordingly. 5. Updated TEST.groups JBS: https://bugs.openjdk.java.net/browse/JDK-8199252 Webrev: http://cr.openjdk.java.net/~mseledtsov/8199252.01.open/ Testing: 1. Ran the following tests on open-only repository and build, using "make run-test" (Linux-x64) vmTestbase_nsk_sysdict All PASS 2. Automated multip-platform test system (usual 4 platforms): - vmTestbase_nsk_sysdict - hs-tier{1,2} In progress Thank you, Misha From erik.joelsson at oracle.com Fri May 11 21:33:20 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 11 May 2018 14:33:20 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: References: Message-ID: Hello, For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 /Erik On 2018-05-10 16:11, Jiangli Zhou wrote: > Hi, > > Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation & discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). > > webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ > RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 > Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 > > The webrev includes the following three main parts: > > 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). > > 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. > > 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. > > Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. > > Thanks, > Jiangli > From jiangli.zhou at oracle.com Fri May 11 21:44:21 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 11 May 2018 14:44:21 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: References: Message-ID: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> Hi Erik, Thank you so much for investigating a proper fix for the vm_version.o issue. I will withdraw the build change from my original webrev. Thanks again for taking over the bug! Jiangli > On May 11, 2018, at 2:33 PM, Erik Joelsson wrote: > > Received: from [10.132.185.167] (/10.132.185.167) > by default (Oracle Beehive Gateway v4.0) > with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 > Subject: Re: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path > matching check > To: Jiangli Zhou , > "hotspot-runtime-dev at openjdk.java.net runtime" > , > build-dev > References: > From: Erik Joelsson > Organization: Oracle Corporation > Message-ID: > Date: Fri, 11 May 2018 14:33:20 -0700 > User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) > Gecko/20100101 Thunderbird/52.7.0 > MIME-Version: 1.0 > In-Reply-To: > Content-Type: text/plain; charset=utf-8; format=flowed > Content-Transfer-Encoding: 8bit > Content-Language: en-US > > Hello, > > For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. > > I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. > > Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 > > /Erik > > >> On 2018-05-10 16:11, Jiangli Zhou wrote: >> Hi, >> >> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation & discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >> >> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >> >> The webrev includes the following three main parts: >> >> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >> >> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >> >> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >> >> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >> >> Thanks, >> Jiangli >> > > Hello, > > For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. > > I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. > > Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 > > /Erik > > >> On 2018-05-10 16:11, Jiangli Zhou wrote: >> Hi, >> >> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation & discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >> >> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >> >> The webrev includes the following three main parts: >> >> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >> >> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >> >> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >> >> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >> >> Thanks, >> Jiangli >> > From jiangli.zhou at oracle.com Fri May 11 22:15:36 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 11 May 2018 15:15:36 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> References: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> Message-ID: <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> I?ve withdrawn my original weberv and removed the build change. Here is the updated webrev that only addresses 8199807. webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.00/ RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 Thanks, Jiangli > On May 11, 2018, at 2:44 PM, Jiangli Zhou wrote: > > Hi Erik, > > Thank you so much for investigating a proper fix for the vm_version.o issue. I will withdraw the build change from my original webrev. > > Thanks again for taking over the bug! > Jiangli > >> On May 11, 2018, at 2:33 PM, Erik Joelsson wrote: >> >> Received: from [10.132.185.167] (/10.132.185.167) >> by default (Oracle Beehive Gateway v4.0) >> with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 >> Subject: Re: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path >> matching check >> To: Jiangli Zhou , >> "hotspot-runtime-dev at openjdk.java.net runtime" >> , >> build-dev >> References: >> From: Erik Joelsson >> Organization: Oracle Corporation >> Message-ID: >> Date: Fri, 11 May 2018 14:33:20 -0700 >> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) >> Gecko/20100101 Thunderbird/52.7.0 >> MIME-Version: 1.0 >> In-Reply-To: >> Content-Type: text/plain; charset=utf-8; format=flowed >> Content-Transfer-Encoding: 8bit >> Content-Language: en-US >> >> Hello, >> >> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >> >> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >> >> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >> >> /Erik >> >> >>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>> Hi, >>> >>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation & discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>> >>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>> >>> The webrev includes the following three main parts: >>> >>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>> >>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>> >>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>> >>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>> >>> Thanks, >>> Jiangli >>> >> >> Hello, >> >> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >> >> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >> >> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >> >> /Erik >> >> >>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>> Hi, >>> >>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation & discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>> >>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>> >>> The webrev includes the following three main parts: >>> >>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>> >>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>> >>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>> >>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>> >>> Thanks, >>> Jiangli >>> >> > From calvin.cheung at oracle.com Fri May 11 22:20:23 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Fri, 11 May 2018 15:20:23 -0700 Subject: RFR(xs): 8202669: Intermittent crash in ClassLoadingService::compute_class_size() Message-ID: <5AF61727.5060206@oracle.com> bug: https://bugs.openjdk.java.net/browse/JDK-8202669 webrev: http://cr.openjdk.java.net/~ccheung/8202669/webrev.00/ This bug surfaced after the change for JDK-8200466. The proposed change is simply null checking the _transitive_interfaces before accessing its size(). I ran the affected tests with the change 30 times on Oracle platforms and didn't see any failures. thanks, Calvin From calvin.cheung at oracle.com Fri May 11 23:32:51 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Fri, 11 May 2018 16:32:51 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> References: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> Message-ID: <5AF62823.7050804@oracle.com> Hi Jiangli, Thanks for doing this useful change. I have a few minor comments: 1) sharedPathsMiscInfo.cpp 199 dp ++; Should the above be similar to line 194 as follows? dp += path_sep_len; 2) filemap.cpp 244 if (ClassLoader::is_modules_image(name)) { I think the above could be: if (is_modules_image()) { The is_modules_image() is a member function of SharedClassPathEntry. Btw, why is it necessary to get the runtime modules image path again in lines 244 - 246? 3) BootClassPathMismatch.java Could you move line 103 to right above line 107 (opts.addPrefix(?))? Because the archive dumping part of the test doesn?t use appJar. thanks, Calvin On 5/11/18, 3:15 PM, Jiangli Zhou wrote: > I?ve withdrawn my original weberv and removed the build change. Here is the updated webrev that only addresses 8199807. > > webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.00/ > RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 > > Thanks, > Jiangli > >> On May 11, 2018, at 2:44 PM, Jiangli Zhou wrote: >> >> Hi Erik, >> >> Thank you so much for investigating a proper fix for the vm_version.o issue. I will withdraw the build change from my original webrev. >> >> Thanks again for taking over the bug! >> Jiangli >> >>> On May 11, 2018, at 2:33 PM, Erik Joelsson wrote: >>> >>> Received: from [10.132.185.167] (/10.132.185.167) >>> by default (Oracle Beehive Gateway v4.0) >>> with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 >>> Subject: Re: RFR: 8199807& 8202738: AppCDS performs overly restrictive path >>> matching check >>> To: Jiangli Zhou, >>> "hotspot-runtime-dev at openjdk.java.net runtime" >>> , >>> build-dev >>> References: >>> From: Erik Joelsson >>> Organization: Oracle Corporation >>> Message-ID: >>> Date: Fri, 11 May 2018 14:33:20 -0700 >>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) >>> Gecko/20100101 Thunderbird/52.7.0 >>> MIME-Version: 1.0 >>> In-Reply-To: >>> Content-Type: text/plain; charset=utf-8; format=flowed >>> Content-Transfer-Encoding: 8bit >>> Content-Language: en-US >>> >>> Hello, >>> >>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>> >>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>> >>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>> >>> /Erik >>> >>> >>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>> Hi, >>>> >>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>> >>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>> >>>> The webrev includes the following three main parts: >>>> >>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>> >>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>> >>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>> >>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>> >>>> Thanks, >>>> Jiangli >>>> >>> Hello, >>> >>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>> >>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>> >>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>> >>> /Erik >>> >>> >>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>> Hi, >>>> >>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>> >>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>> >>>> The webrev includes the following three main parts: >>>> >>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>> >>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>> >>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>> >>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>> >>>> Thanks, >>>> Jiangli >>>> From ioi.lam at oracle.com Fri May 11 23:41:20 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 11 May 2018 16:41:20 -0700 Subject: RFR(xs): 8202669: Intermittent crash in ClassLoadingService::compute_class_size() In-Reply-To: <5AF61727.5060206@oracle.com> References: <5AF61727.5060206@oracle.com> Message-ID: <8eee5c1a-7787-5ba2-4492-03b38bb17798@oracle.com> Looks good. Thanks for fixing! - Ioi On 5/11/18 3:20 PM, Calvin Cheung wrote: > bug: https://bugs.openjdk.java.net/browse/JDK-8202669 > > webrev: http://cr.openjdk.java.net/~ccheung/8202669/webrev.00/ > > This bug surfaced after the change for JDK-8200466. > The proposed change is simply null checking the _transitive_interfaces > before accessing its size(). > > I ran the affected tests with the change 30 times on Oracle platforms > and didn't see any failures. > > thanks, > Calvin From jiangli.zhou at Oracle.COM Sat May 12 01:07:19 2018 From: jiangli.zhou at Oracle.COM (Jiangli Zhou) Date: Fri, 11 May 2018 18:07:19 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: <5AF62823.7050804@oracle.com> References: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> <5AF62823.7050804@oracle.com> Message-ID: <70F6AEB0-9103-4899-87D9-DC845BF66C51@oracle.com> Hi Calvin, Thanks for the review. > On May 11, 2018, at 4:32 PM, Calvin Cheung wrote: > > Hi Jiangli, > > Thanks for doing this useful change. > > I have a few minor comments: > > 1) sharedPathsMiscInfo.cpp > > 199 dp ++; > > Should the above be similar to line 194 as follows? > dp += path_sep_len; Good catch. Will fix. > > 2) filemap.cpp > > 244 if (ClassLoader::is_modules_image(name)) { > > I think the above could be: > if (is_modules_image()) { > Let?s keep it this way as is_modules_image() is a wrapper of 'ClassLoader::is_modules_image(name())?. We already have the ?name? in this case. > The is_modules_image() is a member function of SharedClassPathEntry. > > Btw, why is it necessary to get the runtime modules image path again in lines 244 - 246? As the runtime modules image path could be different from the dump time if the JDK image is moved/copied after archive generation, we need to make sure to use the correct file for file size check. The recored path in the archive is the dump time modules image path, which may no longer be existing at runtime. I will add some comments if that?s helpful. > > 3) BootClassPathMismatch.java > > Could you move line 103 to right above line 107 (opts.addPrefix(?))? > Because the archive dumping part of the test doesn?t use appJar. Sure. Thanks! Jiangli > > thanks, > Calvin > > On 5/11/18, 3:15 PM, Jiangli Zhou wrote: >> I?ve withdrawn my original weberv and removed the build change. Here is the updated webrev that only addresses 8199807. >> >> webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.00/ >> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >> >> Thanks, >> Jiangli >> >>> On May 11, 2018, at 2:44 PM, Jiangli Zhou wrote: >>> >>> Hi Erik, >>> >>> Thank you so much for investigating a proper fix for the vm_version.o issue. I will withdraw the build change from my original webrev. >>> >>> Thanks again for taking over the bug! >>> Jiangli >>> >>>> On May 11, 2018, at 2:33 PM, Erik Joelsson wrote: >>>> >>>> Received: from [10.132.185.167] (/10.132.185.167) >>>> by default (Oracle Beehive Gateway v4.0) >>>> with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 >>>> Subject: Re: RFR: 8199807& 8202738: AppCDS performs overly restrictive path >>>> matching check >>>> To: Jiangli Zhou, >>>> "hotspot-runtime-dev at openjdk.java.net runtime" >>>> , >>>> build-dev >>>> References: >>>> From: Erik Joelsson >>>> Organization: Oracle Corporation >>>> Message-ID: >>>> Date: Fri, 11 May 2018 14:33:20 -0700 >>>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) >>>> Gecko/20100101 Thunderbird/52.7.0 >>>> MIME-Version: 1.0 >>>> In-Reply-To: >>>> Content-Type: text/plain; charset=utf-8; format=flowed >>>> Content-Transfer-Encoding: 8bit >>>> Content-Language: en-US >>>> >>>> Hello, >>>> >>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>> >>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>> >>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>> >>>> /Erik >>>> >>>> >>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>> Hi, >>>>> >>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>> >>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>> >>>>> The webrev includes the following three main parts: >>>>> >>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>> >>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>> >>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>> >>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>> >>>>> Thanks, >>>>> Jiangli >>>>> >>>> Hello, >>>> >>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>> >>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>> >>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>> >>>> /Erik >>>> >>>> >>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>> Hi, >>>>> >>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>> >>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>> >>>>> The webrev includes the following three main parts: >>>>> >>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>> >>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>> >>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>> >>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>> >>>>> Thanks, >>>>> Jiangli >>>>> From markus.gronlund at oracle.com Sat May 12 04:51:04 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Sat, 12 May 2018 06:51:04 +0200 Subject: RFR(xs): 8202669: Intermittent crash in ClassLoadingService::compute_class_size() In-Reply-To: <5AF61727.5060206@oracle.com> References: <5AF61727.5060206@oracle.com> Message-ID: <6D631B48-2BE1-44C5-BA89-B51A7E595761@oracle.com> Hi Calvin, Could you please attempt to provide some explanation as to why this manifested as a consequence of JDK-8200466? The state in the core/.mdmp indicate the class to be corrupt / not yet initialized at all. On what invariant do we expect to be able to determine the size() of a klass if it is in a corrupt / not yet initialized state? I suspect there is a deeper issue here, did some unloading behavior change recently? Thanks Markus Sent from my iPhone > On 12 May 2018, at 00:20, Calvin Cheung wrote: > > bug: https://bugs.openjdk.java.net/browse/JDK-8202669 > > webrev: http://cr.openjdk.java.net/~ccheung/8202669/webrev.00/ > > This bug surfaced after the change for JDK-8200466. > The proposed change is simply null checking the _transitive_interfaces before accessing its size(). > > I ran the affected tests with the change 30 times on Oracle platforms and didn't see any failures. > > thanks, > Calvin From jini.george at oracle.com Sat May 12 05:05:23 2018 From: jini.george at oracle.com (Jini George) Date: Sat, 12 May 2018 10:35:23 +0530 Subject: RFR: (S): SA: clhsdb 'where -a' throws Assertion Failure with illegal code 236 when CDS is used In-Reply-To: <08b8fe9b-b662-f395-7bc4-b021795219b7@oracle.com> References: <82c69a0d-02f3-5db8-fedd-d52b00571a55@oracle.com> <41204e1d-11e1-41a9-af90-7cabd12098aa@oracle.com> <466139cc-b2c4-9abd-5f41-d7664291b2f6@oracle.com> <3cfea08f-2dca-4777-f6ff-fa643e9a5f01@oracle.com> <08b8fe9b-b662-f395-7bc4-b021795219b7@oracle.com> Message-ID: Thanks, Ioi. - Jini. On 5/11/2018 10:04 PM, Ioi Lam wrote: > I think all code that looks at the top of stack (e.g., users of > BytecodeStream) operate with the original bytecode (_aload_0), not the > quicken one (_fast_aload_0 or _nofast_aload_0), so this bug is never > discovered. But I think this is still a bug and should be fixed. > > I filed https://bugs.openjdk.java.net/browse/JDK-8203005 > > Thanks > > - Ioi > > > > > On 5/11/18 12:45 AM, Chris Plummer wrote: >> I'm not too sure how that field ends up getting used. I'd say what's >> likely in this case is that it's wrong but doesn't matter. Hopefully >> someone with a better understanding of the use of this field will >> chime in. >> >> Chris >> >> On 5/11/18 12:29 AM, Jini George wrote: >>> Thank you very much, Chris, for taking a look. >>> >>> The bytecodes table initialization in SA's Bytecodes.java mimics the >>> bytecodes table initialization in hotspot's Bytecodes::initialize() from >>> bytecodes.cpp. In bytecodes.cpp, we have the initialization of the >>> _nofast* bytecodes thus: >>> >>> >>> ? def(_nofast_aload_0????? , "nofast_aload_0"????? , "b"??? , NULL , >>> T_ILLEGAL,? 1, true , _aload_0??????? ); >>> ? def(_nofast_iload??????? , "nofast_iload"??????? , "bi"?? , NULL , >>> T_ILLEGAL,? 1, false, _iload????????? ); >>> >>> So looks like the result type in hotspot's bytecodes.cpp also needs >>> to be changed ? >>> >>> Thanks, >>> Jini. >>> >>> >>> On 5/8/2018 11:43 PM, Chris Plummer wrote: >>>> Hi Jini, >>>> >>>> Why are _nofast_aload_0 and _nofast_iload using return type >>>> BasicType.getTIllegal() when the return type is known? >>>> >>>> The test changes look good. >>>> >>>> thanks, >>>> >>>> Chris >>>> >>>> On 5/8/18 8:53 AM, Jini George wrote: >>>>> Thanks, Ioi. Could I get one more reviewer to take a look at this >>>>> ? >>>>> >>>>> Thanks, Jini. >>>>> >>>>> On 5/8/2018 8:55 PM, Ioi Lam wrote: >>>>>> Looks good. Thanks! >>>>>> >>>>>> - Ioi >>>>>> >>>>>> >>>>>> On 5/7/18 8:38 PM, Jini George wrote: >>>>>>> Thank you very much, Ioi, for the review and for the >>>>>>> clarifications and help provided offline. I have added the >>>>>>> checks for _nofast_getfield and _nofast_putfield. SA has a bug >>>>>>> due to which for iload, only the base bytecode (iload) gets >>>>>>> displayed -- fast_iload and nofast_iload do not get displayed. >>>>>>> JDK-8202693 (SA: clhsdb printall only displays the base >>>>>>> bytecode for iload) has been filed for this. I would add the >>>>>>> test for nofast_iload along with the fix for JDK-8202693. >>>>>>> >>>>>>> The modified webrev is at: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~jgeorge/8174995/webrev.01/ >>>>>>> >>>>>>> Thanks, Jini. >>>>>>> >>>>>>> On 4/27/2018 1:54 AM, Ioi Lam wrote: >>>>>>>> HI Jini, >>>>>>>> >>>>>>>> [1] "_nofast_aload" should be "_nofast_aload_0": aload and >>>>>>>> aload_0 are two different bytecodes. >>>>>>>> >>>>>>>> [2] Only the _nofast_aload_0 bytecode is tested. For >>>>>>>> completeness, do you think it makes sense to add test cases >>>>>>>> for these other 3 bytecodes? >>>>>>>> >>>>>>>> _nofast_getfield _nofast_putfield _nofast_iload >>>>>>>> >>>>>>>> >>>>>>>> Thanks - Ioi >>>>>>>> >>>>>>>> On 4/26/18 11:15 AM, Jini George wrote: >>>>>>>>> Hello all, >>>>>>>>> >>>>>>>>> Please review the following proposed fix for the issue: >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8174995 >>>>>>>>> >>>>>>>>> Webrev: >>>>>>>>> http://cr.openjdk.java.net/~jgeorge/8174995/webrev.00/ >>>>>>>>> >>>>>>>>> Issue: Clhsdb commands like 'where -a', 'printall' would >>>>>>>>> throw an illegal code assertion failure when CDS is used. >>>>>>>>> >>>>>>>>> Root cause and proposed fix: SA has been unaware of the new >>>>>>>>> ?bytecodes introduced for rewriting at CDS dump time >>>>>>>>> (_nofast* bytecodes). The fix is to make SA aware of these >>>>>>>>> new _nofast* bytecodes. >>>>>>>>> >>>>>>>>> Tests Run and Passed: SA tests on Mach5 (including the >>>>>>>>> tests modified to test this fix). >>>>>>>>> >>>>>>>>> Thank you, Jini. >>>>>>>> >>>>>> >>>> >>>> >> >> > From jini.george at oracle.com Sat May 12 05:06:31 2018 From: jini.george at oracle.com (Jini George) Date: Sat, 12 May 2018 10:36:31 +0530 Subject: RFR: (S): SA: clhsdb 'where -a' throws Assertion Failure with illegal code 236 when CDS is used In-Reply-To: <6227ef6c-45bd-f44a-c34b-37e84d243e0a@oracle.com> References: <82c69a0d-02f3-5db8-fedd-d52b00571a55@oracle.com> <41204e1d-11e1-41a9-af90-7cabd12098aa@oracle.com> <466139cc-b2c4-9abd-5f41-d7664291b2f6@oracle.com> <3cfea08f-2dca-4777-f6ff-fa643e9a5f01@oracle.com> <08b8fe9b-b662-f395-7bc4-b021795219b7@oracle.com> <6227ef6c-45bd-f44a-c34b-37e84d243e0a@oracle.com> Message-ID: <53371ed3-bbc3-4eb2-be50-90b37cafcbc7@oracle.com> Thanks, Chris. - Jini. On 5/11/2018 11:26 PM, Chris Plummer wrote: > Thanks Ioi. > > Jini, you can count me as a reviewer for your changes. > > thanks, > > Chris > > On 5/11/18 9:34 AM, Ioi Lam wrote: >> I think all code that looks at the top of stack (e.g., users of >> BytecodeStream) operate with the original bytecode (_aload_0), not the >> quicken one (_fast_aload_0 or _nofast_aload_0), so this bug is never >> discovered. But I think this is still a bug and should be fixed. >> >> I filed https://bugs.openjdk.java.net/browse/JDK-8203005 >> >> Thanks >> >> - Ioi >> >> >> >> >> On 5/11/18 12:45 AM, Chris Plummer wrote: >>> I'm not too sure how that field ends up getting used. I'd say what's >>> likely in this case is that it's wrong but doesn't matter. Hopefully >>> someone with a better understanding of the use of this field will >>> chime in. >>> >>> Chris >>> >>> On 5/11/18 12:29 AM, Jini George wrote: >>>> Thank you very much, Chris, for taking a look. >>>> >>>> The bytecodes table initialization in SA's Bytecodes.java mimics the >>>> bytecodes table initialization in hotspot's Bytecodes::initialize() >>>> from >>>> bytecodes.cpp. In bytecodes.cpp, we have the initialization of the >>>> _nofast* bytecodes thus: >>>> >>>> >>>> ? def(_nofast_aload_0????? , "nofast_aload_0"????? , "b"??? , >>>> NULL??? , T_ILLEGAL,? 1, true , _aload_0??????? ); >>>> ? def(_nofast_iload??????? , "nofast_iload"??????? , "bi"?? , >>>> NULL??? , T_ILLEGAL,? 1, false, _iload????????? ); >>>> >>>> So looks like the result type in hotspot's bytecodes.cpp also needs >>>> to be changed ? >>>> >>>> Thanks, >>>> Jini. >>>> >>>> >>>> On 5/8/2018 11:43 PM, Chris Plummer wrote: >>>>> Hi Jini, >>>>> >>>>> Why are _nofast_aload_0 and _nofast_iload using return type >>>>> BasicType.getTIllegal() when the return type is known? >>>>> >>>>> The test changes look good. >>>>> >>>>> thanks, >>>>> >>>>> Chris >>>>> >>>>> On 5/8/18 8:53 AM, Jini George wrote: >>>>>> Thanks, Ioi. Could I get one more reviewer to take a look at this >>>>>> ? >>>>>> >>>>>> Thanks, Jini. >>>>>> >>>>>> On 5/8/2018 8:55 PM, Ioi Lam wrote: >>>>>>> Looks good. Thanks! >>>>>>> >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> On 5/7/18 8:38 PM, Jini George wrote: >>>>>>>> Thank you very much, Ioi, for the review and for the >>>>>>>> clarifications and help provided offline. I have added the >>>>>>>> checks for _nofast_getfield and _nofast_putfield. SA has a bug >>>>>>>> due to which for iload, only the base bytecode (iload) gets >>>>>>>> displayed -- fast_iload and nofast_iload do not get displayed. >>>>>>>> JDK-8202693 (SA: clhsdb printall only displays the base >>>>>>>> bytecode for iload) has been filed for this. I would add the >>>>>>>> test for nofast_iload along with the fix for JDK-8202693. >>>>>>>> >>>>>>>> The modified webrev is at: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~jgeorge/8174995/webrev.01/ >>>>>>>> >>>>>>>> Thanks, Jini. >>>>>>>> >>>>>>>> On 4/27/2018 1:54 AM, Ioi Lam wrote: >>>>>>>>> HI Jini, >>>>>>>>> >>>>>>>>> [1] "_nofast_aload" should be "_nofast_aload_0": aload and >>>>>>>>> aload_0 are two different bytecodes. >>>>>>>>> >>>>>>>>> [2] Only the _nofast_aload_0 bytecode is tested. For >>>>>>>>> completeness, do you think it makes sense to add test cases >>>>>>>>> for these other 3 bytecodes? >>>>>>>>> >>>>>>>>> _nofast_getfield _nofast_putfield _nofast_iload >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks - Ioi >>>>>>>>> >>>>>>>>> On 4/26/18 11:15 AM, Jini George wrote: >>>>>>>>>> Hello all, >>>>>>>>>> >>>>>>>>>> Please review the following proposed fix for the issue: >>>>>>>>>> >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8174995 >>>>>>>>>> >>>>>>>>>> Webrev: >>>>>>>>>> http://cr.openjdk.java.net/~jgeorge/8174995/webrev.00/ >>>>>>>>>> >>>>>>>>>> Issue: Clhsdb commands like 'where -a', 'printall' would >>>>>>>>>> throw an illegal code assertion failure when CDS is used. >>>>>>>>>> >>>>>>>>>> Root cause and proposed fix: SA has been unaware of the new >>>>>>>>>> ?bytecodes introduced for rewriting at CDS dump time >>>>>>>>>> (_nofast* bytecodes). The fix is to make SA aware of these >>>>>>>>>> new _nofast* bytecodes. >>>>>>>>>> >>>>>>>>>> Tests Run and Passed: SA tests on Mach5 (including the >>>>>>>>>> tests modified to test this fix). >>>>>>>>>> >>>>>>>>>> Thank you, Jini. >>>>>>>>> >>>>>>> >>>>> >>>>> >>> >>> >> > > From ioi.lam at oracle.com Sat May 12 05:43:34 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 11 May 2018 22:43:34 -0700 Subject: Improving AppCDS for Custom Loaders In-Reply-To: <32beda94-2187-7b80-a933-ca42dddf63ca@oracle.com> References: <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> <732ce957-b07c-bff2-277b-ca335c348753@oracle.com> <660A1644-3A08-455D-931D-D95069C39853@oracle.com> <16902EC1-7A5F-4DBC-A815-855BA04509BA@oracle.com> <32beda94-2187-7b80-a933-ca42dddf63ca@oracle.com> Message-ID: On 5/10/18 4:05 PM, Ioi Lam wrote: > > > On 5/10/18 3:34 PM, Volker Simonis wrote: >> >> yumin qi > schrieb am >> Do. 10. Mai 2018 um 22:20: >> >> ??? Hi, Jiangli >> >> ??? ? Thanks for the info. >> >> ??? On Thu, May 10, 2018 at 11:39 AM, Jiangli Zhou >> ??? > wrote: >> >> ??????? Hi Yumin and Volker, >> >> ??????? I?ve create an umbrella RFE JDK-8202854 to keep track this >> ??????? effort. New sub-RFEs will be created and targeted for specific >> ??????? JDK releases when each sub-item matures. Please refer to >> ??????? JDK-8202854 for future progress. >> >> ??????? Yumin, please see below for comments/questions. >> >> ??????? > Agree not including objects in java heap at now --- thanks >> Jiangli for answering my question >> ??????? in other thread. >> >> ??????? I conclude from above that you are not using java heap object >> ??????? archiving due to different GC algorithm being used(?). I >> ??????? would like to understand more about your GC choices. Could you >> ??????? please provide more information on that? G1 GC is the default >> ??????? GC in JDK. Thanks to our GC team, the performance of G1 has >> ??????? been consistently improving over the recent JDK releases, >> ??????? especially in JDK 9 and 10. I strongly encourage you to do >> ??????? performance measurement/comparison and reevaluate your GC >> ??????? choice if G1 is not used. >> >> >> ??? Most of our applications still using CMS, we are pushing to switch >> ??? to G1GC at this time but still most of them still with CMS. I know >> ??? CMS will be removed so it will not be your focus. >> >> >> ??????? >? ?I think for #1, it does not conflict with the two layer >> ??????? archive solution. We can skip the classes from base CDS >> ??????? archive, only dump the non-base loaded classes into second >> ??????? archive, this gives one more option for user to choose. Also >> ??????? it will be good after dump archive to let the application >> ??????? continue to run. >> ??????? > >> ??????? >? ?Can we do a Full GC before dump at exit? it may unload not >> ??????? referenced classes, or it is not necessary since CDS wants to >> ??????? resolve the startup performance for all loaded classes? >> >> ??????? For classes loaded by the builtin loaders, they can be kept >> ??????? alive when archiving phase start. For classes loaded by user >> ??????? defined class loaders, we also need to prevent them from being >> ??????? unloaded before archiving. I have a preliminary idea for how >> ??????? to do that. I can share more information on that when it >> matures. >> >> >> ??? Yes, if classes with custom class loader cleaned, the class loader >> ??? itself will be purged. Full GC before exit may not be a good idea, >> ??? but we can skip unloading classes --- in case dump objects in java >> ??? heap. >> >> >> Why not dump the classes iteratively, while they are loaded. Class >> loading time would be the most natural time for dumping a class, and >> it?s the time when we know the most about a class (i.e. full class >> bytes). >> > > The archive is divided into RO/RW sections. Mutable metadata such as > InstanceKlasses are stored in RW, while non-mutable ones such as > ConstMethod and Symbol are stored into the RO section. In order to > have a single section each for RW and RO, currently we load all the > classes first, and then calculate the sizes of the 2 sections. > > One possible solution is to copy the loaded classes into a temp memory buffer, as the classes are being loaded. Then, when we decide to dump the archive, we can iterate over all the classes in the temp memory buffer. - Ioi >> ??? Thanks >> ??? Yumin >> >> > From volker.simonis at gmail.com Sat May 12 05:54:33 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Sat, 12 May 2018 07:54:33 +0200 Subject: Improving AppCDS for Custom Loaders In-Reply-To: References: <1f5d19a9-de3b-02ec-1f82-58028edb30c8@oracle.com> <732ce957-b07c-bff2-277b-ca335c348753@oracle.com> <660A1644-3A08-455D-931D-D95069C39853@oracle.com> <16902EC1-7A5F-4DBC-A815-855BA04509BA@oracle.com> <32beda94-2187-7b80-a933-ca42dddf63ca@oracle.com> Message-ID: On Sat, May 12, 2018 at 7:43 AM, Ioi Lam wrote: > > > On 5/10/18 4:05 PM, Ioi Lam wrote: >> >> >> >> On 5/10/18 3:34 PM, Volker Simonis wrote: >>> >>> >>> yumin qi > schrieb am Do. >>> 10. Mai 2018 um 22:20: >>> >>> Hi, Jiangli >>> >>> Thanks for the info. >>> >>> On Thu, May 10, 2018 at 11:39 AM, Jiangli Zhou >>> > wrote: >>> >>> Hi Yumin and Volker, >>> >>> I?ve create an umbrella RFE JDK-8202854 to keep track this >>> effort. New sub-RFEs will be created and targeted for specific >>> JDK releases when each sub-item matures. Please refer to >>> JDK-8202854 for future progress. >>> >>> Yumin, please see below for comments/questions. >>> >>> > Agree not including objects in java heap at now --- thanks >>> Jiangli for answering my question >>> in other thread. >>> >>> I conclude from above that you are not using java heap object >>> archiving due to different GC algorithm being used(?). I >>> would like to understand more about your GC choices. Could you >>> please provide more information on that? G1 GC is the default >>> GC in JDK. Thanks to our GC team, the performance of G1 has >>> been consistently improving over the recent JDK releases, >>> especially in JDK 9 and 10. I strongly encourage you to do >>> performance measurement/comparison and reevaluate your GC >>> choice if G1 is not used. >>> >>> >>> Most of our applications still using CMS, we are pushing to switch >>> to G1GC at this time but still most of them still with CMS. I know >>> CMS will be removed so it will not be your focus. >>> >>> >>> > I think for #1, it does not conflict with the two layer >>> archive solution. We can skip the classes from base CDS >>> archive, only dump the non-base loaded classes into second >>> archive, this gives one more option for user to choose. Also >>> it will be good after dump archive to let the application >>> continue to run. >>> > >>> > Can we do a Full GC before dump at exit? it may unload not >>> referenced classes, or it is not necessary since CDS wants to >>> resolve the startup performance for all loaded classes? >>> >>> For classes loaded by the builtin loaders, they can be kept >>> alive when archiving phase start. For classes loaded by user >>> defined class loaders, we also need to prevent them from being >>> unloaded before archiving. I have a preliminary idea for how >>> to do that. I can share more information on that when it matures. >>> >>> >>> Yes, if classes with custom class loader cleaned, the class loader >>> itself will be purged. Full GC before exit may not be a good idea, >>> but we can skip unloading classes --- in case dump objects in java >>> heap. >>> >>> >>> Why not dump the classes iteratively, while they are loaded. Class >>> loading time would be the most natural time for dumping a class, and it?s >>> the time when we know the most about a class (i.e. full class bytes). >>> >> >> The archive is divided into RO/RW sections. Mutable metadata such as >> InstanceKlasses are stored in RW, while non-mutable ones such as ConstMethod >> and Symbol are stored into the RO section. In order to have a single section >> each for RW and RO, currently we load all the classes first, and then >> calculate the sizes of the 2 sections. >> >> > One possible solution is to copy the loaded classes into a temp memory > buffer, as the classes are being loaded. Then, when we decide to dump the > archive, we can iterate over all the classes in the temp memory buffer. > This sounds like a good a good approach. I wanted to propose something similar but hadn't time to look at the possible implementation details more closely until now :) > - Ioi > > >>> Thanks >>> Yumin >>> >>> >> > From erik.osterlund at oracle.com Sun May 13 15:07:49 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Sun, 13 May 2018 16:07:49 +0100 Subject: RFR(M): 8202713: Create a MacroAssembler::access_load/store_at wrapper for S390 and PPC In-Reply-To: References: Message-ID: <551DD750-9B66-437F-BF71-5B8750CA9757@oracle.com> Hi Martin, Thank you for doing this. Looks good! Thanks, /Erik > On 11 May 2018, at 11:43, Doerr, Martin wrote: > > Hi Erik, > > thanks for creating the bug. It makes sense to use load/store_heap_oop like on the other platforms. > It will especially be useful when we add support for a GC with load barriers. > > Here?s the webrev: > http://cr.openjdk.java.net/~mdoerr/8202713_ppc64_s390_masm_access/webrev.00/ > > Please review. > > Best regards, > Martin > From jiangli.zhou at oracle.com Mon May 14 00:31:37 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Sun, 13 May 2018 17:31:37 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: <70F6AEB0-9103-4899-87D9-DC845BF66C51@oracle.com> References: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> <5AF62823.7050804@oracle.com> <70F6AEB0-9103-4899-87D9-DC845BF66C51@oracle.com> Message-ID: <4CBA8376-54A2-4863-8732-B16E1AE9C38A@oracle.com> Here is the updated webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.01/ Thanks, Jiangli > On May 11, 2018, at 6:07 PM, Jiangli Zhou wrote: > > Hi Calvin, > > Thanks for the review. > >> On May 11, 2018, at 4:32 PM, Calvin Cheung wrote: >> >> Hi Jiangli, >> >> Thanks for doing this useful change. >> >> I have a few minor comments: >> >> 1) sharedPathsMiscInfo.cpp >> >> 199 dp ++; >> >> Should the above be similar to line 194 as follows? >> dp += path_sep_len; > > Good catch. Will fix. > >> >> 2) filemap.cpp >> >> 244 if (ClassLoader::is_modules_image(name)) { >> >> I think the above could be: >> if (is_modules_image()) { >> > > Let?s keep it this way as is_modules_image() is a wrapper of 'ClassLoader::is_modules_image(name())?. We already have the ?name? in this case. > >> The is_modules_image() is a member function of SharedClassPathEntry. >> >> Btw, why is it necessary to get the runtime modules image path again in lines 244 - 246? > > As the runtime modules image path could be different from the dump time if the JDK image is moved/copied after archive generation, we need to make sure to use the correct file for file size check. The recored path in the archive is the dump time modules image path, which may no longer be existing at runtime. I will add some comments if that?s helpful. > >> >> 3) BootClassPathMismatch.java >> >> Could you move line 103 to right above line 107 (opts.addPrefix(?))? >> Because the archive dumping part of the test doesn?t use appJar. > > Sure. > > Thanks! > > Jiangli > >> >> thanks, >> Calvin >> >> On 5/11/18, 3:15 PM, Jiangli Zhou wrote: >>> I?ve withdrawn my original weberv and removed the build change. Here is the updated webrev that only addresses 8199807. >>> >>> webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.00/ >>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>> >>> Thanks, >>> Jiangli >>> >>>> On May 11, 2018, at 2:44 PM, Jiangli Zhou wrote: >>>> >>>> Hi Erik, >>>> >>>> Thank you so much for investigating a proper fix for the vm_version.o issue. I will withdraw the build change from my original webrev. >>>> >>>> Thanks again for taking over the bug! >>>> Jiangli >>>> >>>>> On May 11, 2018, at 2:33 PM, Erik Joelsson wrote: >>>>> >>>>> Received: from [10.132.185.167] (/10.132.185.167) >>>>> by default (Oracle Beehive Gateway v4.0) >>>>> with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 >>>>> Subject: Re: RFR: 8199807& 8202738: AppCDS performs overly restrictive path >>>>> matching check >>>>> To: Jiangli Zhou, >>>>> "hotspot-runtime-dev at openjdk.java.net runtime" >>>>> , >>>>> build-dev >>>>> References: >>>>> From: Erik Joelsson >>>>> Organization: Oracle Corporation >>>>> Message-ID: >>>>> Date: Fri, 11 May 2018 14:33:20 -0700 >>>>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) >>>>> Gecko/20100101 Thunderbird/52.7.0 >>>>> MIME-Version: 1.0 >>>>> In-Reply-To: >>>>> Content-Type: text/plain; charset=utf-8; format=flowed >>>>> Content-Transfer-Encoding: 8bit >>>>> Content-Language: en-US >>>>> >>>>> Hello, >>>>> >>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>> >>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>> >>>>> /Erik >>>>> >>>>> >>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>> Hi, >>>>>> >>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>> >>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>> >>>>>> The webrev includes the following three main parts: >>>>>> >>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>> >>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>> >>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>> >>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>> >>>>>> Thanks, >>>>>> Jiangli >>>>>> >>>>> Hello, >>>>> >>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>> >>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>> >>>>> /Erik >>>>> >>>>> >>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>> Hi, >>>>>> >>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>> >>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>> >>>>>> The webrev includes the following three main parts: >>>>>> >>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>> >>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>> >>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>> >>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>> >>>>>> Thanks, >>>>>> Jiangli From thomas.stuefe at gmail.com Mon May 14 09:52:42 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 14 May 2018 11:52:42 +0200 Subject: RFR(xxs): 8203032: PrintMetaspaceDcmd fails: 'Non-Class:' missing from stdout/stderr Message-ID: Hi all, may I please have reviews for this small fix: Bug: https://bugs.openjdk.java.net/browse/JDK-8203032 Fix: diff -r 2f79462aab9b test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java --- a/test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java Mon May 07 09:11:21 2018 +0200 +++ b/test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java Mon May 14 11:46:23 2018 +0200 @@ -33,8 +33,8 @@ * @library /test/lib * @modules java.base/jdk.internal.misc * java.management - * @run main/othervm -XX:MaxMetaspaceSize=201M -XX:+VerifyMetaspace -XX:+UseCompressedClassPointers PrintMetaspaceDcmd with-compressed-class-space - * @run main/othervm -XX:MaxMetaspaceSize=201M -XX:+VerifyMetaspace -XX:-UseCompressedClassPointers PrintMetaspaceDcmd without-compressed-class-space + * @run main/othervm -XX:MaxMetaspaceSize=201M -Xmx100M -XX:+UseCompressedOops -XX:+UseCompressedClassPointers PrintMetaspaceDcmd with-compressed-class-space + * @run main/othervm -XX:MaxMetaspaceSize=201M -Xmx100M -XX:-UseCompressedOops -XX:-UseCompressedClassPointers PrintMetaspaceDcmd without-compressed-class-space */ There had been two issues: 1) I let the test run with -XX:+VerifyMetaspace, but that one is a debug only option, so the test fails in release build. Well, that was just stupid :) My fix is to just remove the option, since I do not know how to specify an option for debug VMs only. 2) The test requires the VM to be started with +UseCompressedClassPointers, and assumed +UseCompressedOops would be always set, because it is default. However, the test is started with CompressedOops explicitly turned off. I do not understand why but my assumtion is that it has to do with -XX:MaxRAMPercentage=8 and running on a big machine, where 8% of the available RAM would bring the VM above the max. 32G heapsize needed to have compressed oops. My fix is to first, specify -XX:+UseCompressedOops explicitly, and second to limit the max heap size to -Xmx100M. I hope this will overrule the -XX:MaxRAMPercentage=8 handed down from above? But since I do not know exactly via which way this option came down, I cannot reproduce and test. So, I would be obliged if someone could test if this fix fixes the issues on Oracles tests. Thanks, Thomas From thomas.stuefe at gmail.com Mon May 14 13:31:55 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 14 May 2018 15:31:55 +0200 Subject: RFR(xxs): 8203032: PrintMetaspaceDcmd fails: 'Non-Class:' missing from stdout/stderr In-Reply-To: References: Message-ID: (webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203032-PrintMetaspaceDCmd-fails/webrev.00/webrev/) Thanks, Thomas On Mon, May 14, 2018 at 11:52 AM, Thomas St?fe wrote: > Hi all, > > may I please have reviews for this small fix: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203032 > > Fix: > > diff -r 2f79462aab9b > test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java > --- a/test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java > Mon May 07 09:11:21 2018 +0200 > +++ b/test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java > Mon May 14 11:46:23 2018 +0200 > @@ -33,8 +33,8 @@ > * @library /test/lib > * @modules java.base/jdk.internal.misc > * java.management > - * @run main/othervm -XX:MaxMetaspaceSize=201M -XX:+VerifyMetaspace > -XX:+UseCompressedClassPointers PrintMetaspaceDcmd > with-compressed-class-space > - * @run main/othervm -XX:MaxMetaspaceSize=201M -XX:+VerifyMetaspace > -XX:-UseCompressedClassPointers PrintMetaspaceDcmd > without-compressed-class-space > + * @run main/othervm -XX:MaxMetaspaceSize=201M -Xmx100M > -XX:+UseCompressedOops -XX:+UseCompressedClassPointers > PrintMetaspaceDcmd with-compressed-class-space > + * @run main/othervm -XX:MaxMetaspaceSize=201M -Xmx100M > -XX:-UseCompressedOops -XX:-UseCompressedClassPointers > PrintMetaspaceDcmd without-compressed-class-space > */ > > > There had been two issues: > > 1) I let the test run with -XX:+VerifyMetaspace, but that one is a > debug only option, so the test fails in release build. Well, that was > just stupid :) > > My fix is to just remove the option, since I do not know how to > specify an option for debug VMs only. > > 2) The test requires the VM to be started with > +UseCompressedClassPointers, and assumed +UseCompressedOops would be > always set, because it is default. However, the test is started with > CompressedOops explicitly turned off. I do not understand why but my > assumtion is that it has to do with -XX:MaxRAMPercentage=8 and running > on a big machine, where 8% of the available RAM would bring the VM > above the max. 32G heapsize needed to have compressed oops. > > My fix is to first, specify -XX:+UseCompressedOops explicitly, and > second to limit the max heap size to -Xmx100M. I hope this will > overrule the -XX:MaxRAMPercentage=8 handed down from above? But since > I do not know exactly via which way this option came down, I cannot > reproduce and test. So, I would be obliged if someone could test if > this fix fixes the issues on Oracles tests. > > > Thanks, Thomas From thomas.stuefe at gmail.com Mon May 14 13:33:45 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 14 May 2018 15:33:45 +0200 Subject: RFR(L): 8176808: Split up metaspace.cpp In-Reply-To: References: Message-ID: Ping... Is there anything I can do to make review easier? This is really mostly code shuffling around (out of metaspace.cpp into new files), so with a bit of effort I could cook up some diffs for those parts. ..Thomas On Fri, May 11, 2018 at 8:31 AM, Thomas St?fe wrote: > All builds are fixed, jdk-submit tests ran through sucessfully. > > Latest webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace-cpp/webrev.01/webrev/ > > Only difference to the first webrev is the placement of a single > semicolon in occupancyMap.hpp, to satisfy the Solaris compiler. > > Thanks, Thomas > > > > On Wed, May 9, 2018 at 5:08 PM, Thomas St?fe wrote: >> Hi all, >> >> This was a long time coming. Lets cut up this beast :) >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8176808 >> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace-cpp/webrev.00/webrev/ >> >> This change splits up metaspace.cpp into multiple parts. Note that I >> did not change much code (exceptions see below in details) - so this >> is mostly moving code around. >> >> This change follows the scheme tentatively outlined in JDK-8201572: >> >> - metaspace internal classes go into the sub directory >> share/memory/metaspace. Ideally, those should never be accessed from >> outside, apart from other metaspace classes and metaspace tests. They >> also go into the namespace ::metaspace::internals. >> >> - metaspace external classes (MetaspaceUtils, ClassLoaderMetaspace, >> etc) remain inside share/memory and, for now, remain in the global >> namespace. >> >> Note: the original idea was to move metaspace external classes to >> namespace ::metaspace. But I shied away from that, since that would >> mean fixing up all usages of these classes either with metaspace:: >> scope or with using declarations. I had to make a cut at some point. >> >> So here is something to decide: >> - should we then get rid of the ::metaspace::internals namespace, move >> metaspace-internal classes to the enclosing ::metaspace namespace and >> leave external classes in the global namespace ? >> - or should we follow through (maybe in a future patch): internal >> classes in ::metaspace::internal, and move external classes to >> ::metaspace ? >> >> Some more details: >> >> - the following classes moved out of metaspace.cpp into namespace >> "metaspace::internal" and the metaspace sub directory: >> MetaDebug, ChunkManager, SpaceManager, BlockFreeList and SmallBlocks, >> OccupancyMap, VirtualSpaceNode and VirtualSpaceList, the >> PrintCLDMetaspaceInfoClosure. >> >> - I also moved metachunk.hpp to internals, since class Metachunk is >> not used externally. What is used externally is Metablock though, so I >> split up metachunk.hpp into metabase.hpp, metablock.hpp and >> metachunk.hpp, holding Metabase, Metablock and Metachunk, >> respectively. >> >> - Now we see some ugly seams: metaspace.hpp is supposed to be the >> outside API contract, however, it contains implementation details >> which should not be there and which manifest now by having to use the >> metaspace::internals namespace here and there. If we care, we could >> solve this with a bit redesigning. >> >> - The ChunkManager gtest and the SpaceManager gtest had been >> implemented inside metaspace.cpp, since it was not possible to access >> the internal classes those tests needed in any other way. Since that >> is now possible, I moved the coding out to gtest-land and made them >> real gtests (exchanging the asserts for real gtest ASSERTS, among >> other things). >> Note that there are some tests left in metaspace.cpp >> (TestMetaspaceUtilsTest, TestVirtualSpaceNodeTest) - those were no >> gtests-in-disguise but were part of -XX:+ExecuteInternalVMTests. I >> guess these tests precede the gtest framework? I leave it up to others >> to decide what to do with them and to potentially move them out of >> metaspace.cpp. >> Side note: -XX:+ExecuteInternalVMTests seems to have bitrotted, see >> https://bugs.openjdk.java.net/browse/JDK-8202848. Does this get tested >> regularly? >> >> - metaspace.cpp is quite a bit smaller now - from ~5000 loc down to >> ~1700 loc. Arguably, one could split out more and clean up more, but I >> think this is a good start. >> >> --- >> >> I built locally on linux (release, fastdebug with and without pch, >> zero) and windows (64, 32bit). jdk-submit tests ran through with a >> build error on solaris sparc - I currently try to reproduce that build >> error with our very slow solaris machine. >> >> Thanks, Thomas From axel.siebenborn at sap.com Mon May 14 13:40:28 2018 From: axel.siebenborn at sap.com (Siebenborn, Axel) Date: Mon, 14 May 2018 13:40:28 +0000 Subject: RFR(xxs): 8203032: PrintMetaspaceDcmd fails: 'Non-Class:' missing from stdout/stderr In-Reply-To: References: Message-ID: Hi Thomas, your patch looks good and fixes the failing test. ?? Thanks for that quick fix! Cheers, Axel > -----Original Message----- > From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- > bounces at openjdk.java.net] On Behalf Of Thomas St?fe > Sent: Montag, 14. Mai 2018 11:53 > To: Hotspot dev runtime > Subject: RFR(xxs): 8203032: PrintMetaspaceDcmd fails: 'Non-Class:' missing > from stdout/stderr > > Hi all, > > may I please have reviews for this small fix: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203032 > > Fix: > > diff -r 2f79462aab9b > test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java > --- a/test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java > Mon May 07 09:11:21 2018 +0200 > +++ b/test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java > Mon May 14 11:46:23 2018 +0200 > @@ -33,8 +33,8 @@ > * @library /test/lib > * @modules java.base/jdk.internal.misc > * java.management > - * @run main/othervm -XX:MaxMetaspaceSize=201M - > XX:+VerifyMetaspace > -XX:+UseCompressedClassPointers PrintMetaspaceDcmd > with-compressed-class-space > - * @run main/othervm -XX:MaxMetaspaceSize=201M - > XX:+VerifyMetaspace > -XX:-UseCompressedClassPointers PrintMetaspaceDcmd > without-compressed-class-space > + * @run main/othervm -XX:MaxMetaspaceSize=201M -Xmx100M > -XX:+UseCompressedOops -XX:+UseCompressedClassPointers > PrintMetaspaceDcmd with-compressed-class-space > + * @run main/othervm -XX:MaxMetaspaceSize=201M -Xmx100M > -XX:-UseCompressedOops -XX:-UseCompressedClassPointers > PrintMetaspaceDcmd without-compressed-class-space > */ > > > There had been two issues: > > 1) I let the test run with -XX:+VerifyMetaspace, but that one is a > debug only option, so the test fails in release build. Well, that was > just stupid :) > > My fix is to just remove the option, since I do not know how to > specify an option for debug VMs only. > > 2) The test requires the VM to be started with > +UseCompressedClassPointers, and assumed +UseCompressedOops would > be > always set, because it is default. However, the test is started with > CompressedOops explicitly turned off. I do not understand why but my > assumtion is that it has to do with -XX:MaxRAMPercentage=8 and running > on a big machine, where 8% of the available RAM would bring the VM > above the max. 32G heapsize needed to have compressed oops. > > My fix is to first, specify -XX:+UseCompressedOops explicitly, and > second to limit the max heap size to -Xmx100M. I hope this will > overrule the -XX:MaxRAMPercentage=8 handed down from above? But > since > I do not know exactly via which way this option came down, I cannot > reproduce and test. So, I would be obliged if someone could test if > this fix fixes the issues on Oracles tests. > > > Thanks, Thomas From thomas.stuefe at gmail.com Mon May 14 13:41:10 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 14 May 2018 15:41:10 +0200 Subject: RFR(xxs): 8203032: PrintMetaspaceDcmd fails: 'Non-Class:' missing from stdout/stderr In-Reply-To: References: Message-ID: Hi Axel, great, thanks for checking :) ..Thomas On Mon, May 14, 2018 at 3:40 PM, Siebenborn, Axel wrote: > Hi Thomas, > your patch looks good and fixes the failing test. ?? > Thanks for that quick fix! > > Cheers, > Axel > >> -----Original Message----- >> From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- >> bounces at openjdk.java.net] On Behalf Of Thomas St?fe >> Sent: Montag, 14. Mai 2018 11:53 >> To: Hotspot dev runtime >> Subject: RFR(xxs): 8203032: PrintMetaspaceDcmd fails: 'Non-Class:' missing >> from stdout/stderr >> >> Hi all, >> >> may I please have reviews for this small fix: >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203032 >> >> Fix: >> >> diff -r 2f79462aab9b >> test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java >> --- a/test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java >> Mon May 07 09:11:21 2018 +0200 >> +++ b/test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java >> Mon May 14 11:46:23 2018 +0200 >> @@ -33,8 +33,8 @@ >> * @library /test/lib >> * @modules java.base/jdk.internal.misc >> * java.management >> - * @run main/othervm -XX:MaxMetaspaceSize=201M - >> XX:+VerifyMetaspace >> -XX:+UseCompressedClassPointers PrintMetaspaceDcmd >> with-compressed-class-space >> - * @run main/othervm -XX:MaxMetaspaceSize=201M - >> XX:+VerifyMetaspace >> -XX:-UseCompressedClassPointers PrintMetaspaceDcmd >> without-compressed-class-space >> + * @run main/othervm -XX:MaxMetaspaceSize=201M -Xmx100M >> -XX:+UseCompressedOops -XX:+UseCompressedClassPointers >> PrintMetaspaceDcmd with-compressed-class-space >> + * @run main/othervm -XX:MaxMetaspaceSize=201M -Xmx100M >> -XX:-UseCompressedOops -XX:-UseCompressedClassPointers >> PrintMetaspaceDcmd without-compressed-class-space >> */ >> >> >> There had been two issues: >> >> 1) I let the test run with -XX:+VerifyMetaspace, but that one is a >> debug only option, so the test fails in release build. Well, that was >> just stupid :) >> >> My fix is to just remove the option, since I do not know how to >> specify an option for debug VMs only. >> >> 2) The test requires the VM to be started with >> +UseCompressedClassPointers, and assumed +UseCompressedOops would >> be >> always set, because it is default. However, the test is started with >> CompressedOops explicitly turned off. I do not understand why but my >> assumtion is that it has to do with -XX:MaxRAMPercentage=8 and running >> on a big machine, where 8% of the available RAM would bring the VM >> above the max. 32G heapsize needed to have compressed oops. >> >> My fix is to first, specify -XX:+UseCompressedOops explicitly, and >> second to limit the max heap size to -Xmx100M. I hope this will >> overrule the -XX:MaxRAMPercentage=8 handed down from above? But >> since >> I do not know exactly via which way this option came down, I cannot >> reproduce and test. So, I would be obliged if someone could test if >> this fix fixes the issues on Oracles tests. >> >> >> Thanks, Thomas From magnus.ihse.bursie at oracle.com Mon May 14 21:56:37 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 14 May 2018 23:56:37 +0200 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: References: Message-ID: On 2018-05-11 23:33, Erik Joelsson wrote: > Hello, > > For the build change, it's very undesirable to always have to relink > libjvm on every incremental build. Such a change cannot be accepted. > > I have a counter suggestion, which is still a bit of a hack, but it > will cause vm_version.cpp to be recompiled (almost) every time > libjvm.so needs to be relinked. The drawback is that compiling > vm_version.cpp is now bound to happen absolutely last and so cannot > happen in parallel with other compilations. > > Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html This looks as good as it can get for a simple fix, but I'd still like to get on the record that I think the way we handle both __TIME__/__DATE__ in hotspot, and ad-hoc version strings in general, is broken and leave much to be desired. /Magnus > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 > > /Erik > > > On 2018-05-10 16:11, Jiangli Zhou wrote: >> Hi, >> >> Please review the following webrev that addresses the issue of >> copied/moved JDK image after generating a CDS archive. Thanks Karen >> Kinnear and Alan Baterman for initiating the investigation & >> discussions in this area (especially the ease of usage). Thanks Ioi >> for implementing a test case for moved JDK (JDK-8202935). >> >> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >> >> The webrev includes the following three main parts: >> >> 1. Reduced check for JDK ?modules? image file at runtime. The runtime >> path to the ?modules? image is no longer required to the the same as >> dump time path. Runtime performs file size check only for the >> ?modules? image, which must match with the dump time ?modules? size. >> Invalidation of an outdated archive is achieved by the existing >> vm_version string check (the archived vm_version string must match >> with the runtime vm_version string). >> >> 2. Boot path check are now performed based on the content of the >> archive. Also added a new test case in BootClassPathMismatch.java and >> add more comments for existing test cases. >> >> 3. Fixed the stale vm_version string issue with incremental build. >> The issue was discovered during the work of 8199807. CDS uses >> vm_version string as part of the runtime validation check for >> archive. A stale vm_version string causes the CDS runtime to >> mistakenly accept an outdated archive. The fix is to make sure >> vm_version.o is recompiled properly when the library/vm is rebuilt. >> >> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK >> image manually after generating an archive. Also tested with Ioi?s >> test both locally and via Mach5. >> >> Thanks, >> Jiangli >> > From jiangli.zhou at oracle.com Mon May 14 22:33:43 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Mon, 14 May 2018 15:33:43 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: References: Message-ID: > On May 14, 2018, at 2:56 PM, Magnus Ihse Bursie wrote: > > On 2018-05-11 23:33, Erik Joelsson wrote: >> Hello, >> >> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >> >> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >> >> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html > > This looks as good as it can get for a simple fix, but I'd still like to get on the record that I think the way we handle both __TIME__/__DATE__ in hotspot, and ad-hoc version strings in general, is broken and leave much to be desired. +1 on the above (both the review and the usage of __TIME__/__DATE__). Thanks! Jiangli > > /Magnus > > > >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >> >> /Erik >> >> >> On 2018-05-10 16:11, Jiangli Zhou wrote: >>> Hi, >>> >>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation & discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>> >>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>> >>> The webrev includes the following three main parts: >>> >>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>> >>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>> >>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>> >>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>> >>> Thanks, >>> Jiangli >>> >> > From calvin.cheung at oracle.com Tue May 15 00:31:45 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Mon, 14 May 2018 17:31:45 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: <4CBA8376-54A2-4863-8732-B16E1AE9C38A@oracle.com> References: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> <5AF62823.7050804@oracle.com> <70F6AEB0-9103-4899-87D9-DC845BF66C51@oracle.com> <4CBA8376-54A2-4863-8732-B16E1AE9C38A@oracle.com> Message-ID: <5AFA2A71.2000006@oracle.com> On 5/13/18, 5:31 PM, Jiangli Zhou wrote: > Here is the updated webrev: > http://cr.openjdk.java.net/~jiangli/8199807/webrev.01/ > Looks good. Could you clarify the following comment in filemap.cpp? 245 // We need to validate the runtime modules image file size against the archived 246 // size information, obtain the runtime modules image path. I'd suggest something like: // In order to validate the runtime modules image file size against the archived // size information, we need to obtain the runtime modules image path. I don't need to see another webrev for the above comment change. thanks, Calvin > > Thanks, > Jiangli > >> On May 11, 2018, at 6:07 PM, Jiangli Zhou >> wrote: >> >> Hi Calvin, >> >> Thanks for the review. >> >>> On May 11, 2018, at 4:32 PM, Calvin Cheung >>> wrote: >>> >>> Hi Jiangli, >>> >>> Thanks for doing this useful change. >>> >>> I have a few minor comments: >>> >>> 1) sharedPathsMiscInfo.cpp >>> >>> 199 dp ++; >>> >>> Should the above be similar to line 194 as follows? >>> dp += path_sep_len; >> >> Good catch. Will fix. >> >>> >>> 2) filemap.cpp >>> >>> 244 if (ClassLoader::is_modules_image(name)) { >>> >>> I think the above could be: >>> if (is_modules_image()) { >>> >> >> Let?s keep it this way as is_modules_image() is a wrapper of >> 'ClassLoader::is_modules_image(name())?. We already have the ?name? >> in this case. >> >>> The is_modules_image() is a member function of SharedClassPathEntry. >>> >>> Btw, why is it necessary to get the runtime modules image path again >>> in lines 244 - 246? >> >> As the runtime modules image path could be different from the dump >> time if the JDK image is moved/copied after archive generation, we >> need to make sure to use the correct file for file size check. The >> recored path in the archive is the dump time modules image path, >> which may no longer be existing at runtime. I will add some comments >> if that?s helpful. >> >>> >>> 3) BootClassPathMismatch.java >>> >>> Could you move line 103 to right above line 107 (opts.addPrefix(?))? >>> Because the archive dumping part of the test doesn?t use appJar. >> >> Sure. >> >> Thanks! >> >> Jiangli >> >>> >>> thanks, >>> Calvin >>> >>> On 5/11/18, 3:15 PM, Jiangli Zhou wrote: >>>> I?ve withdrawn my original weberv and removed the build change. >>>> Here is the updated webrev that only addresses 8199807. >>>> >>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.00/ >>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>> >>>> Thanks, >>>> Jiangli >>>> >>>>> On May 11, 2018, at 2:44 PM, Jiangli Zhou >>>>> wrote: >>>>> >>>>> Hi Erik, >>>>> >>>>> Thank you so much for investigating a proper fix for the >>>>> vm_version.o issue. I will withdraw the build change from my >>>>> original webrev. >>>>> >>>>> Thanks again for taking over the bug! >>>>> Jiangli >>>>> >>>>>> On May 11, 2018, at 2:33 PM, Erik >>>>>> Joelsson wrote: >>>>>> >>>>>> Received: from [10.132.185.167] (/10.132.185.167) >>>>>> by default (Oracle Beehive Gateway v4.0) >>>>>> with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 >>>>>> Subject: Re: RFR: 8199807& 8202738: AppCDS performs overly >>>>>> restrictive path >>>>>> matching check >>>>>> To: Jiangli Zhou, >>>>>> "hotspot-runtime-dev at openjdk.java.net runtime" >>>>>> , >>>>>> build-dev >>>>>> References: >>>>>> From: Erik Joelsson >>>>>> Organization: Oracle Corporation >>>>>> Message-ID: >>>>>> Date: Fri, 11 May 2018 14:33:20 -0700 >>>>>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) >>>>>> Gecko/20100101 Thunderbird/52.7.0 >>>>>> MIME-Version: 1.0 >>>>>> In-Reply-To: >>>>>> Content-Type: text/plain; charset=utf-8; format=flowed >>>>>> Content-Transfer-Encoding: 8bit >>>>>> Content-Language: en-US >>>>>> >>>>>> Hello, >>>>>> >>>>>> For the build change, it's very undesirable to always have to >>>>>> relink libjvm on every incremental build. Such a change cannot be >>>>>> accepted. >>>>>> >>>>>> I have a counter suggestion, which is still a bit of a hack, but >>>>>> it will cause vm_version.cpp to be recompiled (almost) every time >>>>>> libjvm.so needs to be relinked. The drawback is that compiling >>>>>> vm_version.cpp is now bound to happen absolutely last and so >>>>>> cannot happen in parallel with other compilations. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>> >>>>>> /Erik >>>>>> >>>>>> >>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Please review the following webrev that addresses the issue of >>>>>>> copied/moved JDK image after generating a CDS archive. Thanks >>>>>>> Karen Kinnear and Alan Baterman for initiating the >>>>>>> investigation& discussions in this area (especially the ease of >>>>>>> usage). Thanks Ioi for implementing a test case for moved JDK >>>>>>> (JDK-8202935). >>>>>>> >>>>>>> webrev: >>>>>>> http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>> >>>>>>> The webrev includes the following three main parts: >>>>>>> >>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The >>>>>>> runtime path to the ?modules? image is no longer required to the >>>>>>> the same as dump time path. Runtime performs file size check >>>>>>> only for the ?modules? image, which must match with the dump >>>>>>> time ?modules? size. Invalidation of an outdated archive is >>>>>>> achieved by the existing vm_version string check (the archived >>>>>>> vm_version string must match with the runtime vm_version string). >>>>>>> >>>>>>> 2. Boot path check are now performed based on the content of the >>>>>>> archive. Also added a new test case in >>>>>>> BootClassPathMismatch.java and add more comments for existing >>>>>>> test cases. >>>>>>> >>>>>>> 3. Fixed the stale vm_version string issue with incremental >>>>>>> build. The issue was discovered during the work of 8199807. CDS >>>>>>> uses vm_version string as part of the runtime validation check >>>>>>> for archive. A stale vm_version string causes the CDS runtime to >>>>>>> mistakenly accept an outdated archive. The fix is to make sure >>>>>>> vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>> >>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the >>>>>>> JDK image manually after generating an archive. Also tested with >>>>>>> Ioi?s test both locally and via Mach5. >>>>>>> >>>>>>> Thanks, >>>>>>> Jiangli >>>>>>> >>>>>> Hello, >>>>>> >>>>>> For the build change, it's very undesirable to always have to >>>>>> relink libjvm on every incremental build. Such a change cannot be >>>>>> accepted. >>>>>> >>>>>> I have a counter suggestion, which is still a bit of a hack, but >>>>>> it will cause vm_version.cpp to be recompiled (almost) every time >>>>>> libjvm.so needs to be relinked. The drawback is that compiling >>>>>> vm_version.cpp is now bound to happen absolutely last and so >>>>>> cannot happen in parallel with other compilations. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>> >>>>>> /Erik >>>>>> >>>>>> >>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Please review the following webrev that addresses the issue of >>>>>>> copied/moved JDK image after generating a CDS archive. Thanks >>>>>>> Karen Kinnear and Alan Baterman for initiating the >>>>>>> investigation& discussions in this area (especially the ease of >>>>>>> usage). Thanks Ioi for implementing a test case for moved JDK >>>>>>> (JDK-8202935). >>>>>>> >>>>>>> webrev: >>>>>>> http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>> >>>>>>> The webrev includes the following three main parts: >>>>>>> >>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The >>>>>>> runtime path to the ?modules? image is no longer required to the >>>>>>> the same as dump time path. Runtime performs file size check >>>>>>> only for the ?modules? image, which must match with the dump >>>>>>> time ?modules? size. Invalidation of an outdated archive is >>>>>>> achieved by the existing vm_version string check (the archived >>>>>>> vm_version string must match with the runtime vm_version string). >>>>>>> >>>>>>> 2. Boot path check are now performed based on the content of the >>>>>>> archive. Also added a new test case in >>>>>>> BootClassPathMismatch.java and add more comments for existing >>>>>>> test cases. >>>>>>> >>>>>>> 3. Fixed the stale vm_version string issue with incremental >>>>>>> build. The issue was discovered during the work of 8199807. CDS >>>>>>> uses vm_version string as part of the runtime validation check >>>>>>> for archive. A stale vm_version string causes the CDS runtime to >>>>>>> mistakenly accept an outdated archive. The fix is to make sure >>>>>>> vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>> >>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the >>>>>>> JDK image manually after generating an archive. Also tested with >>>>>>> Ioi?s test both locally and via Mach5. >>>>>>> >>>>>>> Thanks, >>>>>>> Jiangli > From jiangli.zhou at oracle.com Tue May 15 00:43:33 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Mon, 14 May 2018 17:43:33 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: <5AFA2A71.2000006@oracle.com> References: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> <5AF62823.7050804@oracle.com> <70F6AEB0-9103-4899-87D9-DC845BF66C51@oracle.com> <4CBA8376-54A2-4863-8732-B16E1AE9C38A@oracle.com> <5AFA2A71.2000006@oracle.com> Message-ID: <7CEF4628-4CDF-4082-BCD1-BEA5AB8C3A6E@oracle.com> > On May 14, 2018, at 5:31 PM, Calvin Cheung wrote: > > > >> On 5/13/18, 5:31 PM, Jiangli Zhou wrote: >> Here is the updated webrev: >> http://cr.openjdk.java.net/~jiangli/8199807/webrev.01/ > Looks good. > > Could you clarify the following comment in filemap.cpp? > > 245 // We need to validate the runtime modules image file size against the archived > 246 // size information, obtain the runtime modules image path. > > I'd suggest something like: > > // In order to validate the runtime modules image file size against the archived > // size information, we need to obtain the runtime modules image path. > > I don't need to see another webrev for the above comment change. Will clarify the comments a bit. Thank, Jiangli > > thanks, > Calvin >> >> Thanks, >> Jiangli >> >>> On May 11, 2018, at 6:07 PM, Jiangli Zhou wrote: >>> >>> Hi Calvin, >>> >>> Thanks for the review. >>> >>>> On May 11, 2018, at 4:32 PM, Calvin Cheung wrote: >>>> >>>> Hi Jiangli, >>>> >>>> Thanks for doing this useful change. >>>> >>>> I have a few minor comments: >>>> >>>> 1) sharedPathsMiscInfo.cpp >>>> >>>> 199 dp ++; >>>> >>>> Should the above be similar to line 194 as follows? >>>> dp += path_sep_len; >>> >>> Good catch. Will fix. >>> >>>> >>>> 2) filemap.cpp >>>> >>>> 244 if (ClassLoader::is_modules_image(name)) { >>>> >>>> I think the above could be: >>>> if (is_modules_image()) { >>>> >>> >>> Let?s keep it this way as is_modules_image() is a wrapper of 'ClassLoader::is_modules_image(name())?. We already have the ?name? in this case. >>> >>>> The is_modules_image() is a member function of SharedClassPathEntry. >>>> >>>> Btw, why is it necessary to get the runtime modules image path again in lines 244 - 246? >>> >>> As the runtime modules image path could be different from the dump time if the JDK image is moved/copied after archive generation, we need to make sure to use the correct file for file size check. The recored path in the archive is the dump time modules image path, which may no longer be existing at runtime. I will add some comments if that?s helpful. >>> >>>> >>>> 3) BootClassPathMismatch.java >>>> >>>> Could you move line 103 to right above line 107 (opts.addPrefix(?))? >>>> Because the archive dumping part of the test doesn?t use appJar. >>> >>> Sure. >>> >>> Thanks! >>> >>> Jiangli >>> >>>> >>>> thanks, >>>> Calvin >>>> >>>>> On 5/11/18, 3:15 PM, Jiangli Zhou wrote: >>>>> I?ve withdrawn my original weberv and removed the build change. Here is the updated webrev that only addresses 8199807. >>>>> >>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.00/ >>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>> >>>>> Thanks, >>>>> Jiangli >>>>> >>>>>> On May 11, 2018, at 2:44 PM, Jiangli Zhou wrote: >>>>>> >>>>>> Hi Erik, >>>>>> >>>>>> Thank you so much for investigating a proper fix for the vm_version.o issue. I will withdraw the build change from my original webrev. >>>>>> >>>>>> Thanks again for taking over the bug! >>>>>> Jiangli >>>>>> >>>>>>> On May 11, 2018, at 2:33 PM, Erik Joelsson wrote: >>>>>>> >>>>>>> Received: from [10.132.185.167] (/10.132.185.167) >>>>>>> by default (Oracle Beehive Gateway v4.0) >>>>>>> with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 >>>>>>> Subject: Re: RFR: 8199807& 8202738: AppCDS performs overly restrictive path >>>>>>> matching check >>>>>>> To: Jiangli Zhou, >>>>>>> "hotspot-runtime-dev at openjdk.java.net runtime" >>>>>>> , >>>>>>> build-dev >>>>>>> References: >>>>>>> From: Erik Joelsson >>>>>>> Organization: Oracle Corporation >>>>>>> Message-ID: >>>>>>> Date: Fri, 11 May 2018 14:33:20 -0700 >>>>>>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) >>>>>>> Gecko/20100101 Thunderbird/52.7.0 >>>>>>> MIME-Version: 1.0 >>>>>>> In-Reply-To: >>>>>>> Content-Type: text/plain; charset=utf-8; format=flowed >>>>>>> Content-Transfer-Encoding: 8bit >>>>>>> Content-Language: en-US >>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>> >>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>> >>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>> >>>>>>> /Erik >>>>>>> >>>>>>> >>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>> >>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>> >>>>>>>> The webrev includes the following three main parts: >>>>>>>> >>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>> >>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>> >>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>> >>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jiangli >>>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>> >>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>> >>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>> >>>>>>> /Erik >>>>>>> >>>>>>> >>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>> >>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>> >>>>>>>> The webrev includes the following three main parts: >>>>>>>> >>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>> >>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>> >>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>> >>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jiangli >> From ioi.lam at oracle.com Tue May 15 06:07:22 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 14 May 2018 23:07:22 -0700 Subject: RFR(XS) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful Message-ID: https://bugs.openjdk.java.net/browse/JDK-8189140 http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ Summary: 1. Removed the forced initialization of a few classes used by AppCDS at JVM start-up. ?? Instead, initialize these class on demand by calling InstanceKlass::initialize, which ?? is a quick no-op if the class is already initialized. 2. The only initialization left is that of a global lock. So I renamed the function ?? to SystemDictionaryShared::initialize_locks(). 3. I moved the call of this function from SystemDictionary::compute_java_loaders() to SystemDictionary::initialize() where it seems to fit. Testing with hs-tiers 1 and 2. Thanks - Ioi From david.holmes at oracle.com Tue May 15 06:25:25 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 15 May 2018 16:25:25 +1000 Subject: RFR(XS) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: References: Message-ID: <3c685cc7-cb21-49fd-8c5a-67a6c65c575f@oracle.com> Hi Ioi, On 15/05/2018 4:07 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8189140 > http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ > > > Summary: > > 1. Removed the forced initialization of a few classes used by AppCDS at > JVM start-up. > ?? Instead, initialize these class on demand by calling > InstanceKlass::initialize, which > ?? is a quick no-op if the class is already initialized. Any reason not to just add them to the set of pre-loaded and pre-initialized classes used with shared spaces in initialize_preloaded_classes()? Aside: It looks a little odd to lazy-init a couple of classes but then assume/expect others (like SecureClassLoader) are already initialized. But this seems functionally fine and a good tidy up (the class init stuff really didn't belong as a side-effect). The only thing I'd be double checking is that the same thread (presumably the main thread) is guaranteed to be the one doing the initialization. Thanks, David > 2. The only initialization left is that of a global lock. So I renamed > the function > ?? to SystemDictionaryShared::initialize_locks(). > > 3. I moved the call of this function from > SystemDictionary::compute_java_loaders() to > SystemDictionary::initialize() where it seems to fit. > > Testing with hs-tiers 1 and 2. > > Thanks > - Ioi From ioi.lam at oracle.com Tue May 15 07:44:01 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 15 May 2018 00:44:01 -0700 Subject: RFR(XS) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: <3c685cc7-cb21-49fd-8c5a-67a6c65c575f@oracle.com> References: <3c685cc7-cb21-49fd-8c5a-67a6c65c575f@oracle.com> Message-ID: On 5/14/18 11:25 PM, David Holmes wrote: > Hi Ioi, > > On 15/05/2018 4:07 PM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8189140 >> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >> >> >> Summary: >> >> 1. Removed the forced initialization of a few classes used by AppCDS >> at JVM start-up. >> ??? Instead, initialize these class on demand by calling >> InstanceKlass::initialize, which >> ??? is a quick no-op if the class is already initialized. > > Any reason not to just add them to the set of pre-loaded and > pre-initialized classes used with shared spaces in > initialize_preloaded_classes()? > Because initialize_preloaded_classes(), despite its name, doesn't actually call InstanceKlass::initialize(). It just resolves the klasses. If you try to call InstanceKlass::initialize() here, the VM would crash. A few classes are initialized inside functions like Threads::initialize_java_lang_classes(), which seems a bit like black magic to me, so I didn't want to muck with them. > Aside: It looks a little odd to lazy-init a couple of classes but then > assume/expect others (like SecureClassLoader) are already initialized. > The explicit init is only necessary before calling InstanceKlass::allocate_instance_handle(), which, unlike the 'new' bytecodes, does not implicitly initialize the class. The other code in SystemDictionaryShared.cpp access the classes through JavaCalls::call_{static,special,virtual}, where the class would be initialized implicitly as necessary in exactly the same way as the invoke{static,special,virtual} bytecodes. Perhaps we should add a JavaCalls::new_instance() method that implicitly initializes the class, just like the 'new' bytecode? Should I do that in this changeset? > But this seems functionally fine and a good tidy up (the class init > stuff really didn't belong as a side-effect). The only thing I'd be > double checking is that the same thread (presumably the main thread) > is guaranteed to be the one doing the initialization. > There's no such requirement on the threads. If another thread is already in the middle of initializing the class, the thread that class InstanceKlass::initialize() will just wait for the class initialization to complete. Thanks - Ioi > Thanks, > David > >> 2. The only initialization left is that of a global lock. So I >> renamed the function >> ??? to SystemDictionaryShared::initialize_locks(). >> >> 3. I moved the call of this function from >> SystemDictionary::compute_java_loaders() to >> SystemDictionary::initialize() where it seems to fit. >> >> Testing with hs-tiers 1 and 2. >> >> Thanks >> - Ioi From david.holmes at oracle.com Tue May 15 08:31:56 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 15 May 2018 18:31:56 +1000 Subject: RFR(XS) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: References: <3c685cc7-cb21-49fd-8c5a-67a6c65c575f@oracle.com> Message-ID: <151df8fe-7c36-5028-fe0e-37faa891969d@oracle.com> On 15/05/2018 5:44 PM, Ioi Lam wrote: > On 5/14/18 11:25 PM, David Holmes wrote: >> Hi Ioi, >> >> On 15/05/2018 4:07 PM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>> >>> >>> Summary: >>> >>> 1. Removed the forced initialization of a few classes used by AppCDS >>> at JVM start-up. >>> ??? Instead, initialize these class on demand by calling >>> InstanceKlass::initialize, which >>> ??? is a quick no-op if the class is already initialized. >> >> Any reason not to just add them to the set of pre-loaded and >> pre-initialized classes used with shared spaces in >> initialize_preloaded_classes()? >> > Because initialize_preloaded_classes(), despite its name, doesn't > actually call InstanceKlass::initialize(). It just resolves the klasses. > If you try to call InstanceKlass::initialize() here, the VM would crash. So much from truth in advertising ;-) > A few classes are initialized inside functions like > Threads::initialize_java_lang_classes(), which seems a bit like black > magic to me, so I didn't want to muck with them. Yes - the initialization order is not something to be trifled with. >> Aside: It looks a little odd to lazy-init a couple of classes but then >> assume/expect others (like SecureClassLoader) are already initialized. >> > The explicit init is only necessary before calling > InstanceKlass::allocate_instance_handle(), which, unlike the 'new' > bytecodes, does not implicitly initialize the class. What is allocate_instance_handle() supposed to do? Is it just a raw allocator? The class must be initialized before any constructor is called. > The other code in SystemDictionaryShared.cpp access the classes through > JavaCalls::call_{static,special,virtual}, where the class would be > initialized implicitly as necessary in exactly the same way as the > invoke{static,special,virtual} bytecodes. > > Perhaps we should add a JavaCalls::new_instance() method that implicitly > initializes the class, just like the 'new' bytecode? Should I do that in > this changeset? I'm not sure where you would do this or why you need to create an instance of these classes ?? >> But this seems functionally fine and a good tidy up (the class init >> stuff really didn't belong as a side-effect). The only thing I'd be >> double checking is that the same thread (presumably the main thread) >> is guaranteed to be the one doing the initialization. >> > There's no such requirement on the threads. If another thread is already > in the middle of initializing the class, the thread that class > InstanceKlass::initialize() will just wait for the class initialization > to complete. I was thinking purely from the perspective of not changing anything in the initialization order - and making sure you can't inadvertently do something on a thread that isn't allowed to load/initialize klasses. This is intended to be a small code cleanup so I presumed you didn't want to perturb things too much - ideally not at all. David > Thanks > - Ioi > >> Thanks, >> David >> >>> 2. The only initialization left is that of a global lock. So I >>> renamed the function >>> ??? to SystemDictionaryShared::initialize_locks(). >>> >>> 3. I moved the call of this function from >>> SystemDictionary::compute_java_loaders() to >>> SystemDictionary::initialize() where it seems to fit. >>> >>> Testing with hs-tiers 1 and 2. >>> >>> Thanks >>> - Ioi > From david.holmes at oracle.com Tue May 15 08:39:11 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 15 May 2018 18:39:11 +1000 Subject: RFR(XS) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: <151df8fe-7c36-5028-fe0e-37faa891969d@oracle.com> References: <3c685cc7-cb21-49fd-8c5a-67a6c65c575f@oracle.com> <151df8fe-7c36-5028-fe0e-37faa891969d@oracle.com> Message-ID: <5a5a99c8-df1d-c53b-03b7-05112df2bc3d@oracle.com> Correcting silly comment ... On 15/05/2018 6:31 PM, David Holmes wrote: > On 15/05/2018 5:44 PM, Ioi Lam wrote: >> On 5/14/18 11:25 PM, David Holmes wrote: >>> Hi Ioi, >>> >>> On 15/05/2018 4:07 PM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>>> >>>> >>>> Summary: >>>> >>>> 1. Removed the forced initialization of a few classes used by AppCDS >>>> at JVM start-up. >>>> ??? Instead, initialize these class on demand by calling >>>> InstanceKlass::initialize, which >>>> ??? is a quick no-op if the class is already initialized. >>> >>> Any reason not to just add them to the set of pre-loaded and >>> pre-initialized classes used with shared spaces in >>> initialize_preloaded_classes()? >>> >> Because initialize_preloaded_classes(), despite its name, doesn't >> actually call InstanceKlass::initialize(). It just resolves the >> klasses. If you try to call InstanceKlass::initialize() here, the VM >> would crash. > > So much from truth in advertising ;-) > >> A few classes are initialized inside functions like >> Threads::initialize_java_lang_classes(), which seems a bit like black >> magic to me, so I didn't want to muck with them. > > Yes - the initialization order is not something to be trifled with. > >>> Aside: It looks a little odd to lazy-init a couple of classes but >>> then assume/expect others (like SecureClassLoader) are already >>> initialized. >>> >> The explicit init is only necessary before calling >> InstanceKlass::allocate_instance_handle(), which, unlike the 'new' >> bytecodes, does not implicitly initialize the class. > > What is allocate_instance_handle() supposed to do? Is it just a raw > allocator? The class must be initialized before any constructor is called. > >> The other code in SystemDictionaryShared.cpp access the classes >> through JavaCalls::call_{static,special,virtual}, where the class >> would be initialized implicitly as necessary in exactly the same way >> as the invoke{static,special,virtual} bytecodes. >> >> Perhaps we should add a JavaCalls::new_instance() method that >> implicitly initializes the class, just like the 'new' bytecode? Should >> I do that in this changeset? > > I'm not sure where you would do this or why you need to create an > instance of these classes ?? Of course you are creating instances of these classes. But you're using JavaCalls::call_special to invoke the constructor, so isn't that already ensuring the class is initialized as you just described ?? David ----- >>> But this seems functionally fine and a good tidy up (the class init >>> stuff really didn't belong as a side-effect). The only thing I'd be >>> double checking is that the same thread (presumably the main thread) >>> is guaranteed to be the one doing the initialization. >>> >> There's no such requirement on the threads. If another thread is >> already in the middle of initializing the class, the thread that class >> InstanceKlass::initialize() will just wait for the class >> initialization to complete. > > I was thinking purely from the perspective of not changing anything in > the initialization order - and making sure you can't inadvertently do > something on a thread that isn't allowed to load/initialize klasses. > This is intended to be a small code cleanup so I presumed you didn't > want to perturb things too much - ideally not at all. > > David > >> Thanks >> - Ioi >> >>> Thanks, >>> David >>> >>>> 2. The only initialization left is that of a global lock. So I >>>> renamed the function >>>> ??? to SystemDictionaryShared::initialize_locks(). >>>> >>>> 3. I moved the call of this function from >>>> SystemDictionary::compute_java_loaders() to >>>> SystemDictionary::initialize() where it seems to fit. >>>> >>>> Testing with hs-tiers 1 and 2. >>>> >>>> Thanks >>>> - Ioi >> From thomas.stuefe at gmail.com Tue May 15 10:06:33 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 15 May 2018 12:06:33 +0200 Subject: Question about Reflection details Message-ID: Hi all, I have two questions about the non-native reflection mechanism in the VM: 1) For each method invocation we generate a new child class of internal.reflect.MagicAccessorImpl, which is loaded by its own instance of DelegatingClassLoader. The comment in jdk.internal.reflect.ClassDefiner states the reasons for this: There are two primary reasons for creating a new loader instead of defining these bytecodes directly into the defining loader of the target class: first, it avoids any possible security risk of having these bytecodes in the same loader. Second, it allows the generated bytecodes to be unloaded earlier than would otherwise be possible, decreasing run-time footprint. Do I understand this correctly: the lifetime of the MagicAccessorImpl instance, its class and its loading DelegatingClassLoader are defined by the lifetime of the java.lang.reflect.Method/Constructor object which keeps the MagicAccessorImpl instance in its methodAccessor/constructorAccessor field? So, if that Method/Constructor object is collected, its MagicAccessorImpl instance is collected, and since it is the only instance its class too, and since it is the only class in that DelegatingClassLoader that gets collected as well? 2) I see in my traces that for each Method.invoke(obj.foo()) we generate - one GeneratedMethodAccessorImplXXX class living in its own DelegatingClassLoader instance, which invokes obj.foo - and then one additional GeneratedConstructorAccessorXXX, again living in its very own DelegatingClassLoader instance, which invokes the constructor for the just generated GeneratedMethodAccessorImplXXX. The latter I see only if I switch off inflation. The very first (and only) time GeneratedMethodAccessorImplXXX is instantiated it will cause GeneratedConstructorAccessorXXX to be created for that one newInstance() call. But surely, in that case we could save one class loader and let the GeneratedConstructorAccessorXXX get loaded by the DelegatingClassLoader which also loaded the associated GeneratedMethodAccessorImplXXX? Or is it too much bother, since we are in an artificial scenario (noInflation=true)? ---- In general, I have the following question: Will the 1:1 relationship between MagicAccessorImpl class and DelegatingClassLoader instance always hold true? Or is there work in progress somewhere which would maybe bundle MagicAccessorImpl classes in one loader (e.g. (2) would be a candidate for this), or maybe do it without loaders? Thanks! Thomas From david.holmes at oracle.com Tue May 15 11:45:29 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 15 May 2018 21:45:29 +1000 Subject: Question about Reflection details In-Reply-To: References: Message-ID: Hi Thomas, This is really a core-libs question not a hotspot one. On 15/05/2018 8:06 PM, Thomas St?fe wrote: > Hi all, > > I have two questions about the non-native reflection mechanism in the VM: > > 1) For each method invocation we generate a new child class of > internal.reflect.MagicAccessorImpl, which is loaded by its own > instance of DelegatingClassLoader. > > The comment in jdk.internal.reflect.ClassDefiner states the reasons for this: > > > There are two primary reasons for creating a new loader > instead of defining these bytecodes directly into the defining > loader of the target class: first, it avoids any possible > security risk of having these bytecodes in the same loader. > Second, it allows the generated bytecodes to be unloaded earlier > than would otherwise be possible, decreasing run-time > footprint. > > > Do I understand this correctly: > > the lifetime of the MagicAccessorImpl instance, its class and its > loading DelegatingClassLoader are defined by the lifetime of the > java.lang.reflect.Method/Constructor object which keeps the > MagicAccessorImpl instance in its methodAccessor/constructorAccessor > field? > > So, if that Method/Constructor object is collected, its > MagicAccessorImpl instance is collected, and since it is the only > instance its class too, and since it is the only class in that > DelegatingClassLoader that gets collected as well? Yes. If we didn't use new classloaders we'd accumulate these generated classes and never be able to unload them. > > 2) I see in my traces that for each Method.invoke(obj.foo()) we generate > - one GeneratedMethodAccessorImplXXX class living in its own > DelegatingClassLoader instance, which invokes obj.foo > - and then one additional GeneratedConstructorAccessorXXX, again > living in its very own DelegatingClassLoader instance, which invokes > the constructor for the just generated GeneratedMethodAccessorImplXXX. That's a level of detail I'd have to go and look up. :) > The latter I see only if I switch off inflation. The very first (and > only) time GeneratedMethodAccessorImplXXX is instantiated it will > cause GeneratedConstructorAccessorXXX to be created for that one > newInstance() call. > > But surely, in that case we could save one class loader and let the > GeneratedConstructorAccessorXXX get loaded by the > DelegatingClassLoader which also loaded the associated > GeneratedMethodAccessorImplXXX? Or is it too much bother, since we are > in an artificial scenario (noInflation=true)? Don't know. > ---- > > In general, I have the following question: > > Will the 1:1 relationship between MagicAccessorImpl class and > DelegatingClassLoader instance always hold true? > > Or is there work in progress somewhere which would maybe bundle > MagicAccessorImpl classes in one loader (e.g. (2) would be a candidate > for this), or maybe do it without loaders? I'm not aware of any impending changes in this area, but it is a pretty old area ... the new reflection was added in 1.4. You could say that this core reflection is legacy and that MethodHandles should be used instead. As I said really a question for core-libs. Cheers, David > Thanks! > > Thomas > From thomas.stuefe at gmail.com Tue May 15 12:04:53 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 15 May 2018 14:04:53 +0200 Subject: RFR(xxs): 8203032: PrintMetaspaceDcmd fails: 'Non-Class:' missing from stdout/stderr In-Reply-To: References: Message-ID: Hi, I was able to reproduce the original bug by passing -javaoptions:-XX:-UseCompressedOops to the jtreg harness, and I verified that my proposed fix actually works. No firther tests needed. Now I only need a second (Reviewer) review, since Axel is only Committer. ..Thomas On Mon, May 14, 2018 at 3:31 PM, Thomas St?fe wrote: > (webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203032-PrintMetaspaceDCmd-fails/webrev.00/webrev/) > > Thanks, Thomas > > On Mon, May 14, 2018 at 11:52 AM, Thomas St?fe wrote: >> Hi all, >> >> may I please have reviews for this small fix: >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203032 >> >> Fix: >> >> diff -r 2f79462aab9b >> test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java >> --- a/test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java >> Mon May 07 09:11:21 2018 +0200 >> +++ b/test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java >> Mon May 14 11:46:23 2018 +0200 >> @@ -33,8 +33,8 @@ >> * @library /test/lib >> * @modules java.base/jdk.internal.misc >> * java.management >> - * @run main/othervm -XX:MaxMetaspaceSize=201M -XX:+VerifyMetaspace >> -XX:+UseCompressedClassPointers PrintMetaspaceDcmd >> with-compressed-class-space >> - * @run main/othervm -XX:MaxMetaspaceSize=201M -XX:+VerifyMetaspace >> -XX:-UseCompressedClassPointers PrintMetaspaceDcmd >> without-compressed-class-space >> + * @run main/othervm -XX:MaxMetaspaceSize=201M -Xmx100M >> -XX:+UseCompressedOops -XX:+UseCompressedClassPointers >> PrintMetaspaceDcmd with-compressed-class-space >> + * @run main/othervm -XX:MaxMetaspaceSize=201M -Xmx100M >> -XX:-UseCompressedOops -XX:-UseCompressedClassPointers >> PrintMetaspaceDcmd without-compressed-class-space >> */ >> >> >> There had been two issues: >> >> 1) I let the test run with -XX:+VerifyMetaspace, but that one is a >> debug only option, so the test fails in release build. Well, that was >> just stupid :) >> >> My fix is to just remove the option, since I do not know how to >> specify an option for debug VMs only. >> >> 2) The test requires the VM to be started with >> +UseCompressedClassPointers, and assumed +UseCompressedOops would be >> always set, because it is default. However, the test is started with >> CompressedOops explicitly turned off. I do not understand why but my >> assumtion is that it has to do with -XX:MaxRAMPercentage=8 and running >> on a big machine, where 8% of the available RAM would bring the VM >> above the max. 32G heapsize needed to have compressed oops. >> >> My fix is to first, specify -XX:+UseCompressedOops explicitly, and >> second to limit the max heap size to -Xmx100M. I hope this will >> overrule the -XX:MaxRAMPercentage=8 handed down from above? But since >> I do not know exactly via which way this option came down, I cannot >> reproduce and test. So, I would be obliged if someone could test if >> this fix fixes the issues on Oracles tests. >> >> >> Thanks, Thomas From thomas.stuefe at gmail.com Tue May 15 12:19:06 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 15 May 2018 14:19:06 +0200 Subject: Question about Reflection details In-Reply-To: References: Message-ID: Thank you David! I will take a closer look at MethodHandles and how they are implemented. And I will redirect this question to core-libs. Best Regards, Thomas On Tue, May 15, 2018 at 1:45 PM, David Holmes wrote: > Hi Thomas, > > This is really a core-libs question not a hotspot one. > > > On 15/05/2018 8:06 PM, Thomas St?fe wrote: >> >> Hi all, >> >> I have two questions about the non-native reflection mechanism in the VM: >> >> 1) For each method invocation we generate a new child class of >> internal.reflect.MagicAccessorImpl, which is loaded by its own >> instance of DelegatingClassLoader. >> >> The comment in jdk.internal.reflect.ClassDefiner states the reasons for >> this: >> >> >> There are two primary reasons for creating a new loader >> instead of defining these bytecodes directly into the defining >> loader of the target class: first, it avoids any possible >> security risk of having these bytecodes in the same loader. >> Second, it allows the generated bytecodes to be unloaded earlier >> than would otherwise be possible, decreasing run-time >> footprint. >> >> >> Do I understand this correctly: >> >> the lifetime of the MagicAccessorImpl instance, its class and its >> loading DelegatingClassLoader are defined by the lifetime of the >> java.lang.reflect.Method/Constructor object which keeps the >> MagicAccessorImpl instance in its methodAccessor/constructorAccessor >> field? >> >> So, if that Method/Constructor object is collected, its >> MagicAccessorImpl instance is collected, and since it is the only >> instance its class too, and since it is the only class in that >> DelegatingClassLoader that gets collected as well? > > > Yes. If we didn't use new classloaders we'd accumulate these generated > classes and never be able to unload them. > >> >> 2) I see in my traces that for each Method.invoke(obj.foo()) we generate >> - one GeneratedMethodAccessorImplXXX class living in its own >> DelegatingClassLoader instance, which invokes obj.foo >> - and then one additional GeneratedConstructorAccessorXXX, again >> living in its very own DelegatingClassLoader instance, which invokes >> the constructor for the just generated GeneratedMethodAccessorImplXXX. > > > That's a level of detail I'd have to go and look up. :) > >> The latter I see only if I switch off inflation. The very first (and >> only) time GeneratedMethodAccessorImplXXX is instantiated it will >> cause GeneratedConstructorAccessorXXX to be created for that one >> newInstance() call. >> >> But surely, in that case we could save one class loader and let the >> GeneratedConstructorAccessorXXX get loaded by the >> DelegatingClassLoader which also loaded the associated >> GeneratedMethodAccessorImplXXX? Or is it too much bother, since we are >> in an artificial scenario (noInflation=true)? > > > Don't know. > >> ---- >> >> In general, I have the following question: >> >> Will the 1:1 relationship between MagicAccessorImpl class and >> DelegatingClassLoader instance always hold true? >> >> Or is there work in progress somewhere which would maybe bundle >> MagicAccessorImpl classes in one loader (e.g. (2) would be a candidate >> for this), or maybe do it without loaders? > > > I'm not aware of any impending changes in this area, but it is a pretty old > area ... the new reflection was added in 1.4. You could say that this core > reflection is legacy and that MethodHandles should be used instead. > > As I said really a question for core-libs. > > Cheers, > David > >> Thanks! >> >> Thomas >> > From zgu at redhat.com Tue May 15 12:22:04 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 15 May 2018 08:22:04 -0400 Subject: RFR(xxs): 8203032: PrintMetaspaceDcmd fails: 'Non-Class:' missing from stdout/stderr In-Reply-To: References: Message-ID: Looks good to me. Thanks, -Zhengyu On 05/15/2018 08:04 AM, Thomas St?fe wrote: > Hi, > > I was able to reproduce the original bug by passing > -javaoptions:-XX:-UseCompressedOops to the jtreg harness, and I > verified that my proposed fix actually works. No firther tests needed. > > Now I only need a second (Reviewer) review, since Axel is only Committer. > > ..Thomas > > On Mon, May 14, 2018 at 3:31 PM, Thomas St?fe wrote: >> (webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203032-PrintMetaspaceDCmd-fails/webrev.00/webrev/) >> >> Thanks, Thomas >> >> On Mon, May 14, 2018 at 11:52 AM, Thomas St?fe wrote: >>> Hi all, >>> >>> may I please have reviews for this small fix: >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203032 >>> >>> Fix: >>> >>> diff -r 2f79462aab9b >>> test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java >>> --- a/test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java >>> Mon May 07 09:11:21 2018 +0200 >>> +++ b/test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java >>> Mon May 14 11:46:23 2018 +0200 >>> @@ -33,8 +33,8 @@ >>> * @library /test/lib >>> * @modules java.base/jdk.internal.misc >>> * java.management >>> - * @run main/othervm -XX:MaxMetaspaceSize=201M -XX:+VerifyMetaspace >>> -XX:+UseCompressedClassPointers PrintMetaspaceDcmd >>> with-compressed-class-space >>> - * @run main/othervm -XX:MaxMetaspaceSize=201M -XX:+VerifyMetaspace >>> -XX:-UseCompressedClassPointers PrintMetaspaceDcmd >>> without-compressed-class-space >>> + * @run main/othervm -XX:MaxMetaspaceSize=201M -Xmx100M >>> -XX:+UseCompressedOops -XX:+UseCompressedClassPointers >>> PrintMetaspaceDcmd with-compressed-class-space >>> + * @run main/othervm -XX:MaxMetaspaceSize=201M -Xmx100M >>> -XX:-UseCompressedOops -XX:-UseCompressedClassPointers >>> PrintMetaspaceDcmd without-compressed-class-space >>> */ >>> >>> >>> There had been two issues: >>> >>> 1) I let the test run with -XX:+VerifyMetaspace, but that one is a >>> debug only option, so the test fails in release build. Well, that was >>> just stupid :) >>> >>> My fix is to just remove the option, since I do not know how to >>> specify an option for debug VMs only. >>> >>> 2) The test requires the VM to be started with >>> +UseCompressedClassPointers, and assumed +UseCompressedOops would be >>> always set, because it is default. However, the test is started with >>> CompressedOops explicitly turned off. I do not understand why but my >>> assumtion is that it has to do with -XX:MaxRAMPercentage=8 and running >>> on a big machine, where 8% of the available RAM would bring the VM >>> above the max. 32G heapsize needed to have compressed oops. >>> >>> My fix is to first, specify -XX:+UseCompressedOops explicitly, and >>> second to limit the max heap size to -Xmx100M. I hope this will >>> overrule the -XX:MaxRAMPercentage=8 handed down from above? But since >>> I do not know exactly via which way this option came down, I cannot >>> reproduce and test. So, I would be obliged if someone could test if >>> this fix fixes the issues on Oracles tests. >>> >>> >>> Thanks, Thomas From thomas.stuefe at gmail.com Tue May 15 12:33:08 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 15 May 2018 14:33:08 +0200 Subject: RFR(xxs): 8203032: PrintMetaspaceDcmd fails: 'Non-Class:' missing from stdout/stderr In-Reply-To: References: Message-ID: Thank you Zhengyu! On Tue, May 15, 2018 at 2:22 PM, Zhengyu Gu wrote: > Looks good to me. > > Thanks, > > -Zhengyu > > > On 05/15/2018 08:04 AM, Thomas St?fe wrote: >> >> Hi, >> >> I was able to reproduce the original bug by passing >> -javaoptions:-XX:-UseCompressedOops to the jtreg harness, and I >> verified that my proposed fix actually works. No firther tests needed. >> >> Now I only need a second (Reviewer) review, since Axel is only Committer. >> >> ..Thomas >> >> On Mon, May 14, 2018 at 3:31 PM, Thomas St?fe >> wrote: >>> >>> (webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203032-PrintMetaspaceDCmd-fails/webrev.00/webrev/) >>> >>> Thanks, Thomas >>> >>> On Mon, May 14, 2018 at 11:52 AM, Thomas St?fe >>> wrote: >>>> >>>> Hi all, >>>> >>>> may I please have reviews for this small fix: >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203032 >>>> >>>> Fix: >>>> >>>> diff -r 2f79462aab9b >>>> test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java >>>> --- a/test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java >>>> Mon May 07 09:11:21 2018 +0200 >>>> +++ b/test/hotspot/jtreg/runtime/Metaspace/PrintMetaspaceDcmd.java >>>> Mon May 14 11:46:23 2018 +0200 >>>> @@ -33,8 +33,8 @@ >>>> * @library /test/lib >>>> * @modules java.base/jdk.internal.misc >>>> * java.management >>>> - * @run main/othervm -XX:MaxMetaspaceSize=201M -XX:+VerifyMetaspace >>>> -XX:+UseCompressedClassPointers PrintMetaspaceDcmd >>>> with-compressed-class-space >>>> - * @run main/othervm -XX:MaxMetaspaceSize=201M -XX:+VerifyMetaspace >>>> -XX:-UseCompressedClassPointers PrintMetaspaceDcmd >>>> without-compressed-class-space >>>> + * @run main/othervm -XX:MaxMetaspaceSize=201M -Xmx100M >>>> -XX:+UseCompressedOops -XX:+UseCompressedClassPointers >>>> PrintMetaspaceDcmd with-compressed-class-space >>>> + * @run main/othervm -XX:MaxMetaspaceSize=201M -Xmx100M >>>> -XX:-UseCompressedOops -XX:-UseCompressedClassPointers >>>> PrintMetaspaceDcmd without-compressed-class-space >>>> */ >>>> >>>> >>>> There had been two issues: >>>> >>>> 1) I let the test run with -XX:+VerifyMetaspace, but that one is a >>>> debug only option, so the test fails in release build. Well, that was >>>> just stupid :) >>>> >>>> My fix is to just remove the option, since I do not know how to >>>> specify an option for debug VMs only. >>>> >>>> 2) The test requires the VM to be started with >>>> +UseCompressedClassPointers, and assumed +UseCompressedOops would be >>>> always set, because it is default. However, the test is started with >>>> CompressedOops explicitly turned off. I do not understand why but my >>>> assumtion is that it has to do with -XX:MaxRAMPercentage=8 and running >>>> on a big machine, where 8% of the available RAM would bring the VM >>>> above the max. 32G heapsize needed to have compressed oops. >>>> >>>> My fix is to first, specify -XX:+UseCompressedOops explicitly, and >>>> second to limit the max heap size to -Xmx100M. I hope this will >>>> overrule the -XX:MaxRAMPercentage=8 handed down from above? But since >>>> I do not know exactly via which way this option came down, I cannot >>>> reproduce and test. So, I would be obliged if someone could test if >>>> this fix fixes the issues on Oracles tests. >>>> >>>> >>>> Thanks, Thomas From goetz.lindenmaier at sap.com Tue May 15 12:42:47 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 15 May 2018 12:42:47 +0000 Subject: RFR(M): 8202713: Create a MacroAssembler::access_load/store_at wrapper for S390 and PPC In-Reply-To: References: Message-ID: <63f242b9552844508c77017d240d42b1@sap.com> Hi Martin, I had a look at your code. This also is the first time I look at the new barrier handling. I find it confusing this is called "barrier". It handles not only the barriers but does the accesses, too. Thus I would have called the whole functionality OopAccessAssembler or JavaHeapAccessAssembler. Why should I call BarrierSetAssembler::load_at if I want to load a value from the heap? It would read better as OopAccessAssembler::load. Further, I miss some comment of what the changed functions are doing, e.g. for BarrierSetAssembler::load_at(): "Generate code to load a value of Java type "type" from address into register dst. Issue gc barriers, memory ordering, oop compression, null handling etc. acording to decorators." The functions are quite complex, thus a bit of comment would help. The documentation should be similar on all platforms. Also, I don't like the directory structure in the cpu directory. There are no subdirectories for interpreter, c1, c2 etc. either. But this is not subject of your change. Detailed review: barrierSetAssembler_ppc.cpp The name of the label is_null is misleading. It sounds as the name of a boolean variable. Maybe "L_handle_null"? And please use explicit compare for pointers: if (is_null == NULL) { Shouldn't there be an assertion that no other decorators are set than those handled here? Like all the memory ordering ones? macroAssembler_ppc.hpp: This header uses DecoratorSet thus should #include accessDecorators.hpp access_load/store_at() This is a misleading name for this function. I would think this should go to nativeInst_ppc.cpp :) Why would you access a store?? But this needs to be changed on all platforms. So not scope of this review. The function is not really needed. Make it private, or inline it into load/store_heap_oop. load_heap_oop_not_null: Please remove this on ppc, too. It's the only platform that still uses this function. There are only 6 uses. Please also remove the dead definitions of it on aarch64 and x86, as well as the comment in s390.ad mentioning it. (not sure about aarch64 as you can't easily test this). Please extend the comment above load_heap_oop. It gained new functionality, and after removing the _not_null functions along with their documentation the current comment is pointless. Mention what the label is good for. Rename the label as above. macroAssembler_ppc.inline.hpp Please rename the boolean variable as_raw to without_gc_barrier. Actually, the DecoratorSet name AS_RAW is misleading, there is still encode/decode etc. But this is not in the scope of this change. stubGenerator_ppc.cpp Remove comment // Load the oop. Makes the line too long, and not too much information gained :) Similar comments hold for the s390 case. Best regards, Goetz. > -----Original Message----- > From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- > bounces at openjdk.java.net] On Behalf Of Doerr, Martin > Sent: Freitag, 11. Mai 2018 12:43 > To: Erik ?sterlund ; hotspot-runtime- > dev at openjdk.java.net > Subject: [CAUTION] RFR(M): 8202713: Create a > MacroAssembler::access_load/store_at wrapper for S390 and PPC > > Hi Erik, > > thanks for creating the bug. It makes sense to use load/store_heap_oop like > on the other platforms. > It will especially be useful when we add support for a GC with load barriers. > > Here's the webrev: > http://cr.openjdk.java.net/~mdoerr/8202713_ppc64_s390_masm_access/w > ebrev.00/ > > Please review. > > Best regards, > Martin From ioi.lam at oracle.com Tue May 15 16:13:31 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 15 May 2018 09:13:31 -0700 Subject: RFR(XS) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: <5a5a99c8-df1d-c53b-03b7-05112df2bc3d@oracle.com> References: <3c685cc7-cb21-49fd-8c5a-67a6c65c575f@oracle.com> <151df8fe-7c36-5028-fe0e-37faa891969d@oracle.com> <5a5a99c8-df1d-c53b-03b7-05112df2bc3d@oracle.com> Message-ID: <45077fa1-5ae5-6b2b-3b0f-0c2b7f13dde6@oracle.com> On 5/15/18 1:39 AM, David Holmes wrote: > Correcting silly comment ... > > On 15/05/2018 6:31 PM, David Holmes wrote: >> On 15/05/2018 5:44 PM, Ioi Lam wrote: >>> On 5/14/18 11:25 PM, David Holmes wrote: >>>> Hi Ioi, >>>> >>>> On 15/05/2018 4:07 PM, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>>>> >>>>> >>>>> Summary: >>>>> >>>>> 1. Removed the forced initialization of a few classes used by >>>>> AppCDS at JVM start-up. >>>>> ??? Instead, initialize these class on demand by calling >>>>> InstanceKlass::initialize, which >>>>> ??? is a quick no-op if the class is already initialized. >>>> >>>> Any reason not to just add them to the set of pre-loaded and >>>> pre-initialized classes used with shared spaces in >>>> initialize_preloaded_classes()? >>>> >>> Because initialize_preloaded_classes(), despite its name, doesn't >>> actually call InstanceKlass::initialize(). It just resolves the >>> klasses. If you try to call InstanceKlass::initialize() here, the VM >>> would crash. >> >> So much from truth in advertising ;-) >> >>> A few classes are initialized inside functions like >>> Threads::initialize_java_lang_classes(), which seems a bit like >>> black magic to me, so I didn't want to muck with them. >> >> Yes - the initialization order is not something to be trifled with. >> >>>> Aside: It looks a little odd to lazy-init a couple of classes but >>>> then assume/expect others (like SecureClassLoader) are already >>>> initialized. >>>> >>> The explicit init is only necessary before calling >>> InstanceKlass::allocate_instance_handle(), which, unlike the 'new' >>> bytecodes, does not implicitly initialize the class. >> >> What is allocate_instance_handle() supposed to do? Is it just a raw >> allocator? The class must be initialized before any constructor is >> called. >> >>> The other code in SystemDictionaryShared.cpp access the classes >>> through JavaCalls::call_{static,special,virtual}, where the class >>> would be initialized implicitly as necessary in exactly the same way >>> as the invoke{static,special,virtual} bytecodes. >>> >>> Perhaps we should add a JavaCalls::new_instance() method that >>> implicitly initializes the class, just like the 'new' bytecode? >>> Should I do that in this changeset? >> >> I'm not sure where you would do this or why you need to create an >> instance of these classes ?? > > Of course you are creating instances of these classes. But you're > using JavaCalls::call_special to invoke the constructor, so isn't that > already ensuring the class is initialized as you just described ?? > I should have clarified what I said above about the invoke bytecodes: if I remember correctly, the only 4 bytecodes that implicitly initialize the operand's class are - new, invokestatic, getstatic and putstatic. invokespecial and invokevirtual do not perform initialization, because they operate on an object, which has already been new'ed, which means its class has already been initialized by the 'new' bytecode. > >>>> But this seems functionally fine and a good tidy up (the class init >>>> stuff really didn't belong as a side-effect). The only thing I'd be >>>> double checking is that the same thread (presumably the main >>>> thread) is guaranteed to be the one doing the initialization. >>>> >>> There's no such requirement on the threads. If another thread is >>> already in the middle of initializing the class, the thread that >>> class InstanceKlass::initialize() will just wait for the class >>> initialization to complete. >> >> I was thinking purely from the perspective of not changing anything >> in the initialization order - and making sure you can't inadvertently >> do something on a thread that isn't allowed to load/initialize >> klasses. This is intended to be a small code cleanup so I presumed >> you didn't want to perturb things too much - ideally not at all. For, the 3 places where the klass->initialize() is added, they all immediately execute Java code, which could have any side effect, including loading/initializing more classes. So I think it's safe to add the initialization call there. Thanks - Ioi >> >> David >> >>> Thanks >>> - Ioi >>> >>>> Thanks, >>>> David >>>> >>>>> 2. The only initialization left is that of a global lock. So I >>>>> renamed the function >>>>> ??? to SystemDictionaryShared::initialize_locks(). >>>>> >>>>> 3. I moved the call of this function from >>>>> SystemDictionary::compute_java_loaders() to >>>>> SystemDictionary::initialize() where it seems to fit. >>>>> >>>>> Testing with hs-tiers 1 and 2. >>>>> >>>>> Thanks >>>>> - Ioi >>> From calvin.cheung at oracle.com Tue May 15 17:57:04 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Tue, 15 May 2018 10:57:04 -0700 Subject: RFR(XS) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: References: Message-ID: <5AFB1F70.6060206@oracle.com> Hi Ioi, I saw that you've removed the initialization of File_klass from systemDictionaryShared.cpp. I'm wondering if it can also be removed from systemDictionary.hpp? 183 /* support for CDS */ \ 184 do_klass(ByteArrayInputStream_klass, java_io_ByteArrayInputStream, Pre ) \ 185 do_klass(*File_klass*, java_io_File, Pre ) \ thanks, Calvin On 5/14/18, 11:07 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8189140 > http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ > > > Summary: > > 1. Removed the forced initialization of a few classes used by AppCDS > at JVM start-up. > Instead, initialize these class on demand by calling > InstanceKlass::initialize, which > is a quick no-op if the class is already initialized. > > 2. The only initialization left is that of a global lock. So I renamed > the function > to SystemDictionaryShared::initialize_locks(). > > 3. I moved the call of this function from > SystemDictionary::compute_java_loaders() to > SystemDictionary::initialize() where it seems to fit. > > Testing with hs-tiers 1 and 2. > > Thanks > - Ioi From gerard.ziemski at oracle.com Tue May 15 18:24:25 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Tue, 15 May 2018 13:24:25 -0500 Subject: RFR(M): 8199252: [TESTBUG] Open source VM testbase system dictionary tests In-Reply-To: <5AF606AF.3020404@oracle.com> References: <5AF606AF.3020404@oracle.com> Message-ID: hi Misha, Looks fine. Just to be sure: did you have to make any functional changes to the test source code, or was it pretty much a straightforward case of moving the files? Thanks for doing this. cheers > On May 11, 2018, at 4:10 PM, Mikhailo Seledtsov wrote: > > Please review this change open sourcing VM system dictionary tests. These tests have been used internally for a while, and are now being open sourced. > Since this is not an creation of new tests, simply open sourcing existing tests, we would like to keep the changes during this review to an absolute required minimum. If you have any feedback on improvements of these tests, please file RFE(s) that will be addressed later in order of priority. > > > Here is what was done for this change: > 1. Moved the tests to OpenJDK repository preserving relative directory location and structure. > 3. Updated Copyright statements accordingly. > 4. Updated "@library" statements accordingly. > 5. Updated TEST.groups > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199252 > Webrev: http://cr.openjdk.java.net/~mseledtsov/8199252.01.open/ > > Testing: > 1. Ran the following tests on open-only repository and build, using "make run-test" > (Linux-x64) > vmTestbase_nsk_sysdict > All PASS > > 2. Automated multip-platform test system (usual 4 platforms): > - vmTestbase_nsk_sysdict > - hs-tier{1,2} > In progress > > Thank you, > Misha > From coleen.phillimore at oracle.com Tue May 15 20:25:03 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 15 May 2018 16:25:03 -0400 Subject: RFR: 8202978: Incorrect tmp register passed to MacroAssembler::load_mirror() In-Reply-To: <506c8615-5d79-be76-5e09-9331ec50465e@oracle.com> References: <70f41abc-9f3f-98fa-a66d-76f931640d02@oracle.com> <506c8615-5d79-be76-5e09-9331ec50465e@oracle.com> Message-ID: <993a387a-574d-43f2-3c61-47388a1bc459@oracle.com> Hi Per, It's calling a static native function so might not be totally performance neutral. It looks like rax is scratch there.? Can you try that??? Sorry I didn't figure out that r11 is rscratch2. Thanks, Coleen On 5/11/18 8:48 AM, Per Liden wrote: > On 05/11/2018 02:32 PM, Vladimir Kozlov wrote: >> Looks good. > > Thanks Vladimir! > >> Is this new code? Should we backport it otherwise? > > Yes, this is new code, i.e. it's not in JDK 10 or older. Also, it just > so happens that this bug doesn't affect the existing GC, only GCs > which do load barriers (like ZGC), which is why it hasn't been caught > before. > > /Per > >> >> Thanks, >> Vladimir K >> >> On 5/11/18 1:31 AM, Per Liden wrote: >>> On x86, MacroAssembler::load_mirror() defaults to using rscratch2 as >>> tmp register, unless something else is specified. In >>> TemplateInterpreterGenerator::generate_native_entry() we call >>> load_mirror(), but the rscratch2 register is already in use in this >>> context, which obviously leads to problems. This is not a >>> performance critical path, so we should just pass noreg as the tmp >>> register. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202978 >>> Webrev: http://cr.openjdk.java.net/~pliden/8202978/webrev.0 >>> >>> Testing: passes hs-tier{1,2} >>> >>> /Per From jiangli.zhou at oracle.com Tue May 15 20:29:43 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Tue, 15 May 2018 13:29:43 -0700 Subject: RFR(XS) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: <45077fa1-5ae5-6b2b-3b0f-0c2b7f13dde6@oracle.com> References: <3c685cc7-cb21-49fd-8c5a-67a6c65c575f@oracle.com> <151df8fe-7c36-5028-fe0e-37faa891969d@oracle.com> <5a5a99c8-df1d-c53b-03b7-05112df2bc3d@oracle.com> <45077fa1-5ae5-6b2b-3b0f-0c2b7f13dde6@oracle.com> Message-ID: > On May 15, 2018, at 9:13 AM, Ioi Lam wrote: > > > > On 5/15/18 1:39 AM, David Holmes wrote: >> Correcting silly comment ... >> >> On 15/05/2018 6:31 PM, David Holmes wrote: >>> On 15/05/2018 5:44 PM, Ioi Lam wrote: >>>> On 5/14/18 11:25 PM, David Holmes wrote: >>>>> Hi Ioi, >>>>> >>>>> On 15/05/2018 4:07 PM, Ioi Lam wrote: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>>>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>>>>> >>>>>> Summary: >>>>>> >>>>>> 1. Removed the forced initialization of a few classes used by AppCDS at JVM start-up. >>>>>> Instead, initialize these class on demand by calling InstanceKlass::initialize, which >>>>>> is a quick no-op if the class is already initialized. >>>>> >>>>> Any reason not to just add them to the set of pre-loaded and pre-initialized classes used with shared spaces in initialize_preloaded_classes()? >>>>> >>>> Because initialize_preloaded_classes(), despite its name, doesn't actually call InstanceKlass::initialize(). It just resolves the klasses. If you try to call InstanceKlass::initialize() here, the VM would crash. >>> >>> So much from truth in advertising ;-) >>> >>>> A few classes are initialized inside functions like Threads::initialize_java_lang_classes(), which seems a bit like black magic to me, so I didn't want to muck with them. >>> >>> Yes - the initialization order is not something to be trifled with. >>> >>>>> Aside: It looks a little odd to lazy-init a couple of classes but then assume/expect others (like SecureClassLoader) are already initialized. >>>>> >>>> The explicit init is only necessary before calling InstanceKlass::allocate_instance_handle(), which, unlike the 'new' bytecodes, does not implicitly initialize the class. >>> >>> What is allocate_instance_handle() supposed to do? Is it just a raw allocator? The class must be initialized before any constructor is called. >>> >>>> The other code in SystemDictionaryShared.cpp access the classes through JavaCalls::call_{static,special,virtual}, where the class would be initialized implicitly as necessary in exactly the same way as the invoke{static,special,virtual} bytecodes. >>>> >>>> Perhaps we should add a JavaCalls::new_instance() method that implicitly initializes the class, just like the 'new' bytecode? Should I do that in this changeset? >>> >>> I'm not sure where you would do this or why you need to create an instance of these classes ?? >> >> Of course you are creating instances of these classes. But you're using JavaCalls::call_special to invoke the constructor, so isn't that already ensuring the class is initialized as you just described ?? >> > > I should have clarified what I said above about the invoke bytecodes: if I remember correctly, the only 4 bytecodes that implicitly initialize the operand's class are - new, invokestatic, getstatic and putstatic. invokespecial and invokevirtual do not perform initialization, because they operate on an object, which has already been new'ed, which means its class has already been initialized by the 'new' byte code. Right, class initialization is triggered ?implicitly? with invokestatic, putstatic, getstatic, new, etc. Before doing invokespecial, an explicit initialize() call is made for a class. Looks like most existing JavaCalls::call_special() call sites don?t do the class initialize(). Those classes? initialize() are done earlier however, either during VM initialization or the initialization of a component. So the original code did follow the convention. Calling initialize() in get_shared_jar_manifest() and get_protection_domain_from_classloader() would be a little less efficient. Maybe add a new API in SystemDictionaryShared for initializing the classes used by AppCDS and add it after the SystemDictionary::compute_java_loaders call (so the original initialization sequence is not changed)? Thanks, Jiangli > > >> >>>>> But this seems functionally fine and a good tidy up (the class init stuff really didn't belong as a side-effect). The only thing I'd be double checking is that the same thread (presumably the main thread) is guaranteed to be the one doing the initialization. >>>>> >>>> There's no such requirement on the threads. If another thread is already in the middle of initializing the class, the thread that class InstanceKlass::initialize() will just wait for the class initialization to complete. >>> >>> I was thinking purely from the perspective of not changing anything in the initialization order - and making sure you can't inadvertently do something on a thread that isn't allowed to load/initialize klasses. This is intended to be a small code cleanup so I presumed you didn't want to perturb things too much - ideally not at all. > > For, the 3 places where the klass->initialize() is added, they all immediately execute Java code, which could have any side effect, including loading/initializing more classes. So I think it's safe to add the initialization call there. > > Thanks > - Ioi > >>> >>> David >>> >>>> Thanks >>>> - Ioi >>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> 2. The only initialization left is that of a global lock. So I renamed the function >>>>>> to SystemDictionaryShared::initialize_locks(). >>>>>> >>>>>> 3. I moved the call of this function from SystemDictionary::compute_java_loaders() to >>>>>> SystemDictionary::initialize() where it seems to fit. >>>>>> >>>>>> Testing with hs-tiers 1 and 2. >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>> > From igor.ignatyev at oracle.com Tue May 15 20:41:22 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 15 May 2018 13:41:22 -0700 Subject: RFR(XS) : 8203250 : runtime/LoadClass/test-classes/Hello.java has wrong legal notice Message-ID: <5E09489B-CB48-48A2-8CD5-F68FE4FFCEA4@oracle.com> http://cr.openjdk.java.net/~iignatyev//8203250/webrev.00/index.html > 20 lines changed: 18 ins; 0 del; 2 mod; Hi all, could you please review this trivial patch which replaced legal notice in runtime/LoadClass/test-classes/Hello.java ? webrev: http://cr.openjdk.java.net/~iignatyev//8203250/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8203250 testing: runtime/LoadClass/test-classes/ tests Thanks, -- Igor From calvin.cheung at oracle.com Tue May 15 20:46:04 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Tue, 15 May 2018 13:46:04 -0700 Subject: RFR(XS) : 8203250 : runtime/LoadClass/test-classes/Hello.java has wrong legal notice In-Reply-To: <5E09489B-CB48-48A2-8CD5-F68FE4FFCEA4@oracle.com> References: <5E09489B-CB48-48A2-8CD5-F68FE4FFCEA4@oracle.com> Message-ID: <5AFB470C.4090601@oracle.com> Hi Igor, The patch looks good. Thanks for fixing it. Calvin On 5/15/18, 1:41 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8203250/webrev.00/index.html >> 20 lines changed: 18 ins; 0 del; 2 mod; > Hi all, > > could you please review this trivial patch which replaced legal notice in runtime/LoadClass/test-classes/Hello.java ? > > webrev: http://cr.openjdk.java.net/~iignatyev//8203250/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8203250 > testing: runtime/LoadClass/test-classes/ tests > > Thanks, > -- Igor From igor.ignatyev at oracle.com Tue May 15 20:56:14 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 15 May 2018 13:56:14 -0700 Subject: RFR(XS) : 8203250 : runtime/LoadClass/test-classes/Hello.java has wrong legal notice In-Reply-To: <5AFB470C.4090601@oracle.com> References: <5E09489B-CB48-48A2-8CD5-F68FE4FFCEA4@oracle.com> <5AFB470C.4090601@oracle.com> Message-ID: <0DB9A637-0352-486F-971B-E4A28727B60A@oracle.com> Hi Calvin, thanks for your review! -- Igor > On May 15, 2018, at 1:46 PM, Calvin Cheung wrote: > > Hi Igor, > > The patch looks good. > Thanks for fixing it. > > Calvin > > On 5/15/18, 1:41 PM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8203250/webrev.00/index.html >>> 20 lines changed: 18 ins; 0 del; 2 mod; >> Hi all, >> >> could you please review this trivial patch which replaced legal notice in runtime/LoadClass/test-classes/Hello.java ? >> >> webrev: http://cr.openjdk.java.net/~iignatyev//8203250/webrev.00/index.html >> JBS: https://bugs.openjdk.java.net/browse/JDK-8203250 >> testing: runtime/LoadClass/test-classes/ tests >> >> Thanks, >> -- Igor From coleen.phillimore at oracle.com Tue May 15 21:00:59 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 15 May 2018 17:00:59 -0400 Subject: RFR(XS) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: References: Message-ID: http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html This looks good.? This is a pattern that's used in other places, and it would be better to not initialize these at startup in thread.cpp. Coleen On 5/15/18 2:07 AM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8189140 > http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ > > > Summary: > > 1. Removed the forced initialization of a few classes used by AppCDS > at JVM start-up. > ?? Instead, initialize these class on demand by calling > InstanceKlass::initialize, which > ?? is a quick no-op if the class is already initialized. > > 2. The only initialization left is that of a global lock. So I renamed > the function > ?? to SystemDictionaryShared::initialize_locks(). > > 3. I moved the call of this function from > SystemDictionary::compute_java_loaders() to > SystemDictionary::initialize() where it seems to fit. > > Testing with hs-tiers 1 and 2. > > Thanks > - Ioi From ioi.lam at oracle.com Tue May 15 21:47:47 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 15 May 2018 14:47:47 -0700 Subject: RFR(S) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: References: Message-ID: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> I've updated the webrev: http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/ 1. Added JavaCalls::new_instance so we can avoid all the boiler plate code for allocating ?? the instance andinvoking the constructor. JavaCalls::new_instance calls InstanceKlass->initialize. This is just a quick op after ?? the class is already initialized. Also, JavaCalls::call_static also internally call ?? into InstanceKlass->initialize, so I am just following the existing pattern as Coleen ?? mentioned below. ?? Doing the initialization on demand also means for cases where JAR manifest is not used ?? (all code is loaded from the system image or directories), we get faster start-up. 2. I also took the time to removed a bunch of "// One oop argument" comments which ?? probably meant something to the person who wrote it, but seems useless to everyone ?? else. 3. As Calvin suggested, I removed the File_klass and also ParseUtil_klass from ?? the system dictionary since they are not used anywhere. This hopefully improves start-up ?? by a little bit, since these 2 classes are no longer resolved at start-up. (BTW, I changed the RFR subject line from XS to S due to the extend of change :-) Thanks - Ioi On 5/15/18 2:00 PM, coleen.phillimore at oracle.com wrote: > > http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html > > > This looks good.? This is a pattern that's used in other places, and > it would be better to not initialize these at startup in thread.cpp. > > Coleen > > On 5/15/18 2:07 AM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8189140 >> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >> >> >> Summary: >> >> 1. Removed the forced initialization of a few classes used by AppCDS >> at JVM start-up. >> ?? Instead, initialize these class on demand by calling >> InstanceKlass::initialize, which >> ?? is a quick no-op if the class is already initialized. >> >> 2. The only initialization left is that of a global lock. So I >> renamed the function >> ?? to SystemDictionaryShared::initialize_locks(). >> >> 3. I moved the call of this function from >> SystemDictionary::compute_java_loaders() to >> SystemDictionary::initialize() where it seems to fit. >> >> Testing with hs-tiers 1 and 2. >> >> Thanks >> - Ioi > From coleen.phillimore at oracle.com Tue May 15 22:22:11 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 15 May 2018 18:22:11 -0400 Subject: RFR(S) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> References: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> Message-ID: <6d03e69d-1b84-254a-9056-7c38c5053975@oracle.com> On 5/15/18 5:47 PM, Ioi Lam wrote: > I've updated the webrev: > > http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/ > > > 1. Added JavaCalls::new_instance so we can avoid all the boiler plate > code for allocating > ?? the instance andinvoking the constructor. > > JavaCalls::new_instance calls InstanceKlass->initialize. This is just > a quick op after > ?? the class is already initialized. Also, JavaCalls::call_static also > internally call > ?? into InstanceKlass->initialize, so I am just following the existing > pattern as Coleen > ?? mentioned below. > I think I like your last version better.? I don't think JavaCalls::new_instance() can nicely generalize the calls to object init functions that I found around the vm.? I think just doing what you had would be better.? It's only 3 klass->initialize calls. You could make the new_instance() function local to systemDictionaryShared to save code if they are all the same, but I don't see much advantage. Also a general new_instance() function should have the call to this to be the same as the others (see reflection.cpp). ? klass->check_valid_for_instantiation(false, CHECK_NULL); If it were possible to replace all klass->allocate_instance() calls with JavaCalls::new_instance(), that seems like the API would be worth it, but it's not possible.?? So people can still make mistakes and not call the object init function, like I think someone did in jvm.cpp. > Doing the initialization on demand also means for cases where JAR > manifest is not used > ?? (all code is loaded from the system image or directories), we get > faster start-up. > > 2. I also took the time to removed a bunch of "// One oop argument" > comments which > ?? probably meant something to the person who wrote it, but seems > useless to everyone > ?? else. Not sure what that would mean myself :) > > 3. As Calvin suggested, I removed the File_klass and also > ParseUtil_klass from > ?? the system dictionary since they are not used anywhere. This > hopefully improves start-up > ?? by a little bit, since these 2 classes are no longer resolved at > start-up. > > > (BTW, I changed the RFR subject line from XS to S due to the extend of > change :-) > Make it S again! Coleen > Thanks > - Ioi > > > > > On 5/15/18 2:00 PM, coleen.phillimore at oracle.com wrote: >> >> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >> >> >> This looks good.? This is a pattern that's used in other places, and >> it would be better to not initialize these at startup in thread.cpp. >> >> Coleen >> >> On 5/15/18 2:07 AM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>> >>> >>> Summary: >>> >>> 1. Removed the forced initialization of a few classes used by AppCDS >>> at JVM start-up. >>> ?? Instead, initialize these class on demand by calling >>> InstanceKlass::initialize, which >>> ?? is a quick no-op if the class is already initialized. >>> >>> 2. The only initialization left is that of a global lock. So I >>> renamed the function >>> ?? to SystemDictionaryShared::initialize_locks(). >>> >>> 3. I moved the call of this function from >>> SystemDictionary::compute_java_loaders() to >>> SystemDictionary::initialize() where it seems to fit. >>> >>> Testing with hs-tiers 1 and 2. >>> >>> Thanks >>> - Ioi >> > From coleen.phillimore at oracle.com Tue May 15 22:23:42 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 15 May 2018 18:23:42 -0400 Subject: RFR(S) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: <6d03e69d-1b84-254a-9056-7c38c5053975@oracle.com> References: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> <6d03e69d-1b84-254a-9056-7c38c5053975@oracle.com> Message-ID: <0b31c733-79a6-4086-215b-62cabdcf47b9@oracle.com> On 5/15/18 6:22 PM, coleen.phillimore at oracle.com wrote: > > > > On 5/15/18 5:47 PM, Ioi Lam wrote: >> I've updated the webrev: >> >> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/ >> >> >> 1. Added JavaCalls::new_instance so we can avoid all the boiler plate >> code for allocating >> ?? the instance andinvoking the constructor. >> >> JavaCalls::new_instance calls InstanceKlass->initialize. This is just >> a quick op after >> ?? the class is already initialized. Also, JavaCalls::call_static >> also internally call >> ?? into InstanceKlass->initialize, so I am just following the >> existing pattern as Coleen >> ?? mentioned below. >> > > I think I like your last version better.? I don't think > JavaCalls::new_instance() can nicely generalize the calls to object > init functions that I found around the vm.? I think just doing what > you had would be better.? It's only 3 klass->initialize calls. You > could make the new_instance() function local to systemDictionaryShared > to save code if they are all the same, but I don't see much advantage. > > Also a general new_instance() function should have the call to this to > be the same as the others (see reflection.cpp). > > ? klass->check_valid_for_instantiation(false, CHECK_NULL); > > If it were possible to replace all klass->allocate_instance() calls > with JavaCalls::new_instance(), that seems like the API would be worth > it, but it's not possible.?? So people can still make mistakes and not > call the object init function, like I think someone did in jvm.cpp. > > >> Doing the initialization on demand also means for cases where JAR >> manifest is not used >> ?? (all code is loaded from the system image or directories), we get >> faster start-up. >> >> 2. I also took the time to removed a bunch of "// One oop argument" >> comments which >> ?? probably meant something to the person who wrote it, but seems >> useless to everyone >> ?? else. > > Not sure what that would mean myself :) >> >> 3. As Calvin suggested, I removed the File_klass and also >> ParseUtil_klass from >> ?? the system dictionary since they are not used anywhere. This >> hopefully improves start-up >> ?? by a little bit, since these 2 classes are no longer resolved at >> start-up. >> >> >> (BTW, I changed the RFR subject line from XS to S due to the extend >> of change :-) >> > Make it S again! I mean XS! Coleen > > Coleen > >> Thanks >> - Ioi >> >> >> >> >> On 5/15/18 2:00 PM, coleen.phillimore at oracle.com wrote: >>> >>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >>> >>> >>> This looks good.? This is a pattern that's used in other places, and >>> it would be better to not initialize these at startup in thread.cpp. >>> >>> Coleen >>> >>> On 5/15/18 2:07 AM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>>> >>>> >>>> Summary: >>>> >>>> 1. Removed the forced initialization of a few classes used by >>>> AppCDS at JVM start-up. >>>> ?? Instead, initialize these class on demand by calling >>>> InstanceKlass::initialize, which >>>> ?? is a quick no-op if the class is already initialized. >>>> >>>> 2. The only initialization left is that of a global lock. So I >>>> renamed the function >>>> ?? to SystemDictionaryShared::initialize_locks(). >>>> >>>> 3. I moved the call of this function from >>>> SystemDictionary::compute_java_loaders() to >>>> SystemDictionary::initialize() where it seems to fit. >>>> >>>> Testing with hs-tiers 1 and 2. >>>> >>>> Thanks >>>> - Ioi >>> >> > From ioi.lam at oracle.com Tue May 15 23:02:26 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 15 May 2018 16:02:26 -0700 Subject: RFR(S) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: <6d03e69d-1b84-254a-9056-7c38c5053975@oracle.com> References: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> <6d03e69d-1b84-254a-9056-7c38c5053975@oracle.com> Message-ID: <8319ecf5-a2d8-fade-91f7-c33f636d4a71@oracle.com> On 5/15/18 3:22 PM, coleen.phillimore at oracle.com wrote: > > > > On 5/15/18 5:47 PM, Ioi Lam wrote: >> I've updated the webrev: >> >> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/ >> >> >> 1. Added JavaCalls::new_instance so we can avoid all the boiler plate >> code for allocating >> ?? the instance andinvoking the constructor. >> >> JavaCalls::new_instance calls InstanceKlass->initialize. This is just >> a quick op after >> ?? the class is already initialized. Also, JavaCalls::call_static >> also internally call >> ?? into InstanceKlass->initialize, so I am just following the >> existing pattern as Coleen >> ?? mentioned below. >> > > I think I like your last version better.? I don't think > JavaCalls::new_instance() can nicely generalize the calls to object > init functions that I found around the vm.? I think just doing what > you had would be better.? It's only 3 klass->initialize calls. You > could make the new_instance() function local to systemDictionaryShared > to save code if they are all the same, but I don't see much advantage. > > Also a general new_instance() function should have the call to this to > be the same as the others (see reflection.cpp). > > ? klass->check_valid_for_instantiation(false, CHECK_NULL); > javaCalls is for cases where "the VM knows what it's doing", and reflection.cpp is to implement java.lang.reflect. So the former doesn't do access checks while the latter does. You can compare JavaCalls::call_static() vs Reflection::invoke_method(). > If it were possible to replace all klass->allocate_instance() calls > with JavaCalls::new_instance(), that seems like the API would be worth > it, but it's not possible.?? So people can still make mistakes and not > call the object init function, like I think someone did in jvm.cpp. > Why is it not possible? I think many places can be cleaned up with this API, e.g., in ./share/services/{management.cpp, memoryService.cpp, attachListener.cpp and gcNotifier.cpp). I can do the cleanup in a separate RFE. Some places call klass->allocate_instance() without calling the constructor, on purpose, to simulate the behavior of the 'new' bytecode. E.g., in jvmciRuntime.cpp. But if what you want to do is "x = new Foo(args ...)" in the C code, JavaCalls::new_instance() seems like a clean way to do it. Thanks - Ioi > >> Doing the initialization on demand also means for cases where JAR >> manifest is not used >> ?? (all code is loaded from the system image or directories), we get >> faster start-up. >> >> 2. I also took the time to removed a bunch of "// One oop argument" >> comments which >> ?? probably meant something to the person who wrote it, but seems >> useless to everyone >> ?? else. > > Not sure what that would mean myself :) >> >> 3. As Calvin suggested, I removed the File_klass and also >> ParseUtil_klass from >> ?? the system dictionary since they are not used anywhere. This >> hopefully improves start-up >> ?? by a little bit, since these 2 classes are no longer resolved at >> start-up. >> >> >> (BTW, I changed the RFR subject line from XS to S due to the extend >> of change :-) >> > Make it S again! > > Coleen > >> Thanks >> - Ioi >> >> >> >> >> On 5/15/18 2:00 PM, coleen.phillimore at oracle.com wrote: >>> >>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >>> >>> >>> This looks good.? This is a pattern that's used in other places, and >>> it would be better to not initialize these at startup in thread.cpp. >>> >>> Coleen >>> >>> On 5/15/18 2:07 AM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>>> >>>> >>>> Summary: >>>> >>>> 1. Removed the forced initialization of a few classes used by >>>> AppCDS at JVM start-up. >>>> ?? Instead, initialize these class on demand by calling >>>> InstanceKlass::initialize, which >>>> ?? is a quick no-op if the class is already initialized. >>>> >>>> 2. The only initialization left is that of a global lock. So I >>>> renamed the function >>>> ?? to SystemDictionaryShared::initialize_locks(). >>>> >>>> 3. I moved the call of this function from >>>> SystemDictionary::compute_java_loaders() to >>>> SystemDictionary::initialize() where it seems to fit. >>>> >>>> Testing with hs-tiers 1 and 2. >>>> >>>> Thanks >>>> - Ioi >>> >> > From igor.ignatyev at oracle.com Tue May 15 23:16:17 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 15 May 2018 16:16:17 -0700 Subject: RFR(M) : 8202946 : [TESTBUG] Open source VM testbase OOM tests Message-ID: <2DD8F9C6-8471-4BF6-8573-0DA3F2B6C66B@oracle.com> http://cr.openjdk.java.net/~iignatyev//8202946/webrev.00/index.html > 1619 lines changed: 1619 ins; 0 del; 0 mod; Hi all, could you please review this patch which open sources OOM tests from VM testbase? these tests test OutOfMemoryError throwing in different scenarios. As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. JBS: https://bugs.openjdk.java.net/browse/JDK-8202946 webrev: http://cr.openjdk.java.net/~iignatyev//8202946/webrev.00/index.html testing: :vmTestbase_vm_oom test group Thanks, -- Igor From ioi.lam at oracle.com Tue May 15 23:33:07 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 15 May 2018 16:33:07 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: <4CBA8376-54A2-4863-8732-B16E1AE9C38A@oracle.com> References: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> <5AF62823.7050804@oracle.com> <70F6AEB0-9103-4899-87D9-DC845BF66C51@oracle.com> <4CBA8376-54A2-4863-8732-B16E1AE9C38A@oracle.com> Message-ID: Hi Jiangli, ?245?? // We need to validate the runtime modules image file size against the archived ?246?? // size information, obtain the runtime modules image path. The recorded dump ?247?? // time modules image path in the archive may be different from the runtime path ?248?? // if the JDK image has beed moved after generating the archive. ?249?? if (ClassLoader::is_modules_image(name)) { ?250???? name = ClassLoader::get_jrt_entry()->name(); ?251?? } What happens if someone has a JAR file called foo.modules, and dumped with ??? java -Xshare:dump -cp foo.modules I think it's better to check that it's #0 in the classpath so we know for sure it's the system modules file. There's a similar problem here: ?220?????? if (!ClassLoader::is_modules_image(name)) { In the following, instead of doing a copy, maybe it's better to add a length argument for os::file_name_strcmp, since sharedPathsMiscInfo.cpp is the only place in the VM that calls this function. ?162?????? if (relaxed_check) { ?163???????? // only check the begining portion of the runtime boot path, up to ?164???????? // the length of the dump time boot path ?165???????? size_t len = strlen(path); ?166???????? runtime_boot_path = NEW_RESOURCE_ARRAY(char, len + 1); ?167???????? strncpy(runtime_boot_path, Arguments::get_sysclasspath(), len); ?168???????? runtime_boot_path[len] = '\0'; ?169?????? } else { ?170???????? // check the full runtime boot path ?171???????? runtime_boot_path = Arguments::get_sysclasspath(); ?172?????? } ?173 ?174?????? // Do a quick check first with a simple ?175?????? // strcmp(dump_time_boot_path, runtime_boot_path). If the paths are the ?176?????? // same, the check has succeeded. ?177?????? if (os::file_name_strcmp(path, runtime_boot_path) == 0) { ?178???????? break; // ok ?179?????? } Also, the copy could be wrong for the following output: ??? path = /tmp/foo/modules ??? Arguments::get_sysclasspath() = /tmp/foo/modulesxyz:/aaa:/bbb What is runtime_boot_path supposed to contain here? I am not sure if this code gives the intended output or not, but it's not easy to understand what it actually does by operating on an truncated pathname. String manipulation code is always hard to read. I would suggest braking it up into smaller functions: 1. Get the first entry of dump time and run time boot path -> d0 and r0 ??? -> ignore d0 ??? -> r0 same as ClassLoader::get_jrt_entry()->name(); 2. Get the remaining part of the dump time and run time boot path -> d_remain, r_remain ??? -> relaxed_check: d_remain must be a prefix of r_remain ??? -> !relaxed_check: They must be identical. By the way, I think the test case in JDK-8202935 should be reviewed together inside this RFR, since the test validates the feature implemented here. I don't think we need a separate bug ID. Otherwise it will be hard to track the test cases. Thanks - Ioi On 5/13/18 5:31 PM, Jiangli Zhou wrote: > Here is the updated webrev: > http://cr.openjdk.java.net/~jiangli/8199807/webrev.01/ > > Thanks, > Jiangli > >> On May 11, 2018, at 6:07 PM, Jiangli Zhou wrote: >> >> Hi Calvin, >> >> Thanks for the review. >> >>> On May 11, 2018, at 4:32 PM, Calvin Cheung wrote: >>> >>> Hi Jiangli, >>> >>> Thanks for doing this useful change. >>> >>> I have a few minor comments: >>> >>> 1) sharedPathsMiscInfo.cpp >>> >>> 199 dp ++; >>> >>> Should the above be similar to line 194 as follows? >>> dp += path_sep_len; >> Good catch. Will fix. >> >>> 2) filemap.cpp >>> >>> 244 if (ClassLoader::is_modules_image(name)) { >>> >>> I think the above could be: >>> if (is_modules_image()) { >>> >> Let?s keep it this way as is_modules_image() is a wrapper of 'ClassLoader::is_modules_image(name())?. We already have the ?name? in this case. >> >>> The is_modules_image() is a member function of SharedClassPathEntry. >>> >>> Btw, why is it necessary to get the runtime modules image path again in lines 244 - 246? >> As the runtime modules image path could be different from the dump time if the JDK image is moved/copied after archive generation, we need to make sure to use the correct file for file size check. The recored path in the archive is the dump time modules image path, which may no longer be existing at runtime. I will add some comments if that?s helpful. >> >>> 3) BootClassPathMismatch.java >>> >>> Could you move line 103 to right above line 107 (opts.addPrefix(?))? >>> Because the archive dumping part of the test doesn?t use appJar. >> Sure. >> >> Thanks! >> >> Jiangli >> >>> thanks, >>> Calvin >>> >>> On 5/11/18, 3:15 PM, Jiangli Zhou wrote: >>>> I?ve withdrawn my original weberv and removed the build change. Here is the updated webrev that only addresses 8199807. >>>> >>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.00/ >>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>> >>>> Thanks, >>>> Jiangli >>>> >>>>> On May 11, 2018, at 2:44 PM, Jiangli Zhou wrote: >>>>> >>>>> Hi Erik, >>>>> >>>>> Thank you so much for investigating a proper fix for the vm_version.o issue. I will withdraw the build change from my original webrev. >>>>> >>>>> Thanks again for taking over the bug! >>>>> Jiangli >>>>> >>>>>> On May 11, 2018, at 2:33 PM, Erik Joelsson wrote: >>>>>> >>>>>> Received: from [10.132.185.167] (/10.132.185.167) >>>>>> by default (Oracle Beehive Gateway v4.0) >>>>>> with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 >>>>>> Subject: Re: RFR: 8199807& 8202738: AppCDS performs overly restrictive path >>>>>> matching check >>>>>> To: Jiangli Zhou, >>>>>> "hotspot-runtime-dev at openjdk.java.net runtime" >>>>>> , >>>>>> build-dev >>>>>> References: >>>>>> From: Erik Joelsson >>>>>> Organization: Oracle Corporation >>>>>> Message-ID: >>>>>> Date: Fri, 11 May 2018 14:33:20 -0700 >>>>>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) >>>>>> Gecko/20100101 Thunderbird/52.7.0 >>>>>> MIME-Version: 1.0 >>>>>> In-Reply-To: >>>>>> Content-Type: text/plain; charset=utf-8; format=flowed >>>>>> Content-Transfer-Encoding: 8bit >>>>>> Content-Language: en-US >>>>>> >>>>>> Hello, >>>>>> >>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>> >>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>> >>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>> >>>>>> /Erik >>>>>> >>>>>> >>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>> >>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>> >>>>>>> The webrev includes the following three main parts: >>>>>>> >>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>> >>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>> >>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>> >>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>> >>>>>>> Thanks, >>>>>>> Jiangli >>>>>>> >>>>>> Hello, >>>>>> >>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>> >>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>> >>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>> >>>>>> /Erik >>>>>> >>>>>> >>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>> Hi, >>>>>>> >>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>> >>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>> >>>>>>> The webrev includes the following three main parts: >>>>>>> >>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>> >>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>> >>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>> >>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>> >>>>>>> Thanks, >>>>>>> Jiangli From coleen.phillimore at oracle.com Tue May 15 23:34:39 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 15 May 2018 19:34:39 -0400 Subject: RFR(S) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: <8319ecf5-a2d8-fade-91f7-c33f636d4a71@oracle.com> References: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> <6d03e69d-1b84-254a-9056-7c38c5053975@oracle.com> <8319ecf5-a2d8-fade-91f7-c33f636d4a71@oracle.com> Message-ID: <6c7b37e4-3c13-a151-1bb7-24bb0b63e506@oracle.com> On 5/15/18 7:02 PM, Ioi Lam wrote: > > > On 5/15/18 3:22 PM, coleen.phillimore at oracle.com wrote: >> >> >> >> On 5/15/18 5:47 PM, Ioi Lam wrote: >>> I've updated the webrev: >>> >>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/ >>> >>> >>> 1. Added JavaCalls::new_instance so we can avoid all the boiler >>> plate code for allocating >>> ?? the instance andinvoking the constructor. >>> >>> JavaCalls::new_instance calls InstanceKlass->initialize. This is >>> just a quick op after >>> ?? the class is already initialized. Also, JavaCalls::call_static >>> also internally call >>> ?? into InstanceKlass->initialize, so I am just following the >>> existing pattern as Coleen >>> ?? mentioned below. >>> >> >> I think I like your last version better.? I don't think >> JavaCalls::new_instance() can nicely generalize the calls to object >> init functions that I found around the vm.? I think just doing what >> you had would be better.? It's only 3 klass->initialize calls. You >> could make the new_instance() function local to >> systemDictionaryShared to save code if they are all the same, but I >> don't see much advantage. >> >> Also a general new_instance() function should have the call to this >> to be the same as the others (see reflection.cpp). >> >> ? klass->check_valid_for_instantiation(false, CHECK_NULL); >> > javaCalls is for cases where "the VM knows what it's doing", and > reflection.cpp is to implement java.lang.reflect. So the former > doesn't do access checks while the latter does. You can compare > JavaCalls::call_static() vs Reflection::invoke_method(). > >> If it were possible to replace all klass->allocate_instance() calls >> with JavaCalls::new_instance(), that seems like the API would be >> worth it, but it's not possible.?? So people can still make mistakes >> and not call the object init function, like I think someone did in >> jvm.cpp. >> > Why is it not possible? I think many places can be cleaned up with > this API, e.g., in ./share/services/{management.cpp, > memoryService.cpp, attachListener.cpp and gcNotifier.cpp). I can do > the cleanup in a separate RFE. I don't see how these would be any cleaner.? One example you sited calls a static function initialize_ThreadInfo_constructor_arguments which pushes 9 arguments to the constructor.?? You would have a version of JavaCalls::new_instance with 1,2,3,...,9 potential arguments??? It wouldn't be less code or nicer. > > Some places call klass->allocate_instance() without calling the > constructor, on purpose, to simulate the behavior of the 'new' > bytecode. E.g., in jvmciRuntime.cpp. Yes, so you cannot replace all calls to allocate_instance(). > > But if what you want to do is "x = new Foo(args ...)" in the C code, > JavaCalls::new_instance() seems like a clean way to do it. This name seems too ambiguous with allocate_instance() which I thought called the constructor until I just looked.? If new_instance had a sentence name, maybe it would indicate when one should use it vs. allocate_instance.?? This doesn't seem like it helps. If you want to add a local one to systemDictionaryShared for the three calls, that would be fine with me.? Then you can see whether you can cleanly replace all of the places with a function that encapsulates calling the klass->initialize function, allocating an object, and calling the constructor with a variable length argument list.?? My initial thought was to have the encapsulating function pass JavaCallArgs with a NULL for the object argument which is replaced inside the call after it's allocated.? Still sort of ugly but better than 10 cascading functions. Coleen > > Thanks > - Ioi >> >>> Doing the initialization on demand also means for cases where JAR >>> manifest is not used >>> ?? (all code is loaded from the system image or directories), we get >>> faster start-up. >>> >>> 2. I also took the time to removed a bunch of "// One oop argument" >>> comments which >>> ?? probably meant something to the person who wrote it, but seems >>> useless to everyone >>> ?? else. >> >> Not sure what that would mean myself :) >>> >>> 3. As Calvin suggested, I removed the File_klass and also >>> ParseUtil_klass from >>> ?? the system dictionary since they are not used anywhere. This >>> hopefully improves start-up >>> ?? by a little bit, since these 2 classes are no longer resolved at >>> start-up. >>> >>> >>> (BTW, I changed the RFR subject line from XS to S due to the extend >>> of change :-) >>> >> Make it S again! >> >> Coleen >> >>> Thanks >>> - Ioi >>> >>> >>> >>> >>> On 5/15/18 2:00 PM, coleen.phillimore at oracle.com wrote: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >>>> >>>> >>>> This looks good.? This is a pattern that's used in other places, >>>> and it would be better to not initialize these at startup in >>>> thread.cpp. >>>> >>>> Coleen >>>> >>>> On 5/15/18 2:07 AM, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>>>> >>>>> >>>>> Summary: >>>>> >>>>> 1. Removed the forced initialization of a few classes used by >>>>> AppCDS at JVM start-up. >>>>> ?? Instead, initialize these class on demand by calling >>>>> InstanceKlass::initialize, which >>>>> ?? is a quick no-op if the class is already initialized. >>>>> >>>>> 2. The only initialization left is that of a global lock. So I >>>>> renamed the function >>>>> ?? to SystemDictionaryShared::initialize_locks(). >>>>> >>>>> 3. I moved the call of this function from >>>>> SystemDictionary::compute_java_loaders() to >>>>> SystemDictionary::initialize() where it seems to fit. >>>>> >>>>> Testing with hs-tiers 1 and 2. >>>>> >>>>> Thanks >>>>> - Ioi >>>> >>> >> > From ioi.lam at oracle.com Tue May 15 23:37:14 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 15 May 2018 16:37:14 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: References: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> <5AF62823.7050804@oracle.com> <70F6AEB0-9103-4899-87D9-DC845BF66C51@oracle.com> <4CBA8376-54A2-4863-8732-B16E1AE9C38A@oracle.com> Message-ID: On 5/15/18 4:33 PM, Ioi Lam wrote: > Hi Jiangli, > > ?245?? // We need to validate the runtime modules image file size > against the archived > ?246?? // size information, obtain the runtime modules image path. The > recorded dump > ?247?? // time modules image path in the archive may be different from > the runtime path > ?248?? // if the JDK image has beed moved after generating the archive. > ?249?? if (ClassLoader::is_modules_image(name)) { > ?250???? name = ClassLoader::get_jrt_entry()->name(); > ?251?? } > > What happens if someone has a JAR file called foo.modules, and dumped > with > > ??? java -Xshare:dump -cp foo.modules > > Another test case would be: ??? java -Xshare:dump -cp hello.jar:$JAVA_HOME/lib/modules:hi.jar Again, I am not sure what the behavior should be, but calling ClassLoader::is_modules_image(name) here makes it even confusing what you're actually checking. Thanks - Ioi > I think it's better to check that it's #0 in the classpath so we know > for sure it's the system modules file. > > There's a similar problem here: > > ?220?????? if (!ClassLoader::is_modules_image(name)) { > > In the following, instead of doing a copy, maybe it's better to add a > length argument for os::file_name_strcmp, since > sharedPathsMiscInfo.cpp is the only place in the VM that calls this > function. > > ?162?????? if (relaxed_check) { > ?163???????? // only check the begining portion of the runtime boot > path, up to > ?164???????? // the length of the dump time boot path > ?165???????? size_t len = strlen(path); > ?166???????? runtime_boot_path = NEW_RESOURCE_ARRAY(char, len + 1); > ?167???????? strncpy(runtime_boot_path, Arguments::get_sysclasspath(), > len); > ?168???????? runtime_boot_path[len] = '\0'; > ?169?????? } else { > ?170???????? // check the full runtime boot path > ?171???????? runtime_boot_path = Arguments::get_sysclasspath(); > ?172?????? } > ?173 > ?174?????? // Do a quick check first with a simple > ?175?????? // strcmp(dump_time_boot_path, runtime_boot_path). If the > paths are the > ?176?????? // same, the check has succeeded. > ?177?????? if (os::file_name_strcmp(path, runtime_boot_path) == 0) { > ?178???????? break; // ok > ?179?????? } > > > Also, the copy could be wrong for the following output: > > ??? path = /tmp/foo/modules > ??? Arguments::get_sysclasspath() = /tmp/foo/modulesxyz:/aaa:/bbb > > What is runtime_boot_path supposed to contain here? I am not sure if > this code gives the intended output or not, but it's not easy to > understand what it actually does by operating on an truncated pathname. > > String manipulation code is always hard to read. I would suggest > braking it up into smaller functions: > > 1. Get the first entry of dump time and run time boot path -> d0 and r0 > ??? -> ignore d0 > ??? -> r0 same as ClassLoader::get_jrt_entry()->name(); > > 2. Get the remaining part of the dump time and run time boot path -> > d_remain, r_remain > ??? -> relaxed_check: d_remain must be a prefix of r_remain > ??? -> !relaxed_check: They must be identical. > > > By the way, I think the test case in JDK-8202935 should be reviewed > together inside this RFR, since the test validates the feature > implemented here. I don't think we need a separate bug ID. Otherwise > it will be hard to track the test cases. > > > Thanks > - Ioi > > > On 5/13/18 5:31 PM, Jiangli Zhou wrote: >> Here is the updated webrev: >> http://cr.openjdk.java.net/~jiangli/8199807/webrev.01/ >> >> Thanks, >> Jiangli >> >>> On May 11, 2018, at 6:07 PM, Jiangli Zhou >>> wrote: >>> >>> Hi Calvin, >>> >>> Thanks for the review. >>> >>>> On May 11, 2018, at 4:32 PM, Calvin Cheung >>>> wrote: >>>> >>>> Hi Jiangli, >>>> >>>> Thanks for doing this useful change. >>>> >>>> I have a few minor comments: >>>> >>>> 1) sharedPathsMiscInfo.cpp >>>> >>>> 199???????? dp ++; >>>> >>>> Should the above be similar to line 194 as follows? >>>> ???????????? dp += path_sep_len; >>> Good catch. Will fix. >>> >>>> 2) filemap.cpp >>>> >>>> 244?? if (ClassLoader::is_modules_image(name)) { >>>> >>>> I think the above could be: >>>> ??????? if (is_modules_image()) { >>>> >>> Let?s keep it this way as is_modules_image() is a wrapper of >>> 'ClassLoader::is_modules_image(name())?. We already have the ?name? >>> in this case. >>> >>>> The is_modules_image() is a member function of SharedClassPathEntry. >>>> >>>> Btw, why is it necessary to get the runtime modules image path >>>> again in lines 244 - 246? >>> As the runtime modules image path could be different from the dump >>> time if the JDK image is moved/copied after archive generation, we >>> need to make sure to use the correct file for file size check. The >>> recored path in the archive is the dump time modules image path, >>> which may no longer be existing at runtime. I will add some comments >>> if that?s helpful. >>> >>>> 3) BootClassPathMismatch.java >>>> >>>> Could you move line 103 to right above line 107 (opts.addPrefix(?))? >>>> Because the archive dumping part of the test doesn?t use appJar. >>> Sure. >>> >>> Thanks! >>> >>> Jiangli >>> >>>> thanks, >>>> Calvin >>>> >>>> On 5/11/18, 3:15 PM, Jiangli Zhou wrote: >>>>> I?ve withdrawn my original weberv and removed the build change. >>>>> Here is the updated webrev that only addresses 8199807. >>>>> >>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.00/ >>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>> >>>>> Thanks, >>>>> Jiangli >>>>> >>>>>> On May 11, 2018, at 2:44 PM, Jiangli >>>>>> Zhou wrote: >>>>>> >>>>>> Hi Erik, >>>>>> >>>>>> Thank you so much for investigating a proper fix for the >>>>>> vm_version.o issue. I will withdraw the build change from my >>>>>> original webrev. >>>>>> >>>>>> Thanks again for taking over the bug! >>>>>> Jiangli >>>>>> >>>>>>> On May 11, 2018, at 2:33 PM, Erik >>>>>>> Joelsson? wrote: >>>>>>> >>>>>>> Received: from [10.132.185.167] (/10.132.185.167) >>>>>>> ? by default (Oracle Beehive Gateway v4.0) >>>>>>> ? with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 >>>>>>> Subject: Re: RFR: 8199807&? 8202738: AppCDS performs overly >>>>>>> restrictive path >>>>>>> matching check >>>>>>> To: Jiangli Zhou, >>>>>>> "hotspot-runtime-dev at openjdk.java.net runtime" >>>>>>> , >>>>>>> build-dev >>>>>>> References: >>>>>>> From: Erik Joelsson >>>>>>> Organization: Oracle Corporation >>>>>>> Message-ID: >>>>>>> Date: Fri, 11 May 2018 14:33:20 -0700 >>>>>>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) >>>>>>> Gecko/20100101 Thunderbird/52.7.0 >>>>>>> MIME-Version: 1.0 >>>>>>> In-Reply-To: >>>>>>> Content-Type: text/plain; charset=utf-8; format=flowed >>>>>>> Content-Transfer-Encoding: 8bit >>>>>>> Content-Language: en-US >>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> For the build change, it's very undesirable to always have to >>>>>>> relink libjvm on every incremental build. Such a change cannot >>>>>>> be accepted. >>>>>>> >>>>>>> I have a counter suggestion, which is still a bit of a hack, but >>>>>>> it will cause vm_version.cpp to be recompiled (almost) every >>>>>>> time libjvm.so needs to be relinked. The drawback is that >>>>>>> compiling vm_version.cpp is now bound to happen absolutely last >>>>>>> and so cannot happen in parallel with other compilations. >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>> >>>>>>> /Erik >>>>>>> >>>>>>> >>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Please review the following webrev that addresses the issue of >>>>>>>> copied/moved JDK image after generating a CDS archive. Thanks >>>>>>>> Karen Kinnear and Alan Baterman for initiating the >>>>>>>> investigation& discussions in this area (especially the ease of >>>>>>>> usage). Thanks Ioi for implementing a test case for moved JDK >>>>>>>> (JDK-8202935). >>>>>>>> >>>>>>>> webrev: >>>>>>>> http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>> >>>>>>>> The webrev includes the following three main parts: >>>>>>>> >>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The >>>>>>>> runtime path to the ?modules? image is no longer required to >>>>>>>> the the same as dump time path. Runtime performs file size >>>>>>>> check only for the ?modules? image, which must match with the >>>>>>>> dump time ?modules? size. Invalidation of an outdated archive >>>>>>>> is achieved by the existing vm_version string check (the >>>>>>>> archived vm_version string must match with the runtime >>>>>>>> vm_version string). >>>>>>>> >>>>>>>> 2. Boot path check are now performed based on the content of >>>>>>>> the archive. Also added a new test case in >>>>>>>> BootClassPathMismatch.java and add more comments for existing >>>>>>>> test cases. >>>>>>>> >>>>>>>> 3. Fixed the stale vm_version string issue with incremental >>>>>>>> build. The issue was discovered during the work of 8199807. CDS >>>>>>>> uses vm_version string as part of the runtime validation check >>>>>>>> for archive. A stale vm_version string causes the CDS runtime >>>>>>>> to mistakenly accept an outdated archive. The fix is to make >>>>>>>> sure vm_version.o is recompiled properly when the library/vm is >>>>>>>> rebuilt. >>>>>>>> >>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating >>>>>>>> the JDK image manually after generating an archive. Also tested >>>>>>>> with Ioi?s test both locally and via Mach5. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jiangli >>>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> For the build change, it's very undesirable to always have to >>>>>>> relink libjvm on every incremental build. Such a change cannot >>>>>>> be accepted. >>>>>>> >>>>>>> I have a counter suggestion, which is still a bit of a hack, but >>>>>>> it will cause vm_version.cpp to be recompiled (almost) every >>>>>>> time libjvm.so needs to be relinked. The drawback is that >>>>>>> compiling vm_version.cpp is now bound to happen absolutely last >>>>>>> and so cannot happen in parallel with other compilations. >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>> >>>>>>> /Erik >>>>>>> >>>>>>> >>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Please review the following webrev that addresses the issue of >>>>>>>> copied/moved JDK image after generating a CDS archive. Thanks >>>>>>>> Karen Kinnear and Alan Baterman for initiating the >>>>>>>> investigation& discussions in this area (especially the ease of >>>>>>>> usage). Thanks Ioi for implementing a test case for moved JDK >>>>>>>> (JDK-8202935). >>>>>>>> >>>>>>>> webrev: >>>>>>>> http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>> >>>>>>>> The webrev includes the following three main parts: >>>>>>>> >>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The >>>>>>>> runtime path to the ?modules? image is no longer required to >>>>>>>> the the same as dump time path. Runtime performs file size >>>>>>>> check only for the ?modules? image, which must match with the >>>>>>>> dump time ?modules? size. Invalidation of an outdated archive >>>>>>>> is achieved by the existing vm_version string check (the >>>>>>>> archived vm_version string must match with the runtime >>>>>>>> vm_version string). >>>>>>>> >>>>>>>> 2. Boot path check are now performed based on the content of >>>>>>>> the archive. Also added a new test case in >>>>>>>> BootClassPathMismatch.java and add more comments for existing >>>>>>>> test cases. >>>>>>>> >>>>>>>> 3. Fixed the stale vm_version string issue with incremental >>>>>>>> build. The issue was discovered during the work of 8199807. CDS >>>>>>>> uses vm_version string as part of the runtime validation check >>>>>>>> for archive. A stale vm_version string causes the CDS runtime >>>>>>>> to mistakenly accept an outdated archive. The fix is to make >>>>>>>> sure vm_version.o is recompiled properly when the library/vm is >>>>>>>> rebuilt. >>>>>>>> >>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating >>>>>>>> the JDK image manually after generating an archive. Also tested >>>>>>>> with Ioi?s test both locally and via Mach5. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jiangli > From ioi.lam at oracle.com Tue May 15 23:42:43 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 15 May 2018 16:42:43 -0700 Subject: RFR(S) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: <6c7b37e4-3c13-a151-1bb7-24bb0b63e506@oracle.com> References: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> <6d03e69d-1b84-254a-9056-7c38c5053975@oracle.com> <8319ecf5-a2d8-fade-91f7-c33f636d4a71@oracle.com> <6c7b37e4-3c13-a151-1bb7-24bb0b63e506@oracle.com> Message-ID: <2d4b3803-4e98-058a-d365-40f9d697e1ae@oracle.com> On 5/15/18 4:34 PM, coleen.phillimore at oracle.com wrote: > > > On 5/15/18 7:02 PM, Ioi Lam wrote: >> >> >> On 5/15/18 3:22 PM, coleen.phillimore at oracle.com wrote: >>> >>> >>> >>> On 5/15/18 5:47 PM, Ioi Lam wrote: >>>> I've updated the webrev: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/ >>>> >>>> >>>> 1. Added JavaCalls::new_instance so we can avoid all the boiler >>>> plate code for allocating >>>> ?? the instance andinvoking the constructor. >>>> >>>> JavaCalls::new_instance calls InstanceKlass->initialize. This is >>>> just a quick op after >>>> ?? the class is already initialized. Also, JavaCalls::call_static >>>> also internally call >>>> ?? into InstanceKlass->initialize, so I am just following the >>>> existing pattern as Coleen >>>> ?? mentioned below. >>>> >>> >>> I think I like your last version better.? I don't think >>> JavaCalls::new_instance() can nicely generalize the calls to object >>> init functions that I found around the vm.? I think just doing what >>> you had would be better.? It's only 3 klass->initialize calls. You >>> could make the new_instance() function local to >>> systemDictionaryShared to save code if they are all the same, but I >>> don't see much advantage. >>> >>> Also a general new_instance() function should have the call to this >>> to be the same as the others (see reflection.cpp). >>> >>> ? klass->check_valid_for_instantiation(false, CHECK_NULL); >>> >> javaCalls is for cases where "the VM knows what it's doing", and >> reflection.cpp is to implement java.lang.reflect. So the former >> doesn't do access checks while the latter does. You can compare >> JavaCalls::call_static() vs Reflection::invoke_method(). >> >>> If it were possible to replace all klass->allocate_instance() calls >>> with JavaCalls::new_instance(), that seems like the API would be >>> worth it, but it's not possible.?? So people can still make mistakes >>> and not call the object init function, like I think someone did in >>> jvm.cpp. >>> >> Why is it not possible? I think many places can be cleaned up with >> this API, e.g., in ./share/services/{management.cpp, >> memoryService.cpp, attachListener.cpp and gcNotifier.cpp). I can do >> the cleanup in a separate RFE. > > I don't see how these would be any cleaner.? One example you sited > calls a static function initialize_ThreadInfo_constructor_arguments > which pushes 9 arguments to the constructor.?? You would have a > version of JavaCalls::new_instance with 1,2,3,...,9 potential > arguments??? It wouldn't be less code or nicer. > There is a catch-all version, just like the JavaCalls::call_static() variants: ? // Allocate instance + invoke constructor. This is equivalent to "new Klass(args ...)" expression in Java code. ? static Handle new_instance(InstanceKlass* klass, Symbol* constructor_signature, JavaCallArguments* args, TRAPS); ? static Handle new_instance(InstanceKlass* klass, Symbol* constructor_signature, TRAPS); ? static Handle new_instance(InstanceKlass* klass, Symbol* constructor_signature, Handle arg1, TRAPS); ? static Handle new_instance(InstanceKlass* klass, Symbol* constructor_signature, Handle arg1, Handle arg2, TRAPS); >> >> Some places call klass->allocate_instance() without calling the >> constructor, on purpose, to simulate the behavior of the 'new' >> bytecode. E.g., in jvmciRuntime.cpp. > > Yes, so you cannot replace all calls to allocate_instance(). > >> >> But if what you want to do is "x = new Foo(args ...)" in the C code, >> JavaCalls::new_instance() seems like a clean way to do it. > > This name seems too ambiguous with allocate_instance() which I thought > called the constructor until I just looked.? If new_instance had a > sentence name, maybe it would indicate when one should use it vs. > allocate_instance.?? This doesn't seem like it helps. > How about JavaCalls::construct_instance(...)? > If you want to add a local one to systemDictionaryShared for the three > calls, that would be fine with me.? Then you can see whether you can > cleanly replace all of the places with a function that encapsulates > calling the klass->initialize function, allocating an object, and > calling the constructor with a variable length argument list.?? My > initial thought was to have the encapsulating function pass > JavaCallArgs with a NULL for the object argument which is replaced > inside the call after it's allocated.? Still sort of ugly but better > than 10 cascading functions. > It's done here: Handle JavaCalls::new_instance(InstanceKlass* klass, Symbol* constructor_signature, JavaCallArguments* args, TRAPS) { ? klass->initialize(CHECK_NH); // Quick no-op if already initialized. ? Handle obj = klass->allocate_instance_handle(CHECK_NH); ? JavaValue void_result(T_VOID); ? args->set_receiver(obj); // inserts as the first argument. <<<<< ? JavaCalls::call_special(&void_result, klass, ????????????????????????? vmSymbols::object_initializer_name(), ????????????????????????? constructor_signature, args, CHECK_NH); ? return obj; } set_receiver does what you mentioned, It's ugly but I didn't write it :-) Thanks - Ioi > Coleen >> >> Thanks >> - Ioi >>> >>>> Doing the initialization on demand also means for cases where JAR >>>> manifest is not used >>>> ?? (all code is loaded from the system image or directories), we >>>> get faster start-up. >>>> >>>> 2. I also took the time to removed a bunch of "// One oop argument" >>>> comments which >>>> ?? probably meant something to the person who wrote it, but seems >>>> useless to everyone >>>> ?? else. >>> >>> Not sure what that would mean myself :) >>>> >>>> 3. As Calvin suggested, I removed the File_klass and also >>>> ParseUtil_klass from >>>> ?? the system dictionary since they are not used anywhere. This >>>> hopefully improves start-up >>>> ?? by a little bit, since these 2 classes are no longer resolved at >>>> start-up. >>>> >>>> >>>> (BTW, I changed the RFR subject line from XS to S due to the extend >>>> of change :-) >>>> >>> Make it S again! >>> >>> Coleen >>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> >>>> >>>> On 5/15/18 2:00 PM, coleen.phillimore at oracle.com wrote: >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >>>>> >>>>> >>>>> This looks good.? This is a pattern that's used in other places, >>>>> and it would be better to not initialize these at startup in >>>>> thread.cpp. >>>>> >>>>> Coleen >>>>> >>>>> On 5/15/18 2:07 AM, Ioi Lam wrote: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>>>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>>>>> >>>>>> >>>>>> Summary: >>>>>> >>>>>> 1. Removed the forced initialization of a few classes used by >>>>>> AppCDS at JVM start-up. >>>>>> ?? Instead, initialize these class on demand by calling >>>>>> InstanceKlass::initialize, which >>>>>> ?? is a quick no-op if the class is already initialized. >>>>>> >>>>>> 2. The only initialization left is that of a global lock. So I >>>>>> renamed the function >>>>>> ?? to SystemDictionaryShared::initialize_locks(). >>>>>> >>>>>> 3. I moved the call of this function from >>>>>> SystemDictionary::compute_java_loaders() to >>>>>> SystemDictionary::initialize() where it seems to fit. >>>>>> >>>>>> Testing with hs-tiers 1 and 2. >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>> >>>> >>> >> > From coleen.phillimore at oracle.com Tue May 15 23:50:15 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 15 May 2018 19:50:15 -0400 Subject: RFR(S) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: <2d4b3803-4e98-058a-d365-40f9d697e1ae@oracle.com> References: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> <6d03e69d-1b84-254a-9056-7c38c5053975@oracle.com> <8319ecf5-a2d8-fade-91f7-c33f636d4a71@oracle.com> <6c7b37e4-3c13-a151-1bb7-24bb0b63e506@oracle.com> <2d4b3803-4e98-058a-d365-40f9d697e1ae@oracle.com> Message-ID: <0199dcf8-b85e-b17c-3828-4f9849ffdf7b@oracle.com> On 5/15/18 7:42 PM, Ioi Lam wrote: > > > On 5/15/18 4:34 PM, coleen.phillimore at oracle.com wrote: >> >> >> On 5/15/18 7:02 PM, Ioi Lam wrote: >>> >>> >>> On 5/15/18 3:22 PM, coleen.phillimore at oracle.com wrote: >>>> >>>> >>>> >>>> On 5/15/18 5:47 PM, Ioi Lam wrote: >>>>> I've updated the webrev: >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/ >>>>> >>>>> >>>>> 1. Added JavaCalls::new_instance so we can avoid all the boiler >>>>> plate code for allocating >>>>> ?? the instance andinvoking the constructor. >>>>> >>>>> JavaCalls::new_instance calls InstanceKlass->initialize. This is >>>>> just a quick op after >>>>> ?? the class is already initialized. Also, JavaCalls::call_static >>>>> also internally call >>>>> ?? into InstanceKlass->initialize, so I am just following the >>>>> existing pattern as Coleen >>>>> ?? mentioned below. >>>>> >>>> >>>> I think I like your last version better.? I don't think >>>> JavaCalls::new_instance() can nicely generalize the calls to object >>>> init functions that I found around the vm.? I think just doing what >>>> you had would be better.? It's only 3 klass->initialize calls. You >>>> could make the new_instance() function local to >>>> systemDictionaryShared to save code if they are all the same, but I >>>> don't see much advantage. >>>> >>>> Also a general new_instance() function should have the call to this >>>> to be the same as the others (see reflection.cpp). >>>> >>>> ? klass->check_valid_for_instantiation(false, CHECK_NULL); >>>> >>> javaCalls is for cases where "the VM knows what it's doing", and >>> reflection.cpp is to implement java.lang.reflect. So the former >>> doesn't do access checks while the latter does. You can compare >>> JavaCalls::call_static() vs Reflection::invoke_method(). >>> >>>> If it were possible to replace all klass->allocate_instance() calls >>>> with JavaCalls::new_instance(), that seems like the API would be >>>> worth it, but it's not possible.?? So people can still make >>>> mistakes and not call the object init function, like I think >>>> someone did in jvm.cpp. >>>> >>> Why is it not possible? I think many places can be cleaned up with >>> this API, e.g., in ./share/services/{management.cpp, >>> memoryService.cpp, attachListener.cpp and gcNotifier.cpp). I can do >>> the cleanup in a separate RFE. >> >> I don't see how these would be any cleaner.? One example you sited >> calls a static function initialize_ThreadInfo_constructor_arguments >> which pushes 9 arguments to the constructor.?? You would have a >> version of JavaCalls::new_instance with 1,2,3,...,9 potential >> arguments? It wouldn't be less code or nicer. >> > > There is a catch-all version, just like the JavaCalls::call_static() > variants: > > ? // Allocate instance + invoke constructor. This is equivalent to > "new Klass(args ...)" expression in Java code. > ? static Handle new_instance(InstanceKlass* klass, Symbol* > constructor_signature, JavaCallArguments* args, TRAPS); > > ? static Handle new_instance(InstanceKlass* klass, Symbol* > constructor_signature, TRAPS); > ? static Handle new_instance(InstanceKlass* klass, Symbol* > constructor_signature, Handle arg1, TRAPS); > ? static Handle new_instance(InstanceKlass* klass, Symbol* > constructor_signature, Handle arg1, Handle arg2, TRAPS); > > >>> >>> Some places call klass->allocate_instance() without calling the >>> constructor, on purpose, to simulate the behavior of the 'new' >>> bytecode. E.g., in jvmciRuntime.cpp. >> >> Yes, so you cannot replace all calls to allocate_instance(). >> >>> >>> But if what you want to do is "x = new Foo(args ...)" in the C code, >>> JavaCalls::new_instance() seems like a clean way to do it. >> >> This name seems too ambiguous with allocate_instance() which I >> thought called the constructor until I just looked.? If new_instance >> had a sentence name, maybe it would indicate when one should use it >> vs. allocate_instance.?? This doesn't seem like it helps. >> > How about JavaCalls::construct_instance(...)? JavaCalls::construct_new_instance.? Sold. > >> If you want to add a local one to systemDictionaryShared for the >> three calls, that would be fine with me.? Then you can see whether >> you can cleanly replace all of the places with a function that >> encapsulates calling the klass->initialize function, allocating an >> object, and calling the constructor with a variable length argument >> list.?? My initial thought was to have the encapsulating function >> pass JavaCallArgs with a NULL for the object argument which is >> replaced inside the call after it's allocated.? Still sort of ugly >> but better than 10 cascading functions. >> > It's done here: > > Handle JavaCalls::new_instance(InstanceKlass* klass, Symbol* > constructor_signature, JavaCallArguments* args, TRAPS) { > ? klass->initialize(CHECK_NH); // Quick no-op if already initialized. > ? Handle obj = klass->allocate_instance_handle(CHECK_NH); > ? JavaValue void_result(T_VOID); > ? args->set_receiver(obj); // inserts as the first argument. <<<<< > ? JavaCalls::call_special(&void_result, klass, > ????????????????????????? vmSymbols::object_initializer_name(), > ????????????????????????? constructor_signature, args, CHECK_NH); > ? return obj; > } > > set_receiver does what you mentioned, It's ugly but I didn't write it :-) I see that now.? I was distracted by the other versions that I didn't like. Coleen > > Thanks > - Ioi > >> Coleen >>> >>> Thanks >>> - Ioi >>>> >>>>> Doing the initialization on demand also means for cases where JAR >>>>> manifest is not used >>>>> ?? (all code is loaded from the system image or directories), we >>>>> get faster start-up. >>>>> >>>>> 2. I also took the time to removed a bunch of "// One oop >>>>> argument" comments which >>>>> ?? probably meant something to the person who wrote it, but seems >>>>> useless to everyone >>>>> ?? else. >>>> >>>> Not sure what that would mean myself :) >>>>> >>>>> 3. As Calvin suggested, I removed the File_klass and also >>>>> ParseUtil_klass from >>>>> ?? the system dictionary since they are not used anywhere. This >>>>> hopefully improves start-up >>>>> ?? by a little bit, since these 2 classes are no longer resolved >>>>> at start-up. >>>>> >>>>> >>>>> (BTW, I changed the RFR subject line from XS to S due to the >>>>> extend of change :-) >>>>> >>>> Make it S again! >>>> >>>> Coleen >>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> >>>>> >>>>> On 5/15/18 2:00 PM, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >>>>>> >>>>>> >>>>>> This looks good.? This is a pattern that's used in other places, >>>>>> and it would be better to not initialize these at startup in >>>>>> thread.cpp. >>>>>> >>>>>> Coleen >>>>>> >>>>>> On 5/15/18 2:07 AM, Ioi Lam wrote: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>>>>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>>>>>> >>>>>>> >>>>>>> Summary: >>>>>>> >>>>>>> 1. Removed the forced initialization of a few classes used by >>>>>>> AppCDS at JVM start-up. >>>>>>> ?? Instead, initialize these class on demand by calling >>>>>>> InstanceKlass::initialize, which >>>>>>> ?? is a quick no-op if the class is already initialized. >>>>>>> >>>>>>> 2. The only initialization left is that of a global lock. So I >>>>>>> renamed the function >>>>>>> ?? to SystemDictionaryShared::initialize_locks(). >>>>>>> >>>>>>> 3. I moved the call of this function from >>>>>>> SystemDictionary::compute_java_loaders() to >>>>>>> SystemDictionary::initialize() where it seems to fit. >>>>>>> >>>>>>> Testing with hs-tiers 1 and 2. >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>> >>>>> >>>> >>> >> > From david.holmes at oracle.com Tue May 15 23:50:27 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 16 May 2018 09:50:27 +1000 Subject: RFR(S) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> References: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> Message-ID: <619cc18a-8c66-e817-3c79-c33140a121fd@oracle.com> Hi Ioi, You seem to have missed my query in earlier email. If JavaCalls::call_static already ensures that the class is initialized then why do we need to explicitly initialize it? I don't see anything in allocate_instance that seems to need the class to be initialized. Thanks, David On 16/05/2018 7:47 AM, Ioi Lam wrote: > I've updated the webrev: > > http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/ > > > 1. Added JavaCalls::new_instance so we can avoid all the boiler plate > code for allocating > ?? the instance andinvoking the constructor. > > JavaCalls::new_instance calls InstanceKlass->initialize. This is just a > quick op after > ?? the class is already initialized. Also, JavaCalls::call_static also > internally call > ?? into InstanceKlass->initialize, so I am just following the existing > pattern as Coleen > ?? mentioned below. > > ?? Doing the initialization on demand also means for cases where JAR > manifest is not used > ?? (all code is loaded from the system image or directories), we get > faster start-up. > > 2. I also took the time to removed a bunch of "// One oop argument" > comments which > ?? probably meant something to the person who wrote it, but seems > useless to everyone > ?? else. > > 3. As Calvin suggested, I removed the File_klass and also > ParseUtil_klass from > ?? the system dictionary since they are not used anywhere. This > hopefully improves start-up > ?? by a little bit, since these 2 classes are no longer resolved at > start-up. > > > (BTW, I changed the RFR subject line from XS to S due to the extend of > change :-) > > Thanks > - Ioi > > > > > On 5/15/18 2:00 PM, coleen.phillimore at oracle.com wrote: >> >> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >> >> >> This looks good.? This is a pattern that's used in other places, and >> it would be better to not initialize these at startup in thread.cpp. >> >> Coleen >> >> On 5/15/18 2:07 AM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>> >>> >>> Summary: >>> >>> 1. Removed the forced initialization of a few classes used by AppCDS >>> at JVM start-up. >>> ?? Instead, initialize these class on demand by calling >>> InstanceKlass::initialize, which >>> ?? is a quick no-op if the class is already initialized. >>> >>> 2. The only initialization left is that of a global lock. So I >>> renamed the function >>> ?? to SystemDictionaryShared::initialize_locks(). >>> >>> 3. I moved the call of this function from >>> SystemDictionary::compute_java_loaders() to >>> SystemDictionary::initialize() where it seems to fit. >>> >>> Testing with hs-tiers 1 and 2. >>> >>> Thanks >>> - Ioi >> > From david.holmes at oracle.com Tue May 15 23:54:41 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 16 May 2018 09:54:41 +1000 Subject: RFR(S) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: <619cc18a-8c66-e817-3c79-c33140a121fd@oracle.com> References: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> <619cc18a-8c66-e817-3c79-c33140a121fd@oracle.com> Message-ID: <089124a2-4553-65c6-6ad5-e2c18f6e7e66@oracle.com> Never mind I missed the change in the subject line - Please don't do that! ;-) David On 16/05/2018 9:50 AM, David Holmes wrote: > Hi Ioi, > > You seem to have missed my query in earlier email. > > If JavaCalls::call_static already ensures that the class is initialized > then why do we need to explicitly initialize it? I don't see anything in > allocate_instance that seems to need the class to be initialized. > > Thanks, > David > > On 16/05/2018 7:47 AM, Ioi Lam wrote: >> I've updated the webrev: >> >> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/ >> >> >> 1. Added JavaCalls::new_instance so we can avoid all the boiler plate >> code for allocating >> ??? the instance andinvoking the constructor. >> >> JavaCalls::new_instance calls InstanceKlass->initialize. This is just >> a quick op after >> ??? the class is already initialized. Also, JavaCalls::call_static >> also internally call >> ??? into InstanceKlass->initialize, so I am just following the >> existing pattern as Coleen >> ??? mentioned below. >> >> ??? Doing the initialization on demand also means for cases where JAR >> manifest is not used >> ??? (all code is loaded from the system image or directories), we get >> faster start-up. >> >> 2. I also took the time to removed a bunch of "// One oop argument" >> comments which >> ??? probably meant something to the person who wrote it, but seems >> useless to everyone >> ??? else. >> >> 3. As Calvin suggested, I removed the File_klass and also >> ParseUtil_klass from >> ??? the system dictionary since they are not used anywhere. This >> hopefully improves start-up >> ??? by a little bit, since these 2 classes are no longer resolved at >> start-up. >> >> >> (BTW, I changed the RFR subject line from XS to S due to the extend of >> change :-) >> >> Thanks >> - Ioi >> >> >> >> >> On 5/15/18 2:00 PM, coleen.phillimore at oracle.com wrote: >>> >>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >>> >>> >>> This looks good.? This is a pattern that's used in other places, and >>> it would be better to not initialize these at startup in thread.cpp. >>> >>> Coleen >>> >>> On 5/15/18 2:07 AM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>>> >>>> >>>> Summary: >>>> >>>> 1. Removed the forced initialization of a few classes used by AppCDS >>>> at JVM start-up. >>>> ?? Instead, initialize these class on demand by calling >>>> InstanceKlass::initialize, which >>>> ?? is a quick no-op if the class is already initialized. >>>> >>>> 2. The only initialization left is that of a global lock. So I >>>> renamed the function >>>> ?? to SystemDictionaryShared::initialize_locks(). >>>> >>>> 3. I moved the call of this function from >>>> SystemDictionary::compute_java_loaders() to >>>> SystemDictionary::initialize() where it seems to fit. >>>> >>>> Testing with hs-tiers 1 and 2. >>>> >>>> Thanks >>>> - Ioi >>> >> From ioi.lam at oracle.com Wed May 16 00:03:21 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 15 May 2018 17:03:21 -0700 Subject: RFR(S) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: <619cc18a-8c66-e817-3c79-c33140a121fd@oracle.com> References: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> <619cc18a-8c66-e817-3c79-c33140a121fd@oracle.com> Message-ID: <52b7f8a0-0064-f194-6e56-02f6d4ff23fb@oracle.com> On 5/15/18 4:50 PM, David Holmes wrote: > Hi Ioi, > > You seem to have missed my query in earlier email. > > If JavaCalls::call_static already ensures that the class is > initialized then why do we need to explicitly initialize it? I don't > see anything in allocate_instance that seems to need the class to be > initialized. > I think you meant JavaCall::call_special here. JavaCall::call_special does not ensure the class is initialized, just like its counterpart, the invokespecial bytecode. If I skip the initialization, I'll get an assert: #6? 0x00007ffff612eb83 in JavaCalls::call_helper (result=0x7ffff7fc7680, method=..., args=0x7ffff7fc7690, __the_thread__=0x7ffff001b800) ??? at /home/iklam/jdk/abe/open/src/hotspot/share/runtime/javaCalls.cpp:382 382?? ???? assert(holder->is_linked(), "rewriting must have taken place"); Thanks - Ioi > Thanks, > David > > On 16/05/2018 7:47 AM, Ioi Lam wrote: >> I've updated the webrev: >> >> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/ >> >> >> 1. Added JavaCalls::new_instance so we can avoid all the boiler plate >> code for allocating >> ??? the instance andinvoking the constructor. >> >> JavaCalls::new_instance calls InstanceKlass->initialize. This is just >> a quick op after >> ??? the class is already initialized. Also, JavaCalls::call_static >> also internally call >> ??? into InstanceKlass->initialize, so I am just following the >> existing pattern as Coleen >> ??? mentioned below. >> >> ??? Doing the initialization on demand also means for cases where JAR >> manifest is not used >> ??? (all code is loaded from the system image or directories), we get >> faster start-up. >> >> 2. I also took the time to removed a bunch of "// One oop argument" >> comments which >> ??? probably meant something to the person who wrote it, but seems >> useless to everyone >> ??? else. >> >> 3. As Calvin suggested, I removed the File_klass and also >> ParseUtil_klass from >> ??? the system dictionary since they are not used anywhere. This >> hopefully improves start-up >> ??? by a little bit, since these 2 classes are no longer resolved at >> start-up. >> >> >> (BTW, I changed the RFR subject line from XS to S due to the extend >> of change :-) >> >> Thanks >> - Ioi >> >> >> >> >> On 5/15/18 2:00 PM, coleen.phillimore at oracle.com wrote: >>> >>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >>> >>> >>> This looks good.? This is a pattern that's used in other places, and >>> it would be better to not initialize these at startup in thread.cpp. >>> >>> Coleen >>> >>> On 5/15/18 2:07 AM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>>> >>>> >>>> Summary: >>>> >>>> 1. Removed the forced initialization of a few classes used by >>>> AppCDS at JVM start-up. >>>> ?? Instead, initialize these class on demand by calling >>>> InstanceKlass::initialize, which >>>> ?? is a quick no-op if the class is already initialized. >>>> >>>> 2. The only initialization left is that of a global lock. So I >>>> renamed the function >>>> ?? to SystemDictionaryShared::initialize_locks(). >>>> >>>> 3. I moved the call of this function from >>>> SystemDictionary::compute_java_loaders() to >>>> SystemDictionary::initialize() where it seems to fit. >>>> >>>> Testing with hs-tiers 1 and 2. >>>> >>>> Thanks >>>> - Ioi >>> >> From coleen.phillimore at oracle.com Wed May 16 00:05:15 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 15 May 2018 20:05:15 -0400 Subject: RFR(S) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> References: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> Message-ID: Hi, I had another question/comment that I forgot to ask: http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html SharedDictionary_lock = new Mutex(Mutex::leaf, "SharedDictionary_lock", true); We want to put all the global locks in mutexLocker.cpp so that we can better see their relative ranking.? Can you put this there?? I don't think it's necessary to make it CDS_ONLY() really, it doesn't take an interesting amount of space. Or was there a reason this was here that I don't see? thanks, Coleen On 5/15/18 5:47 PM, Ioi Lam wrote: > I've updated the webrev: > > http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/ > > > 1. Added JavaCalls::new_instance so we can avoid all the boiler plate > code for allocating > ?? the instance andinvoking the constructor. > > JavaCalls::new_instance calls InstanceKlass->initialize. This is just > a quick op after > ?? the class is already initialized. Also, JavaCalls::call_static also > internally call > ?? into InstanceKlass->initialize, so I am just following the existing > pattern as Coleen > ?? mentioned below. > > ?? Doing the initialization on demand also means for cases where JAR > manifest is not used > ?? (all code is loaded from the system image or directories), we get > faster start-up. > > 2. I also took the time to removed a bunch of "// One oop argument" > comments which > ?? probably meant something to the person who wrote it, but seems > useless to everyone > ?? else. > > 3. As Calvin suggested, I removed the File_klass and also > ParseUtil_klass from > ?? the system dictionary since they are not used anywhere. This > hopefully improves start-up > ?? by a little bit, since these 2 classes are no longer resolved at > start-up. > > > (BTW, I changed the RFR subject line from XS to S due to the extend of > change :-) > > Thanks > - Ioi > > > > > On 5/15/18 2:00 PM, coleen.phillimore at oracle.com wrote: >> >> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >> >> >> This looks good.? This is a pattern that's used in other places, and >> it would be better to not initialize these at startup in thread.cpp. >> >> Coleen >> >> On 5/15/18 2:07 AM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>> >>> >>> Summary: >>> >>> 1. Removed the forced initialization of a few classes used by AppCDS >>> at JVM start-up. >>> ?? Instead, initialize these class on demand by calling >>> InstanceKlass::initialize, which >>> ?? is a quick no-op if the class is already initialized. >>> >>> 2. The only initialization left is that of a global lock. So I >>> renamed the function >>> ?? to SystemDictionaryShared::initialize_locks(). >>> >>> 3. I moved the call of this function from >>> SystemDictionary::compute_java_loaders() to >>> SystemDictionary::initialize() where it seems to fit. >>> >>> Testing with hs-tiers 1 and 2. >>> >>> Thanks >>> - Ioi >> > From jiangli.zhou at oracle.com Wed May 16 01:10:26 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Tue, 15 May 2018 18:10:26 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: References: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> <5AF62823.7050804@oracle.com> <70F6AEB0-9103-4899-87D9-DC845BF66C51@oracle.com> <4CBA8376-54A2-4863-8732-B16E1AE9C38A@oracle.com> Message-ID: Hi Ioi, Thank you very much for the review. Good catches for those edge cases and thanks for the test cases. I will resolves them. Thanks! Jiangli > On May 15, 2018, at 4:33 PM, Ioi Lam wrote: > > Hi Jiangli, > > 245 // We need to validate the runtime modules image file size against the archived > 246 // size information, obtain the runtime modules image path. The recorded dump > 247 // time modules image path in the archive may be different from the runtime path > 248 // if the JDK image has beed moved after generating the archive. > 249 if (ClassLoader::is_modules_image(name)) { > 250 name = ClassLoader::get_jrt_entry()->name(); > 251 } > > What happens if someone has a JAR file called foo.modules, and dumped with > > java -Xshare:dump -cp foo.modules > > > I think it's better to check that it's #0 in the classpath so we know for sure it's the system modules file. > > There's a similar problem here: > > 220 if (!ClassLoader::is_modules_image(name)) { > > In the following, instead of doing a copy, maybe it's better to add a length argument for os::file_name_strcmp, since sharedPathsMiscInfo.cpp is the only place in the VM that calls this function. > > 162 if (relaxed_check) { > 163 // only check the begining portion of the runtime boot path, up to > 164 // the length of the dump time boot path > 165 size_t len = strlen(path); > 166 runtime_boot_path = NEW_RESOURCE_ARRAY(char, len + 1); > 167 strncpy(runtime_boot_path, Arguments::get_sysclasspath(), len); > 168 runtime_boot_path[len] = '\0'; > 169 } else { > 170 // check the full runtime boot path > 171 runtime_boot_path = Arguments::get_sysclasspath(); > 172 } > 173 > 174 // Do a quick check first with a simple > 175 // strcmp(dump_time_boot_path, runtime_boot_path). If the paths are the > 176 // same, the check has succeeded. > 177 if (os::file_name_strcmp(path, runtime_boot_path) == 0) { > 178 break; // ok > 179 } > > > Also, the copy could be wrong for the following output: > > path = /tmp/foo/modules > Arguments::get_sysclasspath() = /tmp/foo/modulesxyz:/aaa:/bbb > > What is runtime_boot_path supposed to contain here? I am not sure if this code gives the intended output or not, but it's not easy to understand what it actually does by operating on an truncated pathname. > > String manipulation code is always hard to read. I would suggest braking it up into smaller functions: > > 1. Get the first entry of dump time and run time boot path -> d0 and r0 > -> ignore d0 > -> r0 same as ClassLoader::get_jrt_entry()->name(); > > 2. Get the remaining part of the dump time and run time boot path -> d_remain, r_remain > -> relaxed_check: d_remain must be a prefix of r_remain > -> !relaxed_check: They must be identical. > > > By the way, I think the test case in JDK-8202935 should be reviewed together inside this RFR, since the test validates the feature implemented here. I don't think we need a separate bug ID. Otherwise it will be hard to track the test cases. > > > Thanks > - Ioi > > > On 5/13/18 5:31 PM, Jiangli Zhou wrote: >> Here is the updated webrev: >> http://cr.openjdk.java.net/~jiangli/8199807/webrev.01/ >> >> Thanks, >> Jiangli >> >>> On May 11, 2018, at 6:07 PM, Jiangli Zhou wrote: >>> >>> Hi Calvin, >>> >>> Thanks for the review. >>> >>>> On May 11, 2018, at 4:32 PM, Calvin Cheung wrote: >>>> >>>> Hi Jiangli, >>>> >>>> Thanks for doing this useful change. >>>> >>>> I have a few minor comments: >>>> >>>> 1) sharedPathsMiscInfo.cpp >>>> >>>> 199 dp ++; >>>> >>>> Should the above be similar to line 194 as follows? >>>> dp += path_sep_len; >>> Good catch. Will fix. >>> >>>> 2) filemap.cpp >>>> >>>> 244 if (ClassLoader::is_modules_image(name)) { >>>> >>>> I think the above could be: >>>> if (is_modules_image()) { >>>> >>> Let?s keep it this way as is_modules_image() is a wrapper of 'ClassLoader::is_modules_image(name())?. We already have the ?name? in this case. >>> >>>> The is_modules_image() is a member function of SharedClassPathEntry. >>>> >>>> Btw, why is it necessary to get the runtime modules image path again in lines 244 - 246? >>> As the runtime modules image path could be different from the dump time if the JDK image is moved/copied after archive generation, we need to make sure to use the correct file for file size check. The recored path in the archive is the dump time modules image path, which may no longer be existing at runtime. I will add some comments if that?s helpful. >>> >>>> 3) BootClassPathMismatch.java >>>> >>>> Could you move line 103 to right above line 107 (opts.addPrefix(?))? >>>> Because the archive dumping part of the test doesn?t use appJar. >>> Sure. >>> >>> Thanks! >>> >>> Jiangli >>> >>>> thanks, >>>> Calvin >>>> >>>> On 5/11/18, 3:15 PM, Jiangli Zhou wrote: >>>>> I?ve withdrawn my original weberv and removed the build change. Here is the updated webrev that only addresses 8199807. >>>>> >>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.00/ >>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>> >>>>> Thanks, >>>>> Jiangli >>>>> >>>>>> On May 11, 2018, at 2:44 PM, Jiangli Zhou wrote: >>>>>> >>>>>> Hi Erik, >>>>>> >>>>>> Thank you so much for investigating a proper fix for the vm_version.o issue. I will withdraw the build change from my original webrev. >>>>>> >>>>>> Thanks again for taking over the bug! >>>>>> Jiangli >>>>>> >>>>>>> On May 11, 2018, at 2:33 PM, Erik Joelsson wrote: >>>>>>> >>>>>>> Received: from [10.132.185.167] (/10.132.185.167) >>>>>>> by default (Oracle Beehive Gateway v4.0) >>>>>>> with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 >>>>>>> Subject: Re: RFR: 8199807& 8202738: AppCDS performs overly restrictive path >>>>>>> matching check >>>>>>> To: Jiangli Zhou, >>>>>>> "hotspot-runtime-dev at openjdk.java.net runtime" >>>>>>> , >>>>>>> build-dev >>>>>>> References: >>>>>>> From: Erik Joelsson >>>>>>> Organization: Oracle Corporation >>>>>>> Message-ID: >>>>>>> Date: Fri, 11 May 2018 14:33:20 -0700 >>>>>>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) >>>>>>> Gecko/20100101 Thunderbird/52.7.0 >>>>>>> MIME-Version: 1.0 >>>>>>> In-Reply-To: >>>>>>> Content-Type: text/plain; charset=utf-8; format=flowed >>>>>>> Content-Transfer-Encoding: 8bit >>>>>>> Content-Language: en-US >>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>> >>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>> >>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>> >>>>>>> /Erik >>>>>>> >>>>>>> >>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>> >>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>> >>>>>>>> The webrev includes the following three main parts: >>>>>>>> >>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>> >>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>> >>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>> >>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jiangli >>>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>> >>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>> >>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>> >>>>>>> /Erik >>>>>>> >>>>>>> >>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>> >>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>> >>>>>>>> The webrev includes the following three main parts: >>>>>>>> >>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>> >>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>> >>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>> >>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jiangli > From coleen.phillimore at oracle.com Wed May 16 01:23:05 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 15 May 2018 21:23:05 -0400 Subject: RFR (XS) 8202014: Possible to receive signal before signal semaphore created Message-ID: <3d3ea98c-60c9-71de-f0c7-e156207b24ca@oracle.com> Summary: Don't assert for null semaphore because it can be null before initialization is complete. Tested this with a shell script, thanks Robbin, and mach5 tier1,2 all Oracle platforms.? AIX doesn't use the semaphore class so doesn't have this bug. open webrev at http://cr.openjdk.java.net/~coleenp/8202014.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8202014 Thanks, Coleen From mikhailo.seledtsov at oracle.com Wed May 16 01:49:00 2018 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Tue, 15 May 2018 18:49:00 -0700 Subject: RFR(M): 8199252: [TESTBUG] Open source VM testbase system dictionary tests In-Reply-To: References: <5AF606AF.3020404@oracle.com> Message-ID: <5AFB8E0C.9000200@oracle.com> Hi Gerard, On 5/15/18, 11:24 AM, Gerard Ziemski wrote: > hi Misha, > > Looks fine. Thank you for review. > > Just to be sure: did you have to make any functional changes to the test source code, or was it pretty much a straightforward case of moving the files? No functional changes. The change was moving the files, updating copyright statements, updating @library statements (removing extra "open" at the beginning of the path) and moving a test group from closed to open. All "mechanical" and fairly trivial changes. > > Thanks for doing this. No problem. Thank you, Misha > > > cheers > >> On May 11, 2018, at 4:10 PM, Mikhailo Seledtsov wrote: >> >> Please review this change open sourcing VM system dictionary tests. These tests have been used internally for a while, and are now being open sourced. >> Since this is not an creation of new tests, simply open sourcing existing tests, we would like to keep the changes during this review to an absolute required minimum. If you have any feedback on improvements of these tests, please file RFE(s) that will be addressed later in order of priority. >> >> >> Here is what was done for this change: >> 1. Moved the tests to OpenJDK repository preserving relative directory location and structure. >> 3. Updated Copyright statements accordingly. >> 4. Updated "@library" statements accordingly. >> 5. Updated TEST.groups >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8199252 >> Webrev: http://cr.openjdk.java.net/~mseledtsov/8199252.01.open/ >> >> Testing: >> 1. Ran the following tests on open-only repository and build, using "make run-test" >> (Linux-x64) >> vmTestbase_nsk_sysdict >> All PASS >> >> 2. Automated multip-platform test system (usual 4 platforms): >> - vmTestbase_nsk_sysdict >> - hs-tier{1,2} >> In progress >> >> Thank you, >> Misha >> From david.holmes at oracle.com Wed May 16 01:53:04 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 16 May 2018 11:53:04 +1000 Subject: RFR(S) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> References: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> Message-ID: <6d89d9c6-47ef-75fd-5aed-cfea6389d922@oracle.com> On 16/05/2018 7:47 AM, Ioi Lam wrote: > I've updated the webrev: > > http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/ > > > 1. Added JavaCalls::new_instance so we can avoid all the boiler plate > code for allocating > ?? the instance andinvoking the constructor. > > JavaCalls::new_instance calls InstanceKlass->initialize. This is just a > quick op after > ?? the class is already initialized. Also, JavaCalls::call_static also > internally call > ?? into InstanceKlass->initialize, so I am just following the existing > pattern as Coleen > ?? mentioned below. > > ?? Doing the initialization on demand also means for cases where JAR > manifest is not used > ?? (all code is loaded from the system image or directories), we get > faster start-up. That all looks good - nice cleanup. And we should look at using the new_instance in more places if appropriate. > 2. I also took the time to removed a bunch of "// One oop argument" > comments which > ?? probably meant something to the person who wrote it, but seems > useless to everyone else. Especially when the coding pattern is odd anyway. I mean why have: JavaCallArguments args(receiver); args.push_oop(arg1); args.push_oop(arg2); instead of JavaCallArguments args; args.push_oop(receiver); args.push_oop(arg1); args.push_oop(arg2); or JavaCallArguments args(receiver, oop1, oop2); ?? (Not something I expect you to change of course!) > > 3. As Calvin suggested, I removed the File_klass and also > ParseUtil_klass from > ?? the system dictionary since they are not used anywhere. This > hopefully improves start-up > ?? by a little bit, since these 2 classes are no longer resolved at > start-up. Ok. > > (BTW, I changed the RFR subject line from XS to S due to the extend of > change :-) Please don't do that as it breaks the flow when sorting/threading by subject! Modulo Colleen's query on the lock initialization placement this all seems fine to me. Thanks, David > Thanks > - Ioi > > > > > On 5/15/18 2:00 PM, coleen.phillimore at oracle.com wrote: >> >> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >> >> >> This looks good.? This is a pattern that's used in other places, and >> it would be better to not initialize these at startup in thread.cpp. >> >> Coleen >> >> On 5/15/18 2:07 AM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>> >>> >>> Summary: >>> >>> 1. Removed the forced initialization of a few classes used by AppCDS >>> at JVM start-up. >>> ?? Instead, initialize these class on demand by calling >>> InstanceKlass::initialize, which >>> ?? is a quick no-op if the class is already initialized. >>> >>> 2. The only initialization left is that of a global lock. So I >>> renamed the function >>> ?? to SystemDictionaryShared::initialize_locks(). >>> >>> 3. I moved the call of this function from >>> SystemDictionary::compute_java_loaders() to >>> SystemDictionary::initialize() where it seems to fit. >>> >>> Testing with hs-tiers 1 and 2. >>> >>> Thanks >>> - Ioi >> > From david.holmes at oracle.com Wed May 16 02:40:17 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 16 May 2018 12:40:17 +1000 Subject: RFR (XS) 8202014: Possible to receive signal before signal semaphore created In-Reply-To: <3d3ea98c-60c9-71de-f0c7-e156207b24ca@oracle.com> References: <3d3ea98c-60c9-71de-f0c7-e156207b24ca@oracle.com> Message-ID: <3e24a240-9e6b-65a0-64ca-c81db396f6cb@oracle.com> Hi Coleen, On 16/05/2018 11:23 AM, coleen.phillimore at oracle.com wrote: > Summary: Don't assert for null semaphore because it can be null before > initialization is complete. Sorry I don't agree with this. Surely we should not install our signal handler until such time as everything is properly initialized? Or the handler itself should have some kind of initialization check to guide how it responds. Thanks, David > Tested this with a shell script, thanks Robbin, and mach5 tier1,2 all > Oracle platforms.? AIX doesn't use the semaphore class so doesn't have > this bug. > > open webrev at http://cr.openjdk.java.net/~coleenp/8202014.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8202014 > > Thanks, > Coleen From ioi.lam at oracle.com Wed May 16 04:06:20 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 15 May 2018 21:06:20 -0700 Subject: RFR(S) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: <6d89d9c6-47ef-75fd-5aed-cfea6389d922@oracle.com> References: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> <6d89d9c6-47ef-75fd-5aed-cfea6389d922@oracle.com> Message-ID: I've updated the webrev: http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v03/ 1. Moved SharedDictionary_lock to mutexLocker.cpp. 2. There's no more need for SystemDictionaryShared::initialize. Removed. 3. Renamed to JavaCalls::construct_new_instance per Coleen. If this goes in, I'll file a separate RFE to convert some other code to use JavaCalls::construct_new_instance. Thanks - Ioi On 5/15/18 6:53 PM, David Holmes wrote: > On 16/05/2018 7:47 AM, Ioi Lam wrote: >> I've updated the webrev: >> >> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/ >> >> >> 1. Added JavaCalls::new_instance so we can avoid all the boiler plate >> code for allocating >> ??? the instance andinvoking the constructor. >> >> JavaCalls::new_instance calls InstanceKlass->initialize. This is just >> a quick op after >> ??? the class is already initialized. Also, JavaCalls::call_static >> also internally call >> ??? into InstanceKlass->initialize, so I am just following the >> existing pattern as Coleen >> ??? mentioned below. >> >> ??? Doing the initialization on demand also means for cases where JAR >> manifest is not used >> ??? (all code is loaded from the system image or directories), we get >> faster start-up. > > That all looks good - nice cleanup. And we should look at using the > new_instance in more places if appropriate. > >> 2. I also took the time to removed a bunch of "// One oop argument" >> comments which >> ??? probably meant something to the person who wrote it, but seems >> useless to everyone else. > > Especially when the coding pattern is odd anyway. I mean why have: > > ??? JavaCallArguments args(receiver); > ??? args.push_oop(arg1); > ??? args.push_oop(arg2); > > instead of > > ??? JavaCallArguments args; > ??? args.push_oop(receiver); > ??? args.push_oop(arg1); > ??? args.push_oop(arg2); > > or > > ??? JavaCallArguments args(receiver, oop1, oop2); > > ?? (Not something I expect you to change of course!) > >> >> 3. As Calvin suggested, I removed the File_klass and also >> ParseUtil_klass from >> ??? the system dictionary since they are not used anywhere. This >> hopefully improves start-up >> ??? by a little bit, since these 2 classes are no longer resolved at >> start-up. > > Ok. > >> >> (BTW, I changed the RFR subject line from XS to S due to the extend >> of change :-) > > Please don't do that as it breaks the flow when sorting/threading by > subject! > > Modulo Colleen's query on the lock initialization placement this all > seems fine to me. > > Thanks, > David > >> Thanks >> - Ioi >> >> >> >> >> On 5/15/18 2:00 PM, coleen.phillimore at oracle.com wrote: >>> >>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >>> >>> >>> This looks good.? This is a pattern that's used in other places, and >>> it would be better to not initialize these at startup in thread.cpp. >>> >>> Coleen >>> >>> On 5/15/18 2:07 AM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>>> >>>> >>>> Summary: >>>> >>>> 1. Removed the forced initialization of a few classes used by >>>> AppCDS at JVM start-up. >>>> ?? Instead, initialize these class on demand by calling >>>> InstanceKlass::initialize, which >>>> ?? is a quick no-op if the class is already initialized. >>>> >>>> 2. The only initialization left is that of a global lock. So I >>>> renamed the function >>>> ?? to SystemDictionaryShared::initialize_locks(). >>>> >>>> 3. I moved the call of this function from >>>> SystemDictionary::compute_java_loaders() to >>>> SystemDictionary::initialize() where it seems to fit. >>>> >>>> Testing with hs-tiers 1 and 2. >>>> >>>> Thanks >>>> - Ioi >>> >> From david.holmes at oracle.com Wed May 16 04:21:42 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 16 May 2018 14:21:42 +1000 Subject: RFR(S) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: References: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> <6d89d9c6-47ef-75fd-5aed-cfea6389d922@oracle.com> Message-ID: Looks good! Thanks, David On 16/05/2018 2:06 PM, Ioi Lam wrote: > I've updated the webrev: > > http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v03/ > > > 1. Moved SharedDictionary_lock to mutexLocker.cpp. > > 2. There's no more need for SystemDictionaryShared::initialize. Removed. > > 3. Renamed to JavaCalls::construct_new_instance per Coleen. > > If this goes in, I'll file a separate RFE to convert some other code to > use JavaCalls::construct_new_instance. > > Thanks > > - Ioi > > > > On 5/15/18 6:53 PM, David Holmes wrote: >> On 16/05/2018 7:47 AM, Ioi Lam wrote: >>> I've updated the webrev: >>> >>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/ >>> >>> >>> 1. Added JavaCalls::new_instance so we can avoid all the boiler plate >>> code for allocating >>> ??? the instance andinvoking the constructor. >>> >>> JavaCalls::new_instance calls InstanceKlass->initialize. This is just >>> a quick op after >>> ??? the class is already initialized. Also, JavaCalls::call_static >>> also internally call >>> ??? into InstanceKlass->initialize, so I am just following the >>> existing pattern as Coleen >>> ??? mentioned below. >>> >>> ??? Doing the initialization on demand also means for cases where JAR >>> manifest is not used >>> ??? (all code is loaded from the system image or directories), we get >>> faster start-up. >> >> That all looks good - nice cleanup. And we should look at using the >> new_instance in more places if appropriate. >> >>> 2. I also took the time to removed a bunch of "// One oop argument" >>> comments which >>> ??? probably meant something to the person who wrote it, but seems >>> useless to everyone else. >> >> Especially when the coding pattern is odd anyway. I mean why have: >> >> ??? JavaCallArguments args(receiver); >> ??? args.push_oop(arg1); >> ??? args.push_oop(arg2); >> >> instead of >> >> ??? JavaCallArguments args; >> ??? args.push_oop(receiver); >> ??? args.push_oop(arg1); >> ??? args.push_oop(arg2); >> >> or >> >> ??? JavaCallArguments args(receiver, oop1, oop2); >> >> ?? (Not something I expect you to change of course!) >> >>> >>> 3. As Calvin suggested, I removed the File_klass and also >>> ParseUtil_klass from >>> ??? the system dictionary since they are not used anywhere. This >>> hopefully improves start-up >>> ??? by a little bit, since these 2 classes are no longer resolved at >>> start-up. >> >> Ok. >> >>> >>> (BTW, I changed the RFR subject line from XS to S due to the extend >>> of change :-) >> >> Please don't do that as it breaks the flow when sorting/threading by >> subject! >> >> Modulo Colleen's query on the lock initialization placement this all >> seems fine to me. >> >> Thanks, >> David >> >>> Thanks >>> - Ioi >>> >>> >>> >>> >>> On 5/15/18 2:00 PM, coleen.phillimore at oracle.com wrote: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >>>> >>>> >>>> This looks good.? This is a pattern that's used in other places, and >>>> it would be better to not initialize these at startup in thread.cpp. >>>> >>>> Coleen >>>> >>>> On 5/15/18 2:07 AM, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>>>> >>>>> >>>>> Summary: >>>>> >>>>> 1. Removed the forced initialization of a few classes used by >>>>> AppCDS at JVM start-up. >>>>> ?? Instead, initialize these class on demand by calling >>>>> InstanceKlass::initialize, which >>>>> ?? is a quick no-op if the class is already initialized. >>>>> >>>>> 2. The only initialization left is that of a global lock. So I >>>>> renamed the function >>>>> ?? to SystemDictionaryShared::initialize_locks(). >>>>> >>>>> 3. I moved the call of this function from >>>>> SystemDictionary::compute_java_loaders() to >>>>> SystemDictionary::initialize() where it seems to fit. >>>>> >>>>> Testing with hs-tiers 1 and 2. >>>>> >>>>> Thanks >>>>> - Ioi >>>> >>> > From thomas.stuefe at gmail.com Wed May 16 06:04:34 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 16 May 2018 08:04:34 +0200 Subject: RFR (XS) 8202014: Possible to receive signal before signal semaphore created In-Reply-To: <3e24a240-9e6b-65a0-64ca-c81db396f6cb@oracle.com> References: <3d3ea98c-60c9-71de-f0c7-e156207b24ca@oracle.com> <3e24a240-9e6b-65a0-64ca-c81db396f6cb@oracle.com> Message-ID: Hi, I agree with David. Signal handlers are installed in os::init_2(), but the semaphore sometime later in the (imho misleadingly named) void os::signal_init(). This looks like an initialization error. As a side note, can we make the "void signalHandler(int sig, siginfo_t* info, void* uc)" functions in os_xxx.cpp static, since they are only file local marshalling points? Kind Regards, Thomas On Wed, May 16, 2018 at 4:40 AM, David Holmes wrote: > Hi Coleen, > > On 16/05/2018 11:23 AM, coleen.phillimore at oracle.com wrote: >> >> Summary: Don't assert for null semaphore because it can be null before >> initialization is complete. > > > Sorry I don't agree with this. Surely we should not install our signal > handler until such time as everything is properly initialized? Or the > handler itself should have some kind of initialization check to guide how it > responds. > > Thanks, > David > > >> Tested this with a shell script, thanks Robbin, and mach5 tier1,2 all >> Oracle platforms. AIX doesn't use the semaphore class so doesn't have this >> bug. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8202014.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8202014 >> >> Thanks, >> Coleen From per.liden at oracle.com Wed May 16 06:58:36 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 16 May 2018 08:58:36 +0200 Subject: RFR: 8202978: Incorrect tmp register passed to MacroAssembler::load_mirror() In-Reply-To: <993a387a-574d-43f2-3c61-47388a1bc459@oracle.com> References: <70f41abc-9f3f-98fa-a66d-76f931640d02@oracle.com> <506c8615-5d79-be76-5e09-9331ec50465e@oracle.com> <993a387a-574d-43f2-3c61-47388a1bc459@oracle.com> Message-ID: <2470a80e-e709-5f56-991f-efc79bc40b5c@oracle.com> On 05/15/2018 10:25 PM, coleen.phillimore at oracle.com wrote: > > Hi Per, Hi Coleen, > > It's calling a static native function so might not be totally > performance neutral. > It looks like rax is scratch there.? Can you try that??? Sorry I didn't > figure out that r11 is rscratch2. I looks like you're right, rax seems to be used as scratch here. I'll run some tests with that and get back. Thanks! /Per > > Thanks, > Coleen > > On 5/11/18 8:48 AM, Per Liden wrote: >> On 05/11/2018 02:32 PM, Vladimir Kozlov wrote: >>> Looks good. >> >> Thanks Vladimir! >> >>> Is this new code? Should we backport it otherwise? >> >> Yes, this is new code, i.e. it's not in JDK 10 or older. Also, it just >> so happens that this bug doesn't affect the existing GC, only GCs >> which do load barriers (like ZGC), which is why it hasn't been caught >> before. >> >> /Per >> >>> >>> Thanks, >>> Vladimir K >>> >>> On 5/11/18 1:31 AM, Per Liden wrote: >>>> On x86, MacroAssembler::load_mirror() defaults to using rscratch2 as >>>> tmp register, unless something else is specified. In >>>> TemplateInterpreterGenerator::generate_native_entry() we call >>>> load_mirror(), but the rscratch2 register is already in use in this >>>> context, which obviously leads to problems. This is not a >>>> performance critical path, so we should just pass noreg as the tmp >>>> register. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202978 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8202978/webrev.0 >>>> >>>> Testing: passes hs-tier{1,2} >>>> >>>> /Per > From per.liden at oracle.com Wed May 16 08:22:31 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 16 May 2018 10:22:31 +0200 Subject: RFR: 8202978: Incorrect tmp register passed to MacroAssembler::load_mirror() In-Reply-To: <2470a80e-e709-5f56-991f-efc79bc40b5c@oracle.com> References: <70f41abc-9f3f-98fa-a66d-76f931640d02@oracle.com> <506c8615-5d79-be76-5e09-9331ec50465e@oracle.com> <993a387a-574d-43f2-3c61-47388a1bc459@oracle.com> <2470a80e-e709-5f56-991f-efc79bc40b5c@oracle.com> Message-ID: On 05/16/2018 08:58 AM, Per Liden wrote: > On 05/15/2018 10:25 PM, coleen.phillimore at oracle.com wrote: >> >> Hi Per, > > Hi Coleen, > >> >> It's calling a static native function so might not be totally >> performance neutral. >> It looks like rax is scratch there.? Can you try that??? Sorry I >> didn't figure out that r11 is rscratch2. > > I looks like you're right, rax seems to be used as scratch here. I'll > run some tests with that and get back. Ok, changed to rax and ran it through testing without finding any problems. Updated webrev: http://cr.openjdk.java.net/~pliden/8202978/webrev.1 /Per > > Thanks! > > /Per > >> >> Thanks, >> Coleen >> >> On 5/11/18 8:48 AM, Per Liden wrote: >>> On 05/11/2018 02:32 PM, Vladimir Kozlov wrote: >>>> Looks good. >>> >>> Thanks Vladimir! >>> >>>> Is this new code? Should we backport it otherwise? >>> >>> Yes, this is new code, i.e. it's not in JDK 10 or older. Also, it >>> just so happens that this bug doesn't affect the existing GC, only >>> GCs which do load barriers (like ZGC), which is why it hasn't been >>> caught before. >>> >>> /Per >>> >>>> >>>> Thanks, >>>> Vladimir K >>>> >>>> On 5/11/18 1:31 AM, Per Liden wrote: >>>>> On x86, MacroAssembler::load_mirror() defaults to using rscratch2 >>>>> as tmp register, unless something else is specified. In >>>>> TemplateInterpreterGenerator::generate_native_entry() we call >>>>> load_mirror(), but the rscratch2 register is already in use in this >>>>> context, which obviously leads to problems. This is not a >>>>> performance critical path, so we should just pass noreg as the tmp >>>>> register. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202978 >>>>> Webrev: http://cr.openjdk.java.net/~pliden/8202978/webrev.0 >>>>> >>>>> Testing: passes hs-tier{1,2} >>>>> >>>>> /Per >> From martin.doerr at sap.com Wed May 16 12:30:22 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 16 May 2018 12:30:22 +0000 Subject: RFR(M): 8202713: Create a MacroAssembler::access_load/store_at wrapper for S390 and PPC In-Reply-To: <63f242b9552844508c77017d240d42b1@sap.com> References: <63f242b9552844508c77017d240d42b1@sap.com> Message-ID: Hi G?tz, > barrierSetAssembler_ppc.cpp > > The name of the label is_null is misleading. It sounds > as the name of a boolean variable. Maybe "L_handle_null"? > And please use explicit compare for pointers: if (is_null == NULL) { > Done. > Shouldn't there be an assertion that no other decorators > are set than those handled here? Like all the memory ordering > ones? > Done. > macroAssembler_ppc.hpp: > > This header uses DecoratorSet thus should #include accessDecorators.hpp > Done. > access_load/store_at() > This is a misleading name for this function. I would think this > should go to nativeInst_ppc.cpp :) Why would you access a store?? > But this needs to be changed on all platforms. So not > scope of this review. > The function is not really needed. Make it private, or inline > it into load/store_heap_oop. > Made private. > load_heap_oop_not_null: > Please remove this on ppc, too. It's the only platform > that still uses this function. There are only 6 uses. > Please also remove the dead definitions of it on aarch64 and x86, > as well as the comment in s390.ad mentioning it. (not sure about > aarch64 as you can't easily test this). > Changed PPC64 and s390 code to use the full featured version only. Removed old functions. I think it would be kind of unfair to change x86/aarch64 in this change because x86 folks will probably not read this review. > Please extend the comment above load_heap_oop. It gained > new functionality, and after removing the _not_null functions > along with their documentation the current comment is pointless. > Mention what the label is good for. Rename the label as above. > Done. > macroAssembler_ppc.inline.hpp > > Please rename the boolean variable as_raw to without_gc_barrier. > Actually, the DecoratorSet name AS_RAW is misleading, there is still > encode/decode etc. But this is not in the scope of this change. > Not done because I'd like to keep PPC64 and s390 implementation consistent to the other platforms. A cleanup change for all platforms could be done in a separate change without s390 or PPC in its title. > stubGenerator_ppc.cpp > Remove comment // Load the oop. Makes the line too long, and > not too much information gained :) > Done. New webrev: http://cr.openjdk.java.net/~mdoerr/8202713_ppc64_s390_masm_access/webrev.01/ Best regards, Martin From goetz.lindenmaier at sap.com Wed May 16 13:04:27 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 16 May 2018 13:04:27 +0000 Subject: RFR(M): 8202713: Create a MacroAssembler::access_load/store_at wrapper for S390 and PPC In-Reply-To: References: <63f242b9552844508c77017d240d42b1@sap.com> Message-ID: <8bc21bb49b194ab2aa779c742f2e6e13@sap.com> Hi Martin, thanks a lot for your adaptions, looks good now. Best regards, Goetz. > -----Original Message----- > From: Doerr, Martin > Sent: Mittwoch, 16. Mai 2018 14:30 > To: Lindenmaier, Goetz ; Erik ?sterlund > ; hotspot-runtime-dev at openjdk.java.net > Subject: RE: RFR(M): 8202713: Create a > MacroAssembler::access_load/store_at wrapper for S390 and PPC > > Hi G?tz, > > > barrierSetAssembler_ppc.cpp > > > > The name of the label is_null is misleading. It sounds > > as the name of a boolean variable. Maybe "L_handle_null"? > > And please use explicit compare for pointers: if (is_null == NULL) { > > > Done. > > > Shouldn't there be an assertion that no other decorators > > are set than those handled here? Like all the memory ordering > > ones? > > > Done. > > > macroAssembler_ppc.hpp: > > > > This header uses DecoratorSet thus should #include accessDecorators.hpp > > > Done. > > > access_load/store_at() > > This is a misleading name for this function. I would think this > > should go to nativeInst_ppc.cpp :) Why would you access a store?? > > But this needs to be changed on all platforms. So not > > scope of this review. > > The function is not really needed. Make it private, or inline > > it into load/store_heap_oop. > > > Made private. > > > load_heap_oop_not_null: > > Please remove this on ppc, too. It's the only platform > > that still uses this function. There are only 6 uses. > > Please also remove the dead definitions of it on aarch64 and x86, > > as well as the comment in s390.ad mentioning it. (not sure about > > aarch64 as you can't easily test this). > > > Changed PPC64 and s390 code to use the full featured version only. > Removed old functions. > I think it would be kind of unfair to change x86/aarch64 in this change > because x86 folks will probably not read this review. > > > Please extend the comment above load_heap_oop. It gained > > new functionality, and after removing the _not_null functions > > along with their documentation the current comment is pointless. > > Mention what the label is good for. Rename the label as above. > > > Done. > > > macroAssembler_ppc.inline.hpp > > > > Please rename the boolean variable as_raw to without_gc_barrier. > > Actually, the DecoratorSet name AS_RAW is misleading, there is still > > encode/decode etc. But this is not in the scope of this change. > > > Not done because I'd like to keep PPC64 and s390 implementation consistent > to the other platforms. A cleanup change for all platforms could be done in a > separate change without s390 or PPC in its title. > > > stubGenerator_ppc.cpp > > Remove comment // Load the oop. Makes the line too long, and > > not too much information gained :) > > > Done. > > New webrev: > http://cr.openjdk.java.net/~mdoerr/8202713_ppc64_s390_masm_access/w > ebrev.01/ > > Best regards, > Martin From matthias.baesken at sap.com Wed May 16 13:12:56 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 16 May 2018 13:12:56 +0000 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: References: Message-ID: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> Ping : could I get a second review ? Bug : https://bugs.openjdk.java.net/browse/JDK-8202427 Change : http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > We could also just print the MB value , let's see what other think. > Another option might be to have a flexible output (kB for smaller memory > values , MB (or GB) for larger ) ? Martin suggested to just print the MB values, what do you think ? Thanks, Matthias > -----Original Message----- > From: Baesken, Matthias > Sent: Mittwoch, 2. Mai 2018 13:00 > To: Doerr, Martin ; 'hotspot- > dev at openjdk.java.net' ; hotspot- > runtime-dev at openjdk.java.net > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows > > Hi Martin, thanks for your input . > I add hotspot-runtime-dev . > > > > > I wonder if we really need the sizes in kB in addition to MB. Maybe other > > reviewers would like to comment on this, too. > > > > We could also just print the MB value , let's see what other think. > Another option might be to have a flexible output (kB for smaller memory > values , MB (or GB) for larger ) ? > > Best regards, Matthias > > > > -----Original Message----- > > From: Doerr, Martin > > Sent: Mittwoch, 2. Mai 2018 12:53 > > To: Baesken, Matthias ; 'hotspot- > > dev at openjdk.java.net' > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows > > > > Hi Matthias, > > > > looks like a nice enhancement. We can get substantially more information. > > > > I wonder if we really need the sizes in kB in addition to MB. Maybe other > > reviewers would like to comment on this, too. > > > > We should have line breaks. > > > > Best regards, > > Martin > > > > > > -----Original Message----- > > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > > Behalf Of Baesken, Matthias > > Sent: Montag, 30. April 2018 16:53 > > To: 'hotspot-dev at openjdk.java.net' > > Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows > > > > On Windows, > > the os::print_memory_info misses a few memory-related infos that might > > be interesting : > > - current and peak WorkingSet size (= physical memory assigned to the > > process) > > - current and peak commit charge (also known as "private bytes" in > Windows > > tools) > > - on 32bit Windows : > > user-mode portion/free user mode portion of virtual address-space > > (Total/AvailVirtual) because it shows how close do we get to the 2-4 GB per > > process border. > > > > > > - the current naming of "swap/free-swap" memory is a bit misleading; > > in the Windows world swap is a special page file used for UWP apps. > > (see Windows Internals : "The swap file" (chapter 5 Memory > management) > > Windows 8 added another page file called a swap file. It is ... exclusively > used > > for UWP (Universal Windows Platform) apps. > > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the UWP > > related values, it is about page file sizes > > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just virtual > > memory might be more appropriate). > > > > > > https://msdn.microsoft.com/de- > > de/library/windows/desktop/aa366770(v=vs.85).aspx > > > > documents it in the following way: > > ullTotalPageFile: > > The current committed memory limit for the system or the current process, > > whichever is smaller,... > > > > ullAvailPageFile : > > The maximum amount of memory the current process can commit, in > bytes. > > This value is equal to or smaller than the system-wide available commit > value > > > > > > > > Aditionally I suggest having output of the various memory-values in M > > (megabyte) as well , the k (kilobyte) output sometimes gives very high and > > unreadable numbers). > > > > > > Could you please review my change ? > > > > > > Bug : > > > > https://bugs.openjdk.java.net/browse/JDK-8202427 > > > > > > Change : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > > > > > > Thanks, Matthias > > From goetz.lindenmaier at sap.com Wed May 16 13:30:22 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 16 May 2018 13:30:22 +0000 Subject: RFR(M): 8199940: Print more information about class loaders in IllegalAccessErrors. Message-ID: <515b3476674f45eabcbb7fac5044a459@sap.com> Hi, I finally prepared the webrev for this change. I simplified the messages wrt. to my first intent that was found too verbose. The IllegalAccessErrors in classFileParser report more verbose information about the class loaders. As I understand, the class loaders are the reason for the error here, so I think information about them is useful. See also the tests 1-3. I added reporting whether the class is abstract. In linkResolver, I just switch to class_loader_and_module_name() to report a more verbose class name as Lois requested. I removed mentioning the resolved class in the method case. I report the modifiers of methods/fields. But as I understand, even here the class loader can be the reason of the Error, see tests 6-8. Please review. I'm happy to improve the messages further :) http://cr.openjdk.java.net/~goetz/wr18/8199940-exMsg-IllegalAccess/01/ Best regards, Goetz. From coleen.phillimore at oracle.com Wed May 16 13:34:51 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 16 May 2018 09:34:51 -0400 Subject: RFR (XS) 8202014: Possible to receive signal before signal semaphore created In-Reply-To: <3e24a240-9e6b-65a0-64ca-c81db396f6cb@oracle.com> References: <3d3ea98c-60c9-71de-f0c7-e156207b24ca@oracle.com> <3e24a240-9e6b-65a0-64ca-c81db396f6cb@oracle.com> Message-ID: <8eee838e-d472-da80-703d-d25656ae9c3e@oracle.com> On 5/15/18 10:40 PM, David Holmes wrote: > Hi Coleen, > > On 16/05/2018 11:23 AM, coleen.phillimore at oracle.com wrote: >> Summary: Don't assert for null semaphore because it can be null >> before initialization is complete. > > Sorry I don't agree with this. Surely we should not install our signal > handler until such time as everything is properly initialized? Or the > handler itself should have some kind of initialization check to guide > how it responds. Ok.? I'll try to move the initialization then. Coleen > > Thanks, > David > >> Tested this with a shell script, thanks Robbin, and mach5 tier1,2 all >> Oracle platforms.? AIX doesn't use the semaphore class so doesn't >> have this bug. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8202014.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8202014 >> >> Thanks, >> Coleen From coleen.phillimore at oracle.com Wed May 16 13:35:46 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 16 May 2018 09:35:46 -0400 Subject: RFR (XS) 8202014: Possible to receive signal before signal semaphore created In-Reply-To: References: <3d3ea98c-60c9-71de-f0c7-e156207b24ca@oracle.com> <3e24a240-9e6b-65a0-64ca-c81db396f6cb@oracle.com> Message-ID: <7f1598a8-dad8-249f-4c34-512afd6afc48@oracle.com> On 5/16/18 2:04 AM, Thomas St?fe wrote: > Hi, > > I agree with David. Signal handlers are installed in os::init_2(), but > the semaphore sometime later in the (imho misleadingly named) void > os::signal_init(). This looks like an initialization error. > > As a side note, can we make the "void signalHandler(int sig, > siginfo_t* info, void* uc)" functions in os_xxx.cpp static, since they > are only file local marshalling points? Sure, I can do that. Coleen > > Kind Regards, Thomas > > > > > On Wed, May 16, 2018 at 4:40 AM, David Holmes wrote: >> Hi Coleen, >> >> On 16/05/2018 11:23 AM, coleen.phillimore at oracle.com wrote: >>> Summary: Don't assert for null semaphore because it can be null before >>> initialization is complete. >> >> Sorry I don't agree with this. Surely we should not install our signal >> handler until such time as everything is properly initialized? Or the >> handler itself should have some kind of initialization check to guide how it >> responds. >> >> Thanks, >> David >> >> >>> Tested this with a shell script, thanks Robbin, and mach5 tier1,2 all >>> Oracle platforms. AIX doesn't use the semaphore class so doesn't have this >>> bug. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8202014.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8202014 >>> >>> Thanks, >>> Coleen From coleen.phillimore at oracle.com Wed May 16 13:38:16 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 16 May 2018 09:38:16 -0400 Subject: RFR: 8202978: Incorrect tmp register passed to MacroAssembler::load_mirror() In-Reply-To: References: <70f41abc-9f3f-98fa-a66d-76f931640d02@oracle.com> <506c8615-5d79-be76-5e09-9331ec50465e@oracle.com> <993a387a-574d-43f2-3c61-47388a1bc459@oracle.com> <2470a80e-e709-5f56-991f-efc79bc40b5c@oracle.com> Message-ID: <0d85792f-c8ba-100c-9780-761f6b12ea8b@oracle.com> Yes, this looks good.? I'm glad that works out. thanks, Coleen On 5/16/18 4:22 AM, Per Liden wrote: > On 05/16/2018 08:58 AM, Per Liden wrote: >> On 05/15/2018 10:25 PM, coleen.phillimore at oracle.com wrote: >>> >>> Hi Per, >> >> Hi Coleen, >> >>> >>> It's calling a static native function so might not be totally >>> performance neutral. >>> It looks like rax is scratch there.? Can you try that??? Sorry I >>> didn't figure out that r11 is rscratch2. >> >> I looks like you're right, rax seems to be used as scratch here. I'll >> run some tests with that and get back. > > Ok, changed to rax and ran it through testing without finding any > problems. > > Updated webrev: http://cr.openjdk.java.net/~pliden/8202978/webrev.1 > > /Per > >> >> Thanks! >> >> /Per >> >>> >>> Thanks, >>> Coleen >>> >>> On 5/11/18 8:48 AM, Per Liden wrote: >>>> On 05/11/2018 02:32 PM, Vladimir Kozlov wrote: >>>>> Looks good. >>>> >>>> Thanks Vladimir! >>>> >>>>> Is this new code? Should we backport it otherwise? >>>> >>>> Yes, this is new code, i.e. it's not in JDK 10 or older. Also, it >>>> just so happens that this bug doesn't affect the existing GC, only >>>> GCs which do load barriers (like ZGC), which is why it hasn't been >>>> caught before. >>>> >>>> /Per >>>> >>>>> >>>>> Thanks, >>>>> Vladimir K >>>>> >>>>> On 5/11/18 1:31 AM, Per Liden wrote: >>>>>> On x86, MacroAssembler::load_mirror() defaults to using rscratch2 >>>>>> as tmp register, unless something else is specified. In >>>>>> TemplateInterpreterGenerator::generate_native_entry() we call >>>>>> load_mirror(), but the rscratch2 register is already in use in >>>>>> this context, which obviously leads to problems. This is not a >>>>>> performance critical path, so we should just pass noreg as the >>>>>> tmp register. >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202978 >>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8202978/webrev.0 >>>>>> >>>>>> Testing: passes hs-tier{1,2} >>>>>> >>>>>> /Per >>> From goetz.lindenmaier at sap.com Wed May 16 13:48:48 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 16 May 2018 13:48:48 +0000 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> References: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> Message-ID: Hi Matthias, I appreciate the additional information in the hs_err file. Could you please add an example output (of the final version) to the bug description? As Martin, I would like more line feeds, both in the code and the output. Currently, the output in the hs_err file looks like this (where the first line is too long): Memory: 4k page, system-wide physical 16776692k [16383M] (8795032k [8588M] free), TotalPageFile size 49949460k [48778M] (AvailPageFile size 38942152k [38029M]), user-mode portion of virtual address-space 2097024k [2047M] (6940k [6M] free) current process WorkingSet (physical memory assigned to process): 991804k [968M], peak: 991804k [968M] current process commit charge ("private bytes"): 1523632k [1487M], peak: 1523632k [1487M] I also think that one number is enough. Adaptive numbers would be great. I know about src/hotspot/share/memory/metaspace/metaspaceCommon.hpp: print_human_readable_size() for this, but this seems a bit hidden in metaspace. I don't like usage of _M_IX86. Common in this file seems #ifndef _WIN64 for 32-bit windows dependent code. But don't care here. Best regards, Goetz. > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > Behalf Of Baesken, Matthias > Sent: Mittwoch, 16. Mai 2018 15:13 > To: Doerr, Martin ; 'hotspot- > dev at openjdk.java.net' ; hotspot- > runtime-dev at openjdk.java.net > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows > > Ping : could I get a second review ? > > Bug : > https://bugs.openjdk.java.net/browse/JDK-8202427 > Change : > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > > > > We could also just print the MB value , let's see what other think. > > Another option might be to have a flexible output (kB for smaller > memory > > values , MB (or GB) for larger ) ? > > Martin suggested to just print the MB values, what do you think ? > > > Thanks, Matthias > > > > -----Original Message----- > > From: Baesken, Matthias > > Sent: Mittwoch, 2. Mai 2018 13:00 > > To: Doerr, Martin ; 'hotspot- > > dev at openjdk.java.net' ; hotspot- > > runtime-dev at openjdk.java.net > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows > > > > Hi Martin, thanks for your input . > > I add hotspot-runtime-dev . > > > > > > > > I wonder if we really need the sizes in kB in addition to MB. Maybe other > > > reviewers would like to comment on this, too. > > > > > > > We could also just print the MB value , let's see what other think. > > Another option might be to have a flexible output (kB for smaller > memory > > values , MB (or GB) for larger ) ? > > > > Best regards, Matthias > > > > > > > -----Original Message----- > > > From: Doerr, Martin > > > Sent: Mittwoch, 2. Mai 2018 12:53 > > > To: Baesken, Matthias ; 'hotspot- > > > dev at openjdk.java.net' > > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows > > > > > > Hi Matthias, > > > > > > looks like a nice enhancement. We can get substantially more > information. > > > > > > I wonder if we really need the sizes in kB in addition to MB. Maybe other > > > reviewers would like to comment on this, too. > > > > > > We should have line breaks. > > > > > > Best regards, > > > Martin > > > > > > > > > -----Original Message----- > > > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > > > Behalf Of Baesken, Matthias > > > Sent: Montag, 30. April 2018 16:53 > > > To: 'hotspot-dev at openjdk.java.net' > > > Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows > > > > > > On Windows, > > > the os::print_memory_info misses a few memory-related infos that > might > > > be interesting : > > > - current and peak WorkingSet size (= physical memory assigned to the > > > process) > > > - current and peak commit charge (also known as "private bytes" in > > Windows > > > tools) > > > - on 32bit Windows : > > > user-mode portion/free user mode portion of virtual address-space > > > (Total/AvailVirtual) because it shows how close do we get to the 2-4 GB > per > > > process border. > > > > > > > > > - the current naming of "swap/free-swap" memory is a bit misleading; > > > in the Windows world swap is a special page file used for UWP apps. > > > (see Windows Internals : "The swap file" (chapter 5 Memory > > management) > > > Windows 8 added another page file called a swap file. It is ... exclusively > > used > > > for UWP (Universal Windows Platform) apps. > > > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the UWP > > > related values, it is about page file sizes > > > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just > virtual > > > memory might be more appropriate). > > > > > > > > > https://msdn.microsoft.com/de- > > > de/library/windows/desktop/aa366770(v=vs.85).aspx > > > > > > documents it in the following way: > > > ullTotalPageFile: > > > The current committed memory limit for the system or the current > process, > > > whichever is smaller,... > > > > > > ullAvailPageFile : > > > The maximum amount of memory the current process can commit, in > > bytes. > > > This value is equal to or smaller than the system-wide available commit > > value > > > > > > > > > > > > Aditionally I suggest having output of the various memory-values in M > > > (megabyte) as well , the k (kilobyte) output sometimes gives very high > and > > > unreadable numbers). > > > > > > > > > Could you please review my change ? > > > > > > > > > Bug : > > > > > > https://bugs.openjdk.java.net/browse/JDK-8202427 > > > > > > > > > Change : > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > > > > > > > > > Thanks, Matthias > > > From coleen.phillimore at oracle.com Wed May 16 14:00:07 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 16 May 2018 10:00:07 -0400 Subject: RFR(S) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: References: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> <6d89d9c6-47ef-75fd-5aed-cfea6389d922@oracle.com> Message-ID: <4471fb3c-9d98-fce2-3898-1329d548f99e@oracle.com> This looks great! Thank you! Coleen On 5/16/18 12:06 AM, Ioi Lam wrote: > I've updated the webrev: > > http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v03/ > > > 1. Moved SharedDictionary_lock to mutexLocker.cpp. > > 2. There's no more need for SystemDictionaryShared::initialize. Removed. > > 3. Renamed to JavaCalls::construct_new_instance per Coleen. > > If this goes in, I'll file a separate RFE to convert some other code > to use JavaCalls::construct_new_instance. > > Thanks > > - Ioi > > > > On 5/15/18 6:53 PM, David Holmes wrote: >> On 16/05/2018 7:47 AM, Ioi Lam wrote: >>> I've updated the webrev: >>> >>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/ >>> >>> >>> 1. Added JavaCalls::new_instance so we can avoid all the boiler >>> plate code for allocating >>> ??? the instance andinvoking the constructor. >>> >>> JavaCalls::new_instance calls InstanceKlass->initialize. This is >>> just a quick op after >>> ??? the class is already initialized. Also, JavaCalls::call_static >>> also internally call >>> ??? into InstanceKlass->initialize, so I am just following the >>> existing pattern as Coleen >>> ??? mentioned below. >>> >>> ??? Doing the initialization on demand also means for cases where >>> JAR manifest is not used >>> ??? (all code is loaded from the system image or directories), we >>> get faster start-up. >> >> That all looks good - nice cleanup. And we should look at using the >> new_instance in more places if appropriate. >> >>> 2. I also took the time to removed a bunch of "// One oop argument" >>> comments which >>> ??? probably meant something to the person who wrote it, but seems >>> useless to everyone else. >> >> Especially when the coding pattern is odd anyway. I mean why have: >> >> ??? JavaCallArguments args(receiver); >> ??? args.push_oop(arg1); >> ??? args.push_oop(arg2); >> >> instead of >> >> ??? JavaCallArguments args; >> ??? args.push_oop(receiver); >> ??? args.push_oop(arg1); >> ??? args.push_oop(arg2); >> >> or >> >> ??? JavaCallArguments args(receiver, oop1, oop2); >> >> ?? (Not something I expect you to change of course!) >> >>> >>> 3. As Calvin suggested, I removed the File_klass and also >>> ParseUtil_klass from >>> ??? the system dictionary since they are not used anywhere. This >>> hopefully improves start-up >>> ??? by a little bit, since these 2 classes are no longer resolved at >>> start-up. >> >> Ok. >> >>> >>> (BTW, I changed the RFR subject line from XS to S due to the extend >>> of change :-) >> >> Please don't do that as it breaks the flow when sorting/threading by >> subject! >> >> Modulo Colleen's query on the lock initialization placement this all >> seems fine to me. >> >> Thanks, >> David >> >>> Thanks >>> - Ioi >>> >>> >>> >>> >>> On 5/15/18 2:00 PM, coleen.phillimore at oracle.com wrote: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >>>> >>>> >>>> This looks good.? This is a pattern that's used in other places, >>>> and it would be better to not initialize these at startup in >>>> thread.cpp. >>>> >>>> Coleen >>>> >>>> On 5/15/18 2:07 AM, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>>>> >>>>> >>>>> Summary: >>>>> >>>>> 1. Removed the forced initialization of a few classes used by >>>>> AppCDS at JVM start-up. >>>>> ?? Instead, initialize these class on demand by calling >>>>> InstanceKlass::initialize, which >>>>> ?? is a quick no-op if the class is already initialized. >>>>> >>>>> 2. The only initialization left is that of a global lock. So I >>>>> renamed the function >>>>> ?? to SystemDictionaryShared::initialize_locks(). >>>>> >>>>> 3. I moved the call of this function from >>>>> SystemDictionary::compute_java_loaders() to >>>>> SystemDictionary::initialize() where it seems to fit. >>>>> >>>>> Testing with hs-tiers 1 and 2. >>>>> >>>>> Thanks >>>>> - Ioi >>>> >>> > From thomas.stuefe at gmail.com Wed May 16 14:35:21 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 16 May 2018 16:35:21 +0200 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: References: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> Message-ID: On Wed, May 16, 2018 at 3:48 PM, Lindenmaier, Goetz wrote: > Hi Matthias, > > I appreciate the additional information in the hs_err file. > Could you please add an example output (of the final version) to the bug description? > > As Martin, I would like more line feeds, both in the code and the output. > Currently, the output in the hs_err file looks like this (where the first line is too long): > Memory: 4k page, system-wide physical 16776692k [16383M] (8795032k [8588M] free), TotalPageFile size 49949460k [48778M] (AvailPageFile size 38942152k [38029M]), user-mode portion of virtual address-space 2097024k [2047M] (6940k [6M] free) > current process WorkingSet (physical memory assigned to process): 991804k [968M], peak: 991804k [968M] > > current process commit charge ("private bytes"): 1523632k [1487M], peak: 1523632k [1487M] > > I also think that one number is enough. Adaptive numbers would be great. > I know about > src/hotspot/share/memory/metaspace/metaspaceCommon.hpp: print_human_readable_size() > for this, but this seems a bit hidden in metaspace. > Guys, lets not worry about human readable numbers for now. We can move print_human_readable_size() out from metaspace to something generic and use it to improve this code (and others) in a later patch. That was my original intention when adding print_human_readable_size(). The patch is good. Thanks for doing this. ..Thoams > I don't like usage of _M_IX86. Common in this file seems #ifndef _WIN64 for 32-bit windows > dependent code. But don't care here. > > Best regards, > Goetz. > > >> -----Original Message----- >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On >> Behalf Of Baesken, Matthias >> Sent: Mittwoch, 16. Mai 2018 15:13 >> To: Doerr, Martin ; 'hotspot- >> dev at openjdk.java.net' ; hotspot- >> runtime-dev at openjdk.java.net >> Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows >> >> Ping : could I get a second review ? >> >> Bug : >> https://bugs.openjdk.java.net/browse/JDK-8202427 >> Change : >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ >> >> >> > We could also just print the MB value , let's see what other think. >> > Another option might be to have a flexible output (kB for smaller >> memory >> > values , MB (or GB) for larger ) ? >> >> Martin suggested to just print the MB values, what do you think ? >> >> >> Thanks, Matthias >> >> >> > -----Original Message----- >> > From: Baesken, Matthias >> > Sent: Mittwoch, 2. Mai 2018 13:00 >> > To: Doerr, Martin ; 'hotspot- >> > dev at openjdk.java.net' ; hotspot- >> > runtime-dev at openjdk.java.net >> > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows >> > >> > Hi Martin, thanks for your input . >> > I add hotspot-runtime-dev . >> > >> > > >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe other >> > > reviewers would like to comment on this, too. >> > > >> > >> > We could also just print the MB value , let's see what other think. >> > Another option might be to have a flexible output (kB for smaller >> memory >> > values , MB (or GB) for larger ) ? >> > >> > Best regards, Matthias >> > >> > >> > > -----Original Message----- >> > > From: Doerr, Martin >> > > Sent: Mittwoch, 2. Mai 2018 12:53 >> > > To: Baesken, Matthias ; 'hotspot- >> > > dev at openjdk.java.net' >> > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows >> > > >> > > Hi Matthias, >> > > >> > > looks like a nice enhancement. We can get substantially more >> information. >> > > >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe other >> > > reviewers would like to comment on this, too. >> > > >> > > We should have line breaks. >> > > >> > > Best regards, >> > > Martin >> > > >> > > >> > > -----Original Message----- >> > > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On >> > > Behalf Of Baesken, Matthias >> > > Sent: Montag, 30. April 2018 16:53 >> > > To: 'hotspot-dev at openjdk.java.net' >> > > Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows >> > > >> > > On Windows, >> > > the os::print_memory_info misses a few memory-related infos that >> might >> > > be interesting : >> > > - current and peak WorkingSet size (= physical memory assigned to the >> > > process) >> > > - current and peak commit charge (also known as "private bytes" in >> > Windows >> > > tools) >> > > - on 32bit Windows : >> > > user-mode portion/free user mode portion of virtual address-space >> > > (Total/AvailVirtual) because it shows how close do we get to the 2-4 GB >> per >> > > process border. >> > > >> > > >> > > - the current naming of "swap/free-swap" memory is a bit misleading; >> > > in the Windows world swap is a special page file used for UWP apps. >> > > (see Windows Internals : "The swap file" (chapter 5 Memory >> > management) >> > > Windows 8 added another page file called a swap file. It is ... exclusively >> > used >> > > for UWP (Universal Windows Platform) apps. >> > > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the UWP >> > > related values, it is about page file sizes >> > > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just >> virtual >> > > memory might be more appropriate). >> > > >> > > >> > > https://msdn.microsoft.com/de- >> > > de/library/windows/desktop/aa366770(v=vs.85).aspx >> > > >> > > documents it in the following way: >> > > ullTotalPageFile: >> > > The current committed memory limit for the system or the current >> process, >> > > whichever is smaller,... >> > > >> > > ullAvailPageFile : >> > > The maximum amount of memory the current process can commit, in >> > bytes. >> > > This value is equal to or smaller than the system-wide available commit >> > value >> > > >> > > >> > > >> > > Aditionally I suggest having output of the various memory-values in M >> > > (megabyte) as well , the k (kilobyte) output sometimes gives very high >> and >> > > unreadable numbers). >> > > >> > > >> > > Could you please review my change ? >> > > >> > > >> > > Bug : >> > > >> > > https://bugs.openjdk.java.net/browse/JDK-8202427 >> > > >> > > >> > > Change : >> > > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ >> > > >> > > >> > > Thanks, Matthias >> > > > From calvin.cheung at oracle.com Wed May 16 15:21:09 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 16 May 2018 08:21:09 -0700 Subject: RFR(S) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: References: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> <6d89d9c6-47ef-75fd-5aed-cfea6389d922@oracle.com> Message-ID: <5AFC4C65.4020702@oracle.com> Looks good. thanks, Calvin On 5/15/18, 9:06 PM, Ioi Lam wrote: > I've updated the webrev: > > http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v03/ > > > 1. Moved SharedDictionary_lock to mutexLocker.cpp. > > 2. There's no more need for SystemDictionaryShared::initialize. Removed. > > 3. Renamed to JavaCalls::construct_new_instance per Coleen. > > If this goes in, I'll file a separate RFE to convert some other code > to use JavaCalls::construct_new_instance. > > Thanks > > - Ioi > > > > On 5/15/18 6:53 PM, David Holmes wrote: >> On 16/05/2018 7:47 AM, Ioi Lam wrote: >>> I've updated the webrev: >>> >>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/ >>> >>> >>> 1. Added JavaCalls::new_instance so we can avoid all the boiler >>> plate code for allocating >>> the instance andinvoking the constructor. >>> >>> JavaCalls::new_instance calls InstanceKlass->initialize. This is >>> just a quick op after >>> the class is already initialized. Also, JavaCalls::call_static >>> also internally call >>> into InstanceKlass->initialize, so I am just following the >>> existing pattern as Coleen >>> mentioned below. >>> >>> Doing the initialization on demand also means for cases where >>> JAR manifest is not used >>> (all code is loaded from the system image or directories), we >>> get faster start-up. >> >> That all looks good - nice cleanup. And we should look at using the >> new_instance in more places if appropriate. >> >>> 2. I also took the time to removed a bunch of "// One oop argument" >>> comments which >>> probably meant something to the person who wrote it, but seems >>> useless to everyone else. >> >> Especially when the coding pattern is odd anyway. I mean why have: >> >> JavaCallArguments args(receiver); >> args.push_oop(arg1); >> args.push_oop(arg2); >> >> instead of >> >> JavaCallArguments args; >> args.push_oop(receiver); >> args.push_oop(arg1); >> args.push_oop(arg2); >> >> or >> >> JavaCallArguments args(receiver, oop1, oop2); >> >> ?? (Not something I expect you to change of course!) >> >>> >>> 3. As Calvin suggested, I removed the File_klass and also >>> ParseUtil_klass from >>> the system dictionary since they are not used anywhere. This >>> hopefully improves start-up >>> by a little bit, since these 2 classes are no longer resolved at >>> start-up. >> >> Ok. >> >>> >>> (BTW, I changed the RFR subject line from XS to S due to the extend >>> of change :-) >> >> Please don't do that as it breaks the flow when sorting/threading by >> subject! >> >> Modulo Colleen's query on the lock initialization placement this all >> seems fine to me. >> >> Thanks, >> David >> >>> Thanks >>> - Ioi >>> >>> >>> >>> >>> On 5/15/18 2:00 PM, coleen.phillimore at oracle.com wrote: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >>>> >>>> >>>> This looks good. This is a pattern that's used in other places, >>>> and it would be better to not initialize these at startup in >>>> thread.cpp. >>>> >>>> Coleen >>>> >>>> On 5/15/18 2:07 AM, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>>>> >>>>> >>>>> Summary: >>>>> >>>>> 1. Removed the forced initialization of a few classes used by >>>>> AppCDS at JVM start-up. >>>>> Instead, initialize these class on demand by calling >>>>> InstanceKlass::initialize, which >>>>> is a quick no-op if the class is already initialized. >>>>> >>>>> 2. The only initialization left is that of a global lock. So I >>>>> renamed the function >>>>> to SystemDictionaryShared::initialize_locks(). >>>>> >>>>> 3. I moved the call of this function from >>>>> SystemDictionary::compute_java_loaders() to >>>>> SystemDictionary::initialize() where it seems to fit. >>>>> >>>>> Testing with hs-tiers 1 and 2. >>>>> >>>>> Thanks >>>>> - Ioi >>>> >>> > From adam.farley at uk.ibm.com Wed May 16 15:35:10 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Wed, 16 May 2018 16:35:10 +0100 Subject: RFR: JDK-8190187: C++ code calling JNI_CreateJavaVM can be killed by Java - Final Email Message-ID: Hi All, The code to resolve JDK-8190187 has been added to the bug, in hg_diff.txt. Could a committer please take the fix and: 1) Complete the CSR process for the new JNI Return code. 2) Commit the changes that contain the Return code. And, optionally, 3) Perform 1 and 2 for the jdwp agent changes as well, so it can use this new functionality. Note: the lack of response to this issue over the past few months has been interpreted as a lack of community interest in resolving this defect, so, should this email spark no response, I will not be pursuing the issue further. If and when community interest does resume, hg_diff contains the most up-to-date code. Best Regards Adam Farley Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From jiangli.zhou at oracle.com Wed May 16 15:55:44 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Wed, 16 May 2018 08:55:44 -0700 Subject: RFR(S) 8189140 - SystemDictionaryShared::initialize() should be renamed to be more meaningful In-Reply-To: References: <926d4126-406b-658e-372c-19fdf1e78a49@oracle.com> <6d89d9c6-47ef-75fd-5aed-cfea6389d922@oracle.com> Message-ID: <5B325FDF-F805-4F1A-90EF-BE4D8F21EF12@oracle.com> The cleanup looks really good! Thanks, Jiangli > On May 15, 2018, at 9:06 PM, Ioi Lam wrote: > > I've updated the webrev: > > http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v03/ > > 1. Moved SharedDictionary_lock to mutexLocker.cpp. > > 2. There's no more need for SystemDictionaryShared::initialize. Removed. > > 3. Renamed to JavaCalls::construct_new_instance per Coleen. > > If this goes in, I'll file a separate RFE to convert some other code to use JavaCalls::construct_new_instance. > > Thanks > > - Ioi > > > > On 5/15/18 6:53 PM, David Holmes wrote: >> On 16/05/2018 7:47 AM, Ioi Lam wrote: >>> I've updated the webrev: >>> >>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v02/ >>> >>> 1. Added JavaCalls::new_instance so we can avoid all the boiler plate code for allocating >>> the instance andinvoking the constructor. >>> >>> JavaCalls::new_instance calls InstanceKlass->initialize. This is just a quick op after >>> the class is already initialized. Also, JavaCalls::call_static also internally call >>> into InstanceKlass->initialize, so I am just following the existing pattern as Coleen >>> mentioned below. >>> >>> Doing the initialization on demand also means for cases where JAR manifest is not used >>> (all code is loaded from the system image or directories), we get faster start-up. >> >> That all looks good - nice cleanup. And we should look at using the new_instance in more places if appropriate. >> >>> 2. I also took the time to removed a bunch of "// One oop argument" comments which >>> probably meant something to the person who wrote it, but seems useless to everyone else. >> >> Especially when the coding pattern is odd anyway. I mean why have: >> >> JavaCallArguments args(receiver); >> args.push_oop(arg1); >> args.push_oop(arg2); >> >> instead of >> >> JavaCallArguments args; >> args.push_oop(receiver); >> args.push_oop(arg1); >> args.push_oop(arg2); >> >> or >> >> JavaCallArguments args(receiver, oop1, oop2); >> >> ?? (Not something I expect you to change of course!) >> >>> >>> 3. As Calvin suggested, I removed the File_klass and also ParseUtil_klass from >>> the system dictionary since they are not used anywhere. This hopefully improves start-up >>> by a little bit, since these 2 classes are no longer resolved at start-up. >> >> Ok. >> >>> >>> (BTW, I changed the RFR subject line from XS to S due to the extend of change :-) >> >> Please don't do that as it breaks the flow when sorting/threading by subject! >> >> Modulo Colleen's query on the lock initialization placement this all seems fine to me. >> >> Thanks, >> David >> >>> Thanks >>> - Ioi >>> >>> >>> >>> >>> On 5/15/18 2:00 PM, coleen.phillimore at oracle.com wrote: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >>>> >>>> This looks good. This is a pattern that's used in other places, and it would be better to not initialize these at startup in thread.cpp. >>>> >>>> Coleen >>>> >>>> On 5/15/18 2:07 AM, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8189140 >>>>> http://cr.openjdk.java.net/~iklam/jdk11/8189140-rename-system-dict-shared-initialize.v01/ >>>>> >>>>> Summary: >>>>> >>>>> 1. Removed the forced initialization of a few classes used by AppCDS at JVM start-up. >>>>> Instead, initialize these class on demand by calling InstanceKlass::initialize, which >>>>> is a quick no-op if the class is already initialized. >>>>> >>>>> 2. The only initialization left is that of a global lock. So I renamed the function >>>>> to SystemDictionaryShared::initialize_locks(). >>>>> >>>>> 3. I moved the call of this function from SystemDictionary::compute_java_loaders() to >>>>> SystemDictionary::initialize() where it seems to fit. >>>>> >>>>> Testing with hs-tiers 1 and 2. >>>>> >>>>> Thanks >>>>> - Ioi >>>> >>> > From gerard.ziemski at oracle.com Wed May 16 17:47:43 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Wed, 16 May 2018 12:47:43 -0500 Subject: RFR (XS) 8202360: [TESTBUG] runtime/LoadClass/TestResize.java needs to print output when it fails Message-ID: Hi all, Please review this small enhancement where we print out additional information (i.e. output from PrintSystemDictionaryAtExit), when we detect the failure. https://bugs.openjdk.java.net/browse/JDK-8202360 http://cr.openjdk.java.net/~gziemski/8202360_rev1/ Testing: mach5 hs-tier1,2 (running) and hotspot/jtreg/runtime/LoadClass/TestResize.java via jtreg locally cheers From mikhailo.seledtsov at oracle.com Wed May 16 18:00:44 2018 From: mikhailo.seledtsov at oracle.com (mikhailo) Date: Wed, 16 May 2018 11:00:44 -0700 Subject: RFR (XS) 8202360: [TESTBUG] runtime/LoadClass/TestResize.java needs to print output when it fails In-Reply-To: References: Message-ID: Looks good, Misha On 05/16/2018 10:47 AM, Gerard Ziemski wrote: > Hi all, > > Please review this small enhancement where we print out additional information (i.e. output from PrintSystemDictionaryAtExit), when we detect the failure. > > https://bugs.openjdk.java.net/browse/JDK-8202360 > http://cr.openjdk.java.net/~gziemski/8202360_rev1/ > > Testing: mach5 hs-tier1,2 (running) and hotspot/jtreg/runtime/LoadClass/TestResize.java via jtreg locally > > > cheers From gerard.ziemski at oracle.com Wed May 16 18:06:54 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Wed, 16 May 2018 13:06:54 -0500 Subject: RFR(S) 8200746: Remove FlagGuard class Message-ID: <24EC2A98-BDFA-4788-A306-1432C91C4225@oracle.com> Hi all, Please review this small fix where we remove unused FlagGuard class and FLAG_GUARD macro from hotspot and gtests. https://bugs.openjdk.java.net/browse/JDK-8200746 http://cr.openjdk.java.net/~gziemski/8200746_rev1/ Testing: mach5 hs-tier1,2 cheers From gerard.ziemski at oracle.com Wed May 16 18:09:37 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Wed, 16 May 2018 13:09:37 -0500 Subject: RFR (XS) 8202360: [TESTBUG] runtime/LoadClass/TestResize.java needs to print output when it fails In-Reply-To: References: Message-ID: <933EB741-1E7B-4E20-AE77-A01378314E6D@oracle.com> Thank you Misha! > On May 16, 2018, at 1:00 PM, mikhailo wrote: > > Looks good, > > Misha > > > On 05/16/2018 10:47 AM, Gerard Ziemski wrote: >> Hi all, >> >> Please review this small enhancement where we print out additional information (i.e. output from PrintSystemDictionaryAtExit), when we detect the failure. >> >> https://bugs.openjdk.java.net/browse/JDK-8202360 >> http://cr.openjdk.java.net/~gziemski/8202360_rev1/ >> >> Testing: mach5 hs-tier1,2 (running) and hotspot/jtreg/runtime/LoadClass/TestResize.java via jtreg locally >> >> >> cheers > From coleen.phillimore at oracle.com Wed May 16 18:12:39 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 16 May 2018 14:12:39 -0400 Subject: RFR(S) 8200746: Remove FlagGuard class In-Reply-To: <24EC2A98-BDFA-4788-A306-1432C91C4225@oracle.com> References: <24EC2A98-BDFA-4788-A306-1432C91C4225@oracle.com> Message-ID: <22a40b8e-55cd-b035-82ea-62d88711357d@oracle.com> Looks good!? Thank you for cleaning it up. Coleen On 5/16/18 2:06 PM, Gerard Ziemski wrote: > Hi all, > > Please review this small fix where we remove unused FlagGuard class and FLAG_GUARD macro from hotspot and gtests. > > https://bugs.openjdk.java.net/browse/JDK-8200746 > http://cr.openjdk.java.net/~gziemski/8200746_rev1/ > > Testing: mach5 hs-tier1,2 > > > cheers From gerard.ziemski at oracle.com Wed May 16 18:26:42 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Wed, 16 May 2018 13:26:42 -0500 Subject: RFR(S) 8200746: Remove FlagGuard class In-Reply-To: <22a40b8e-55cd-b035-82ea-62d88711357d@oracle.com> References: <24EC2A98-BDFA-4788-A306-1432C91C4225@oracle.com> <22a40b8e-55cd-b035-82ea-62d88711357d@oracle.com> Message-ID: <1565E0AE-89EC-4DF2-8597-BE0195589D77@oracle.com> Thank you Coleen! > On May 16, 2018, at 1:12 PM, coleen.phillimore at oracle.com wrote: > > Looks good! Thank you for cleaning it up. > Coleen > > On 5/16/18 2:06 PM, Gerard Ziemski wrote: >> Hi all, >> >> Please review this small fix where we remove unused FlagGuard class and FLAG_GUARD macro from hotspot and gtests. >> >> https://bugs.openjdk.java.net/browse/JDK-8200746 >> http://cr.openjdk.java.net/~gziemski/8200746_rev1/ >> >> Testing: mach5 hs-tier1,2 >> >> >> cheers > From harold.seigel at oracle.com Wed May 16 19:04:17 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Wed, 16 May 2018 15:04:17 -0400 Subject: RFR(S) 8200746: Remove FlagGuard class In-Reply-To: <22a40b8e-55cd-b035-82ea-62d88711357d@oracle.com> References: <24EC2A98-BDFA-4788-A306-1432C91C4225@oracle.com> <22a40b8e-55cd-b035-82ea-62d88711357d@oracle.com> Message-ID: <0569e9ac-0b60-fe69-d672-b1cbe070e473@oracle.com> Looks good to me, also. Harold On 5/16/2018 2:12 PM, coleen.phillimore at oracle.com wrote: > Looks good!? Thank you for cleaning it up. > Coleen > > On 5/16/18 2:06 PM, Gerard Ziemski wrote: >> Hi all, >> >> Please review this small fix where we remove unused FlagGuard class >> and FLAG_GUARD macro from hotspot and gtests. >> >> https://bugs.openjdk.java.net/browse/JDK-8200746 >> http://cr.openjdk.java.net/~gziemski/8200746_rev1/ >> >> Testing: mach5 hs-tier1,2 >> >> >> cheers > From igor.ignatyev at oracle.com Wed May 16 21:35:25 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 16 May 2018 14:35:25 -0700 Subject: RFR(M): 8199252: [TESTBUG] Open source VM testbase system dictionary tests In-Reply-To: <5AF606AF.3020404@oracle.com> References: <5AF606AF.3020404@oracle.com> Message-ID: <1BB15086-EC2C-4620-B8FF-A0C35C0A60C2@oracle.com> Hi Misha, looks good to me. Thanks, -- Igor > On May 11, 2018, at 2:10 PM, Mikhailo Seledtsov wrote: > > Please review this change open sourcing VM system dictionary tests. These tests have been used internally for a while, and are now being open sourced. > Since this is not an creation of new tests, simply open sourcing existing tests, we would like to keep the changes during this review to an absolute required minimum. If you have any feedback on improvements of these tests, please file RFE(s) that will be addressed later in order of priority. > > > Here is what was done for this change: > 1. Moved the tests to OpenJDK repository preserving relative directory location and structure. > 3. Updated Copyright statements accordingly. > 4. Updated "@library" statements accordingly. > 5. Updated TEST.groups > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199252 > Webrev: http://cr.openjdk.java.net/~mseledtsov/8199252.01.open/ > > Testing: > 1. Ran the following tests on open-only repository and build, using "make run-test" > (Linux-x64) > vmTestbase_nsk_sysdict > All PASS > > 2. Automated multip-platform test system (usual 4 platforms): > - vmTestbase_nsk_sysdict > - hs-tier{1,2} > In progress > > Thank you, > Misha > From david.holmes at oracle.com Wed May 16 21:54:13 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 17 May 2018 07:54:13 +1000 Subject: RFR (XS) 8202360: [TESTBUG] runtime/LoadClass/TestResize.java needs to print output when it fails In-Reply-To: References: Message-ID: Hi Gerard, On 17/05/2018 3:47 AM, Gerard Ziemski wrote: > Hi all, > > Please review this small enhancement where we print out additional information (i.e. output from PrintSystemDictionaryAtExit), when we detect the failure. > > https://bugs.openjdk.java.net/browse/JDK-8202360 > http://cr.openjdk.java.net/~gziemski/8202360_rev1/ I don't understand what your fix is doing - sorry. What I wanted to see was the complete history of lines of the form "Java dictionary (table_size=107, classes=6)" so that we can see how the table size and number of classes have changed leading up to the failure. But I realize now that we already get that from the: System.out.println("PASS table_size:"+table_size+", classes:"+classes+" OK"); All that is missing is the printing of the table_size and classes when we throw the exception. That can be trivially fixed: if (loadFactor > MAX_LOAD_FACTOR) { + System.out.println("FAIL table_size:"+table_size+", classes:"+classes); throw new RuntimeException("Load factor too high, expected MAX "+MAX_LOAD_FACTOR+", got "+loadFactor); } else { System.out.println("PASS table_size:"+table_size+", classes:"+classes+" OK"); } Thanks, David > Testing: mach5 hs-tier1,2 (running) and hotspot/jtreg/runtime/LoadClass/TestResize.java via jtreg locally > > > cheers > From david.holmes at oracle.com Wed May 16 22:06:47 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 17 May 2018 08:06:47 +1000 Subject: RFR(S) 8200746: Remove FlagGuard class In-Reply-To: <24EC2A98-BDFA-4788-A306-1432C91C4225@oracle.com> References: <24EC2A98-BDFA-4788-A306-1432C91C4225@oracle.com> Message-ID: <2c87ed39-1406-7d0c-c694-70a4e21a1816@oracle.com> Hi Gerard, On 17/05/2018 4:06 AM, Gerard Ziemski wrote: > Hi all, > > Please review this small fix where we remove unused FlagGuard class and FLAG_GUARD macro from hotspot and gtests. Well it seems only unused after you have removed its uses! Why does it turn out not to be needed in test_CollectorPolicy.cpp? Why is the whole test_globals.cpp no longer relevant? The bug report should explain why we don't need these. Thanks, David > https://bugs.openjdk.java.net/browse/JDK-8200746 > http://cr.openjdk.java.net/~gziemski/8200746_rev1/ > > Testing: mach5 hs-tier1,2 > > > cheers > From calvin.cheung at oracle.com Wed May 16 23:28:11 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 16 May 2018 16:28:11 -0700 Subject: RFR(xxs): 8196619: [TESTBUG] restore current version check in runtime/appcds/MultiReleaseJars.java Message-ID: <5AFCBE8B.6000500@oracle.com> JBS: https://bugs.openjdk.java.net/browse/JDK-8196619 webrev: http://cr.openjdk.java.net/~ccheung/8196619/webrev.00/ With javac supports --release 11, the temporary workaround for this test bug can be reverted. Sanity tested with hs-tier1 and hs-tier2. thanks, Calvin From david.holmes at oracle.com Thu May 17 00:02:11 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 17 May 2018 10:02:11 +1000 Subject: RFR(xxs): 8196619: [TESTBUG] restore current version check in runtime/appcds/MultiReleaseJars.java In-Reply-To: <5AFCBE8B.6000500@oracle.com> References: <5AFCBE8B.6000500@oracle.com> Message-ID: <5cdf7672-de7a-2427-0f27-73ca0f9b6808@oracle.com> Looks good - and trivial. Thanks, David On 17/05/2018 9:28 AM, Calvin Cheung wrote: > JBS: https://bugs.openjdk.java.net/browse/JDK-8196619 > > webrev: http://cr.openjdk.java.net/~ccheung/8196619/webrev.00/ > > With javac supports --release 11, the temporary workaround for this test > bug can be reverted. > > Sanity tested with hs-tier1 and hs-tier2. > > thanks, > Calvin From calvin.cheung at oracle.com Thu May 17 00:36:28 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 16 May 2018 17:36:28 -0700 Subject: RFR(xxs): 8196619: [TESTBUG] restore current version check in runtime/appcds/MultiReleaseJars.java In-Reply-To: <5cdf7672-de7a-2427-0f27-73ca0f9b6808@oracle.com> References: <5AFCBE8B.6000500@oracle.com> <5cdf7672-de7a-2427-0f27-73ca0f9b6808@oracle.com> Message-ID: <5AFCCE8C.6050701@oracle.com> Thanks David. Calvin On 5/16/18, 5:02 PM, David Holmes wrote: > Looks good - and trivial. > > Thanks, > David > > On 17/05/2018 9:28 AM, Calvin Cheung wrote: >> JBS: https://bugs.openjdk.java.net/browse/JDK-8196619 >> >> webrev: http://cr.openjdk.java.net/~ccheung/8196619/webrev.00/ >> >> With javac supports --release 11, the temporary workaround for this >> test bug can be reverted. >> >> Sanity tested with hs-tier1 and hs-tier2. >> >> thanks, >> Calvin From jiangli.zhou at oracle.com Thu May 17 03:27:47 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Wed, 16 May 2018 20:27:47 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: References: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> <5AF62823.7050804@oracle.com> <70F6AEB0-9103-4899-87D9-DC845BF66C51@oracle.com> <4CBA8376-54A2-4863-8732-B16E1AE9C38A@oracle.com> Message-ID: <5C996E41-CDA0-4EDA-B289-CDD7C38151B9@oracle.com> Here is the updated webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.02/ At CDS dump time, when SharedClassPathEntry::init() is called from FileMapInfo::allocate_shared_path_table(), we can identify the modules image entry directly. SharedClassPathEntry::init() now takes an extra bool argument to indicate if the current entry is the modules image. We no longer need to test the entry name using ClassLoader::is_modules_image(), which might be wrong for a shared path entry as Ioi pointed out. I added a ?_type? field in SharedClassPathEntry to record the type of shared path entries at dump time. At runtime, the path entry ?_type? is checked to identify the modules image in SharedClassPathEntry::validate(). SharedClassPathEntry::is_modules_image() no longer need to use ClassLoader::is_modules_image(). Reworked the BOOT_PATH case in SharedPathsMiscInfo::check(). I left os::file_name_strcmp() unchanged since it?s also used by APP_PATH. I'll handle it in a separate REF to do more testing. I also included the MoveJDKTest.java written by Ioi in the updated webrev. In the test, I added a new case for a JAR entry named with Hello.modules. Reran all appcds tests including MoveJDKTest.java locally. All tests passed. Running tiered tests. Thanks, Jiangli > On May 15, 2018, at 6:10 PM, Jiangli Zhou wrote: > > Hi Ioi, > > Thank you very much for the review. Good catches for those edge cases and thanks for the test cases. I will resolves them. > > Thanks! > Jiangli > >> On May 15, 2018, at 4:33 PM, Ioi Lam wrote: >> >> Hi Jiangli, >> >> 245 // We need to validate the runtime modules image file size against the archived >> 246 // size information, obtain the runtime modules image path. The recorded dump >> 247 // time modules image path in the archive may be different from the runtime path >> 248 // if the JDK image has beed moved after generating the archive. >> 249 if (ClassLoader::is_modules_image(name)) { >> 250 name = ClassLoader::get_jrt_entry()->name(); >> 251 } >> >> What happens if someone has a JAR file called foo.modules, and dumped with >> >> java -Xshare:dump -cp foo.modules >> >> >> I think it's better to check that it's #0 in the classpath so we know for sure it's the system modules file. >> >> There's a similar problem here: >> >> 220 if (!ClassLoader::is_modules_image(name)) { >> >> In the following, instead of doing a copy, maybe it's better to add a length argument for os::file_name_strcmp, since sharedPathsMiscInfo.cpp is the only place in the VM that calls this function. >> >> 162 if (relaxed_check) { >> 163 // only check the begining portion of the runtime boot path, up to >> 164 // the length of the dump time boot path >> 165 size_t len = strlen(path); >> 166 runtime_boot_path = NEW_RESOURCE_ARRAY(char, len + 1); >> 167 strncpy(runtime_boot_path, Arguments::get_sysclasspath(), len); >> 168 runtime_boot_path[len] = '\0'; >> 169 } else { >> 170 // check the full runtime boot path >> 171 runtime_boot_path = Arguments::get_sysclasspath(); >> 172 } >> 173 >> 174 // Do a quick check first with a simple >> 175 // strcmp(dump_time_boot_path, runtime_boot_path). If the paths are the >> 176 // same, the check has succeeded. >> 177 if (os::file_name_strcmp(path, runtime_boot_path) == 0) { >> 178 break; // ok >> 179 } >> >> >> Also, the copy could be wrong for the following output: >> >> path = /tmp/foo/modules >> Arguments::get_sysclasspath() = /tmp/foo/modulesxyz:/aaa:/bbb >> >> What is runtime_boot_path supposed to contain here? I am not sure if this code gives the intended output or not, but it's not easy to understand what it actually does by operating on an truncated pathname. >> >> String manipulation code is always hard to read. I would suggest braking it up into smaller functions: >> >> 1. Get the first entry of dump time and run time boot path -> d0 and r0 >> -> ignore d0 >> -> r0 same as ClassLoader::get_jrt_entry()->name(); >> >> 2. Get the remaining part of the dump time and run time boot path -> d_remain, r_remain >> -> relaxed_check: d_remain must be a prefix of r_remain >> -> !relaxed_check: They must be identical. >> >> >> By the way, I think the test case in JDK-8202935 should be reviewed together inside this RFR, since the test validates the feature implemented here. I don't think we need a separate bug ID. Otherwise it will be hard to track the test cases. >> >> >> Thanks >> - Ioi >> >> >> On 5/13/18 5:31 PM, Jiangli Zhou wrote: >>> Here is the updated webrev: >>> http://cr.openjdk.java.net/~jiangli/8199807/webrev.01/ >>> >>> Thanks, >>> Jiangli >>> >>>> On May 11, 2018, at 6:07 PM, Jiangli Zhou wrote: >>>> >>>> Hi Calvin, >>>> >>>> Thanks for the review. >>>> >>>>> On May 11, 2018, at 4:32 PM, Calvin Cheung wrote: >>>>> >>>>> Hi Jiangli, >>>>> >>>>> Thanks for doing this useful change. >>>>> >>>>> I have a few minor comments: >>>>> >>>>> 1) sharedPathsMiscInfo.cpp >>>>> >>>>> 199 dp ++; >>>>> >>>>> Should the above be similar to line 194 as follows? >>>>> dp += path_sep_len; >>>> Good catch. Will fix. >>>> >>>>> 2) filemap.cpp >>>>> >>>>> 244 if (ClassLoader::is_modules_image(name)) { >>>>> >>>>> I think the above could be: >>>>> if (is_modules_image()) { >>>>> >>>> Let?s keep it this way as is_modules_image() is a wrapper of 'ClassLoader::is_modules_image(name())?. We already have the ?name? in this case. >>>> >>>>> The is_modules_image() is a member function of SharedClassPathEntry. >>>>> >>>>> Btw, why is it necessary to get the runtime modules image path again in lines 244 - 246? >>>> As the runtime modules image path could be different from the dump time if the JDK image is moved/copied after archive generation, we need to make sure to use the correct file for file size check. The recored path in the archive is the dump time modules image path, which may no longer be existing at runtime. I will add some comments if that?s helpful. >>>> >>>>> 3) BootClassPathMismatch.java >>>>> >>>>> Could you move line 103 to right above line 107 (opts.addPrefix(?))? >>>>> Because the archive dumping part of the test doesn?t use appJar. >>>> Sure. >>>> >>>> Thanks! >>>> >>>> Jiangli >>>> >>>>> thanks, >>>>> Calvin >>>>> >>>>> On 5/11/18, 3:15 PM, Jiangli Zhou wrote: >>>>>> I?ve withdrawn my original weberv and removed the build change. Here is the updated webrev that only addresses 8199807. >>>>>> >>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.00/ >>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>> >>>>>> Thanks, >>>>>> Jiangli >>>>>> >>>>>>> On May 11, 2018, at 2:44 PM, Jiangli Zhou wrote: >>>>>>> >>>>>>> Hi Erik, >>>>>>> >>>>>>> Thank you so much for investigating a proper fix for the vm_version.o issue. I will withdraw the build change from my original webrev. >>>>>>> >>>>>>> Thanks again for taking over the bug! >>>>>>> Jiangli >>>>>>> >>>>>>>> On May 11, 2018, at 2:33 PM, Erik Joelsson wrote: >>>>>>>> >>>>>>>> Received: from [10.132.185.167] (/10.132.185.167) >>>>>>>> by default (Oracle Beehive Gateway v4.0) >>>>>>>> with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 >>>>>>>> Subject: Re: RFR: 8199807& 8202738: AppCDS performs overly restrictive path >>>>>>>> matching check >>>>>>>> To: Jiangli Zhou, >>>>>>>> "hotspot-runtime-dev at openjdk.java.net runtime" >>>>>>>> , >>>>>>>> build-dev >>>>>>>> References: >>>>>>>> From: Erik Joelsson >>>>>>>> Organization: Oracle Corporation >>>>>>>> Message-ID: >>>>>>>> Date: Fri, 11 May 2018 14:33:20 -0700 >>>>>>>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) >>>>>>>> Gecko/20100101 Thunderbird/52.7.0 >>>>>>>> MIME-Version: 1.0 >>>>>>>> In-Reply-To: >>>>>>>> Content-Type: text/plain; charset=utf-8; format=flowed >>>>>>>> Content-Transfer-Encoding: 8bit >>>>>>>> Content-Language: en-US >>>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>>> >>>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>>> >>>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>>> >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>>> >>>>>>>> /Erik >>>>>>>> >>>>>>>> >>>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>>> >>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>>> >>>>>>>>> The webrev includes the following three main parts: >>>>>>>>> >>>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>>> >>>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>>> >>>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>>> >>>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Jiangli >>>>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>>> >>>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>>> >>>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>>> >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>>> >>>>>>>> /Erik >>>>>>>> >>>>>>>> >>>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>>> >>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>>> >>>>>>>>> The webrev includes the following three main parts: >>>>>>>>> >>>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>>> >>>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>>> >>>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>>> >>>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Jiangli >> > From erik.helin at oracle.com Thu May 17 07:04:00 2018 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 17 May 2018 09:04:00 +0200 Subject: RFR(S) 8200746: Remove FlagGuard class In-Reply-To: <2c87ed39-1406-7d0c-c694-70a4e21a1816@oracle.com> References: <24EC2A98-BDFA-4788-A306-1432C91C4225@oracle.com> <2c87ed39-1406-7d0c-c694-70a4e21a1816@oracle.com> Message-ID: On 05/17/2018 12:06 AM, David Holmes wrote: > Hi Gerard, > > On 17/05/2018 4:06 AM, Gerard Ziemski wrote: >> Hi all, >> >> Please review this small fix where we remove unused FlagGuard class >> and FLAG_GUARD macro from hotspot and gtests. > > Well it seems only unused after you have removed its uses! Why does it > turn out not to be needed in test_CollectorPolicy.cpp? I think it is still needed, I don't think this patch is correct. When we execute the gtests we execute the TEST_VM tests one-by-one in serial order in the same process. This means that if one tests changes a global flag, e.g. InitialHeapSize, the next test will see a different value for InitialHeapSize. In order to guard the flags, I introduced the class FlagGuard (and corresponding helper macro FLAG_GUARD). This means that a test, such as test_collectorPolicy.cpp, can easily guard any global flags the test intends to mutate, and then the flags will be restored to their original values when test has finished. > Why is the whole test_globals.cpp no longer relevant? test_globals.cpp currently only tests the FlagGuard class (and FLAG_GUARD) macro in globals.hpp, so since Gerard removed the FlagGuard class, the tests are no longer relevant. However, see my comment above, we can't just remove the FlagGuard class, it is needed! > The bug report should explain why we don't need these. Yes, I would also like to see an explanation why this code can be removed? To me it seems like it is needed in test_collectorPolicy.cpp, but I could be wrong. Thanks, Erik > Thanks, > David > >> https://bugs.openjdk.java.net/browse/JDK-8200746 >> http://cr.openjdk.java.net/~gziemski/8200746_rev1/ >> >> Testing: mach5 hs-tier1,2 >> >> >> cheers >> From per.liden at oracle.com Thu May 17 08:01:41 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 17 May 2018 10:01:41 +0200 Subject: RFR: 8202978: Incorrect tmp register passed to MacroAssembler::load_mirror() In-Reply-To: <0d85792f-c8ba-100c-9780-761f6b12ea8b@oracle.com> References: <70f41abc-9f3f-98fa-a66d-76f931640d02@oracle.com> <506c8615-5d79-be76-5e09-9331ec50465e@oracle.com> <993a387a-574d-43f2-3c61-47388a1bc459@oracle.com> <2470a80e-e709-5f56-991f-efc79bc40b5c@oracle.com> <0d85792f-c8ba-100c-9780-761f6b12ea8b@oracle.com> Message-ID: Thanks Coleen! /Per On 05/16/2018 03:38 PM, coleen.phillimore at oracle.com wrote: > > Yes, this looks good.? I'm glad that works out. > thanks, > Coleen > > On 5/16/18 4:22 AM, Per Liden wrote: >> On 05/16/2018 08:58 AM, Per Liden wrote: >>> On 05/15/2018 10:25 PM, coleen.phillimore at oracle.com wrote: >>>> >>>> Hi Per, >>> >>> Hi Coleen, >>> >>>> >>>> It's calling a static native function so might not be totally >>>> performance neutral. >>>> It looks like rax is scratch there.? Can you try that??? Sorry I >>>> didn't figure out that r11 is rscratch2. >>> >>> I looks like you're right, rax seems to be used as scratch here. I'll >>> run some tests with that and get back. >> >> Ok, changed to rax and ran it through testing without finding any >> problems. >> >> Updated webrev: http://cr.openjdk.java.net/~pliden/8202978/webrev.1 >> >> /Per >> >>> >>> Thanks! >>> >>> /Per >>> >>>> >>>> Thanks, >>>> Coleen >>>> >>>> On 5/11/18 8:48 AM, Per Liden wrote: >>>>> On 05/11/2018 02:32 PM, Vladimir Kozlov wrote: >>>>>> Looks good. >>>>> >>>>> Thanks Vladimir! >>>>> >>>>>> Is this new code? Should we backport it otherwise? >>>>> >>>>> Yes, this is new code, i.e. it's not in JDK 10 or older. Also, it >>>>> just so happens that this bug doesn't affect the existing GC, only >>>>> GCs which do load barriers (like ZGC), which is why it hasn't been >>>>> caught before. >>>>> >>>>> /Per >>>>> >>>>>> >>>>>> Thanks, >>>>>> Vladimir K >>>>>> >>>>>> On 5/11/18 1:31 AM, Per Liden wrote: >>>>>>> On x86, MacroAssembler::load_mirror() defaults to using rscratch2 >>>>>>> as tmp register, unless something else is specified. In >>>>>>> TemplateInterpreterGenerator::generate_native_entry() we call >>>>>>> load_mirror(), but the rscratch2 register is already in use in >>>>>>> this context, which obviously leads to problems. This is not a >>>>>>> performance critical path, so we should just pass noreg as the >>>>>>> tmp register. >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202978 >>>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8202978/webrev.0 >>>>>>> >>>>>>> Testing: passes hs-tier{1,2} >>>>>>> >>>>>>> /Per >>>> > From adam.farley at uk.ibm.com Thu May 17 08:26:13 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Thu, 17 May 2018 09:26:13 +0100 Subject: Bug Request: Please remove out-of-date files from bug In-Reply-To: <5ce54a40-7f2d-f8a2-561b-db21c4a16000@oracle.com> References: <5ce54a40-7f2d-f8a2-561b-db21c4a16000@oracle.com> Message-ID: Hi David, Good catch. Copying to the Hotspot list. Best Regards Adam Farley OpenJDK Team Runtimes IBM From: David Holmes To: Adam Farley8 , core-libs-dev Date: 16/05/2018 22:32 Subject: Re: Bug Request: Please remove out-of-date files from bug Done. Though you sent this to the wrong mailing list for a hotspot bug. David On 17/05/2018 1:36 AM, Adam Farley8 wrote: > Hi All, > > In bug JDK-8190187, hotspot_hg_diff and jdk_hg_diff are out-of-date. > > Please can someone delete these files? > > Best Regards > > Adam Farley > OpenJDK Team > Runtimes > IBM > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From axel.siebenborn at sap.com Thu May 17 10:15:48 2018 From: axel.siebenborn at sap.com (Siebenborn, Axel) Date: Thu, 17 May 2018 10:15:48 +0000 Subject: RFR(L): 8176808: Split up metaspace.cpp In-Reply-To: References: Message-ID: <32826e8139614cd68f0b4548ca8c044c@sap.com> Hi Thomas, The change looks good . I should have read your whole mail before starting looking at the changes, as you explain what you have done and what you didn't do ??. Concerning the decision using ::metaspace vs. ::metaspace::internal: I would use ::metaspace for the internal classes and leave the public classes in the global namespace. This is more consistent to hotspot code where namespaces are rarely used (if this counts as an argument). I would change that in this change, as having having ::metaspace::internal and nothing in ::metaspace seems to be the worst alternative. Altogether, this is really a good refactoring. Cheers, Axel > -----Original Message----- > From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- > bounces at openjdk.java.net] On Behalf Of Thomas St?fe > Sent: Montag, 14. Mai 2018 15:34 > To: Hotspot dev runtime > Subject: Re: RFR(L): 8176808: Split up metaspace.cpp > > Ping... > > Is there anything I can do to make review easier? This is really > mostly code shuffling around (out of metaspace.cpp into new files), so > with a bit of effort I could cook up some diffs for those parts. > > ..Thomas > > On Fri, May 11, 2018 at 8:31 AM, Thomas St?fe > wrote: > > All builds are fixed, jdk-submit tests ran through sucessfully. > > > > Latest webrev: > > http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace- > cpp/webrev.01/webrev/ > > > > Only difference to the first webrev is the placement of a single > > semicolon in occupancyMap.hpp, to satisfy the Solaris compiler. > > > > Thanks, Thomas > > > > > > > > On Wed, May 9, 2018 at 5:08 PM, Thomas St?fe > wrote: > >> Hi all, > >> > >> This was a long time coming. Lets cut up this beast :) > >> > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8176808 > >> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- > metaspace-cpp/webrev.00/webrev/ > >> > >> This change splits up metaspace.cpp into multiple parts. Note that I > >> did not change much code (exceptions see below in details) - so this > >> is mostly moving code around. > >> > >> This change follows the scheme tentatively outlined in JDK-8201572: > >> > >> - metaspace internal classes go into the sub directory > >> share/memory/metaspace. Ideally, those should never be accessed from > >> outside, apart from other metaspace classes and metaspace tests. They > >> also go into the namespace ::metaspace::internals. > >> > >> - metaspace external classes (MetaspaceUtils, ClassLoaderMetaspace, > >> etc) remain inside share/memory and, for now, remain in the global > >> namespace. > >> > >> Note: the original idea was to move metaspace external classes to > >> namespace ::metaspace. But I shied away from that, since that would > >> mean fixing up all usages of these classes either with metaspace:: > >> scope or with using declarations. I had to make a cut at some point. > >> > >> So here is something to decide: > >> - should we then get rid of the ::metaspace::internals namespace, move > >> metaspace-internal classes to the enclosing ::metaspace namespace and > >> leave external classes in the global namespace ? > >> - or should we follow through (maybe in a future patch): internal > >> classes in ::metaspace::internal, and move external classes to > >> ::metaspace ? > >> > >> Some more details: > >> > >> - the following classes moved out of metaspace.cpp into namespace > >> "metaspace::internal" and the metaspace sub directory: > >> MetaDebug, ChunkManager, SpaceManager, BlockFreeList and > SmallBlocks, > >> OccupancyMap, VirtualSpaceNode and VirtualSpaceList, the > >> PrintCLDMetaspaceInfoClosure. > >> > >> - I also moved metachunk.hpp to internals, since class Metachunk is > >> not used externally. What is used externally is Metablock though, so I > >> split up metachunk.hpp into metabase.hpp, metablock.hpp and > >> metachunk.hpp, holding Metabase, Metablock and Metachunk, > >> respectively. > >> > >> - Now we see some ugly seams: metaspace.hpp is supposed to be the > >> outside API contract, however, it contains implementation details > >> which should not be there and which manifest now by having to use the > >> metaspace::internals namespace here and there. If we care, we could > >> solve this with a bit redesigning. > >> > >> - The ChunkManager gtest and the SpaceManager gtest had been > >> implemented inside metaspace.cpp, since it was not possible to access > >> the internal classes those tests needed in any other way. Since that > >> is now possible, I moved the coding out to gtest-land and made them > >> real gtests (exchanging the asserts for real gtest ASSERTS, among > >> other things). > >> Note that there are some tests left in metaspace.cpp > >> (TestMetaspaceUtilsTest, TestVirtualSpaceNodeTest) - those were no > >> gtests-in-disguise but were part of -XX:+ExecuteInternalVMTests. I > >> guess these tests precede the gtest framework? I leave it up to others > >> to decide what to do with them and to potentially move them out of > >> metaspace.cpp. > >> Side note: -XX:+ExecuteInternalVMTests seems to have bitrotted, see > >> https://bugs.openjdk.java.net/browse/JDK-8202848. Does this get tested > >> regularly? > >> > >> - metaspace.cpp is quite a bit smaller now - from ~5000 loc down to > >> ~1700 loc. Arguably, one could split out more and clean up more, but I > >> think this is a good start. > >> > >> --- > >> > >> I built locally on linux (release, fastdebug with and without pch, > >> zero) and windows (64, 32bit). jdk-submit tests ran through with a > >> build error on solaris sparc - I currently try to reproduce that build > >> error with our very slow solaris machine. > >> > >> Thanks, Thomas From thomas.stuefe at gmail.com Thu May 17 10:36:37 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 17 May 2018 12:36:37 +0200 Subject: RFR(L): 8176808: Split up metaspace.cpp In-Reply-To: <32826e8139614cd68f0b4548ca8c044c@sap.com> References: <32826e8139614cd68f0b4548ca8c044c@sap.com> Message-ID: Hi Axel, Thanks a lot for the review! Remarks inline. On Thu, May 17, 2018 at 12:15 PM, Siebenborn, Axel wrote: > Hi Thomas, > The change looks good . > I should have read your whole mail before starting looking at the changes, as you explain what you have done and what you didn't do ??. > Concerning the decision using ::metaspace vs. ::metaspace::internal: > I would use ::metaspace for the internal classes and leave the public classes in the global namespace. > This is more consistent to hotspot code where namespaces are rarely used (if this counts as an argument). > I would change that in this change, as having having ::metaspace::internal and nothing in ::metaspace seems to be the worst alternative. Yeah, I thought so too. I'll wait for other reviewers to chime in though. I am fine either way. > > Altogether, this is really a good refactoring. Thank you! ..Thomas > > Cheers, > Axel > >> -----Original Message----- >> From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- >> bounces at openjdk.java.net] On Behalf Of Thomas St?fe >> Sent: Montag, 14. Mai 2018 15:34 >> To: Hotspot dev runtime >> Subject: Re: RFR(L): 8176808: Split up metaspace.cpp >> >> Ping... >> >> Is there anything I can do to make review easier? This is really >> mostly code shuffling around (out of metaspace.cpp into new files), so >> with a bit of effort I could cook up some diffs for those parts. >> >> ..Thomas >> >> On Fri, May 11, 2018 at 8:31 AM, Thomas St?fe >> wrote: >> > All builds are fixed, jdk-submit tests ran through sucessfully. >> > >> > Latest webrev: >> > http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace- >> cpp/webrev.01/webrev/ >> > >> > Only difference to the first webrev is the placement of a single >> > semicolon in occupancyMap.hpp, to satisfy the Solaris compiler. >> > >> > Thanks, Thomas >> > >> > >> > >> > On Wed, May 9, 2018 at 5:08 PM, Thomas St?fe >> wrote: >> >> Hi all, >> >> >> >> This was a long time coming. Lets cut up this beast :) >> >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8176808 >> >> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- >> metaspace-cpp/webrev.00/webrev/ >> >> >> >> This change splits up metaspace.cpp into multiple parts. Note that I >> >> did not change much code (exceptions see below in details) - so this >> >> is mostly moving code around. >> >> >> >> This change follows the scheme tentatively outlined in JDK-8201572: >> >> >> >> - metaspace internal classes go into the sub directory >> >> share/memory/metaspace. Ideally, those should never be accessed from >> >> outside, apart from other metaspace classes and metaspace tests. They >> >> also go into the namespace ::metaspace::internals. >> >> >> >> - metaspace external classes (MetaspaceUtils, ClassLoaderMetaspace, >> >> etc) remain inside share/memory and, for now, remain in the global >> >> namespace. >> >> >> >> Note: the original idea was to move metaspace external classes to >> >> namespace ::metaspace. But I shied away from that, since that would >> >> mean fixing up all usages of these classes either with metaspace:: >> >> scope or with using declarations. I had to make a cut at some point. >> >> >> >> So here is something to decide: >> >> - should we then get rid of the ::metaspace::internals namespace, move >> >> metaspace-internal classes to the enclosing ::metaspace namespace and >> >> leave external classes in the global namespace ? >> >> - or should we follow through (maybe in a future patch): internal >> >> classes in ::metaspace::internal, and move external classes to >> >> ::metaspace ? >> >> >> >> Some more details: >> >> >> >> - the following classes moved out of metaspace.cpp into namespace >> >> "metaspace::internal" and the metaspace sub directory: >> >> MetaDebug, ChunkManager, SpaceManager, BlockFreeList and >> SmallBlocks, >> >> OccupancyMap, VirtualSpaceNode and VirtualSpaceList, the >> >> PrintCLDMetaspaceInfoClosure. >> >> >> >> - I also moved metachunk.hpp to internals, since class Metachunk is >> >> not used externally. What is used externally is Metablock though, so I >> >> split up metachunk.hpp into metabase.hpp, metablock.hpp and >> >> metachunk.hpp, holding Metabase, Metablock and Metachunk, >> >> respectively. >> >> >> >> - Now we see some ugly seams: metaspace.hpp is supposed to be the >> >> outside API contract, however, it contains implementation details >> >> which should not be there and which manifest now by having to use the >> >> metaspace::internals namespace here and there. If we care, we could >> >> solve this with a bit redesigning. >> >> >> >> - The ChunkManager gtest and the SpaceManager gtest had been >> >> implemented inside metaspace.cpp, since it was not possible to access >> >> the internal classes those tests needed in any other way. Since that >> >> is now possible, I moved the coding out to gtest-land and made them >> >> real gtests (exchanging the asserts for real gtest ASSERTS, among >> >> other things). >> >> Note that there are some tests left in metaspace.cpp >> >> (TestMetaspaceUtilsTest, TestVirtualSpaceNodeTest) - those were no >> >> gtests-in-disguise but were part of -XX:+ExecuteInternalVMTests. I >> >> guess these tests precede the gtest framework? I leave it up to others >> >> to decide what to do with them and to potentially move them out of >> >> metaspace.cpp. >> >> Side note: -XX:+ExecuteInternalVMTests seems to have bitrotted, see >> >> https://bugs.openjdk.java.net/browse/JDK-8202848. Does this get tested >> >> regularly? >> >> >> >> - metaspace.cpp is quite a bit smaller now - from ~5000 loc down to >> >> ~1700 loc. Arguably, one could split out more and clean up more, but I >> >> think this is a good start. >> >> >> >> --- >> >> >> >> I built locally on linux (release, fastdebug with and without pch, >> >> zero) and windows (64, 32bit). jdk-submit tests ran through with a >> >> build error on solaris sparc - I currently try to reproduce that build >> >> error with our very slow solaris machine. >> >> >> >> Thanks, Thomas From martin.doerr at sap.com Thu May 17 12:27:58 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 17 May 2018 12:27:58 +0000 Subject: RFR(M): 8202713: Create a MacroAssembler::access_load/store_at wrapper for S390 and PPC In-Reply-To: <8bc21bb49b194ab2aa779c742f2e6e13@sap.com> References: <63f242b9552844508c77017d240d42b1@sap.com> <8bc21bb49b194ab2aa779c742f2e6e13@sap.com> Message-ID: Hi Erik and G?tz, thanks for the reviews. Pushed. Best regards, Martin -----Original Message----- From: Lindenmaier, Goetz Sent: Mittwoch, 16. Mai 2018 15:04 To: Doerr, Martin ; Erik ?sterlund ; hotspot-runtime-dev at openjdk.java.net Subject: RE: RFR(M): 8202713: Create a MacroAssembler::access_load/store_at wrapper for S390 and PPC Hi Martin, thanks a lot for your adaptions, looks good now. Best regards, Goetz. > -----Original Message----- > From: Doerr, Martin > Sent: Mittwoch, 16. Mai 2018 14:30 > To: Lindenmaier, Goetz ; Erik ?sterlund > ; hotspot-runtime-dev at openjdk.java.net > Subject: RE: RFR(M): 8202713: Create a > MacroAssembler::access_load/store_at wrapper for S390 and PPC > > Hi G?tz, > > > barrierSetAssembler_ppc.cpp > > > > The name of the label is_null is misleading. It sounds > > as the name of a boolean variable. Maybe "L_handle_null"? > > And please use explicit compare for pointers: if (is_null == NULL) { > > > Done. > > > Shouldn't there be an assertion that no other decorators > > are set than those handled here? Like all the memory ordering > > ones? > > > Done. > > > macroAssembler_ppc.hpp: > > > > This header uses DecoratorSet thus should #include accessDecorators.hpp > > > Done. > > > access_load/store_at() > > This is a misleading name for this function. I would think this > > should go to nativeInst_ppc.cpp :) Why would you access a store?? > > But this needs to be changed on all platforms. So not > > scope of this review. > > The function is not really needed. Make it private, or inline > > it into load/store_heap_oop. > > > Made private. > > > load_heap_oop_not_null: > > Please remove this on ppc, too. It's the only platform > > that still uses this function. There are only 6 uses. > > Please also remove the dead definitions of it on aarch64 and x86, > > as well as the comment in s390.ad mentioning it. (not sure about > > aarch64 as you can't easily test this). > > > Changed PPC64 and s390 code to use the full featured version only. > Removed old functions. > I think it would be kind of unfair to change x86/aarch64 in this change > because x86 folks will probably not read this review. > > > Please extend the comment above load_heap_oop. It gained > > new functionality, and after removing the _not_null functions > > along with their documentation the current comment is pointless. > > Mention what the label is good for. Rename the label as above. > > > Done. > > > macroAssembler_ppc.inline.hpp > > > > Please rename the boolean variable as_raw to without_gc_barrier. > > Actually, the DecoratorSet name AS_RAW is misleading, there is still > > encode/decode etc. But this is not in the scope of this change. > > > Not done because I'd like to keep PPC64 and s390 implementation consistent > to the other platforms. A cleanup change for all platforms could be done in a > separate change without s390 or PPC in its title. > > > stubGenerator_ppc.cpp > > Remove comment // Load the oop. Makes the line too long, and > > not too much information gained :) > > > Done. > > New webrev: > http://cr.openjdk.java.net/~mdoerr/8202713_ppc64_s390_masm_access/w > ebrev.01/ > > Best regards, > Martin From matthias.baesken at sap.com Thu May 17 15:20:29 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 17 May 2018 15:20:29 +0000 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: References: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> Message-ID: > > > I also think that one number is enough. Adaptive numbers would be great. Hi Thomas/Goetz/Martin , thanks for looking into it. Adaptive numbers could be done in a follow-up as suggested by Thomas , this would also touch the other platforms . I added more line feeds to the output and reduced output to just printing M . Example output from a Windows desktop PC : Memory: 4k page, system-wide physical 32520M (20165M free) TotalPageFile size 34568M (AvailPageFile size 18170M) current process WorkingSet (physical memory assigned to process): 79M, peak: 79M current process commit charge ("private bytes"): 148M, peak: 148M new webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.1/ Best regards, Matthias > -----Original Message----- > From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] > Sent: Mittwoch, 16. Mai 2018 16:35 > To: Lindenmaier, Goetz > Cc: Baesken, Matthias ; Doerr, Martin > ; hotspot-dev at openjdk.java.net; hotspot- > runtime-dev at openjdk.java.net > Subject: Re: [RFR] 8202427: Enhance os::print_memory_info on Windows > > On Wed, May 16, 2018 at 3:48 PM, Lindenmaier, Goetz > wrote: > > Hi Matthias, > > > > I appreciate the additional information in the hs_err file. > > Could you please add an example output (of the final version) to the bug > description? > > > > As Martin, I would like more line feeds, both in the code and the output. > > Currently, the output in the hs_err file looks like this (where the first line is > too long): > > Memory: 4k page, system-wide physical 16776692k [16383M] (8795032k > [8588M] free), TotalPageFile size 49949460k [48778M] (AvailPageFile size > 38942152k [38029M]), user-mode portion of virtual address-space 2097024k > [2047M] (6940k [6M] free) > > current process WorkingSet (physical memory assigned to process): > 991804k [968M], peak: 991804k [968M] > > > > current process commit charge ("private bytes"): 1523632k [1487M], > peak: 1523632k [1487M] > > > > I also think that one number is enough. Adaptive numbers would be great. > > I know about > > src/hotspot/share/memory/metaspace/metaspaceCommon.hpp: > print_human_readable_size() > > for this, but this seems a bit hidden in metaspace. > > > > Guys, lets not worry about human readable numbers for now. We can move > print_human_readable_size() out from metaspace to something generic > and use it to improve this code (and others) in a later patch. That > was my original intention when adding print_human_readable_size(). > > The patch is good. Thanks for doing this. > > ..Thoams > > > I don't like usage of _M_IX86. Common in this file seems #ifndef _WIN64 > for 32-bit windows > > dependent code. But don't care here. > > > > Best regards, > > Goetz. > > > > > >> -----Original Message----- > >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > >> Behalf Of Baesken, Matthias > >> Sent: Mittwoch, 16. Mai 2018 15:13 > >> To: Doerr, Martin ; 'hotspot- > >> dev at openjdk.java.net' ; hotspot- > >> runtime-dev at openjdk.java.net > >> Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows > >> > >> Ping : could I get a second review ? > >> > >> Bug : > >> https://bugs.openjdk.java.net/browse/JDK-8202427 > >> Change : > >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > >> > >> > >> > We could also just print the MB value , let's see what other think. > >> > Another option might be to have a flexible output (kB for smaller > >> memory > >> > values , MB (or GB) for larger ) ? > >> > >> Martin suggested to just print the MB values, what do you think ? > >> > >> > >> Thanks, Matthias > >> > >> > >> > -----Original Message----- > >> > From: Baesken, Matthias > >> > Sent: Mittwoch, 2. Mai 2018 13:00 > >> > To: Doerr, Martin ; 'hotspot- > >> > dev at openjdk.java.net' ; hotspot- > >> > runtime-dev at openjdk.java.net > >> > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on > Windows > >> > > >> > Hi Martin, thanks for your input . > >> > I add hotspot-runtime-dev . > >> > > >> > > > >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe > other > >> > > reviewers would like to comment on this, too. > >> > > > >> > > >> > We could also just print the MB value , let's see what other think. > >> > Another option might be to have a flexible output (kB for smaller > >> memory > >> > values , MB (or GB) for larger ) ? > >> > > >> > Best regards, Matthias > >> > > >> > > >> > > -----Original Message----- > >> > > From: Doerr, Martin > >> > > Sent: Mittwoch, 2. Mai 2018 12:53 > >> > > To: Baesken, Matthias ; 'hotspot- > >> > > dev at openjdk.java.net' > >> > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on > Windows > >> > > > >> > > Hi Matthias, > >> > > > >> > > looks like a nice enhancement. We can get substantially more > >> information. > >> > > > >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe > other > >> > > reviewers would like to comment on this, too. > >> > > > >> > > We should have line breaks. > >> > > > >> > > Best regards, > >> > > Martin > >> > > > >> > > > >> > > -----Original Message----- > >> > > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] > On > >> > > Behalf Of Baesken, Matthias > >> > > Sent: Montag, 30. April 2018 16:53 > >> > > To: 'hotspot-dev at openjdk.java.net' dev at openjdk.java.net> > >> > > Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows > >> > > > >> > > On Windows, > >> > > the os::print_memory_info misses a few memory-related infos that > >> might > >> > > be interesting : > >> > > - current and peak WorkingSet size (= physical memory assigned to > the > >> > > process) > >> > > - current and peak commit charge (also known as "private bytes" in > >> > Windows > >> > > tools) > >> > > - on 32bit Windows : > >> > > user-mode portion/free user mode portion of virtual address-space > >> > > (Total/AvailVirtual) because it shows how close do we get to the 2-4 > GB > >> per > >> > > process border. > >> > > > >> > > > >> > > - the current naming of "swap/free-swap" memory is a bit misleading; > >> > > in the Windows world swap is a special page file used for UWP apps. > >> > > (see Windows Internals : "The swap file" (chapter 5 Memory > >> > management) > >> > > Windows 8 added another page file called a swap file. It is ... > exclusively > >> > used > >> > > for UWP (Universal Windows Platform) apps. > >> > > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the > UWP > >> > > related values, it is about page file sizes > >> > > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just > >> virtual > >> > > memory might be more appropriate). > >> > > > >> > > > >> > > https://msdn.microsoft.com/de- > >> > > de/library/windows/desktop/aa366770(v=vs.85).aspx > >> > > > >> > > documents it in the following way: > >> > > ullTotalPageFile: > >> > > The current committed memory limit for the system or the current > >> process, > >> > > whichever is smaller,... > >> > > > >> > > ullAvailPageFile : > >> > > The maximum amount of memory the current process can commit, in > >> > bytes. > >> > > This value is equal to or smaller than the system-wide available > commit > >> > value > >> > > > >> > > > >> > > > >> > > Aditionally I suggest having output of the various memory-values in M > >> > > (megabyte) as well , the k (kilobyte) output sometimes gives very > high > >> and > >> > > unreadable numbers). > >> > > > >> > > > >> > > Could you please review my change ? > >> > > > >> > > > >> > > Bug : > >> > > > >> > > https://bugs.openjdk.java.net/browse/JDK-8202427 > >> > > > >> > > > >> > > Change : > >> > > > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > >> > > > >> > > > >> > > Thanks, Matthias > >> > > > > From gerard.ziemski at oracle.com Thu May 17 15:53:04 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Thu, 17 May 2018 10:53:04 -0500 Subject: RFR(S) 8200746: Remove FlagGuard class In-Reply-To: References: <24EC2A98-BDFA-4788-A306-1432C91C4225@oracle.com> <2c87ed39-1406-7d0c-c694-70a4e21a1816@oracle.com> Message-ID: <85AC5FE4-C4A4-4519-8032-2C5469E00F0A@oracle.com> Thank you Erik, David for the review. My motivation for removing FlagGuard is that it?s not used anywhere in the hotspot itself. It is used in test_globals.cpp, but that tests the functionality itself, so as Erik points out without the feature the test is no needed. The test_collectorPolicy.cpp is the real issue here, which I missed, as I assumed the tests would be executed in fresh VM instances, but now I see that we should either: 1. Use ?TEST_OTHER_VM? for the tests that modify the global flags 2. Implement FlagGuard in the test itself. My main issue here is that we have code in hotspot, which is not used by hotspot (the product) itself, but only by one of its test. Furthermore, the code seems simple enough that it shouldn?t be too much trouble to provide an implementation of it in any test that needs it. Erik, as the original author, do you think #1 will work for test_collectorPolicy.cpp, or do we need #2? cheers > On May 17, 2018, at 2:04 AM, Erik Helin wrote: > > On 05/17/2018 12:06 AM, David Holmes wrote: >> Hi Gerard, >> On 17/05/2018 4:06 AM, Gerard Ziemski wrote: >>> Hi all, >>> >>> Please review this small fix where we remove unused FlagGuard class and FLAG_GUARD macro from hotspot and gtests. >> Well it seems only unused after you have removed its uses! Why does it turn out not to be needed in test_CollectorPolicy.cpp? > > I think it is still needed, I don't think this patch is correct. > > When we execute the gtests we execute the TEST_VM tests one-by-one in serial order in the same process. This means that if one tests changes a global flag, e.g. InitialHeapSize, the next test will see a different value for InitialHeapSize. In order to guard the flags, I introduced the class FlagGuard (and corresponding helper macro FLAG_GUARD). This means that a test, such as test_collectorPolicy.cpp, can easily guard any global flags the test intends to mutate, and then the flags will be restored to their original values when test has finished. > >> Why is the whole test_globals.cpp no longer relevant? > > test_globals.cpp currently only tests the FlagGuard class (and FLAG_GUARD) macro in globals.hpp, so since Gerard removed the FlagGuard class, the tests are no longer relevant. However, see my comment above, we can't just remove the FlagGuard class, it is needed! > >> The bug report should explain why we don't need these. > > Yes, I would also like to see an explanation why this code can be removed? To me it seems like it is needed in test_collectorPolicy.cpp, but I could be wrong. > > Thanks, > Erik > >> Thanks, >> David >>> https://bugs.openjdk.java.net/browse/JDK-8200746 >>> http://cr.openjdk.java.net/~gziemski/8200746_rev1/ >>> >>> Testing: mach5 hs-tier1,2 >>> >>> >>> cheers >>> From gerard.ziemski at oracle.com Thu May 17 16:06:02 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Thu, 17 May 2018 11:06:02 -0500 Subject: RFR(S) 8200746: Remove FlagGuard class In-Reply-To: <0569e9ac-0b60-fe69-d672-b1cbe070e473@oracle.com> References: <24EC2A98-BDFA-4788-A306-1432C91C4225@oracle.com> <22a40b8e-55cd-b035-82ea-62d88711357d@oracle.com> <0569e9ac-0b60-fe69-d672-b1cbe070e473@oracle.com> Message-ID: <16933A6C-D62C-42D3-A1DA-56FF117EB38D@oracle.com> Thank you Harold! Looks though like I?ll need to put up a webrev2 shortly though? cheers > On May 16, 2018, at 2:04 PM, Harold David Seigel wrote: > > Looks good to me, also. > > Harold > > > On 5/16/2018 2:12 PM, coleen.phillimore at oracle.com wrote: >> Looks good! Thank you for cleaning it up. >> Coleen >> >> On 5/16/18 2:06 PM, Gerard Ziemski wrote: >>> Hi all, >>> >>> Please review this small fix where we remove unused FlagGuard class and FLAG_GUARD macro from hotspot and gtests. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8200746 >>> http://cr.openjdk.java.net/~gziemski/8200746_rev1/ >>> >>> Testing: mach5 hs-tier1,2 >>> >>> >>> cheers >> > From gerard.ziemski at oracle.com Thu May 17 16:21:43 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Thu, 17 May 2018 11:21:43 -0500 Subject: RFR (XS) 8202360: [TESTBUG] runtime/LoadClass/TestResize.java needs to print output when it fails In-Reply-To: References: Message-ID: hi David, Thank you for the review! > On May 16, 2018, at 4:54 PM, David Holmes wrote: > > Hi Gerard, > > On 17/05/2018 3:47 AM, Gerard Ziemski wrote: >> Hi all, >> Please review this small enhancement where we print out additional information (i.e. output from PrintSystemDictionaryAtExit), when we detect the failure. >> https://bugs.openjdk.java.net/browse/JDK-8202360 >> http://cr.openjdk.java.net/~gziemski/8202360_rev1/ > > I don't understand what your fix is doing - sorry. The fix takes all the available output bytes from the process (using Scanner to grab all tokens at once) and prints all the output from the process (i.e. PrintSystemDictionaryAtExit, and there is a lot of it) at failure. > > What I wanted to see was the complete history of lines of the form > > "Java dictionary (table_size=107, classes=6)" > > so that we can see how the table size and number of classes have changed leading up to the failure. But I realize now that we already get that from the: > > System.out.println("PASS table_size:"+table_size+", classes:"+classes+" OK"); > > All that is missing is the printing of the table_size and classes when we throw the exception. That can be trivially fixed: > > if (loadFactor > MAX_LOAD_FACTOR) { > + System.out.println("FAIL table_size:"+table_size+", classes:"+classes); > throw new RuntimeException("Load factor too high, expected MAX "+MAX_LOAD_FACTOR+", got "+loadFactor); > } else { > System.out.println("PASS table_size:"+table_size+", classes:"+classes+" OK"); > } OK, so you determined that only that one piece of info was missing in the end in your case, but maybe the next person might want to see everything that PrintSystemDictionaryAtExit provides? As long as we?re touching the test - should we then stick with the solution that prints out everything from PrintSystemDictionaryAtExit? cheers From coleen.phillimore at oracle.com Thu May 17 17:29:34 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 17 May 2018 13:29:34 -0400 Subject: RFR(S) 8200746: Remove FlagGuard class In-Reply-To: <85AC5FE4-C4A4-4519-8032-2C5469E00F0A@oracle.com> References: <24EC2A98-BDFA-4788-A306-1432C91C4225@oracle.com> <2c87ed39-1406-7d0c-c694-70a4e21a1816@oracle.com> <85AC5FE4-C4A4-4519-8032-2C5469E00F0A@oracle.com> Message-ID: Or we could just close this as WNF, since it seems useful in test_collectorPolicy.cpp, and not just to test the FlagGuard class. Coleen On 5/17/18 11:53 AM, Gerard Ziemski wrote: > Thank you Erik, David for the review. > > My motivation for removing FlagGuard is that it?s not used anywhere in the hotspot itself. > > It is used in test_globals.cpp, but that tests the functionality itself, so as Erik points out without the feature the test is no needed. > > The test_collectorPolicy.cpp is the real issue here, which I missed, as I assumed the tests would be executed in fresh VM instances, but now I see that we should either: > > 1. Use ?TEST_OTHER_VM? for the tests that modify the global flags > 2. Implement FlagGuard in the test itself. > > My main issue here is that we have code in hotspot, which is not used by hotspot (the product) itself, but only by one of its test. Furthermore, the code seems simple enough that it shouldn?t be too much trouble to provide an implementation of it in any test that needs it. > > Erik, as the original author, do you think #1 will work for test_collectorPolicy.cpp, or do we need #2? > > > cheers > > >> On May 17, 2018, at 2:04 AM, Erik Helin wrote: >> >> On 05/17/2018 12:06 AM, David Holmes wrote: >>> Hi Gerard, >>> On 17/05/2018 4:06 AM, Gerard Ziemski wrote: >>>> Hi all, >>>> >>>> Please review this small fix where we remove unused FlagGuard class and FLAG_GUARD macro from hotspot and gtests. >>> Well it seems only unused after you have removed its uses! Why does it turn out not to be needed in test_CollectorPolicy.cpp? >> I think it is still needed, I don't think this patch is correct. >> >> When we execute the gtests we execute the TEST_VM tests one-by-one in serial order in the same process. This means that if one tests changes a global flag, e.g. InitialHeapSize, the next test will see a different value for InitialHeapSize. In order to guard the flags, I introduced the class FlagGuard (and corresponding helper macro FLAG_GUARD). This means that a test, such as test_collectorPolicy.cpp, can easily guard any global flags the test intends to mutate, and then the flags will be restored to their original values when test has finished. >> >>> Why is the whole test_globals.cpp no longer relevant? >> test_globals.cpp currently only tests the FlagGuard class (and FLAG_GUARD) macro in globals.hpp, so since Gerard removed the FlagGuard class, the tests are no longer relevant. However, see my comment above, we can't just remove the FlagGuard class, it is needed! >> >>> The bug report should explain why we don't need these. >> Yes, I would also like to see an explanation why this code can be removed? To me it seems like it is needed in test_collectorPolicy.cpp, but I could be wrong. >> >> Thanks, >> Erik >> >>> Thanks, >>> David >>>> https://bugs.openjdk.java.net/browse/JDK-8200746 >>>> http://cr.openjdk.java.net/~gziemski/8200746_rev1/ >>>> >>>> Testing: mach5 hs-tier1,2 >>>> >>>> >>>> cheers >>>> From gerard.ziemski at oracle.com Thu May 17 17:42:38 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Thu, 17 May 2018 12:42:38 -0500 Subject: RFR(S) 8200746: Remove FlagGuard class In-Reply-To: References: <24EC2A98-BDFA-4788-A306-1432C91C4225@oracle.com> <2c87ed39-1406-7d0c-c694-70a4e21a1816@oracle.com> <85AC5FE4-C4A4-4519-8032-2C5469E00F0A@oracle.com> Message-ID: <20F1DCD7-B623-4664-8CE9-4EE83927B09E@oracle.com> But the part where FlagGuard is not used by the hotspot itself is not addressed. Is it OK to have code in hotspot that is only used by a test? Especially if the code seems simple and used only by 1 test? > On May 17, 2018, at 12:29 PM, coleen.phillimore at oracle.com wrote: > > > Or we could just close this as WNF, since it seems useful in test_collectorPolicy.cpp, and not just to test the FlagGuard class. > > Coleen > > On 5/17/18 11:53 AM, Gerard Ziemski wrote: >> Thank you Erik, David for the review. >> >> My motivation for removing FlagGuard is that it?s not used anywhere in the hotspot itself. >> >> It is used in test_globals.cpp, but that tests the functionality itself, so as Erik points out without the feature the test is no needed. >> >> The test_collectorPolicy.cpp is the real issue here, which I missed, as I assumed the tests would be executed in fresh VM instances, but now I see that we should either: >> >> 1. Use ?TEST_OTHER_VM? for the tests that modify the global flags >> 2. Implement FlagGuard in the test itself. >> >> My main issue here is that we have code in hotspot, which is not used by hotspot (the product) itself, but only by one of its test. Furthermore, the code seems simple enough that it shouldn?t be too much trouble to provide an implementation of it in any test that needs it. >> >> Erik, as the original author, do you think #1 will work for test_collectorPolicy.cpp, or do we need #2? >> >> >> cheers >> >> >>> On May 17, 2018, at 2:04 AM, Erik Helin wrote: >>> >>> On 05/17/2018 12:06 AM, David Holmes wrote: >>>> Hi Gerard, >>>> On 17/05/2018 4:06 AM, Gerard Ziemski wrote: >>>>> Hi all, >>>>> >>>>> Please review this small fix where we remove unused FlagGuard class and FLAG_GUARD macro from hotspot and gtests. >>>> Well it seems only unused after you have removed its uses! Why does it turn out not to be needed in test_CollectorPolicy.cpp? >>> I think it is still needed, I don't think this patch is correct. >>> >>> When we execute the gtests we execute the TEST_VM tests one-by-one in serial order in the same process. This means that if one tests changes a global flag, e.g. InitialHeapSize, the next test will see a different value for InitialHeapSize. In order to guard the flags, I introduced the class FlagGuard (and corresponding helper macro FLAG_GUARD). This means that a test, such as test_collectorPolicy.cpp, can easily guard any global flags the test intends to mutate, and then the flags will be restored to their original values when test has finished. >>> >>>> Why is the whole test_globals.cpp no longer relevant? >>> test_globals.cpp currently only tests the FlagGuard class (and FLAG_GUARD) macro in globals.hpp, so since Gerard removed the FlagGuard class, the tests are no longer relevant. However, see my comment above, we can't just remove the FlagGuard class, it is needed! >>> >>>> The bug report should explain why we don't need these. >>> Yes, I would also like to see an explanation why this code can be removed? To me it seems like it is needed in test_collectorPolicy.cpp, but I could be wrong. >>> >>> Thanks, >>> Erik >>> >>>> Thanks, >>>> David >>>>> https://bugs.openjdk.java.net/browse/JDK-8200746 >>>>> http://cr.openjdk.java.net/~gziemski/8200746_rev1/ >>>>> >>>>> Testing: mach5 hs-tier1,2 >>>>> >>>>> >>>>> cheers >>>>> > From ioi.lam at oracle.com Thu May 17 18:52:00 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 17 May 2018 11:52:00 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: <5C996E41-CDA0-4EDA-B289-CDD7C38151B9@oracle.com> References: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> <5AF62823.7050804@oracle.com> <70F6AEB0-9103-4899-87D9-DC845BF66C51@oracle.com> <4CBA8376-54A2-4863-8732-B16E1AE9C38A@oracle.com> <5C996E41-CDA0-4EDA-B289-CDD7C38151B9@oracle.com> Message-ID: <6b94fc12-61ae-e328-59b1-40096544cbc1@oracle.com> Hi Jiangli, The changes look good. Just some minor nits. No need for new webrev if you decide to make these changes. ????? // The first entry in boot path is the modules_image (guaranteed by ????? // ClassLoader::setup_boot_search_path()). Skip the first entry. The ????? // path of the runtime modules_image may be different from the dump ????? // time path (e.g. the JDK image is copied to a different location ????? // after generating the shared archive), which is acceptable. For most ????? // common cases, the dump time boot path might contain modules_image only. Perhaps add an assert that #0 of the bootclasspath is the runtime module image? ?444?????? isSigned = stream->check_is_signed(); ?445?????? if (isSigned) { Maybe remove isSigned variable and change to if (stream->check_is_signed()) .... Thanks - Ioi On 5/16/18 8:27 PM, Jiangli Zhou wrote: > Here is the updated webrev: > http://cr.openjdk.java.net/~jiangli/8199807/webrev.02/ > > At CDS dump time, when SharedClassPathEntry::init() is called from FileMapInfo::allocate_shared_path_table(), we can identify the modules image entry directly. SharedClassPathEntry::init() now takes an extra bool argument to indicate if the current entry is the modules image. We no longer need to test the entry name using ClassLoader::is_modules_image(), which might be wrong for a shared path entry as Ioi pointed out. > > I added a ?_type? field in SharedClassPathEntry to record the type of shared path entries at dump time. At runtime, the path entry ?_type? is checked to identify the modules image in SharedClassPathEntry::validate(). SharedClassPathEntry::is_modules_image() no longer need to use ClassLoader::is_modules_image(). > > Reworked the BOOT_PATH case in SharedPathsMiscInfo::check(). I left os::file_name_strcmp() unchanged since it?s also used by APP_PATH. I'll handle it in a separate REF to do more testing. > > I also included the MoveJDKTest.java written by Ioi in the updated webrev. In the test, I added a new case for a JAR entry named with Hello.modules. > > Reran all appcds tests including MoveJDKTest.java locally. All tests passed. Running tiered tests. > > Thanks, > Jiangli > >> On May 15, 2018, at 6:10 PM, Jiangli Zhou wrote: >> >> Hi Ioi, >> >> Thank you very much for the review. Good catches for those edge cases and thanks for the test cases. I will resolves them. >> >> Thanks! >> Jiangli >> >>> On May 15, 2018, at 4:33 PM, Ioi Lam wrote: >>> >>> Hi Jiangli, >>> >>> 245 // We need to validate the runtime modules image file size against the archived >>> 246 // size information, obtain the runtime modules image path. The recorded dump >>> 247 // time modules image path in the archive may be different from the runtime path >>> 248 // if the JDK image has beed moved after generating the archive. >>> 249 if (ClassLoader::is_modules_image(name)) { >>> 250 name = ClassLoader::get_jrt_entry()->name(); >>> 251 } >>> >>> What happens if someone has a JAR file called foo.modules, and dumped with >>> >>> java -Xshare:dump -cp foo.modules >>> >>> >>> I think it's better to check that it's #0 in the classpath so we know for sure it's the system modules file. >>> >>> There's a similar problem here: >>> >>> 220 if (!ClassLoader::is_modules_image(name)) { >>> >>> In the following, instead of doing a copy, maybe it's better to add a length argument for os::file_name_strcmp, since sharedPathsMiscInfo.cpp is the only place in the VM that calls this function. >>> >>> 162 if (relaxed_check) { >>> 163 // only check the begining portion of the runtime boot path, up to >>> 164 // the length of the dump time boot path >>> 165 size_t len = strlen(path); >>> 166 runtime_boot_path = NEW_RESOURCE_ARRAY(char, len + 1); >>> 167 strncpy(runtime_boot_path, Arguments::get_sysclasspath(), len); >>> 168 runtime_boot_path[len] = '\0'; >>> 169 } else { >>> 170 // check the full runtime boot path >>> 171 runtime_boot_path = Arguments::get_sysclasspath(); >>> 172 } >>> 173 >>> 174 // Do a quick check first with a simple >>> 175 // strcmp(dump_time_boot_path, runtime_boot_path). If the paths are the >>> 176 // same, the check has succeeded. >>> 177 if (os::file_name_strcmp(path, runtime_boot_path) == 0) { >>> 178 break; // ok >>> 179 } >>> >>> >>> Also, the copy could be wrong for the following output: >>> >>> path = /tmp/foo/modules >>> Arguments::get_sysclasspath() = /tmp/foo/modulesxyz:/aaa:/bbb >>> >>> What is runtime_boot_path supposed to contain here? I am not sure if this code gives the intended output or not, but it's not easy to understand what it actually does by operating on an truncated pathname. >>> >>> String manipulation code is always hard to read. I would suggest braking it up into smaller functions: >>> >>> 1. Get the first entry of dump time and run time boot path -> d0 and r0 >>> -> ignore d0 >>> -> r0 same as ClassLoader::get_jrt_entry()->name(); >>> >>> 2. Get the remaining part of the dump time and run time boot path -> d_remain, r_remain >>> -> relaxed_check: d_remain must be a prefix of r_remain >>> -> !relaxed_check: They must be identical. >>> >>> >>> By the way, I think the test case in JDK-8202935 should be reviewed together inside this RFR, since the test validates the feature implemented here. I don't think we need a separate bug ID. Otherwise it will be hard to track the test cases. >>> >>> >>> Thanks >>> - Ioi >>> >>> >>> On 5/13/18 5:31 PM, Jiangli Zhou wrote: >>>> Here is the updated webrev: >>>> http://cr.openjdk.java.net/~jiangli/8199807/webrev.01/ >>>> >>>> Thanks, >>>> Jiangli >>>> >>>>> On May 11, 2018, at 6:07 PM, Jiangli Zhou wrote: >>>>> >>>>> Hi Calvin, >>>>> >>>>> Thanks for the review. >>>>> >>>>>> On May 11, 2018, at 4:32 PM, Calvin Cheung wrote: >>>>>> >>>>>> Hi Jiangli, >>>>>> >>>>>> Thanks for doing this useful change. >>>>>> >>>>>> I have a few minor comments: >>>>>> >>>>>> 1) sharedPathsMiscInfo.cpp >>>>>> >>>>>> 199 dp ++; >>>>>> >>>>>> Should the above be similar to line 194 as follows? >>>>>> dp += path_sep_len; >>>>> Good catch. Will fix. >>>>> >>>>>> 2) filemap.cpp >>>>>> >>>>>> 244 if (ClassLoader::is_modules_image(name)) { >>>>>> >>>>>> I think the above could be: >>>>>> if (is_modules_image()) { >>>>>> >>>>> Let?s keep it this way as is_modules_image() is a wrapper of 'ClassLoader::is_modules_image(name())?. We already have the ?name? in this case. >>>>> >>>>>> The is_modules_image() is a member function of SharedClassPathEntry. >>>>>> >>>>>> Btw, why is it necessary to get the runtime modules image path again in lines 244 - 246? >>>>> As the runtime modules image path could be different from the dump time if the JDK image is moved/copied after archive generation, we need to make sure to use the correct file for file size check. The recored path in the archive is the dump time modules image path, which may no longer be existing at runtime. I will add some comments if that?s helpful. >>>>> >>>>>> 3) BootClassPathMismatch.java >>>>>> >>>>>> Could you move line 103 to right above line 107 (opts.addPrefix(?))? >>>>>> Because the archive dumping part of the test doesn?t use appJar. >>>>> Sure. >>>>> >>>>> Thanks! >>>>> >>>>> Jiangli >>>>> >>>>>> thanks, >>>>>> Calvin >>>>>> >>>>>> On 5/11/18, 3:15 PM, Jiangli Zhou wrote: >>>>>>> I?ve withdrawn my original weberv and removed the build change. Here is the updated webrev that only addresses 8199807. >>>>>>> >>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.00/ >>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>> >>>>>>> Thanks, >>>>>>> Jiangli >>>>>>> >>>>>>>> On May 11, 2018, at 2:44 PM, Jiangli Zhou wrote: >>>>>>>> >>>>>>>> Hi Erik, >>>>>>>> >>>>>>>> Thank you so much for investigating a proper fix for the vm_version.o issue. I will withdraw the build change from my original webrev. >>>>>>>> >>>>>>>> Thanks again for taking over the bug! >>>>>>>> Jiangli >>>>>>>> >>>>>>>>> On May 11, 2018, at 2:33 PM, Erik Joelsson wrote: >>>>>>>>> >>>>>>>>> Received: from [10.132.185.167] (/10.132.185.167) >>>>>>>>> by default (Oracle Beehive Gateway v4.0) >>>>>>>>> with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 >>>>>>>>> Subject: Re: RFR: 8199807& 8202738: AppCDS performs overly restrictive path >>>>>>>>> matching check >>>>>>>>> To: Jiangli Zhou, >>>>>>>>> "hotspot-runtime-dev at openjdk.java.net runtime" >>>>>>>>> , >>>>>>>>> build-dev >>>>>>>>> References: >>>>>>>>> From: Erik Joelsson >>>>>>>>> Organization: Oracle Corporation >>>>>>>>> Message-ID: >>>>>>>>> Date: Fri, 11 May 2018 14:33:20 -0700 >>>>>>>>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) >>>>>>>>> Gecko/20100101 Thunderbird/52.7.0 >>>>>>>>> MIME-Version: 1.0 >>>>>>>>> In-Reply-To: >>>>>>>>> Content-Type: text/plain; charset=utf-8; format=flowed >>>>>>>>> Content-Transfer-Encoding: 8bit >>>>>>>>> Content-Language: en-US >>>>>>>>> >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>>>> >>>>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>>>> >>>>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>>>> >>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>>>> >>>>>>>>> /Erik >>>>>>>>> >>>>>>>>> >>>>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>>>> >>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>>>> >>>>>>>>>> The webrev includes the following three main parts: >>>>>>>>>> >>>>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>>>> >>>>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>>>> >>>>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>>>> >>>>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Jiangli >>>>>>>>>> >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>>>> >>>>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>>>> >>>>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>>>> >>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>>>> >>>>>>>>> /Erik >>>>>>>>> >>>>>>>>> >>>>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>>>> >>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>>>> >>>>>>>>>> The webrev includes the following three main parts: >>>>>>>>>> >>>>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>>>> >>>>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>>>> >>>>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>>>> >>>>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Jiangli From jiangli.zhou at oracle.com Thu May 17 18:55:37 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Thu, 17 May 2018 11:55:37 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: <6b94fc12-61ae-e328-59b1-40096544cbc1@oracle.com> References: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> <5AF62823.7050804@oracle.com> <70F6AEB0-9103-4899-87D9-DC845BF66C51@oracle.com> <4CBA8376-54A2-4863-8732-B16E1AE9C38A@oracle.com> <5C996E41-CDA0-4EDA-B289-CDD7C38151B9@oracle.com> <6b94fc12-61ae-e328-59b1-40096544cbc1@oracle.com> Message-ID: <9A7035A6-892E-45E6-8F1A-65D6A344D78C@oracle.com> Thanks, Ioi! Will incorporate your suggestions below before integration. Jiangli > On May 17, 2018, at 11:52 AM, Ioi Lam wrote: > > Hi Jiangli, > > The changes look good. Just some minor nits. No need for new webrev if you decide to make these changes. > > // The first entry in boot path is the modules_image (guaranteed by > > // ClassLoader::setup_boot_search_path()). Skip the first entry. The > // path of the runtime modules_image may be different from the dump > // time path (e.g. the JDK image is copied to a different location > // after generating the shared archive), which is acceptable. For most > // common cases, the dump time boot path might contain modules_image only. > > Perhaps add an assert that #0 of the bootclasspath is the runtime module image? > > > 444 isSigned = stream->check_is_signed(); > 445 if (isSigned) { > > Maybe remove isSigned variable and change to if (stream->check_is_signed()) .... > > Thanks > - Ioi > > > On 5/16/18 8:27 PM, Jiangli Zhou wrote: >> Here is the updated webrev: >> http://cr.openjdk.java.net/~jiangli/8199807/webrev.02/ >> >> At CDS dump time, when SharedClassPathEntry::init() is called from FileMapInfo::allocate_shared_path_table(), we can identify the modules image entry directly. SharedClassPathEntry::init() now takes an extra bool argument to indicate if the current entry is the modules image. We no longer need to test the entry name using ClassLoader::is_modules_image(), which might be wrong for a shared path entry as Ioi pointed out. >> >> I added a ?_type? field in SharedClassPathEntry to record the type of shared path entries at dump time. At runtime, the path entry ?_type? is checked to identify the modules image in SharedClassPathEntry::validate(). SharedClassPathEntry::is_modules_image() no longer need to use ClassLoader::is_modules_image(). >> >> Reworked the BOOT_PATH case in SharedPathsMiscInfo::check(). I left os::file_name_strcmp() unchanged since it?s also used by APP_PATH. I'll handle it in a separate REF to do more testing. >> >> I also included the MoveJDKTest.java written by Ioi in the updated webrev. In the test, I added a new case for a JAR entry named with Hello.modules. >> >> Reran all appcds tests including MoveJDKTest.java locally. All tests passed. Running tiered tests. >> >> Thanks, >> Jiangli >> >>> On May 15, 2018, at 6:10 PM, Jiangli Zhou wrote: >>> >>> Hi Ioi, >>> >>> Thank you very much for the review. Good catches for those edge cases and thanks for the test cases. I will resolves them. >>> >>> Thanks! >>> Jiangli >>> >>>> On May 15, 2018, at 4:33 PM, Ioi Lam wrote: >>>> >>>> Hi Jiangli, >>>> >>>> 245 // We need to validate the runtime modules image file size against the archived >>>> 246 // size information, obtain the runtime modules image path. The recorded dump >>>> 247 // time modules image path in the archive may be different from the runtime path >>>> 248 // if the JDK image has beed moved after generating the archive. >>>> 249 if (ClassLoader::is_modules_image(name)) { >>>> 250 name = ClassLoader::get_jrt_entry()->name(); >>>> 251 } >>>> >>>> What happens if someone has a JAR file called foo.modules, and dumped with >>>> >>>> java -Xshare:dump -cp foo.modules >>>> >>>> >>>> I think it's better to check that it's #0 in the classpath so we know for sure it's the system modules file. >>>> >>>> There's a similar problem here: >>>> >>>> 220 if (!ClassLoader::is_modules_image(name)) { >>>> >>>> In the following, instead of doing a copy, maybe it's better to add a length argument for os::file_name_strcmp, since sharedPathsMiscInfo.cpp is the only place in the VM that calls this function. >>>> >>>> 162 if (relaxed_check) { >>>> 163 // only check the begining portion of the runtime boot path, up to >>>> 164 // the length of the dump time boot path >>>> 165 size_t len = strlen(path); >>>> 166 runtime_boot_path = NEW_RESOURCE_ARRAY(char, len + 1); >>>> 167 strncpy(runtime_boot_path, Arguments::get_sysclasspath(), len); >>>> 168 runtime_boot_path[len] = '\0'; >>>> 169 } else { >>>> 170 // check the full runtime boot path >>>> 171 runtime_boot_path = Arguments::get_sysclasspath(); >>>> 172 } >>>> 173 >>>> 174 // Do a quick check first with a simple >>>> 175 // strcmp(dump_time_boot_path, runtime_boot_path). If the paths are the >>>> 176 // same, the check has succeeded. >>>> 177 if (os::file_name_strcmp(path, runtime_boot_path) == 0) { >>>> 178 break; // ok >>>> 179 } >>>> >>>> >>>> Also, the copy could be wrong for the following output: >>>> >>>> path = /tmp/foo/modules >>>> Arguments::get_sysclasspath() = /tmp/foo/modulesxyz:/aaa:/bbb >>>> >>>> What is runtime_boot_path supposed to contain here? I am not sure if this code gives the intended output or not, but it's not easy to understand what it actually does by operating on an truncated pathname. >>>> >>>> String manipulation code is always hard to read. I would suggest braking it up into smaller functions: >>>> >>>> 1. Get the first entry of dump time and run time boot path -> d0 and r0 >>>> -> ignore d0 >>>> -> r0 same as ClassLoader::get_jrt_entry()->name(); >>>> >>>> 2. Get the remaining part of the dump time and run time boot path -> d_remain, r_remain >>>> -> relaxed_check: d_remain must be a prefix of r_remain >>>> -> !relaxed_check: They must be identical. >>>> >>>> >>>> By the way, I think the test case in JDK-8202935 should be reviewed together inside this RFR, since the test validates the feature implemented here. I don't think we need a separate bug ID. Otherwise it will be hard to track the test cases. >>>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> On 5/13/18 5:31 PM, Jiangli Zhou wrote: >>>>> Here is the updated webrev: >>>>> http://cr.openjdk.java.net/~jiangli/8199807/webrev.01/ >>>>> >>>>> Thanks, >>>>> Jiangli >>>>> >>>>>> On May 11, 2018, at 6:07 PM, Jiangli Zhou wrote: >>>>>> >>>>>> Hi Calvin, >>>>>> >>>>>> Thanks for the review. >>>>>> >>>>>>> On May 11, 2018, at 4:32 PM, Calvin Cheung wrote: >>>>>>> >>>>>>> Hi Jiangli, >>>>>>> >>>>>>> Thanks for doing this useful change. >>>>>>> >>>>>>> I have a few minor comments: >>>>>>> >>>>>>> 1) sharedPathsMiscInfo.cpp >>>>>>> >>>>>>> 199 dp ++; >>>>>>> >>>>>>> Should the above be similar to line 194 as follows? >>>>>>> dp += path_sep_len; >>>>>> Good catch. Will fix. >>>>>> >>>>>>> 2) filemap.cpp >>>>>>> >>>>>>> 244 if (ClassLoader::is_modules_image(name)) { >>>>>>> >>>>>>> I think the above could be: >>>>>>> if (is_modules_image()) { >>>>>>> >>>>>> Let?s keep it this way as is_modules_image() is a wrapper of 'ClassLoader::is_modules_image(name())?. We already have the ?name? in this case. >>>>>> >>>>>>> The is_modules_image() is a member function of SharedClassPathEntry. >>>>>>> >>>>>>> Btw, why is it necessary to get the runtime modules image path again in lines 244 - 246? >>>>>> As the runtime modules image path could be different from the dump time if the JDK image is moved/copied after archive generation, we need to make sure to use the correct file for file size check. The recored path in the archive is the dump time modules image path, which may no longer be existing at runtime. I will add some comments if that?s helpful. >>>>>> >>>>>>> 3) BootClassPathMismatch.java >>>>>>> >>>>>>> Could you move line 103 to right above line 107 (opts.addPrefix(?))? >>>>>>> Because the archive dumping part of the test doesn?t use appJar. >>>>>> Sure. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> Jiangli >>>>>> >>>>>>> thanks, >>>>>>> Calvin >>>>>>> >>>>>>> On 5/11/18, 3:15 PM, Jiangli Zhou wrote: >>>>>>>> I?ve withdrawn my original weberv and removed the build change. Here is the updated webrev that only addresses 8199807. >>>>>>>> >>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.00/ >>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jiangli >>>>>>>> >>>>>>>>> On May 11, 2018, at 2:44 PM, Jiangli Zhou wrote: >>>>>>>>> >>>>>>>>> Hi Erik, >>>>>>>>> >>>>>>>>> Thank you so much for investigating a proper fix for the vm_version.o issue. I will withdraw the build change from my original webrev. >>>>>>>>> >>>>>>>>> Thanks again for taking over the bug! >>>>>>>>> Jiangli >>>>>>>>> >>>>>>>>>> On May 11, 2018, at 2:33 PM, Erik Joelsson wrote: >>>>>>>>>> >>>>>>>>>> Received: from [10.132.185.167] (/10.132.185.167) >>>>>>>>>> by default (Oracle Beehive Gateway v4.0) >>>>>>>>>> with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 >>>>>>>>>> Subject: Re: RFR: 8199807& 8202738: AppCDS performs overly restrictive path >>>>>>>>>> matching check >>>>>>>>>> To: Jiangli Zhou, >>>>>>>>>> "hotspot-runtime-dev at openjdk.java.net runtime" >>>>>>>>>> , >>>>>>>>>> build-dev >>>>>>>>>> References: >>>>>>>>>> From: Erik Joelsson >>>>>>>>>> Organization: Oracle Corporation >>>>>>>>>> Message-ID: >>>>>>>>>> Date: Fri, 11 May 2018 14:33:20 -0700 >>>>>>>>>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) >>>>>>>>>> Gecko/20100101 Thunderbird/52.7.0 >>>>>>>>>> MIME-Version: 1.0 >>>>>>>>>> In-Reply-To: >>>>>>>>>> Content-Type: text/plain; charset=utf-8; format=flowed >>>>>>>>>> Content-Transfer-Encoding: 8bit >>>>>>>>>> Content-Language: en-US >>>>>>>>>> >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>>>>> >>>>>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>>>>> >>>>>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>>>>> >>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>>>>> >>>>>>>>>> /Erik >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>>>>> >>>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>>>>> >>>>>>>>>>> The webrev includes the following three main parts: >>>>>>>>>>> >>>>>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>>>>> >>>>>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>>>>> >>>>>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>>>>> >>>>>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Jiangli >>>>>>>>>>> >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>>>>> >>>>>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>>>>> >>>>>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>>>>> >>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>>>>> >>>>>>>>>> /Erik >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>>>>> >>>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>>>>> >>>>>>>>>>> The webrev includes the following three main parts: >>>>>>>>>>> >>>>>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>>>>> >>>>>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>>>>> >>>>>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>>>>> >>>>>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Jiangli > From coleen.phillimore at oracle.com Thu May 17 19:15:16 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 17 May 2018 15:15:16 -0400 Subject: RFR(S) 8200746: Remove FlagGuard class In-Reply-To: <20F1DCD7-B623-4664-8CE9-4EE83927B09E@oracle.com> References: <24EC2A98-BDFA-4788-A306-1432C91C4225@oracle.com> <2c87ed39-1406-7d0c-c694-70a4e21a1816@oracle.com> <85AC5FE4-C4A4-4519-8032-2C5469E00F0A@oracle.com> <20F1DCD7-B623-4664-8CE9-4EE83927B09E@oracle.com> Message-ID: <06a5b420-2fc9-9bc2-e0b6-814cf6c03465@oracle.com> On 5/17/18 1:42 PM, Gerard Ziemski wrote: > But the part where FlagGuard is not used by the hotspot itself is not addressed. > > Is it OK to have code in hotspot that is only used by a test? Especially if the code seems simple and used only by 1 test? Yes, since it's needed by the test, it might as well be in hotspot since it's simple. Thanks, Coleen > > >> On May 17, 2018, at 12:29 PM, coleen.phillimore at oracle.com wrote: >> >> >> Or we could just close this as WNF, since it seems useful in test_collectorPolicy.cpp, and not just to test the FlagGuard class. >> >> Coleen >> >> On 5/17/18 11:53 AM, Gerard Ziemski wrote: >>> Thank you Erik, David for the review. >>> >>> My motivation for removing FlagGuard is that it?s not used anywhere in the hotspot itself. >>> >>> It is used in test_globals.cpp, but that tests the functionality itself, so as Erik points out without the feature the test is no needed. >>> >>> The test_collectorPolicy.cpp is the real issue here, which I missed, as I assumed the tests would be executed in fresh VM instances, but now I see that we should either: >>> >>> 1. Use ?TEST_OTHER_VM? for the tests that modify the global flags >>> 2. Implement FlagGuard in the test itself. >>> >>> My main issue here is that we have code in hotspot, which is not used by hotspot (the product) itself, but only by one of its test. Furthermore, the code seems simple enough that it shouldn?t be too much trouble to provide an implementation of it in any test that needs it. >>> >>> Erik, as the original author, do you think #1 will work for test_collectorPolicy.cpp, or do we need #2? >>> >>> >>> cheers >>> >>> >>>> On May 17, 2018, at 2:04 AM, Erik Helin wrote: >>>> >>>> On 05/17/2018 12:06 AM, David Holmes wrote: >>>>> Hi Gerard, >>>>> On 17/05/2018 4:06 AM, Gerard Ziemski wrote: >>>>>> Hi all, >>>>>> >>>>>> Please review this small fix where we remove unused FlagGuard class and FLAG_GUARD macro from hotspot and gtests. >>>>> Well it seems only unused after you have removed its uses! Why does it turn out not to be needed in test_CollectorPolicy.cpp? >>>> I think it is still needed, I don't think this patch is correct. >>>> >>>> When we execute the gtests we execute the TEST_VM tests one-by-one in serial order in the same process. This means that if one tests changes a global flag, e.g. InitialHeapSize, the next test will see a different value for InitialHeapSize. In order to guard the flags, I introduced the class FlagGuard (and corresponding helper macro FLAG_GUARD). This means that a test, such as test_collectorPolicy.cpp, can easily guard any global flags the test intends to mutate, and then the flags will be restored to their original values when test has finished. >>>> >>>>> Why is the whole test_globals.cpp no longer relevant? >>>> test_globals.cpp currently only tests the FlagGuard class (and FLAG_GUARD) macro in globals.hpp, so since Gerard removed the FlagGuard class, the tests are no longer relevant. However, see my comment above, we can't just remove the FlagGuard class, it is needed! >>>> >>>>> The bug report should explain why we don't need these. >>>> Yes, I would also like to see an explanation why this code can be removed? To me it seems like it is needed in test_collectorPolicy.cpp, but I could be wrong. >>>> >>>> Thanks, >>>> Erik >>>> >>>>> Thanks, >>>>> David >>>>>> https://bugs.openjdk.java.net/browse/JDK-8200746 >>>>>> http://cr.openjdk.java.net/~gziemski/8200746_rev1/ >>>>>> >>>>>> Testing: mach5 hs-tier1,2 >>>>>> >>>>>> >>>>>> cheers >>>>>> From coleen.phillimore at oracle.com Thu May 17 19:25:22 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 17 May 2018 15:25:22 -0400 Subject: RFR (XS) 8202014: Possible to receive signal before signal semaphore created In-Reply-To: References: <3d3ea98c-60c9-71de-f0c7-e156207b24ca@oracle.com> <3e24a240-9e6b-65a0-64ca-c81db396f6cb@oracle.com> Message-ID: On 5/16/18 2:04 AM, Thomas St?fe wrote: > Hi, > > I agree with David. Signal handlers are installed in os::init_2(), but > the semaphore sometime later in the (imho misleadingly named) void > os::signal_init(). This looks like an initialization error. > > As a side note, can we make the "void signalHandler(int sig, > siginfo_t* info, void* uc)" functions in os_xxx.cpp static, since they > are only file local marshalling points? Okay, I've moved the initialization of the signal semaphore to os::init_2().?? The signal handler for this signal_notify() case is set up by JVM_RegisterSignal early in JDK initialization (initPhase1) which is before os::signal_init() was run.? Now the semaphore is initialized before that. I also made signalHandler static in this change although not strictly related. http://cr.openjdk.java.net/~coleenp/8202014.02/webrev/index.html Reran kill -INT script and mach5 hs_tiers 1 and 2. test/hotspot/jtreg/runtime/signal tests and test/jdk/sun/misc/SunMiscSignalTest.java. Thanks, Coleen > > Kind Regards, Thomas > > > > > On Wed, May 16, 2018 at 4:40 AM, David Holmes wrote: >> Hi Coleen, >> >> On 16/05/2018 11:23 AM, coleen.phillimore at oracle.com wrote: >>> Summary: Don't assert for null semaphore because it can be null before >>> initialization is complete. >> >> Sorry I don't agree with this. Surely we should not install our signal >> handler until such time as everything is properly initialized? Or the >> handler itself should have some kind of initialization check to guide how it >> responds. >> >> Thanks, >> David >> >> >>> Tested this with a shell script, thanks Robbin, and mach5 tier1,2 all >>> Oracle platforms. AIX doesn't use the semaphore class so doesn't have this >>> bug. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8202014.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8202014 >>> >>> Thanks, >>> Coleen From gerard.ziemski at oracle.com Thu May 17 19:45:38 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Thu, 17 May 2018 14:45:38 -0500 Subject: RFR(S) 8200746: Remove FlagGuard class In-Reply-To: <06a5b420-2fc9-9bc2-e0b6-814cf6c03465@oracle.com> References: <24EC2A98-BDFA-4788-A306-1432C91C4225@oracle.com> <2c87ed39-1406-7d0c-c694-70a4e21a1816@oracle.com> <85AC5FE4-C4A4-4519-8032-2C5469E00F0A@oracle.com> <20F1DCD7-B623-4664-8CE9-4EE83927B09E@oracle.com> <06a5b420-2fc9-9bc2-e0b6-814cf6c03465@oracle.com> Message-ID: <38B836B2-321D-449E-A7EE-87A237846BD2@oracle.com> > On 5/17/18 1:42 PM, Gerard Ziemski wrote: >> But the part where FlagGuard is not used by the hotspot itself is not addressed. >> >> Is it OK to have code in hotspot that is only used by a test? Especially if the code seems simple and used only by 1 test? > > Yes, since it's needed by the test, it might as well be in hotspot since it's simple. I respectfully disagree, but I will withdraw it. cheers > > Thanks, > Coleen >> >> >>> On May 17, 2018, at 12:29 PM, coleen.phillimore at oracle.com wrote: >>> >>> >>> Or we could just close this as WNF, since it seems useful in test_collectorPolicy.cpp, and not just to test the FlagGuard class. >>> >>> Coleen >>> >>> On 5/17/18 11:53 AM, Gerard Ziemski wrote: >>>> Thank you Erik, David for the review. >>>> >>>> My motivation for removing FlagGuard is that it?s not used anywhere in the hotspot itself. >>>> >>>> It is used in test_globals.cpp, but that tests the functionality itself, so as Erik points out without the feature the test is no needed. >>>> >>>> The test_collectorPolicy.cpp is the real issue here, which I missed, as I assumed the tests would be executed in fresh VM instances, but now I see that we should either: >>>> >>>> 1. Use ?TEST_OTHER_VM? for the tests that modify the global flags >>>> 2. Implement FlagGuard in the test itself. >>>> >>>> My main issue here is that we have code in hotspot, which is not used by hotspot (the product) itself, but only by one of its test. Furthermore, the code seems simple enough that it shouldn?t be too much trouble to provide an implementation of it in any test that needs it. >>>> >>>> Erik, as the original author, do you think #1 will work for test_collectorPolicy.cpp, or do we need #2? >>>> >>>> >>>> cheers >>>> >>>> >>>>> On May 17, 2018, at 2:04 AM, Erik Helin wrote: >>>>> >>>>> On 05/17/2018 12:06 AM, David Holmes wrote: >>>>>> Hi Gerard, >>>>>> On 17/05/2018 4:06 AM, Gerard Ziemski wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> Please review this small fix where we remove unused FlagGuard class and FLAG_GUARD macro from hotspot and gtests. >>>>>> Well it seems only unused after you have removed its uses! Why does it turn out not to be needed in test_CollectorPolicy.cpp? >>>>> I think it is still needed, I don't think this patch is correct. >>>>> >>>>> When we execute the gtests we execute the TEST_VM tests one-by-one in serial order in the same process. This means that if one tests changes a global flag, e.g. InitialHeapSize, the next test will see a different value for InitialHeapSize. In order to guard the flags, I introduced the class FlagGuard (and corresponding helper macro FLAG_GUARD). This means that a test, such as test_collectorPolicy.cpp, can easily guard any global flags the test intends to mutate, and then the flags will be restored to their original values when test has finished. >>>>> >>>>>> Why is the whole test_globals.cpp no longer relevant? >>>>> test_globals.cpp currently only tests the FlagGuard class (and FLAG_GUARD) macro in globals.hpp, so since Gerard removed the FlagGuard class, the tests are no longer relevant. However, see my comment above, we can't just remove the FlagGuard class, it is needed! >>>>> >>>>>> The bug report should explain why we don't need these. >>>>> Yes, I would also like to see an explanation why this code can be removed? To me it seems like it is needed in test_collectorPolicy.cpp, but I could be wrong. >>>>> >>>>> Thanks, >>>>> Erik >>>>> >>>>>> Thanks, >>>>>> David >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8200746 >>>>>>> http://cr.openjdk.java.net/~gziemski/8200746_rev1/ >>>>>>> >>>>>>> Testing: mach5 hs-tier1,2 >>>>>>> >>>>>>> >>>>>>> cheers >>>>>>> > From coleen.phillimore at oracle.com Thu May 17 20:26:45 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 17 May 2018 16:26:45 -0400 Subject: RFR (S) 8134537: Much nearly duplicated code for sun.misc.Signal support Message-ID: Summary: merged jvm_.cpp into jvm_posix.cpp mostly See bug for more info.? The os_.cpp code is sort of duplicated but different enough to not be worth refactoring, for this feature.? I was in the area, so merged this one part. Tested with hs-tier1,2 runtime/signal tests and sun/misc/SunMiscSignalTest.java. open webrev at http://cr.openjdk.java.net/~coleenp/8134537.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8134537 Thanks, Coleen From mikhailo.seledtsov at oracle.com Thu May 17 20:59:12 2018 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Thu, 17 May 2018 13:59:12 -0700 Subject: RFR(M): 8199252: [TESTBUG] Open source VM testbase system dictionary tests In-Reply-To: <1BB15086-EC2C-4620-B8FF-A0C35C0A60C2@oracle.com> References: <5AF606AF.3020404@oracle.com> <1BB15086-EC2C-4620-B8FF-A0C35C0A60C2@oracle.com> Message-ID: <5AFDED20.6060909@oracle.com> Thank you, Misha On 5/16/18, 2:35 PM, Igor Ignatyev wrote: > Hi Misha, > > looks good to me. > Thanks, > -- Igor > >> On May 11, 2018, at 2:10 PM, Mikhailo Seledtsov wrote: >> >> Please review this change open sourcing VM system dictionary tests. These tests have been used internally for a while, and are now being open sourced. >> Since this is not an creation of new tests, simply open sourcing existing tests, we would like to keep the changes during this review to an absolute required minimum. If you have any feedback on improvements of these tests, please file RFE(s) that will be addressed later in order of priority. >> >> >> Here is what was done for this change: >> 1. Moved the tests to OpenJDK repository preserving relative directory location and structure. >> 3. Updated Copyright statements accordingly. >> 4. Updated "@library" statements accordingly. >> 5. Updated TEST.groups >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8199252 >> Webrev: http://cr.openjdk.java.net/~mseledtsov/8199252.01.open/ >> >> Testing: >> 1. Ran the following tests on open-only repository and build, using "make run-test" >> (Linux-x64) >> vmTestbase_nsk_sysdict >> All PASS >> >> 2. Automated multip-platform test system (usual 4 platforms): >> - vmTestbase_nsk_sysdict >> - hs-tier{1,2} >> In progress >> >> Thank you, >> Misha >> From ioi.lam at oracle.com Fri May 18 01:13:03 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 17 May 2018 18:13:03 -0700 Subject: RFR(S) 8193332 MetaspaceShared::check_shared_class_loader_type is not used during archive creation Message-ID: <4c9edcca-6c94-4933-326b-b9cd88d7033f@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8193332 http://cr.openjdk.java.net/~iklam/jdk11/8193332-check_shared_class_loader_type.v01/ Summary: When we restructured the AppCDS code, the call to MetaspaceShared::check_shared_class_loader_type was somehow left out. I have restored it, and actually found a minor bug related to modules and -Xbootclasspath/a: In classLoader.cpp, at dump time we used to load a class from -Xbootclasspath/a even if this class belongs to a named packaged, which means it would not be loaded by the JVM at runtime. Such dump-time loading seems unnecessary so I removed it, and fixed a related test case to reflect the latest behavior. Tested with hs-tier1,2. Thanks - Ioi From david.holmes at oracle.com Fri May 18 04:37:42 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 18 May 2018 14:37:42 +1000 Subject: RFR (S) 8134537: Much nearly duplicated code for sun.misc.Signal support In-Reply-To: References: Message-ID: <6fd46541-7fb4-81eb-0ea1-36fa78e8525b@oracle.com> Hi Coleen, That all seems fine to me. Nice cleanup. (Plenty more to go ;-) ) Thanks, David On 18/05/2018 6:26 AM, coleen.phillimore at oracle.com wrote: > Summary: merged jvm_.cpp into jvm_posix.cpp mostly > > See bug for more info.? The os_.cpp code is sort of duplicated but > different enough to not be worth refactoring, for this feature.? I was > in the area, so merged this one part. > > Tested with hs-tier1,2 runtime/signal tests and > sun/misc/SunMiscSignalTest.java. > > open webrev at http://cr.openjdk.java.net/~coleenp/8134537.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8134537 > > Thanks, > Coleen From david.holmes at oracle.com Fri May 18 04:49:16 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 17 May 2018 21:49:16 -0700 (PDT) Subject: RFR (S) 8134537: Much nearly duplicated code for sun.misc.Signal support In-Reply-To: <6fd46541-7fb4-81eb-0ea1-36fa78e8525b@oracle.com> References: <6fd46541-7fb4-81eb-0ea1-36fa78e8525b@oracle.com> Message-ID: <8ae60fc9-32af-9481-2096-2d7c73920e68@oracle.com> One minor thing - only noticed because of your other change: src/hotspot/os/posix/jvm_posix.cpp 33 // sun.misc.Signal It isn't sun.misc.Signal anymore :) Thanks, David On 18/05/2018 2:37 PM, David Holmes wrote: > Hi Coleen, > > That all seems fine to me. Nice cleanup. (Plenty more to go ;-) ) > > Thanks, > David > > On 18/05/2018 6:26 AM, coleen.phillimore at oracle.com wrote: >> Summary: merged jvm_.cpp into jvm_posix.cpp mostly >> >> See bug for more info.? The os_.cpp code is sort of duplicated but >> different enough to not be worth refactoring, for this feature.? I was >> in the area, so merged this one part. >> >> Tested with hs-tier1,2 runtime/signal tests and >> sun/misc/SunMiscSignalTest.java. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8134537.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8134537 >> >> Thanks, >> Coleen From david.holmes at oracle.com Fri May 18 05:44:00 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 18 May 2018 15:44:00 +1000 Subject: RFR (XS) 8202014: Possible to receive signal before signal semaphore created In-Reply-To: References: <3d3ea98c-60c9-71de-f0c7-e156207b24ca@oracle.com> <3e24a240-9e6b-65a0-64ca-c81db396f6cb@oracle.com> Message-ID: <031035c7-55f2-23ad-f2e0-0f77c59bd6bf@oracle.com> Hi Coleen, On 18/05/2018 5:25 AM, coleen.phillimore at oracle.com wrote: > > > On 5/16/18 2:04 AM, Thomas St?fe wrote: >> Hi, >> >> I agree with David. Signal handlers are installed in os::init_2(), but >> the semaphore sometime later in the (imho misleadingly named) void >> os::signal_init(). This looks like an initialization error. >> >> As a side note, can we make the "void signalHandler(int sig, >> siginfo_t* info, void* uc)" functions in os_xxx.cpp static, since they >> are only file local marshalling points? > > Okay, I've moved the initialization of the signal semaphore to > os::init_2().?? The signal handler for this signal_notify() case is set > up by JVM_RegisterSignal early in JDK initialization (initPhase1) which > is before os::signal_init() was run.? Now the semaphore is initialized > before that. That seems fine. Thanks, David > I also made signalHandler static in this change although not strictly > related. > > http://cr.openjdk.java.net/~coleenp/8202014.02/webrev/index.html > > Reran kill -INT script and mach5 hs_tiers 1 and 2. > test/hotspot/jtreg/runtime/signal tests and > test/jdk/sun/misc/SunMiscSignalTest.java. > > Thanks, > Coleen > >> >> Kind Regards, Thomas >> >> >> >> >> On Wed, May 16, 2018 at 4:40 AM, David Holmes >> wrote: >>> Hi Coleen, >>> >>> On 16/05/2018 11:23 AM, coleen.phillimore at oracle.com wrote: >>>> Summary: Don't assert for null semaphore because it can be null before >>>> initialization is complete. >>> >>> Sorry I don't agree with this. Surely we should not install our signal >>> handler until such time as everything is properly initialized? Or the >>> handler itself should have some kind of initialization check to guide >>> how it >>> responds. >>> >>> Thanks, >>> David >>> >>> >>>> Tested this with a shell script, thanks Robbin, and mach5 tier1,2 all >>>> Oracle platforms.? AIX doesn't use the semaphore class so doesn't >>>> have this >>>> bug. >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8202014.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8202014 >>>> >>>> Thanks, >>>> Coleen > From goetz.lindenmaier at sap.com Fri May 18 05:45:07 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 18 May 2018 05:45:07 +0000 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: References: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> Message-ID: <1420518caee1469bb7ce92fbebf0b6a9@sap.com> Hi Matthias, The output looks good now. Reviewed. Could you still please break the lines of the st->print statements in the code? Just after the format string for example. I don't need a new webrev for that. Best regards, Goetz > -----Original Message----- > From: Baesken, Matthias > Sent: Thursday, May 17, 2018 5:20 PM > To: Thomas St?fe ; Lindenmaier, Goetz > > Cc: Doerr, Martin ; hotspot-dev at openjdk.java.net; > hotspot-runtime-dev at openjdk.java.net > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows > > > > > I also think that one number is enough. Adaptive numbers would be > great. > > > Hi Thomas/Goetz/Martin , thanks for looking into it. > Adaptive numbers could be done in a follow-up as suggested by Thomas , > this would also touch the other platforms . > > I added more line feeds to the output and reduced output to just printing > M . > > > Example output from a Windows desktop PC : > > Memory: 4k page, system-wide physical 32520M (20165M free) > TotalPageFile size 34568M (AvailPageFile size 18170M) > current process WorkingSet (physical memory assigned to process): 79M, > peak: 79M > current process commit charge ("private bytes"): 148M, peak: 148M > > > new webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.1/ > > > Best regards, Matthias > > > > > -----Original Message----- > > From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] > > Sent: Mittwoch, 16. Mai 2018 16:35 > > To: Lindenmaier, Goetz > > Cc: Baesken, Matthias ; Doerr, Martin > > ; hotspot-dev at openjdk.java.net; hotspot- > > runtime-dev at openjdk.java.net > > Subject: Re: [RFR] 8202427: Enhance os::print_memory_info on Windows > > > > On Wed, May 16, 2018 at 3:48 PM, Lindenmaier, Goetz > > wrote: > > > Hi Matthias, > > > > > > I appreciate the additional information in the hs_err file. > > > Could you please add an example output (of the final version) to the bug > > description? > > > > > > As Martin, I would like more line feeds, both in the code and the output. > > > Currently, the output in the hs_err file looks like this (where the first line > is > > too long): > > > Memory: 4k page, system-wide physical 16776692k [16383M] (8795032k > > [8588M] free), TotalPageFile size 49949460k [48778M] (AvailPageFile size > > 38942152k [38029M]), user-mode portion of virtual address-space 2097024k > > [2047M] (6940k [6M] free) > > > current process WorkingSet (physical memory assigned to process): > > 991804k [968M], peak: 991804k [968M] > > > > > > current process commit charge ("private bytes"): 1523632k [1487M], > > peak: 1523632k [1487M] > > > > > > I also think that one number is enough. Adaptive numbers would be > great. > > > I know about > > > src/hotspot/share/memory/metaspace/metaspaceCommon.hpp: > > print_human_readable_size() > > > for this, but this seems a bit hidden in metaspace. > > > > > > > Guys, lets not worry about human readable numbers for now. We can > move > > print_human_readable_size() out from metaspace to something generic > > and use it to improve this code (and others) in a later patch. That > > was my original intention when adding print_human_readable_size(). > > > > The patch is good. Thanks for doing this. > > > > ..Thoams > > > > > I don't like usage of _M_IX86. Common in this file seems #ifndef _WIN64 > > for 32-bit windows > > > dependent code. But don't care here. > > > > > > Best regards, > > > Goetz. > > > > > > > > >> -----Original Message----- > > >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] > On > > >> Behalf Of Baesken, Matthias > > >> Sent: Mittwoch, 16. Mai 2018 15:13 > > >> To: Doerr, Martin ; 'hotspot- > > >> dev at openjdk.java.net' ; hotspot- > > >> runtime-dev at openjdk.java.net > > >> Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on > Windows > > >> > > >> Ping : could I get a second review ? > > >> > > >> Bug : > > >> https://bugs.openjdk.java.net/browse/JDK-8202427 > > >> Change : > > >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > > >> > > >> > > >> > We could also just print the MB value , let's see what other think. > > >> > Another option might be to have a flexible output (kB for smaller > > >> memory > > >> > values , MB (or GB) for larger ) ? > > >> > > >> Martin suggested to just print the MB values, what do you think ? > > >> > > >> > > >> Thanks, Matthias > > >> > > >> > > >> > -----Original Message----- > > >> > From: Baesken, Matthias > > >> > Sent: Mittwoch, 2. Mai 2018 13:00 > > >> > To: Doerr, Martin ; 'hotspot- > > >> > dev at openjdk.java.net' ; hotspot- > > >> > runtime-dev at openjdk.java.net > > >> > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on > > Windows > > >> > > > >> > Hi Martin, thanks for your input . > > >> > I add hotspot-runtime-dev . > > >> > > > >> > > > > >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe > > other > > >> > > reviewers would like to comment on this, too. > > >> > > > > >> > > > >> > We could also just print the MB value , let's see what other think. > > >> > Another option might be to have a flexible output (kB for smaller > > >> memory > > >> > values , MB (or GB) for larger ) ? > > >> > > > >> > Best regards, Matthias > > >> > > > >> > > > >> > > -----Original Message----- > > >> > > From: Doerr, Martin > > >> > > Sent: Mittwoch, 2. Mai 2018 12:53 > > >> > > To: Baesken, Matthias ; 'hotspot- > > >> > > dev at openjdk.java.net' > > >> > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on > > Windows > > >> > > > > >> > > Hi Matthias, > > >> > > > > >> > > looks like a nice enhancement. We can get substantially more > > >> information. > > >> > > > > >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe > > other > > >> > > reviewers would like to comment on this, too. > > >> > > > > >> > > We should have line breaks. > > >> > > > > >> > > Best regards, > > >> > > Martin > > >> > > > > >> > > > > >> > > -----Original Message----- > > >> > > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] > > On > > >> > > Behalf Of Baesken, Matthias > > >> > > Sent: Montag, 30. April 2018 16:53 > > >> > > To: 'hotspot-dev at openjdk.java.net' > dev at openjdk.java.net> > > >> > > Subject: [RFR] 8202427: Enhance os::print_memory_info on > Windows > > >> > > > > >> > > On Windows, > > >> > > the os::print_memory_info misses a few memory-related infos > that > > >> might > > >> > > be interesting : > > >> > > - current and peak WorkingSet size (= physical memory assigned to > > the > > >> > > process) > > >> > > - current and peak commit charge (also known as "private bytes" in > > >> > Windows > > >> > > tools) > > >> > > - on 32bit Windows : > > >> > > user-mode portion/free user mode portion of virtual address-space > > >> > > (Total/AvailVirtual) because it shows how close do we get to the 2-4 > > GB > > >> per > > >> > > process border. > > >> > > > > >> > > > > >> > > - the current naming of "swap/free-swap" memory is a bit > misleading; > > >> > > in the Windows world swap is a special page file used for UWP apps. > > >> > > (see Windows Internals : "The swap file" (chapter 5 Memory > > >> > management) > > >> > > Windows 8 added another page file called a swap file. It is ... > > exclusively > > >> > used > > >> > > for UWP (Universal Windows Platform) apps. > > >> > > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the > > UWP > > >> > > related values, it is about page file sizes > > >> > > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just > > >> virtual > > >> > > memory might be more appropriate). > > >> > > > > >> > > > > >> > > https://msdn.microsoft.com/de- > > >> > > de/library/windows/desktop/aa366770(v=vs.85).aspx > > >> > > > > >> > > documents it in the following way: > > >> > > ullTotalPageFile: > > >> > > The current committed memory limit for the system or the current > > >> process, > > >> > > whichever is smaller,... > > >> > > > > >> > > ullAvailPageFile : > > >> > > The maximum amount of memory the current process can commit, > in > > >> > bytes. > > >> > > This value is equal to or smaller than the system-wide available > > commit > > >> > value > > >> > > > > >> > > > > >> > > > > >> > > Aditionally I suggest having output of the various memory-values in > M > > >> > > (megabyte) as well , the k (kilobyte) output sometimes gives very > > high > > >> and > > >> > > unreadable numbers). > > >> > > > > >> > > > > >> > > Could you please review my change ? > > >> > > > > >> > > > > >> > > Bug : > > >> > > > > >> > > https://bugs.openjdk.java.net/browse/JDK-8202427 > > >> > > > > >> > > > > >> > > Change : > > >> > > > > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > > >> > > > > >> > > > > >> > > Thanks, Matthias > > >> > > > > > From thomas.stuefe at gmail.com Fri May 18 05:49:14 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 18 May 2018 07:49:14 +0200 Subject: RFR (S) 8134537: Much nearly duplicated code for sun.misc.Signal support In-Reply-To: References: Message-ID: Hi Coleen, fix looks good. You are right, there are many more similarities, but too different to do it within limited time. Thanks, Thomas On Thu, May 17, 2018 at 10:26 PM, wrote: > Summary: merged jvm_.cpp into jvm_posix.cpp mostly > > See bug for more info. The os_.cpp code is sort of duplicated but > different enough to not be worth refactoring, for this feature. I was in > the area, so merged this one part. > > Tested with hs-tier1,2 runtime/signal tests and > sun/misc/SunMiscSignalTest.java. > > open webrev at http://cr.openjdk.java.net/~coleenp/8134537.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8134537 > > Thanks, > Coleen From david.holmes at oracle.com Fri May 18 05:59:14 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 18 May 2018 15:59:14 +1000 Subject: RFR (XS) 8202360: [TESTBUG] runtime/LoadClass/TestResize.java needs to print output when it fails In-Reply-To: References: Message-ID: Hi Gerard, On 18/05/2018 2:21 AM, Gerard Ziemski wrote: > hi David, > > Thank you for the review! > > >> On May 16, 2018, at 4:54 PM, David Holmes wrote: >> >> Hi Gerard, >> >> On 17/05/2018 3:47 AM, Gerard Ziemski wrote: >>> Hi all, >>> Please review this small enhancement where we print out additional information (i.e. output from PrintSystemDictionaryAtExit), when we detect the failure. >>> https://bugs.openjdk.java.net/browse/JDK-8202360 >>> http://cr.openjdk.java.net/~gziemski/8202360_rev1/ >> >> I don't understand what your fix is doing - sorry. > > The fix takes all the available output bytes from the process (using Scanner to grab all tokens at once) and prints all the output from the process (i.e. PrintSystemDictionaryAtExit, and there is a lot of it) at failure. Okay but that's the rest of the output that hasn't yet been processed - correct? So the line of output that caused the failure won't be there as it's already been grabbed by the scanner and processed. >> >> What I wanted to see was the complete history of lines of the form >> >> "Java dictionary (table_size=107, classes=6)" >> >> so that we can see how the table size and number of classes have changed leading up to the failure. But I realize now that we already get that from the: >> >> System.out.println("PASS table_size:"+table_size+", classes:"+classes+" OK"); >> >> All that is missing is the printing of the table_size and classes when we throw the exception. That can be trivially fixed: >> >> if (loadFactor > MAX_LOAD_FACTOR) { >> + System.out.println("FAIL table_size:"+table_size+", classes:"+classes); >> throw new RuntimeException("Load factor too high, expected MAX "+MAX_LOAD_FACTOR+", got "+loadFactor); >> } else { >> System.out.println("PASS table_size:"+table_size+", classes:"+classes+" OK"); >> } > > OK, so you determined that only that one piece of info was missing in the end in your case, but maybe the next person might want to see everything that PrintSystemDictionaryAtExit provides? > > As long as we?re touching the test - should we then stick with the solution that prints out everything from PrintSystemDictionaryAtExit? I'm not sure how anything that comes later in PrintSystemDictionaryAtExit would help with a failure that happened earlier, but if you see value it then that's fine. However I think you need my suggestion as well to get the information that actually caused the failure. Some sample output in the bug report would be useful. Thanks, David > > cheers > > From thomas.stuefe at gmail.com Fri May 18 06:07:20 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 18 May 2018 08:07:20 +0200 Subject: RFR (XS) 8202014: Possible to receive signal before signal semaphore created In-Reply-To: References: <3d3ea98c-60c9-71de-f0c7-e156207b24ca@oracle.com> <3e24a240-9e6b-65a0-64ca-c81db396f6cb@oracle.com> Message-ID: Hi Coleen, looks good, thanks for taking my suggestion. Thoughts for further improvements: - os::signal_init could be renamed to something more to the point, e.g. os::initialize_signal_dispatcher_thread() or os::initialize_jdk_signal_support() or similar - os::signal_init_pd could be removed from os.hpp altogether and made file scope static in the os_xxx.cpp file. It could also have a better name. I leave it up to you if you do this in a follow up item, or even at all. The fix is fine to me in its current form. Thanks, Thomas On Thu, May 17, 2018 at 9:25 PM, wrote: > > > On 5/16/18 2:04 AM, Thomas St?fe wrote: >> >> Hi, >> >> I agree with David. Signal handlers are installed in os::init_2(), but >> the semaphore sometime later in the (imho misleadingly named) void >> os::signal_init(). This looks like an initialization error. >> >> As a side note, can we make the "void signalHandler(int sig, >> siginfo_t* info, void* uc)" functions in os_xxx.cpp static, since they >> are only file local marshalling points? > > > Okay, I've moved the initialization of the signal semaphore to os::init_2(). > The signal handler for this signal_notify() case is set up by > JVM_RegisterSignal early in JDK initialization (initPhase1) which is before > os::signal_init() was run. Now the semaphore is initialized before that. > > I also made signalHandler static in this change although not strictly > related. > > http://cr.openjdk.java.net/~coleenp/8202014.02/webrev/index.html > > Reran kill -INT script and mach5 hs_tiers 1 and 2. > test/hotspot/jtreg/runtime/signal tests and > test/jdk/sun/misc/SunMiscSignalTest.java. > > Thanks, > Coleen > > >> >> Kind Regards, Thomas >> >> >> >> >> On Wed, May 16, 2018 at 4:40 AM, David Holmes >> wrote: >>> >>> Hi Coleen, >>> >>> On 16/05/2018 11:23 AM, coleen.phillimore at oracle.com wrote: >>>> >>>> Summary: Don't assert for null semaphore because it can be null before >>>> initialization is complete. >>> >>> >>> Sorry I don't agree with this. Surely we should not install our signal >>> handler until such time as everything is properly initialized? Or the >>> handler itself should have some kind of initialization check to guide how >>> it >>> responds. >>> >>> Thanks, >>> David >>> >>> >>>> Tested this with a shell script, thanks Robbin, and mach5 tier1,2 all >>>> Oracle platforms. AIX doesn't use the semaphore class so doesn't have >>>> this >>>> bug. >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8202014.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8202014 >>>> >>>> Thanks, >>>> Coleen > > From thomas.stuefe at gmail.com Fri May 18 06:16:10 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 18 May 2018 08:16:10 +0200 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: References: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> Message-ID: Hi Matthias, I took another closer look and have some minor nits. Sorry for not looking better the first time, my head was not clear :) Since that was my fault I leave it up to you if you follow up. The fix in its current form is also fine to me. - I am not a big fan of ">> 20". I would change that to " / M" to make the code more readable. - Strictly speaking GetProcessMemoryInfo could fail (I have never seen that though) and return FALSE. In which case we should probably not output the results. Best Regards, Thomas On Thu, May 17, 2018 at 5:20 PM, Baesken, Matthias wrote: >> > > I also think that one number is enough. Adaptive numbers would be great. > > > Hi Thomas/Goetz/Martin , thanks for looking into it. > Adaptive numbers could be done in a follow-up as suggested by Thomas , this would also touch the other platforms . > > I added more line feeds to the output and reduced output to just printing M . > > > Example output from a Windows desktop PC : > > Memory: 4k page, system-wide physical 32520M (20165M free) > TotalPageFile size 34568M (AvailPageFile size 18170M) > current process WorkingSet (physical memory assigned to process): 79M, peak: 79M > current process commit charge ("private bytes"): 148M, peak: 148M > > > new webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.1/ > > > Best regards, Matthias > > > >> -----Original Message----- >> From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] >> Sent: Mittwoch, 16. Mai 2018 16:35 >> To: Lindenmaier, Goetz >> Cc: Baesken, Matthias ; Doerr, Martin >> ; hotspot-dev at openjdk.java.net; hotspot- >> runtime-dev at openjdk.java.net >> Subject: Re: [RFR] 8202427: Enhance os::print_memory_info on Windows >> >> On Wed, May 16, 2018 at 3:48 PM, Lindenmaier, Goetz >> wrote: >> > Hi Matthias, >> > >> > I appreciate the additional information in the hs_err file. >> > Could you please add an example output (of the final version) to the bug >> description? >> > >> > As Martin, I would like more line feeds, both in the code and the output. >> > Currently, the output in the hs_err file looks like this (where the first line is >> too long): >> > Memory: 4k page, system-wide physical 16776692k [16383M] (8795032k >> [8588M] free), TotalPageFile size 49949460k [48778M] (AvailPageFile size >> 38942152k [38029M]), user-mode portion of virtual address-space 2097024k >> [2047M] (6940k [6M] free) >> > current process WorkingSet (physical memory assigned to process): >> 991804k [968M], peak: 991804k [968M] >> > >> > current process commit charge ("private bytes"): 1523632k [1487M], >> peak: 1523632k [1487M] >> > >> > I also think that one number is enough. Adaptive numbers would be great. >> > I know about >> > src/hotspot/share/memory/metaspace/metaspaceCommon.hpp: >> print_human_readable_size() >> > for this, but this seems a bit hidden in metaspace. >> > >> >> Guys, lets not worry about human readable numbers for now. We can move >> print_human_readable_size() out from metaspace to something generic >> and use it to improve this code (and others) in a later patch. That >> was my original intention when adding print_human_readable_size(). >> >> The patch is good. Thanks for doing this. >> >> ..Thoams >> >> > I don't like usage of _M_IX86. Common in this file seems #ifndef _WIN64 >> for 32-bit windows >> > dependent code. But don't care here. >> > >> > Best regards, >> > Goetz. >> > >> > >> >> -----Original Message----- >> >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On >> >> Behalf Of Baesken, Matthias >> >> Sent: Mittwoch, 16. Mai 2018 15:13 >> >> To: Doerr, Martin ; 'hotspot- >> >> dev at openjdk.java.net' ; hotspot- >> >> runtime-dev at openjdk.java.net >> >> Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows >> >> >> >> Ping : could I get a second review ? >> >> >> >> Bug : >> >> https://bugs.openjdk.java.net/browse/JDK-8202427 >> >> Change : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ >> >> >> >> >> >> > We could also just print the MB value , let's see what other think. >> >> > Another option might be to have a flexible output (kB for smaller >> >> memory >> >> > values , MB (or GB) for larger ) ? >> >> >> >> Martin suggested to just print the MB values, what do you think ? >> >> >> >> >> >> Thanks, Matthias >> >> >> >> >> >> > -----Original Message----- >> >> > From: Baesken, Matthias >> >> > Sent: Mittwoch, 2. Mai 2018 13:00 >> >> > To: Doerr, Martin ; 'hotspot- >> >> > dev at openjdk.java.net' ; hotspot- >> >> > runtime-dev at openjdk.java.net >> >> > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on >> Windows >> >> > >> >> > Hi Martin, thanks for your input . >> >> > I add hotspot-runtime-dev . >> >> > >> >> > > >> >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe >> other >> >> > > reviewers would like to comment on this, too. >> >> > > >> >> > >> >> > We could also just print the MB value , let's see what other think. >> >> > Another option might be to have a flexible output (kB for smaller >> >> memory >> >> > values , MB (or GB) for larger ) ? >> >> > >> >> > Best regards, Matthias >> >> > >> >> > >> >> > > -----Original Message----- >> >> > > From: Doerr, Martin >> >> > > Sent: Mittwoch, 2. Mai 2018 12:53 >> >> > > To: Baesken, Matthias ; 'hotspot- >> >> > > dev at openjdk.java.net' >> >> > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on >> Windows >> >> > > >> >> > > Hi Matthias, >> >> > > >> >> > > looks like a nice enhancement. We can get substantially more >> >> information. >> >> > > >> >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe >> other >> >> > > reviewers would like to comment on this, too. >> >> > > >> >> > > We should have line breaks. >> >> > > >> >> > > Best regards, >> >> > > Martin >> >> > > >> >> > > >> >> > > -----Original Message----- >> >> > > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] >> On >> >> > > Behalf Of Baesken, Matthias >> >> > > Sent: Montag, 30. April 2018 16:53 >> >> > > To: 'hotspot-dev at openjdk.java.net' > dev at openjdk.java.net> >> >> > > Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows >> >> > > >> >> > > On Windows, >> >> > > the os::print_memory_info misses a few memory-related infos that >> >> might >> >> > > be interesting : >> >> > > - current and peak WorkingSet size (= physical memory assigned to >> the >> >> > > process) >> >> > > - current and peak commit charge (also known as "private bytes" in >> >> > Windows >> >> > > tools) >> >> > > - on 32bit Windows : >> >> > > user-mode portion/free user mode portion of virtual address-space >> >> > > (Total/AvailVirtual) because it shows how close do we get to the 2-4 >> GB >> >> per >> >> > > process border. >> >> > > >> >> > > >> >> > > - the current naming of "swap/free-swap" memory is a bit misleading; >> >> > > in the Windows world swap is a special page file used for UWP apps. >> >> > > (see Windows Internals : "The swap file" (chapter 5 Memory >> >> > management) >> >> > > Windows 8 added another page file called a swap file. It is ... >> exclusively >> >> > used >> >> > > for UWP (Universal Windows Platform) apps. >> >> > > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the >> UWP >> >> > > related values, it is about page file sizes >> >> > > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just >> >> virtual >> >> > > memory might be more appropriate). >> >> > > >> >> > > >> >> > > https://msdn.microsoft.com/de- >> >> > > de/library/windows/desktop/aa366770(v=vs.85).aspx >> >> > > >> >> > > documents it in the following way: >> >> > > ullTotalPageFile: >> >> > > The current committed memory limit for the system or the current >> >> process, >> >> > > whichever is smaller,... >> >> > > >> >> > > ullAvailPageFile : >> >> > > The maximum amount of memory the current process can commit, in >> >> > bytes. >> >> > > This value is equal to or smaller than the system-wide available >> commit >> >> > value >> >> > > >> >> > > >> >> > > >> >> > > Aditionally I suggest having output of the various memory-values in M >> >> > > (megabyte) as well , the k (kilobyte) output sometimes gives very >> high >> >> and >> >> > > unreadable numbers). >> >> > > >> >> > > >> >> > > Could you please review my change ? >> >> > > >> >> > > >> >> > > Bug : >> >> > > >> >> > > https://bugs.openjdk.java.net/browse/JDK-8202427 >> >> > > >> >> > > >> >> > > Change : >> >> > > >> >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ >> >> > > >> >> > > >> >> > > Thanks, Matthias >> >> > > >> > From goetz.lindenmaier at sap.com Fri May 18 09:09:33 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 18 May 2018 09:09:33 +0000 Subject: RFR(XS): Fix issue with "8199852: Print more information about class loaders in LinkageErrors." Message-ID: Hi, Unfortunately I need to do a small follow up for my LinkageError change. If parentNameOop is NULL, parentName remains NULL. It should be set to . I actually saw a crash on solaris x86_64 with this. I also remove some dead, leftover variables. Please review: http://cr.openjdk.java.net/~goetz/wr18/8203400-exMsg-fixLinkageError/01/ Best regards, Goetz. From david.holmes at oracle.com Fri May 18 09:27:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 18 May 2018 19:27:23 +1000 Subject: RFR(XS): Fix issue with "8199852: Print more information about class loaders in LinkageErrors." In-Reply-To: References: Message-ID: <9c372241-95a5-48f6-7bf6-9e37a2399ddd@oracle.com> Looks okay but can you add a space back in: ! Symbol*name = k->name(); :) Thanks, David On 18/05/2018 7:09 PM, Lindenmaier, Goetz wrote: > Hi, > > Unfortunately I need to do a small follow up for my LinkageError change. > If parentNameOop is NULL, parentName remains NULL. It should be set to . > I actually saw a crash on solaris x86_64 with this. > I also remove some dead, leftover variables. > > Please review: > http://cr.openjdk.java.net/~goetz/wr18/8203400-exMsg-fixLinkageError/01/ > > Best regards, > Goetz. > From goetz.lindenmaier at sap.com Fri May 18 10:59:00 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 18 May 2018 10:59:00 +0000 Subject: RFR(XS): Fix issue with "8199852: Print more information about class loaders in LinkageErrors." In-Reply-To: <9c372241-95a5-48f6-7bf6-9e37a2399ddd@oracle.com> References: <9c372241-95a5-48f6-7bf6-9e37a2399ddd@oracle.com> Message-ID: <791adb077f51449a96e821d465942ee3@sap.com> Hi David, Thanks for looking at this! > Looks okay but can you add a space back in: Oops, there were two spaces, and I removed both ... added one again. Best regards, Goetz. > -----Original Message----- > From: David Holmes > Sent: Friday, May 18, 2018 11:27 AM > To: Lindenmaier, Goetz ; hotspot-runtime- > dev at openjdk.java.net > Subject: Re: RFR(XS): Fix issue with "8199852: Print more information about > class loaders in LinkageErrors." > > Looks okay but can you add a space back in: > > ! Symbol*name = k->name(); > > :) > > Thanks, > David > > On 18/05/2018 7:09 PM, Lindenmaier, Goetz wrote: > > Hi, > > > > Unfortunately I need to do a small follow up for my LinkageError change. > > If parentNameOop is NULL, parentName remains NULL. It should be set to > . > > I actually saw a crash on solaris x86_64 with this. > > I also remove some dead, leftover variables. > > > > Please review: > > http://cr.openjdk.java.net/~goetz/wr18/8203400-exMsg- > fixLinkageError/01/ > > > > Best regards, > > Goetz. > > From lois.foltan at oracle.com Fri May 18 11:45:42 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Fri, 18 May 2018 07:45:42 -0400 Subject: RFR(XS): Fix issue with "8199852: Print more information about class loaders in LinkageErrors." In-Reply-To: References: Message-ID: Looks good. Lois On 5/18/2018 5:09 AM, Lindenmaier, Goetz wrote: > Hi, > > Unfortunately I need to do a small follow up for my LinkageError change. > If parentNameOop is NULL, parentName remains NULL. It should be set to . > I actually saw a crash on solaris x86_64 with this. > I also remove some dead, leftover variables. > > Please review: > http://cr.openjdk.java.net/~goetz/wr18/8203400-exMsg-fixLinkageError/01/ > > Best regards, > Goetz. From goetz.lindenmaier at sap.com Fri May 18 11:52:59 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 18 May 2018 11:52:59 +0000 Subject: RFR(XS): Fix issue with "8199852: Print more information about class loaders in LinkageErrors." In-Reply-To: References: Message-ID: <50c74d426eab4153a1ae32906a0a2030@sap.com> Thanks Lois! Best regards, Goetz. > -----Original Message----- > From: Lois Foltan > Sent: Friday, May 18, 2018 1:46 PM > To: Lindenmaier, Goetz ; hotspot-runtime- > dev at openjdk.java.net > Subject: Re: RFR(XS): Fix issue with "8199852: Print more information about > class loaders in LinkageErrors." > > Looks good. > Lois > > On 5/18/2018 5:09 AM, Lindenmaier, Goetz wrote: > > Hi, > > > > Unfortunately I need to do a small follow up for my LinkageError change. > > If parentNameOop is NULL, parentName remains NULL. It should be set to > . > > I actually saw a crash on solaris x86_64 with this. > > I also remove some dead, leftover variables. > > > > Please review: > > http://cr.openjdk.java.net/~goetz/wr18/8203400-exMsg- > fixLinkageError/01/ > > > > Best regards, > > Goetz. From coleen.phillimore at oracle.com Fri May 18 12:21:36 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 18 May 2018 08:21:36 -0400 Subject: RFR (S) 8134537: Much nearly duplicated code for sun.misc.Signal support In-Reply-To: <8ae60fc9-32af-9481-2096-2d7c73920e68@oracle.com> References: <6fd46541-7fb4-81eb-0ea1-36fa78e8525b@oracle.com> <8ae60fc9-32af-9481-2096-2d7c73920e68@oracle.com> Message-ID: <7a46242f-33ee-cf84-0def-28209a6b26ef@oracle.com> On 5/18/18 12:49 AM, David Holmes wrote: > One minor thing - only noticed because of your other change: > > src/hotspot/os/posix/jvm_posix.cpp > > 33 // sun.misc.Signal > > It isn't sun.misc.Signal anymore :) Okay, I'll change that.? There are other comments in the file that I didn't want to mess with, since I only newly have read this code. Thanks, Coleen > > Thanks, > David > > On 18/05/2018 2:37 PM, David Holmes wrote: >> Hi Coleen, >> >> That all seems fine to me. Nice cleanup. (Plenty more to go ;-) ) >> >> Thanks, >> David >> >> On 18/05/2018 6:26 AM, coleen.phillimore at oracle.com wrote: >>> Summary: merged jvm_.cpp into jvm_posix.cpp mostly >>> >>> See bug for more info.? The os_.cpp code is sort of duplicated >>> but different enough to not be worth refactoring, for this feature.? >>> I was in the area, so merged this one part. >>> >>> Tested with hs-tier1,2 runtime/signal tests and >>> sun/misc/SunMiscSignalTest.java. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8134537.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8134537 >>> >>> Thanks, >>> Coleen From coleen.phillimore at oracle.com Fri May 18 12:22:07 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 18 May 2018 08:22:07 -0400 Subject: RFR (S) 8134537: Much nearly duplicated code for sun.misc.Signal support In-Reply-To: References: Message-ID: On 5/18/18 1:49 AM, Thomas St?fe wrote: > Hi Coleen, > > fix looks good. You are right, there are many more similarities, but > too different to do it within limited time. Thanks, Thomas. Coleen > > Thanks, Thomas > > > On Thu, May 17, 2018 at 10:26 PM, wrote: >> Summary: merged jvm_.cpp into jvm_posix.cpp mostly >> >> See bug for more info. The os_.cpp code is sort of duplicated but >> different enough to not be worth refactoring, for this feature. I was in >> the area, so merged this one part. >> >> Tested with hs-tier1,2 runtime/signal tests and >> sun/misc/SunMiscSignalTest.java. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8134537.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8134537 >> >> Thanks, >> Coleen From coleen.phillimore at oracle.com Fri May 18 12:38:19 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 18 May 2018 08:38:19 -0400 Subject: RFR (XS) 8202014: Possible to receive signal before signal semaphore created In-Reply-To: References: <3d3ea98c-60c9-71de-f0c7-e156207b24ca@oracle.com> <3e24a240-9e6b-65a0-64ca-c81db396f6cb@oracle.com> Message-ID: <8fd7977f-7dcf-52fa-cbfa-8c36c2ce609b@oracle.com> On 5/18/18 2:07 AM, Thomas St?fe wrote: > Hi Coleen, > > looks good, thanks for taking my suggestion. > > Thoughts for further improvements: > > - os::signal_init could be renamed to something more to the point, > e.g. os::initialize_signal_dispatcher_thread() or > os::initialize_jdk_signal_support() or similar Okay os::signal_init => os::initialize_jdk_signal_support() > > - os::signal_init_pd could be removed from os.hpp altogether and made > file scope static in the os_xxx.cpp file. It could also have a better > name. os::signal_init_pd => jdk_misc_signal_init(). > I leave it up to you if you do this in a follow up item, or even at > all. The fix is fine to me in its current form. I'll make these changes with this minor change.? Hopefully this is agreeable. Thanks, Coleen > > Thanks, Thomas > > > > > On Thu, May 17, 2018 at 9:25 PM, wrote: >> >> On 5/16/18 2:04 AM, Thomas St?fe wrote: >>> Hi, >>> >>> I agree with David. Signal handlers are installed in os::init_2(), but >>> the semaphore sometime later in the (imho misleadingly named) void >>> os::signal_init(). This looks like an initialization error. >>> >>> As a side note, can we make the "void signalHandler(int sig, >>> siginfo_t* info, void* uc)" functions in os_xxx.cpp static, since they >>> are only file local marshalling points? >> >> Okay, I've moved the initialization of the signal semaphore to os::init_2(). >> The signal handler for this signal_notify() case is set up by >> JVM_RegisterSignal early in JDK initialization (initPhase1) which is before >> os::signal_init() was run. Now the semaphore is initialized before that. >> >> I also made signalHandler static in this change although not strictly >> related. >> >> http://cr.openjdk.java.net/~coleenp/8202014.02/webrev/index.html >> >> Reran kill -INT script and mach5 hs_tiers 1 and 2. >> test/hotspot/jtreg/runtime/signal tests and >> test/jdk/sun/misc/SunMiscSignalTest.java. >> >> Thanks, >> Coleen >> >> >>> Kind Regards, Thomas >>> >>> >>> >>> >>> On Wed, May 16, 2018 at 4:40 AM, David Holmes >>> wrote: >>>> Hi Coleen, >>>> >>>> On 16/05/2018 11:23 AM, coleen.phillimore at oracle.com wrote: >>>>> Summary: Don't assert for null semaphore because it can be null before >>>>> initialization is complete. >>>> >>>> Sorry I don't agree with this. Surely we should not install our signal >>>> handler until such time as everything is properly initialized? Or the >>>> handler itself should have some kind of initialization check to guide how >>>> it >>>> responds. >>>> >>>> Thanks, >>>> David >>>> >>>> >>>>> Tested this with a shell script, thanks Robbin, and mach5 tier1,2 all >>>>> Oracle platforms. AIX doesn't use the semaphore class so doesn't have >>>>> this >>>>> bug. >>>>> >>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8202014.01/webrev >>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8202014 >>>>> >>>>> Thanks, >>>>> Coleen >> From coleen.phillimore at oracle.com Fri May 18 12:59:21 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 18 May 2018 08:59:21 -0400 Subject: RFR (XS) 8202014: Possible to receive signal before signal semaphore created In-Reply-To: <8fd7977f-7dcf-52fa-cbfa-8c36c2ce609b@oracle.com> References: <3d3ea98c-60c9-71de-f0c7-e156207b24ca@oracle.com> <3e24a240-9e6b-65a0-64ca-c81db396f6cb@oracle.com> <8fd7977f-7dcf-52fa-cbfa-8c36c2ce609b@oracle.com> Message-ID: <6a713c87-2839-9406-13c7-0e203fc85c91@oracle.com> On 5/18/18 8:38 AM, coleen.phillimore at oracle.com wrote: > > > On 5/18/18 2:07 AM, Thomas St?fe wrote: >> Hi Coleen, >> >> looks good, thanks for taking my suggestion. >> >> Thoughts for further improvements: >> >> - os::signal_init could be renamed to something more to the point, >> e.g. os::initialize_signal_dispatcher_thread() or >> os::initialize_jdk_signal_support() or similar > > Okay os::signal_init => os::initialize_jdk_signal_support() >> >> - os::signal_init_pd could be removed from os.hpp altogether and made >> file scope static in the os_xxx.cpp file. It could also have a better >> name. > > os::signal_init_pd => jdk_misc_signal_init(). >> I leave it up to you if you do this in a follow up item, or even at >> all. The fix is fine to me in its current form. > > I'll make these changes with this minor change.? Hopefully this is > agreeable. http://cr.openjdk.java.net/~coleenp/8202014.03/webrev Rerunning oracle platform tier1,2 to make sure I didn't break anything. thanks, Coleen > > Thanks, > Coleen > >> >> Thanks, Thomas >> >> >> >> >> On Thu, May 17, 2018 at 9:25 PM, wrote: >>> >>> On 5/16/18 2:04 AM, Thomas St?fe wrote: >>>> Hi, >>>> >>>> I agree with David. Signal handlers are installed in os::init_2(), but >>>> the semaphore sometime later in the (imho misleadingly named) void >>>> os::signal_init(). This looks like an initialization error. >>>> >>>> As a side note, can we make the "void signalHandler(int sig, >>>> siginfo_t* info, void* uc)" functions in os_xxx.cpp static, since they >>>> are only file local marshalling points? >>> >>> Okay, I've moved the initialization of the signal semaphore to >>> os::init_2(). >>> The signal handler for this signal_notify() case is set up by >>> JVM_RegisterSignal early in JDK initialization (initPhase1) which is >>> before >>> os::signal_init() was run.? Now the semaphore is initialized before >>> that. >>> >>> I also made signalHandler static in this change although not strictly >>> related. >>> >>> http://cr.openjdk.java.net/~coleenp/8202014.02/webrev/index.html >>> >>> Reran kill -INT script and mach5 hs_tiers 1 and 2. >>> test/hotspot/jtreg/runtime/signal tests and >>> test/jdk/sun/misc/SunMiscSignalTest.java. >>> >>> Thanks, >>> Coleen >>> >>> >>>> Kind Regards, Thomas >>>> >>>> >>>> >>>> >>>> On Wed, May 16, 2018 at 4:40 AM, David Holmes >>>> >>>> wrote: >>>>> Hi Coleen, >>>>> >>>>> On 16/05/2018 11:23 AM, coleen.phillimore at oracle.com wrote: >>>>>> Summary: Don't assert for null semaphore because it can be null >>>>>> before >>>>>> initialization is complete. >>>>> >>>>> Sorry I don't agree with this. Surely we should not install our >>>>> signal >>>>> handler until such time as everything is properly initialized? Or the >>>>> handler itself should have some kind of initialization check to >>>>> guide how >>>>> it >>>>> responds. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> >>>>>> Tested this with a shell script, thanks Robbin, and mach5 tier1,2 >>>>>> all >>>>>> Oracle platforms.? AIX doesn't use the semaphore class so doesn't >>>>>> have >>>>>> this >>>>>> bug. >>>>>> >>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8202014.01/webrev >>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8202014 >>>>>> >>>>>> Thanks, >>>>>> Coleen >>> > From thomas.stuefe at gmail.com Fri May 18 13:30:07 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 18 May 2018 15:30:07 +0200 Subject: RFR (XS) 8202014: Possible to receive signal before signal semaphore created In-Reply-To: <6a713c87-2839-9406-13c7-0e203fc85c91@oracle.com> References: <3d3ea98c-60c9-71de-f0c7-e156207b24ca@oracle.com> <3e24a240-9e6b-65a0-64ca-c81db396f6cb@oracle.com> <8fd7977f-7dcf-52fa-cbfa-8c36c2ce609b@oracle.com> <6a713c87-2839-9406-13c7-0e203fc85c91@oracle.com> Message-ID: Beautiful, thank you :) ..Thomas On Fri, May 18, 2018 at 2:59 PM, wrote: > > > On 5/18/18 8:38 AM, coleen.phillimore at oracle.com wrote: >> >> >> >> On 5/18/18 2:07 AM, Thomas St?fe wrote: >>> >>> Hi Coleen, >>> >>> looks good, thanks for taking my suggestion. >>> >>> Thoughts for further improvements: >>> >>> - os::signal_init could be renamed to something more to the point, >>> e.g. os::initialize_signal_dispatcher_thread() or >>> os::initialize_jdk_signal_support() or similar >> >> >> Okay os::signal_init => os::initialize_jdk_signal_support() >>> >>> >>> - os::signal_init_pd could be removed from os.hpp altogether and made >>> file scope static in the os_xxx.cpp file. It could also have a better >>> name. >> >> >> os::signal_init_pd => jdk_misc_signal_init(). >>> >>> I leave it up to you if you do this in a follow up item, or even at >>> all. The fix is fine to me in its current form. >> >> >> I'll make these changes with this minor change. Hopefully this is >> agreeable. > > > http://cr.openjdk.java.net/~coleenp/8202014.03/webrev > > Rerunning oracle platform tier1,2 to make sure I didn't break anything. > > thanks, > Coleen > > >> >> Thanks, >> Coleen >> >>> >>> Thanks, Thomas >>> >>> >>> >>> >>> On Thu, May 17, 2018 at 9:25 PM, wrote: >>>> >>>> >>>> On 5/16/18 2:04 AM, Thomas St?fe wrote: >>>>> >>>>> Hi, >>>>> >>>>> I agree with David. Signal handlers are installed in os::init_2(), but >>>>> the semaphore sometime later in the (imho misleadingly named) void >>>>> os::signal_init(). This looks like an initialization error. >>>>> >>>>> As a side note, can we make the "void signalHandler(int sig, >>>>> siginfo_t* info, void* uc)" functions in os_xxx.cpp static, since they >>>>> are only file local marshalling points? >>>> >>>> >>>> Okay, I've moved the initialization of the signal semaphore to >>>> os::init_2(). >>>> The signal handler for this signal_notify() case is set up by >>>> JVM_RegisterSignal early in JDK initialization (initPhase1) which is >>>> before >>>> os::signal_init() was run. Now the semaphore is initialized before >>>> that. >>>> >>>> I also made signalHandler static in this change although not strictly >>>> related. >>>> >>>> http://cr.openjdk.java.net/~coleenp/8202014.02/webrev/index.html >>>> >>>> Reran kill -INT script and mach5 hs_tiers 1 and 2. >>>> test/hotspot/jtreg/runtime/signal tests and >>>> test/jdk/sun/misc/SunMiscSignalTest.java. >>>> >>>> Thanks, >>>> Coleen >>>> >>>> >>>>> Kind Regards, Thomas >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, May 16, 2018 at 4:40 AM, David Holmes >>>>> wrote: >>>>>> >>>>>> Hi Coleen, >>>>>> >>>>>> On 16/05/2018 11:23 AM, coleen.phillimore at oracle.com wrote: >>>>>>> >>>>>>> Summary: Don't assert for null semaphore because it can be null >>>>>>> before >>>>>>> initialization is complete. >>>>>> >>>>>> >>>>>> Sorry I don't agree with this. Surely we should not install our signal >>>>>> handler until such time as everything is properly initialized? Or the >>>>>> handler itself should have some kind of initialization check to guide >>>>>> how >>>>>> it >>>>>> responds. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>> >>>>>>> Tested this with a shell script, thanks Robbin, and mach5 tier1,2 all >>>>>>> Oracle platforms. AIX doesn't use the semaphore class so doesn't >>>>>>> have >>>>>>> this >>>>>>> bug. >>>>>>> >>>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8202014.01/webrev >>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8202014 >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>> >>>> >> > From coleen.phillimore at oracle.com Fri May 18 14:27:22 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 18 May 2018 10:27:22 -0400 Subject: RFR (XS) 8202014: Possible to receive signal before signal semaphore created In-Reply-To: References: <3d3ea98c-60c9-71de-f0c7-e156207b24ca@oracle.com> <3e24a240-9e6b-65a0-64ca-c81db396f6cb@oracle.com> <8fd7977f-7dcf-52fa-cbfa-8c36c2ce609b@oracle.com> <6a713c87-2839-9406-13c7-0e203fc85c91@oracle.com> Message-ID: <5b689da2-bc48-41bc-9612-36b774f3dd59@oracle.com> Thanks! Coleen On 5/18/18 9:30 AM, Thomas St?fe wrote: > Beautiful, thank you :) > > ..Thomas > > On Fri, May 18, 2018 at 2:59 PM, wrote: >> >> On 5/18/18 8:38 AM, coleen.phillimore at oracle.com wrote: >>> >>> >>> On 5/18/18 2:07 AM, Thomas St?fe wrote: >>>> Hi Coleen, >>>> >>>> looks good, thanks for taking my suggestion. >>>> >>>> Thoughts for further improvements: >>>> >>>> - os::signal_init could be renamed to something more to the point, >>>> e.g. os::initialize_signal_dispatcher_thread() or >>>> os::initialize_jdk_signal_support() or similar >>> >>> Okay os::signal_init => os::initialize_jdk_signal_support() >>>> >>>> - os::signal_init_pd could be removed from os.hpp altogether and made >>>> file scope static in the os_xxx.cpp file. It could also have a better >>>> name. >>> >>> os::signal_init_pd => jdk_misc_signal_init(). >>>> I leave it up to you if you do this in a follow up item, or even at >>>> all. The fix is fine to me in its current form. >>> >>> I'll make these changes with this minor change. Hopefully this is >>> agreeable. >> >> http://cr.openjdk.java.net/~coleenp/8202014.03/webrev >> >> Rerunning oracle platform tier1,2 to make sure I didn't break anything. >> >> thanks, >> Coleen >> >> >>> Thanks, >>> Coleen >>> >>>> Thanks, Thomas >>>> >>>> >>>> >>>> >>>> On Thu, May 17, 2018 at 9:25 PM, wrote: >>>>> >>>>> On 5/16/18 2:04 AM, Thomas St?fe wrote: >>>>>> Hi, >>>>>> >>>>>> I agree with David. Signal handlers are installed in os::init_2(), but >>>>>> the semaphore sometime later in the (imho misleadingly named) void >>>>>> os::signal_init(). This looks like an initialization error. >>>>>> >>>>>> As a side note, can we make the "void signalHandler(int sig, >>>>>> siginfo_t* info, void* uc)" functions in os_xxx.cpp static, since they >>>>>> are only file local marshalling points? >>>>> >>>>> Okay, I've moved the initialization of the signal semaphore to >>>>> os::init_2(). >>>>> The signal handler for this signal_notify() case is set up by >>>>> JVM_RegisterSignal early in JDK initialization (initPhase1) which is >>>>> before >>>>> os::signal_init() was run. Now the semaphore is initialized before >>>>> that. >>>>> >>>>> I also made signalHandler static in this change although not strictly >>>>> related. >>>>> >>>>> http://cr.openjdk.java.net/~coleenp/8202014.02/webrev/index.html >>>>> >>>>> Reran kill -INT script and mach5 hs_tiers 1 and 2. >>>>> test/hotspot/jtreg/runtime/signal tests and >>>>> test/jdk/sun/misc/SunMiscSignalTest.java. >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> >>>>>> Kind Regards, Thomas >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Wed, May 16, 2018 at 4:40 AM, David Holmes >>>>>> wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> On 16/05/2018 11:23 AM, coleen.phillimore at oracle.com wrote: >>>>>>>> Summary: Don't assert for null semaphore because it can be null >>>>>>>> before >>>>>>>> initialization is complete. >>>>>>> >>>>>>> Sorry I don't agree with this. Surely we should not install our signal >>>>>>> handler until such time as everything is properly initialized? Or the >>>>>>> handler itself should have some kind of initialization check to guide >>>>>>> how >>>>>>> it >>>>>>> responds. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>> >>>>>>>> Tested this with a shell script, thanks Robbin, and mach5 tier1,2 all >>>>>>>> Oracle platforms. AIX doesn't use the semaphore class so doesn't >>>>>>>> have >>>>>>>> this >>>>>>>> bug. >>>>>>>> >>>>>>>> open webrev at http://cr.openjdk.java.net/~coleenp/8202014.01/webrev >>>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8202014 >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>> From matthias.baesken at sap.com Fri May 18 15:10:48 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 18 May 2018 15:10:48 +0000 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: References: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> Message-ID: <219b9940eaa14dd6805ac5360a1733aa@sap.com> Hi Thomas , thanks looking into it. Indeed GetProcessMemoryInfo could fail , and GetProcessMemoryInfo could fail as well . MSDN says for both functions . https://msdn.microsoft.com/de-de/library/windows/desktop/ms683219(v=vs.85).aspx https://msdn.microsoft.com/de-de/library/windows/desktop/aa366589(v=vs.85).aspx .......... Return value If the function succeeds, the return value is nonzero. If the function fails, the return value is zero. I add return code checking for failure cases . (plus some line feeds in the coding which is desired by Goetz ). For GlobalMemoryStatusEx there seem to be quite a few places where return code checking is missing , this should be addressed in a follow up change : OperatingSystemImpl.c 103 GlobalMemoryStatusEx(&ms); 113 GlobalMemoryStatusEx(&ms); 142 GlobalMemoryStatusEx(&ms); 152 GlobalMemoryStatusEx(&ms); os_windows.cpp 736 GlobalMemoryStatusEx(&ms); 748 GlobalMemoryStatusEx(&ms); 1748 GlobalMemoryStatusEx(&ms); 3669 GlobalMemoryStatusEx(&ms); Best regards, Matthias > -----Original Message----- > From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] > Sent: Freitag, 18. Mai 2018 08:16 > To: Baesken, Matthias > Cc: Lindenmaier, Goetz ; Doerr, Martin > ; hotspot-dev at openjdk.java.net; hotspot- > runtime-dev at openjdk.java.net > Subject: Re: [RFR] 8202427: Enhance os::print_memory_info on Windows > > Hi Matthias, > > I took another closer look and have some minor nits. Sorry for not > looking better the first time, my head was not clear :) > > Since that was my fault I leave it up to you if you follow up. The fix > in its current form is also fine to me. > > - I am not a big fan of ">> 20". I would change that to " / M" to make > the code more readable. > - Strictly speaking GetProcessMemoryInfo could fail (I have never seen > that though) and return FALSE. In which case we should probably not > output the results. > > Best Regards, Thomas > > > On Thu, May 17, 2018 at 5:20 PM, Baesken, Matthias > wrote: > >> > > I also think that one number is enough. Adaptive numbers would be > great. > > > > > > Hi Thomas/Goetz/Martin , thanks for looking into it. > > Adaptive numbers could be done in a follow-up as suggested by Thomas , > this would also touch the other platforms . > > > > I added more line feeds to the output and reduced output to just printing > M . > > > > > > Example output from a Windows desktop PC : > > > > Memory: 4k page, system-wide physical 32520M (20165M free) > > TotalPageFile size 34568M (AvailPageFile size 18170M) > > current process WorkingSet (physical memory assigned to process): 79M, > peak: 79M > > current process commit charge ("private bytes"): 148M, peak: 148M > > > > > > new webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.1/ > > > > > > Best regards, Matthias > > > > > > > >> -----Original Message----- > >> From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] > >> Sent: Mittwoch, 16. Mai 2018 16:35 > >> To: Lindenmaier, Goetz > >> Cc: Baesken, Matthias ; Doerr, Martin > >> ; hotspot-dev at openjdk.java.net; hotspot- > >> runtime-dev at openjdk.java.net > >> Subject: Re: [RFR] 8202427: Enhance os::print_memory_info on Windows > >> > >> On Wed, May 16, 2018 at 3:48 PM, Lindenmaier, Goetz > >> wrote: > >> > Hi Matthias, > >> > > >> > I appreciate the additional information in the hs_err file. > >> > Could you please add an example output (of the final version) to the > bug > >> description? > >> > > >> > As Martin, I would like more line feeds, both in the code and the output. > >> > Currently, the output in the hs_err file looks like this (where the first > line is > >> too long): > >> > Memory: 4k page, system-wide physical 16776692k [16383M] > (8795032k > >> [8588M] free), TotalPageFile size 49949460k [48778M] (AvailPageFile size > >> 38942152k [38029M]), user-mode portion of virtual address-space > 2097024k > >> [2047M] (6940k [6M] free) > >> > current process WorkingSet (physical memory assigned to process): > >> 991804k [968M], peak: 991804k [968M] > >> > > >> > current process commit charge ("private bytes"): 1523632k [1487M], > >> peak: 1523632k [1487M] > >> > > >> > I also think that one number is enough. Adaptive numbers would be > great. > >> > I know about > >> > src/hotspot/share/memory/metaspace/metaspaceCommon.hpp: > >> print_human_readable_size() > >> > for this, but this seems a bit hidden in metaspace. > >> > > >> > >> Guys, lets not worry about human readable numbers for now. We can > move > >> print_human_readable_size() out from metaspace to something generic > >> and use it to improve this code (and others) in a later patch. That > >> was my original intention when adding print_human_readable_size(). > >> > >> The patch is good. Thanks for doing this. > >> > >> ..Thoams > >> > >> > I don't like usage of _M_IX86. Common in this file seems #ifndef > _WIN64 > >> for 32-bit windows > >> > dependent code. But don't care here. > >> > > >> > Best regards, > >> > Goetz. > >> > > >> > > >> >> -----Original Message----- > >> >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] > On > >> >> Behalf Of Baesken, Matthias > >> >> Sent: Mittwoch, 16. Mai 2018 15:13 > >> >> To: Doerr, Martin ; 'hotspot- > >> >> dev at openjdk.java.net' ; hotspot- > >> >> runtime-dev at openjdk.java.net > >> >> Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on > Windows > >> >> > >> >> Ping : could I get a second review ? > >> >> > >> >> Bug : > >> >> https://bugs.openjdk.java.net/browse/JDK-8202427 > >> >> Change : > >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > >> >> > >> >> > >> >> > We could also just print the MB value , let's see what other think. > >> >> > Another option might be to have a flexible output (kB for smaller > >> >> memory > >> >> > values , MB (or GB) for larger ) ? > >> >> > >> >> Martin suggested to just print the MB values, what do you think ? > >> >> > >> >> > >> >> Thanks, Matthias > >> >> > >> >> > >> >> > -----Original Message----- > >> >> > From: Baesken, Matthias > >> >> > Sent: Mittwoch, 2. Mai 2018 13:00 > >> >> > To: Doerr, Martin ; 'hotspot- > >> >> > dev at openjdk.java.net' ; hotspot- > >> >> > runtime-dev at openjdk.java.net > >> >> > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on > >> Windows > >> >> > > >> >> > Hi Martin, thanks for your input . > >> >> > I add hotspot-runtime-dev . > >> >> > > >> >> > > > >> >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe > >> other > >> >> > > reviewers would like to comment on this, too. > >> >> > > > >> >> > > >> >> > We could also just print the MB value , let's see what other think. > >> >> > Another option might be to have a flexible output (kB for smaller > >> >> memory > >> >> > values , MB (or GB) for larger ) ? > >> >> > > >> >> > Best regards, Matthias > >> >> > > >> >> > > >> >> > > -----Original Message----- > >> >> > > From: Doerr, Martin > >> >> > > Sent: Mittwoch, 2. Mai 2018 12:53 > >> >> > > To: Baesken, Matthias ; 'hotspot- > >> >> > > dev at openjdk.java.net' > >> >> > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on > >> Windows > >> >> > > > >> >> > > Hi Matthias, > >> >> > > > >> >> > > looks like a nice enhancement. We can get substantially more > >> >> information. > >> >> > > > >> >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe > >> other > >> >> > > reviewers would like to comment on this, too. > >> >> > > > >> >> > > We should have line breaks. > >> >> > > > >> >> > > Best regards, > >> >> > > Martin > >> >> > > > >> >> > > > >> >> > > -----Original Message----- > >> >> > > From: hotspot-dev [mailto:hotspot-dev- > bounces at openjdk.java.net] > >> On > >> >> > > Behalf Of Baesken, Matthias > >> >> > > Sent: Montag, 30. April 2018 16:53 > >> >> > > To: 'hotspot-dev at openjdk.java.net' >> dev at openjdk.java.net> > >> >> > > Subject: [RFR] 8202427: Enhance os::print_memory_info on > Windows > >> >> > > > >> >> > > On Windows, > >> >> > > the os::print_memory_info misses a few memory-related infos > that > >> >> might > >> >> > > be interesting : > >> >> > > - current and peak WorkingSet size (= physical memory assigned to > >> the > >> >> > > process) > >> >> > > - current and peak commit charge (also known as "private bytes" in > >> >> > Windows > >> >> > > tools) > >> >> > > - on 32bit Windows : > >> >> > > user-mode portion/free user mode portion of virtual address- > space > >> >> > > (Total/AvailVirtual) because it shows how close do we get to the 2- > 4 > >> GB > >> >> per > >> >> > > process border. > >> >> > > > >> >> > > > >> >> > > - the current naming of "swap/free-swap" memory is a bit > misleading; > >> >> > > in the Windows world swap is a special page file used for UWP > apps. > >> >> > > (see Windows Internals : "The swap file" (chapter 5 Memory > >> >> > management) > >> >> > > Windows 8 added another page file called a swap file. It is ... > >> exclusively > >> >> > used > >> >> > > for UWP (Universal Windows Platform) apps. > >> >> > > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the > >> UWP > >> >> > > related values, it is about page file sizes > >> >> > > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just > >> >> virtual > >> >> > > memory might be more appropriate). > >> >> > > > >> >> > > > >> >> > > https://msdn.microsoft.com/de- > >> >> > > de/library/windows/desktop/aa366770(v=vs.85).aspx > >> >> > > > >> >> > > documents it in the following way: > >> >> > > ullTotalPageFile: > >> >> > > The current committed memory limit for the system or the current > >> >> process, > >> >> > > whichever is smaller,... > >> >> > > > >> >> > > ullAvailPageFile : > >> >> > > The maximum amount of memory the current process can commit, > in > >> >> > bytes. > >> >> > > This value is equal to or smaller than the system-wide available > >> commit > >> >> > value > >> >> > > > >> >> > > > >> >> > > > >> >> > > Aditionally I suggest having output of the various memory-values > in M > >> >> > > (megabyte) as well , the k (kilobyte) output sometimes gives very > >> high > >> >> and > >> >> > > unreadable numbers). > >> >> > > > >> >> > > > >> >> > > Could you please review my change ? > >> >> > > > >> >> > > > >> >> > > Bug : > >> >> > > > >> >> > > https://bugs.openjdk.java.net/browse/JDK-8202427 > >> >> > > > >> >> > > > >> >> > > Change : > >> >> > > > >> >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > >> >> > > > >> >> > > > >> >> > > Thanks, Matthias > >> >> > > > >> > From harold.seigel at oracle.com Fri May 18 15:15:25 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Fri, 18 May 2018 11:15:25 -0400 Subject: RFR 8203183: vmTestbase/vm/mlvm/anonloader/stress/oome/heap/Test.java times out Message-ID: <24278661-d13e-770c-a8f9-a010b27c8db2@oracle.com> Hi, Please review this fix for JDK-8203183.? The fix is to run the test with -Xmx128m to prevent the test from timing out.? The fix was tested on Linux X64, Mac OS X, Windows, and Solaris. Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8203183/webrev/index.html JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8203183 Thanks, Harold From ioi.lam at oracle.com Fri May 18 16:34:43 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 18 May 2018 09:34:43 -0700 Subject: RFR 8203381 Replace InstanceKlass::allocate_instance_handle with JavaCalls::construct_new_instance Message-ID: <2a449ddb-4512-193f-d6d1-152989b8526c@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8203381 http://cr.openjdk.java.net/~iklam/jdk11/8203381-construct-new-instance.v01/ This patch removes a lot of the boilerplate code for allocating a Java object and calling its constructor. Hopefully the code is cleaner and easier to read. Also: Added assert(InstanceKlass::is_initialized(), ...) in case where we cannot call JavaCalls::construct_new_instance for thread objects. Replaced unnecessary SystemDictionary::resolve_or_null() calls with SystemDictionary::XXX_klass(). Tested with hs-tier[1,2,3,4,5] Thanks - Ioi From gerard.ziemski at oracle.com Fri May 18 18:54:23 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Fri, 18 May 2018 13:54:23 -0500 Subject: RFR (XS) 8202360: [TESTBUG] runtime/LoadClass/TestResize.java needs to print output when it fails In-Reply-To: References: Message-ID: hi David, > On May 18, 2018, at 12:59 AM, David Holmes wrote: > > Hi Gerard, > > On 18/05/2018 2:21 AM, Gerard Ziemski wrote: >> hi David, >> Thank you for the review! >>> On May 16, 2018, at 4:54 PM, David Holmes wrote: >>> >>> Hi Gerard, >>> >>> On 17/05/2018 3:47 AM, Gerard Ziemski wrote: >>>> Hi all, >>>> Please review this small enhancement where we print out additional information (i.e. output from PrintSystemDictionaryAtExit), when we detect the failure. >>>> https://bugs.openjdk.java.net/browse/JDK-8202360 >>>> http://cr.openjdk.java.net/~gziemski/8202360_rev1/ >>> >>> I don't understand what your fix is doing - sorry. >> The fix takes all the available output bytes from the process (using Scanner to grab all tokens at once) and prints all the output from the process (i.e. PrintSystemDictionaryAtExit, and there is a lot of it) at failure. > > Okay but that's the rest of the output that hasn't yet been processed - correct? So the line of output that caused the failure won't be there as it's already been grabbed by the scanner and processed. Good catch, I changed the fix to make sure we print all the output. >>> >>> What I wanted to see was the complete history of lines of the form >>> >>> "Java dictionary (table_size=107, classes=6)" >>> >>> so that we can see how the table size and number of classes have changed leading up to the failure. But I realize now that we already get that from the: >>> >>> System.out.println("PASS table_size:"+table_size+", classes:"+classes+" OK"); >>> >>> All that is missing is the printing of the table_size and classes when we throw the exception. That can be trivially fixed: >>> >>> if (loadFactor > MAX_LOAD_FACTOR) { >>> + System.out.println("FAIL table_size:"+table_size+", classes:"+classes); >>> throw new RuntimeException("Load factor too high, expected MAX "+MAX_LOAD_FACTOR+", got "+loadFactor); >>> } else { >>> System.out.println("PASS table_size:"+table_size+", classes:"+classes+" OK"); >>> } >> OK, so you determined that only that one piece of info was missing in the end in your case, but maybe the next person might want to see everything that PrintSystemDictionaryAtExit provides? >> As long as we?re touching the test - should we then stick with the solution that prints out everything from PrintSystemDictionaryAtExit? > > I'm not sure how anything that comes later in PrintSystemDictionaryAtExit would help with a failure that happened earlier, but if you see value it then that's fine. However I think you need my suggestion as well to get the information that actually caused the failure. We now print all the output from PrintSystemDictionaryAtExit, but also following your suggestion we summarize the failure details in the exception itself like: java.lang.RuntimeException: Load factor too high, expected MAX 5.0, got 467.30841121495325 [table size 107, number of clases 50002] > Some sample output in the bug report would be useful. I attached a sample output to the issue as requested. https://bugs.openjdk.java.net/browse/JDK-8202360 http://cr.openjdk.java.net/~gziemski/8202360_rev2 cheers From jiangli.zhou at Oracle.COM Fri May 18 19:59:05 2018 From: jiangli.zhou at Oracle.COM (Jiangli Zhou) Date: Fri, 18 May 2018 12:59:05 -0700 Subject: RFR(S) 8193332 MetaspaceShared::check_shared_class_loader_type is not used during archive creation In-Reply-To: <4c9edcca-6c94-4933-326b-b9cd88d7033f@oracle.com> References: <4c9edcca-6c94-4933-326b-b9cd88d7033f@oracle.com> Message-ID: <2CBE0467-99F8-4F6A-BCCB-A3C276031BEC@oracle.com> Hi Ioi, - classLoader.cpp I think the change in ClassLoader::load_class() is ok. We used to support --limit-modules at CDS runtime. So an archived ?module? classes from the -Xbootclasspath/a could be used at runtime. That?s the reason why we wanted to make the archived classes as a superset of all runtime loadable classes. In JDK 11, we?ve eliminated the support of --limit-modules from CDS because --limit-modules is only useful for testing mode and it complicates CDS code unnecessary. It?s good that the special case for DumpSharedSpaces in general class loading code is gone. - metaspaceShared.cpp The changes look ok. I also want to note that the loader_type records in archived classes is no longer necessary. The loader type of archived classes including for user defined class loaders can be quickly determined by the shared path indexes. Eliminating the loader type for archived classes will help to clean up the code further and also free 3 bits in InstanceKlass _misc_flags for other usage. I?ll file a RFE for loader type cleanups. All the rest of the changes also look good to me. Thanks, Jiangli > On May 17, 2018, at 6:13 PM, Ioi Lam wrote: > > https://bugs.openjdk.java.net/browse/JDK-8193332 > http://cr.openjdk.java.net/~iklam/jdk11/8193332-check_shared_class_loader_type.v01/ > > Summary: > > When we restructured the AppCDS code, the call to MetaspaceShared::check_shared_class_loader_type > was somehow left out. I have restored it, and actually found a minor bug related to modules > and -Xbootclasspath/a: > > In classLoader.cpp, at dump time we used to load a class from -Xbootclasspath/a even if > this class belongs to a named packaged, which means it would not be loaded by the JVM at runtime. > Such dump-time loading seems unnecessary so I removed it, and fixed a related test case > to reflect the latest behavior. > > Tested with hs-tier1,2. > > Thanks > - Ioi From coleen.phillimore at oracle.com Fri May 18 22:41:42 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 18 May 2018 18:41:42 -0400 Subject: RFR(L): 8176808: Split up metaspace.cpp In-Reply-To: <32826e8139614cd68f0b4548ca8c044c@sap.com> References: <32826e8139614cd68f0b4548ca8c044c@sap.com> Message-ID: <0da5bc3d-3f3a-dd09-c635-fdc67ed7ecaa@oracle.com> On 5/17/18 6:15 AM, Siebenborn, Axel wrote: > Hi Thomas, > The change looks good . > I should have read your whole mail before starting looking at the changes, as you explain what you have done and what you didn't do ??. > Concerning the decision using ::metaspace vs. ::metaspace::internal: > I would use ::metaspace for the internal classes and leave the public classes in the global namespace. > This is more consistent to hotspot code where namespaces are rarely used (if this counts as an argument). > I would change that in this change, as having having ::metaspace::internal and nothing in ::metaspace seems to be the worst alternative. Yes, I agree with this.? One namespace to encapsulate things like ChunkManager and SpaceManager and these things is good, without adding the another nested "internals" namespace.? That's too much for us :) Coleen > > Altogether, this is really a good refactoring. > > Cheers, > Axel > >> -----Original Message----- >> From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- >> bounces at openjdk.java.net] On Behalf Of Thomas St?fe >> Sent: Montag, 14. Mai 2018 15:34 >> To: Hotspot dev runtime >> Subject: Re: RFR(L): 8176808: Split up metaspace.cpp >> >> Ping... >> >> Is there anything I can do to make review easier? This is really >> mostly code shuffling around (out of metaspace.cpp into new files), so >> with a bit of effort I could cook up some diffs for those parts. >> >> ..Thomas >> >> On Fri, May 11, 2018 at 8:31 AM, Thomas St?fe >> wrote: >>> All builds are fixed, jdk-submit tests ran through sucessfully. >>> >>> Latest webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace- >> cpp/webrev.01/webrev/ >>> Only difference to the first webrev is the placement of a single >>> semicolon in occupancyMap.hpp, to satisfy the Solaris compiler. >>> >>> Thanks, Thomas >>> >>> >>> >>> On Wed, May 9, 2018 at 5:08 PM, Thomas St?fe >> wrote: >>>> Hi all, >>>> >>>> This was a long time coming. Lets cut up this beast :) >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8176808 >>>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- >> metaspace-cpp/webrev.00/webrev/ >>>> This change splits up metaspace.cpp into multiple parts. Note that I >>>> did not change much code (exceptions see below in details) - so this >>>> is mostly moving code around. >>>> >>>> This change follows the scheme tentatively outlined in JDK-8201572: >>>> >>>> - metaspace internal classes go into the sub directory >>>> share/memory/metaspace. Ideally, those should never be accessed from >>>> outside, apart from other metaspace classes and metaspace tests. They >>>> also go into the namespace ::metaspace::internals. >>>> >>>> - metaspace external classes (MetaspaceUtils, ClassLoaderMetaspace, >>>> etc) remain inside share/memory and, for now, remain in the global >>>> namespace. >>>> >>>> Note: the original idea was to move metaspace external classes to >>>> namespace ::metaspace. But I shied away from that, since that would >>>> mean fixing up all usages of these classes either with metaspace:: >>>> scope or with using declarations. I had to make a cut at some point. >>>> >>>> So here is something to decide: >>>> - should we then get rid of the ::metaspace::internals namespace, move >>>> metaspace-internal classes to the enclosing ::metaspace namespace and >>>> leave external classes in the global namespace ? >>>> - or should we follow through (maybe in a future patch): internal >>>> classes in ::metaspace::internal, and move external classes to >>>> ::metaspace ? >>>> >>>> Some more details: >>>> >>>> - the following classes moved out of metaspace.cpp into namespace >>>> "metaspace::internal" and the metaspace sub directory: >>>> MetaDebug, ChunkManager, SpaceManager, BlockFreeList and >> SmallBlocks, >>>> OccupancyMap, VirtualSpaceNode and VirtualSpaceList, the >>>> PrintCLDMetaspaceInfoClosure. >>>> >>>> - I also moved metachunk.hpp to internals, since class Metachunk is >>>> not used externally. What is used externally is Metablock though, so I >>>> split up metachunk.hpp into metabase.hpp, metablock.hpp and >>>> metachunk.hpp, holding Metabase, Metablock and Metachunk, >>>> respectively. >>>> >>>> - Now we see some ugly seams: metaspace.hpp is supposed to be the >>>> outside API contract, however, it contains implementation details >>>> which should not be there and which manifest now by having to use the >>>> metaspace::internals namespace here and there. If we care, we could >>>> solve this with a bit redesigning. >>>> >>>> - The ChunkManager gtest and the SpaceManager gtest had been >>>> implemented inside metaspace.cpp, since it was not possible to access >>>> the internal classes those tests needed in any other way. Since that >>>> is now possible, I moved the coding out to gtest-land and made them >>>> real gtests (exchanging the asserts for real gtest ASSERTS, among >>>> other things). >>>> Note that there are some tests left in metaspace.cpp >>>> (TestMetaspaceUtilsTest, TestVirtualSpaceNodeTest) - those were no >>>> gtests-in-disguise but were part of -XX:+ExecuteInternalVMTests. I >>>> guess these tests precede the gtest framework? I leave it up to others >>>> to decide what to do with them and to potentially move them out of >>>> metaspace.cpp. >>>> Side note: -XX:+ExecuteInternalVMTests seems to have bitrotted, see >>>> https://bugs.openjdk.java.net/browse/JDK-8202848. Does this get tested >>>> regularly? >>>> >>>> - metaspace.cpp is quite a bit smaller now - from ~5000 loc down to >>>> ~1700 loc. Arguably, one could split out more and clean up more, but I >>>> think this is a good start. >>>> >>>> --- >>>> >>>> I built locally on linux (release, fastdebug with and without pch, >>>> zero) and windows (64, 32bit). jdk-submit tests ran through with a >>>> build error on solaris sparc - I currently try to reproduce that build >>>> error with our very slow solaris machine. >>>> >>>> Thanks, Thomas From coleen.phillimore at oracle.com Fri May 18 22:58:20 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 18 May 2018 18:58:20 -0400 Subject: RFR(L): 8176808: Split up metaspace.cpp In-Reply-To: References: Message-ID: Thomas, thank you for doing this and sorry about the delay in looking at this. On 5/9/18 11:08 AM, Thomas St?fe wrote: > Hi all, > > This was a long time coming. Lets cut up this beast :) > > Bug: https://bugs.openjdk.java.net/browse/JDK-8176808 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace-cpp/webrev.00/webrev/ > > This change splits up metaspace.cpp into multiple parts. Note that I > did not change much code (exceptions see below in details) - so this > is mostly moving code around. > > This change follows the scheme tentatively outlined in JDK-8201572: > > - metaspace internal classes go into the sub directory > share/memory/metaspace. Ideally, those should never be accessed from > outside, apart from other metaspace classes and metaspace tests. They > also go into the namespace ::metaspace::internals. > > - metaspace external classes (MetaspaceUtils, ClassLoaderMetaspace, > etc) remain inside share/memory and, for now, remain in the global > namespace. > > Note: the original idea was to move metaspace external classes to > namespace ::metaspace. But I shied away from that, since that would > mean fixing up all usages of these classes either with metaspace:: > scope or with using declarations. I had to make a cut at some point. > > So here is something to decide: > - should we then get rid of the ::metaspace::internals namespace, move > metaspace-internal classes to the enclosing ::metaspace namespace and > leave external classes in the global namespace ? > - or should we follow through (maybe in a future patch): internal > classes in ::metaspace::internal, and move external classes to > ::metaspace ? Yes I think "internal" is kind of noisy.? I like just the metaspace namespace to hold all these things, and keep the rest global. > > Some more details: > > - the following classes moved out of metaspace.cpp into namespace > "metaspace::internal" and the metaspace sub directory: > MetaDebug, ChunkManager, SpaceManager, BlockFreeList and SmallBlocks, > OccupancyMap, VirtualSpaceNode and VirtualSpaceList, the > PrintCLDMetaspaceInfoClosure. > > - I also moved metachunk.hpp to internals, since class Metachunk is > not used externally. What is used externally is Metablock though, so I > split up metachunk.hpp into metabase.hpp, metablock.hpp and > metachunk.hpp, holding Metabase, Metablock and Metachunk, > respectively. > > - Now we see some ugly seams: metaspace.hpp is supposed to be the > outside API contract, however, it contains implementation details > which should not be there and which manifest now by having to use the > metaspace::internals namespace here and there. If we care, we could > solve this with a bit redesigning. > > - The ChunkManager gtest and the SpaceManager gtest had been > implemented inside metaspace.cpp, since it was not possible to access > the internal classes those tests needed in any other way. Since that > is now possible, I moved the coding out to gtest-land and made them > real gtests (exchanging the asserts for real gtest ASSERTS, among > other things). > Note that there are some tests left in metaspace.cpp > (TestMetaspaceUtilsTest, TestVirtualSpaceNodeTest) - those were no > gtests-in-disguise but were part of -XX:+ExecuteInternalVMTests. I > guess these tests precede the gtest framework? I leave it up to others > to decide what to do with them and to potentially move them out of > metaspace.cpp. Yes, ExecuteInternalVMTests preceeded gtest and these tests should be eventually moved, now that they can be. > Side note: -XX:+ExecuteInternalVMTests seems to have bitrotted, see > https://bugs.openjdk.java.net/browse/JDK-8202848. Does this get tested > regularly? > > - metaspace.cpp is quite a bit smaller now - from ~5000 loc down to > ~1700 loc. Arguably, one could split out more and clean up more, but I > think this is a good start. I think it's great.?? I clicked on most but since it's just code movement, I reviewed the change. Thanks, Coleen > --- > > I built locally on linux (release, fastdebug with and without pch, > zero) and windows (64, 32bit). jdk-submit tests ran through with a > build error on solaris sparc - I currently try to reproduce that build > error with our very slow solaris machine. > > Thanks, Thomas From ioi.lam at oracle.com Sat May 19 01:46:32 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 18 May 2018 18:46:32 -0700 Subject: RFR 8203381 Replace InstanceKlass::allocate_instance_handle with JavaCalls::construct_new_instance In-Reply-To: References: <45eb3ef9-88c3-1bda-bb9e-122eb888dc35@oracle.com> Message-ID: <7acb71f5-584f-9447-e7c5-e7bc3c06ee99@oracle.com> Hi Yumin, Thanks for the review. In some cases, such as create_initial_thread, we must do ik->allocate_instance_handle() and JavaCalls::call_special in 2 steps, and set some thread states in between the 2 steps. Also, there are other use of JavaCalls::call_special for invoke instance methods that are not constructors. Therefore, we cannot hide JavaCalls::call_special. Thanks - Ioi On 5/18/18 11:04 AM, yumin qi wrote: > Hi, Ioi > > ? Very nice clean-up for Thread creation. > ? One question, should we replace all? the JavaCalls::call_special > with JavaCalls::construct_new_instance (there are three versions) and > make JavaCalls::call_special an internal implementation? I did see > many places using JavaCalls::call_special. > > Thanks > Yumin > > On Fri, May 18, 2018 at 9:36 AM, Ioi Lam > wrote: > > https://bugs.openjdk.java.net/browse/JDK-8203381 > > http://cr.openjdk.java.net/~iklam/jdk11/8203381-construct-new-instance.v01/ > > > This patch removes a lot of the boilerplate code for allocating a > Java object and calling its constructor. Hopefully the code is cleaner > and easier to read. > > Also: > > Added assert(InstanceKlass::is_initialized(), ...) in case where > we cannot call JavaCalls::construct_new_instance for thread objects. > > Replaced unnecessary SystemDictionary::resolve_or_null() calls with > SystemDictionary::XXX_klass(). > > Tested with hs-tier[1,2,3,4,5] > > Thanks > - Ioi > > > > > > From thomas.stuefe at gmail.com Sat May 19 04:55:26 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 19 May 2018 06:55:26 +0200 Subject: RFR(L): 8176808: Split up metaspace.cpp In-Reply-To: <0da5bc3d-3f3a-dd09-c635-fdc67ed7ecaa@oracle.com> References: <32826e8139614cd68f0b4548ca8c044c@sap.com> <0da5bc3d-3f3a-dd09-c635-fdc67ed7ecaa@oracle.com> Message-ID: Hi, Coleen and Axel, thanks a lot for the reviews! I will be very happy once this is removed from my patch queue, since merging the patch is becoming a pain. New webrev. As suggested, I removed the "internals" inner namespace, which leaves us with "metaspace::". Nothing else changed. Still fine? Webrevs: Incremental: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace-cpp/webrev.01-to-02/webrev/ Full: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace-cpp/webrev.02/webrev/ Thanks, Thomas On Sat, May 19, 2018 at 12:41 AM, wrote: > > > On 5/17/18 6:15 AM, Siebenborn, Axel wrote: > >> Hi Thomas, >> The change looks good . >> I should have read your whole mail before starting looking at the >> changes, as you explain what you have done and what you didn't do [image: >> ??]. >> Concerning the decision using ::metaspace vs. ::metaspace::internal: >> I would use ::metaspace for the internal classes and leave the public >> classes in the global namespace. >> This is more consistent to hotspot code where namespaces are rarely used >> (if this counts as an argument). >> I would change that in this change, as having having >> ::metaspace::internal and nothing in ::metaspace seems to be the worst >> alternative. >> > > Yes, I agree with this. One namespace to encapsulate things like > ChunkManager and SpaceManager and these things is good, without adding the > another nested "internals" namespace. That's too much for us :) > > Coleen > > >> Altogether, this is really a good refactoring. >> >> Cheers, >> Axel >> >> -----Original Message----- >>> From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- >>> bounces at openjdk.java.net] On Behalf Of Thomas St?fe >>> Sent: Montag, 14. Mai 2018 15:34 >>> To: Hotspot dev runtime >>> Subject: Re: RFR(L): 8176808: Split up metaspace.cpp >>> >>> Ping... >>> >>> Is there anything I can do to make review easier? This is really >>> mostly code shuffling around (out of metaspace.cpp into new files), so >>> with a bit of effort I could cook up some diffs for those parts. >>> >>> ..Thomas >>> >>> On Fri, May 11, 2018 at 8:31 AM, Thomas St?fe >>> wrote: >>> >>>> All builds are fixed, jdk-submit tests ran through sucessfully. >>>> >>>> Latest webrev: >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace- >>>> >>> cpp/webrev.01/webrev/ >>> >>>> Only difference to the first webrev is the placement of a single >>>> semicolon in occupancyMap.hpp, to satisfy the Solaris compiler. >>>> >>>> Thanks, Thomas >>>> >>>> >>>> >>>> On Wed, May 9, 2018 at 5:08 PM, Thomas St?fe >>>> >>> wrote: >>> >>>> Hi all, >>>>> >>>>> This was a long time coming. Lets cut up this beast :) >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8176808 >>>>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- >>>>> >>>> metaspace-cpp/webrev.00/webrev/ >>> >>>> This change splits up metaspace.cpp into multiple parts. Note that I >>>>> did not change much code (exceptions see below in details) - so this >>>>> is mostly moving code around. >>>>> >>>>> This change follows the scheme tentatively outlined in JDK-8201572: >>>>> >>>>> - metaspace internal classes go into the sub directory >>>>> share/memory/metaspace. Ideally, those should never be accessed from >>>>> outside, apart from other metaspace classes and metaspace tests. They >>>>> also go into the namespace ::metaspace::internals. >>>>> >>>>> - metaspace external classes (MetaspaceUtils, ClassLoaderMetaspace, >>>>> etc) remain inside share/memory and, for now, remain in the global >>>>> namespace. >>>>> >>>>> Note: the original idea was to move metaspace external classes to >>>>> namespace ::metaspace. But I shied away from that, since that would >>>>> mean fixing up all usages of these classes either with metaspace:: >>>>> scope or with using declarations. I had to make a cut at some point. >>>>> >>>>> So here is something to decide: >>>>> - should we then get rid of the ::metaspace::internals namespace, move >>>>> metaspace-internal classes to the enclosing ::metaspace namespace and >>>>> leave external classes in the global namespace ? >>>>> - or should we follow through (maybe in a future patch): internal >>>>> classes in ::metaspace::internal, and move external classes to >>>>> ::metaspace ? >>>>> >>>>> Some more details: >>>>> >>>>> - the following classes moved out of metaspace.cpp into namespace >>>>> "metaspace::internal" and the metaspace sub directory: >>>>> MetaDebug, ChunkManager, SpaceManager, BlockFreeList and >>>>> >>>> SmallBlocks, >>> >>>> OccupancyMap, VirtualSpaceNode and VirtualSpaceList, the >>>>> PrintCLDMetaspaceInfoClosure. >>>>> >>>>> - I also moved metachunk.hpp to internals, since class Metachunk is >>>>> not used externally. What is used externally is Metablock though, so I >>>>> split up metachunk.hpp into metabase.hpp, metablock.hpp and >>>>> metachunk.hpp, holding Metabase, Metablock and Metachunk, >>>>> respectively. >>>>> >>>>> - Now we see some ugly seams: metaspace.hpp is supposed to be the >>>>> outside API contract, however, it contains implementation details >>>>> which should not be there and which manifest now by having to use the >>>>> metaspace::internals namespace here and there. If we care, we could >>>>> solve this with a bit redesigning. >>>>> >>>>> - The ChunkManager gtest and the SpaceManager gtest had been >>>>> implemented inside metaspace.cpp, since it was not possible to access >>>>> the internal classes those tests needed in any other way. Since that >>>>> is now possible, I moved the coding out to gtest-land and made them >>>>> real gtests (exchanging the asserts for real gtest ASSERTS, among >>>>> other things). >>>>> Note that there are some tests left in metaspace.cpp >>>>> (TestMetaspaceUtilsTest, TestVirtualSpaceNodeTest) - those were no >>>>> gtests-in-disguise but were part of -XX:+ExecuteInternalVMTests. I >>>>> guess these tests precede the gtest framework? I leave it up to others >>>>> to decide what to do with them and to potentially move them out of >>>>> metaspace.cpp. >>>>> Side note: -XX:+ExecuteInternalVMTests seems to have bitrotted, see >>>>> https://bugs.openjdk.java.net/browse/JDK-8202848. Does this get tested >>>>> regularly? >>>>> >>>>> - metaspace.cpp is quite a bit smaller now - from ~5000 loc down to >>>>> ~1700 loc. Arguably, one could split out more and clean up more, but I >>>>> think this is a good start. >>>>> >>>>> --- >>>>> >>>>> I built locally on linux (release, fastdebug with and without pch, >>>>> zero) and windows (64, 32bit). jdk-submit tests ran through with a >>>>> build error on solaris sparc - I currently try to reproduce that build >>>>> error with our very slow solaris machine. >>>>> >>>>> Thanks, Thomas >>>>> >>>> > From thomas.stuefe at gmail.com Sat May 19 05:24:06 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 19 May 2018 07:24:06 +0200 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: <219b9940eaa14dd6805ac5360a1733aa@sap.com> References: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> <219b9940eaa14dd6805ac5360a1733aa@sap.com> Message-ID: Hi Matthias, sounds good. ..Thomas On Fri, May 18, 2018 at 5:10 PM, Baesken, Matthias wrote: > Hi Thomas , thanks looking into it. > Indeed GetProcessMemoryInfo could fail , and GetProcessMemoryInfo could fail as well . > > MSDN says for both functions . > > https://msdn.microsoft.com/de-de/library/windows/desktop/ms683219(v=vs.85).aspx > > https://msdn.microsoft.com/de-de/library/windows/desktop/aa366589(v=vs.85).aspx > > .......... > > Return value > If the function succeeds, the return value is nonzero. > If the function fails, the return value is zero. > > > I add return code checking for failure cases . (plus some line feeds in the coding which is desired by Goetz ). > > > For GlobalMemoryStatusEx there seem to be quite a few places where return code checking is missing , this should be addressed in a follow up change : > > > OperatingSystemImpl.c > 103 GlobalMemoryStatusEx(&ms); > 113 GlobalMemoryStatusEx(&ms); > 142 GlobalMemoryStatusEx(&ms); > 152 GlobalMemoryStatusEx(&ms); > > > os_windows.cpp > 736 GlobalMemoryStatusEx(&ms); > 748 GlobalMemoryStatusEx(&ms); > 1748 GlobalMemoryStatusEx(&ms); > 3669 GlobalMemoryStatusEx(&ms); > > > Best regards, Matthias > >> -----Original Message----- >> From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] >> Sent: Freitag, 18. Mai 2018 08:16 >> To: Baesken, Matthias >> Cc: Lindenmaier, Goetz ; Doerr, Martin >> ; hotspot-dev at openjdk.java.net; hotspot- >> runtime-dev at openjdk.java.net >> Subject: Re: [RFR] 8202427: Enhance os::print_memory_info on Windows >> >> Hi Matthias, >> >> I took another closer look and have some minor nits. Sorry for not >> looking better the first time, my head was not clear :) >> >> Since that was my fault I leave it up to you if you follow up. The fix >> in its current form is also fine to me. >> >> - I am not a big fan of ">> 20". I would change that to " / M" to make >> the code more readable. >> - Strictly speaking GetProcessMemoryInfo could fail (I have never seen >> that though) and return FALSE. In which case we should probably not >> output the results. >> >> Best Regards, Thomas >> >> >> On Thu, May 17, 2018 at 5:20 PM, Baesken, Matthias >> wrote: >> >> > > I also think that one number is enough. Adaptive numbers would be >> great. >> > >> > >> > Hi Thomas/Goetz/Martin , thanks for looking into it. >> > Adaptive numbers could be done in a follow-up as suggested by Thomas , >> this would also touch the other platforms . >> > >> > I added more line feeds to the output and reduced output to just printing >> M . >> > >> > >> > Example output from a Windows desktop PC : >> > >> > Memory: 4k page, system-wide physical 32520M (20165M free) >> > TotalPageFile size 34568M (AvailPageFile size 18170M) >> > current process WorkingSet (physical memory assigned to process): 79M, >> peak: 79M >> > current process commit charge ("private bytes"): 148M, peak: 148M >> > >> > >> > new webrev : >> > >> > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.1/ >> > >> > >> > Best regards, Matthias >> > >> > >> > >> >> -----Original Message----- >> >> From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] >> >> Sent: Mittwoch, 16. Mai 2018 16:35 >> >> To: Lindenmaier, Goetz >> >> Cc: Baesken, Matthias ; Doerr, Martin >> >> ; hotspot-dev at openjdk.java.net; hotspot- >> >> runtime-dev at openjdk.java.net >> >> Subject: Re: [RFR] 8202427: Enhance os::print_memory_info on Windows >> >> >> >> On Wed, May 16, 2018 at 3:48 PM, Lindenmaier, Goetz >> >> wrote: >> >> > Hi Matthias, >> >> > >> >> > I appreciate the additional information in the hs_err file. >> >> > Could you please add an example output (of the final version) to the >> bug >> >> description? >> >> > >> >> > As Martin, I would like more line feeds, both in the code and the output. >> >> > Currently, the output in the hs_err file looks like this (where the first >> line is >> >> too long): >> >> > Memory: 4k page, system-wide physical 16776692k [16383M] >> (8795032k >> >> [8588M] free), TotalPageFile size 49949460k [48778M] (AvailPageFile size >> >> 38942152k [38029M]), user-mode portion of virtual address-space >> 2097024k >> >> [2047M] (6940k [6M] free) >> >> > current process WorkingSet (physical memory assigned to process): >> >> 991804k [968M], peak: 991804k [968M] >> >> > >> >> > current process commit charge ("private bytes"): 1523632k [1487M], >> >> peak: 1523632k [1487M] >> >> > >> >> > I also think that one number is enough. Adaptive numbers would be >> great. >> >> > I know about >> >> > src/hotspot/share/memory/metaspace/metaspaceCommon.hpp: >> >> print_human_readable_size() >> >> > for this, but this seems a bit hidden in metaspace. >> >> > >> >> >> >> Guys, lets not worry about human readable numbers for now. We can >> move >> >> print_human_readable_size() out from metaspace to something generic >> >> and use it to improve this code (and others) in a later patch. That >> >> was my original intention when adding print_human_readable_size(). >> >> >> >> The patch is good. Thanks for doing this. >> >> >> >> ..Thoams >> >> >> >> > I don't like usage of _M_IX86. Common in this file seems #ifndef >> _WIN64 >> >> for 32-bit windows >> >> > dependent code. But don't care here. >> >> > >> >> > Best regards, >> >> > Goetz. >> >> > >> >> > >> >> >> -----Original Message----- >> >> >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] >> On >> >> >> Behalf Of Baesken, Matthias >> >> >> Sent: Mittwoch, 16. Mai 2018 15:13 >> >> >> To: Doerr, Martin ; 'hotspot- >> >> >> dev at openjdk.java.net' ; hotspot- >> >> >> runtime-dev at openjdk.java.net >> >> >> Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on >> Windows >> >> >> >> >> >> Ping : could I get a second review ? >> >> >> >> >> >> Bug : >> >> >> https://bugs.openjdk.java.net/browse/JDK-8202427 >> >> >> Change : >> >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ >> >> >> >> >> >> >> >> >> > We could also just print the MB value , let's see what other think. >> >> >> > Another option might be to have a flexible output (kB for smaller >> >> >> memory >> >> >> > values , MB (or GB) for larger ) ? >> >> >> >> >> >> Martin suggested to just print the MB values, what do you think ? >> >> >> >> >> >> >> >> >> Thanks, Matthias >> >> >> >> >> >> >> >> >> > -----Original Message----- >> >> >> > From: Baesken, Matthias >> >> >> > Sent: Mittwoch, 2. Mai 2018 13:00 >> >> >> > To: Doerr, Martin ; 'hotspot- >> >> >> > dev at openjdk.java.net' ; hotspot- >> >> >> > runtime-dev at openjdk.java.net >> >> >> > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on >> >> Windows >> >> >> > >> >> >> > Hi Martin, thanks for your input . >> >> >> > I add hotspot-runtime-dev . >> >> >> > >> >> >> > > >> >> >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe >> >> other >> >> >> > > reviewers would like to comment on this, too. >> >> >> > > >> >> >> > >> >> >> > We could also just print the MB value , let's see what other think. >> >> >> > Another option might be to have a flexible output (kB for smaller >> >> >> memory >> >> >> > values , MB (or GB) for larger ) ? >> >> >> > >> >> >> > Best regards, Matthias >> >> >> > >> >> >> > >> >> >> > > -----Original Message----- >> >> >> > > From: Doerr, Martin >> >> >> > > Sent: Mittwoch, 2. Mai 2018 12:53 >> >> >> > > To: Baesken, Matthias ; 'hotspot- >> >> >> > > dev at openjdk.java.net' >> >> >> > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on >> >> Windows >> >> >> > > >> >> >> > > Hi Matthias, >> >> >> > > >> >> >> > > looks like a nice enhancement. We can get substantially more >> >> >> information. >> >> >> > > >> >> >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe >> >> other >> >> >> > > reviewers would like to comment on this, too. >> >> >> > > >> >> >> > > We should have line breaks. >> >> >> > > >> >> >> > > Best regards, >> >> >> > > Martin >> >> >> > > >> >> >> > > >> >> >> > > -----Original Message----- >> >> >> > > From: hotspot-dev [mailto:hotspot-dev- >> bounces at openjdk.java.net] >> >> On >> >> >> > > Behalf Of Baesken, Matthias >> >> >> > > Sent: Montag, 30. April 2018 16:53 >> >> >> > > To: 'hotspot-dev at openjdk.java.net' > >> dev at openjdk.java.net> >> >> >> > > Subject: [RFR] 8202427: Enhance os::print_memory_info on >> Windows >> >> >> > > >> >> >> > > On Windows, >> >> >> > > the os::print_memory_info misses a few memory-related infos >> that >> >> >> might >> >> >> > > be interesting : >> >> >> > > - current and peak WorkingSet size (= physical memory assigned to >> >> the >> >> >> > > process) >> >> >> > > - current and peak commit charge (also known as "private bytes" in >> >> >> > Windows >> >> >> > > tools) >> >> >> > > - on 32bit Windows : >> >> >> > > user-mode portion/free user mode portion of virtual address- >> space >> >> >> > > (Total/AvailVirtual) because it shows how close do we get to the 2- >> 4 >> >> GB >> >> >> per >> >> >> > > process border. >> >> >> > > >> >> >> > > >> >> >> > > - the current naming of "swap/free-swap" memory is a bit >> misleading; >> >> >> > > in the Windows world swap is a special page file used for UWP >> apps. >> >> >> > > (see Windows Internals : "The swap file" (chapter 5 Memory >> >> >> > management) >> >> >> > > Windows 8 added another page file called a swap file. It is ... >> >> exclusively >> >> >> > used >> >> >> > > for UWP (Universal Windows Platform) apps. >> >> >> > > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the >> >> UWP >> >> >> > > related values, it is about page file sizes >> >> >> > > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just >> >> >> virtual >> >> >> > > memory might be more appropriate). >> >> >> > > >> >> >> > > >> >> >> > > https://msdn.microsoft.com/de- >> >> >> > > de/library/windows/desktop/aa366770(v=vs.85).aspx >> >> >> > > >> >> >> > > documents it in the following way: >> >> >> > > ullTotalPageFile: >> >> >> > > The current committed memory limit for the system or the current >> >> >> process, >> >> >> > > whichever is smaller,... >> >> >> > > >> >> >> > > ullAvailPageFile : >> >> >> > > The maximum amount of memory the current process can commit, >> in >> >> >> > bytes. >> >> >> > > This value is equal to or smaller than the system-wide available >> >> commit >> >> >> > value >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > Aditionally I suggest having output of the various memory-values >> in M >> >> >> > > (megabyte) as well , the k (kilobyte) output sometimes gives very >> >> high >> >> >> and >> >> >> > > unreadable numbers). >> >> >> > > >> >> >> > > >> >> >> > > Could you please review my change ? >> >> >> > > >> >> >> > > >> >> >> > > Bug : >> >> >> > > >> >> >> > > https://bugs.openjdk.java.net/browse/JDK-8202427 >> >> >> > > >> >> >> > > >> >> >> > > Change : >> >> >> > > >> >> >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ >> >> >> > > >> >> >> > > >> >> >> > > Thanks, Matthias >> >> >> > > >> >> > From coleen.phillimore at oracle.com Sat May 19 13:20:29 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Sat, 19 May 2018 09:20:29 -0400 Subject: RFR(L): 8176808: Split up metaspace.cpp In-Reply-To: References: <32826e8139614cd68f0b4548ca8c044c@sap.com> <0da5bc3d-3f3a-dd09-c635-fdc67ed7ecaa@oracle.com> Message-ID: <3c69f828-d3b1-00e8-bc37-6c2796474ee1@oracle.com> http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace-cpp/webrev.02/webrev/src/hotspot/share/memory/metaspace/metaspaceCommon.hpp.udiff.html Not your change but metaspaceCommon.hpp has these with a trailing _ ? #ifndef SHARE_MEMORY_METASPACE_METASPACECOMMON_HPP_ #define SHARE_MEMORY_METASPACE_METASPACECOMMON_HPP_ The other new files have it this too.? I don't think we have this anywhere else and it looks strange.? I don't like the _. http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace-cpp/webrev.02/webrev/src/hotspot/share/memory/metaspace/virtualSpaceList.cpp.html This is missing a Copyright.? Are you supposed to put the SAP line on these new files also? http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace-cpp/webrev.02/webrev/src/hotspot/share/memory/metaspace/occupancyMap.cpp.html 3 * Copyright (c) 2018, SAP. Is this SAP copyright correct?? I thought there was more on the line.? Maybe not. For the _ changes and copyright change, I don't need to see another webrev.? This refactoring and the use of namespace metaspace is really nice.? It does give an appreciation for how complicated the memory management for metaspace became since its start as: class Metaspace : public Arena {}. Thanks! Coleen On 5/19/18 12:55 AM, Thomas St?fe wrote: > Hi, Coleen and Axel, > > thanks a lot for the reviews! I will be very happy once this is > removed from my patch queue, since merging the patch is becoming a pain. > > New webrev. > > As suggested, I removed the "internals" inner namespace, which leaves > us with "metaspace::". Nothing else changed. Still fine? > > Webrevs: > Incremental: > http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace-cpp/webrev.01-to-02/webrev/ > > Full: > http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace-cpp/webrev.02/webrev/ > > > Thanks, Thomas > > > On Sat, May 19, 2018 at 12:41 AM, > wrote: > > > > On 5/17/18 6:15 AM, Siebenborn, Axel wrote: > > Hi Thomas, > The change looks good . > I should have read your whole mail before starting looking at > the changes, as you explain what you have done and what you > didn't do ??. > Concerning the decision using ::metaspace vs. > ::metaspace::internal: > I would use ::metaspace for the internal classes and leave the > public classes in the global namespace. > This is more consistent to hotspot code where namespaces are > rarely used (if this counts as an argument). > I would change that in this change, as having having > ::metaspace::internal and nothing in ::metaspace seems to be > the worst alternative. > > > Yes, I agree with this.? One namespace to encapsulate things like > ChunkManager and SpaceManager and these things is good, without > adding the another nested "internals" namespace. That's too much > for us :) > > Coleen > > > Altogether, this is really a good refactoring. > > Cheers, > Axel > > -----Original Message----- > From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- > > bounces at openjdk.java.net > ] On Behalf Of Thomas St?fe > Sent: Montag, 14. Mai 2018 15:34 > To: Hotspot dev runtime > > > Subject: Re: RFR(L): 8176808: Split up metaspace.cpp > > Ping... > > Is there anything I can do to make review easier? This is > really > mostly code shuffling around (out of metaspace.cpp into > new files), so > with a bit of effort I could cook up some diffs for those > parts. > > ..Thomas > > On Fri, May 11, 2018 at 8:31 AM, Thomas St?fe > > > wrote: > > All builds are fixed, jdk-submit tests ran through > sucessfully. > > Latest webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace- > > > cpp/webrev.01/webrev/ > > Only difference to the first webrev is the placement > of a single > semicolon in occupancyMap.hpp, to satisfy the Solaris > compiler. > > Thanks, Thomas > > > > On Wed, May 9, 2018 at 5:08 PM, Thomas St?fe > > > > wrote: > > Hi all, > > This was a long time coming. Lets cut up this beast :) > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8176808 > > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- > > > metaspace-cpp/webrev.00/webrev/ > > This change splits up metaspace.cpp into multiple > parts. Note that I > did not change much code (exceptions see below in > details) - so this > is mostly moving code around. > > This change follows the scheme tentatively > outlined in JDK-8201572: > > - metaspace internal classes go into the sub directory > share/memory/metaspace. Ideally, those should > never be accessed from > outside, apart from other metaspace classes and > metaspace tests. They > also go into the namespace ::metaspace::internals. > > - metaspace external classes (MetaspaceUtils, > ClassLoaderMetaspace, > etc) remain inside share/memory and, for now, > remain in the global > namespace. > > Note: the original idea was to move metaspace > external classes to > namespace ::metaspace. But I shied away from that, > since that would > mean fixing up all usages of these classes either > with metaspace:: > scope or with using declarations. I had to make a > cut at some point. > > So here is something to decide: > - should we then get rid of the > ::metaspace::internals namespace, move > metaspace-internal classes to the enclosing > ::metaspace namespace and > leave external classes in the global namespace ? > - or should we follow through (maybe in a future > patch): internal > classes in ::metaspace::internal, and move > external classes to > ::metaspace ? > > Some more details: > > - the following classes moved out of metaspace.cpp > into namespace > "metaspace::internal" and the metaspace sub directory: > MetaDebug, ChunkManager, SpaceManager, > BlockFreeList and > > SmallBlocks, > > OccupancyMap, VirtualSpaceNode and > VirtualSpaceList, the > PrintCLDMetaspaceInfoClosure. > > - I also moved metachunk.hpp to internals, since > class Metachunk is > not used externally. What is used externally is > Metablock though, so I > split up metachunk.hpp into metabase.hpp, > metablock.hpp and > metachunk.hpp, holding Metabase, Metablock and > Metachunk, > respectively. > > - Now we see some ugly seams: metaspace.hpp is > supposed to be the > outside API contract, however, it contains > implementation details > which should not be there and which manifest now > by having to use the > metaspace::internals namespace here and there. If > we care, we could > solve this with a bit redesigning. > > - The ChunkManager gtest and the SpaceManager > gtest had been > implemented inside metaspace.cpp, since it was not > possible to access > the internal classes those tests needed in any > other way. Since that > is now possible, I moved the coding out to > gtest-land and made them > real gtests (exchanging the asserts for real gtest > ASSERTS, among > other things). > Note that there are some tests left in metaspace.cpp > (TestMetaspaceUtilsTest, TestVirtualSpaceNodeTest) > - those were no > gtests-in-disguise but were part of > -XX:+ExecuteInternalVMTests. I > guess these tests precede the gtest framework? I > leave it up to others > to decide what to do with them and to potentially > move them out of > metaspace.cpp. > Side note: -XX:+ExecuteInternalVMTests seems to > have bitrotted, see > https://bugs.openjdk.java.net/browse/JDK-8202848 > . > Does this get tested > regularly? > > - metaspace.cpp is quite a bit smaller now - from > ~5000 loc down to > ~1700 loc. Arguably, one could split out more and > clean up more, but I > think this is a good start. > > --- > > I built locally on linux (release, fastdebug with > and without pch, > zero) and windows (64, 32bit). jdk-submit tests > ran through with a > build error on solaris sparc - I currently try to > reproduce that build > error with our very slow solaris machine. > > Thanks, Thomas > > > From thomas.stuefe at gmail.com Sat May 19 15:51:54 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 19 May 2018 17:51:54 +0200 Subject: RFR(L): 8176808: Split up metaspace.cpp In-Reply-To: <3c69f828-d3b1-00e8-bc37-6c2796474ee1@oracle.com> References: <32826e8139614cd68f0b4548ca8c044c@sap.com> <0da5bc3d-3f3a-dd09-c635-fdc67ed7ecaa@oracle.com> <3c69f828-d3b1-00e8-bc37-6c2796474ee1@oracle.com> Message-ID: Hi Coleen, On Sat, May 19, 2018 at 3:20 PM, wrote: > > http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- > metaspace-cpp/webrev.02/webrev/src/hotspot/share/memory/metaspace/ > metaspaceCommon.hpp.udiff.html > > Not your change but metaspaceCommon.hpp has these with a trailing _ ? > > #ifndef SHARE_MEMORY_METASPACE_METASPACECOMMON_HPP_ > #define SHARE_MEMORY_METASPACE_METASPACECOMMON_HPP_ > > The other new files have it this too. I don't think we have this anywhere > else and it looks strange. I don't like the _. > > Sure, I remove it. These lines were autogenerated by CDT. Maybe I should return to good old vi :) > http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- > metaspace-cpp/webrev.02/webrev/src/hotspot/share/memory/metaspace/ > virtualSpaceList.cpp.html > > This is missing a Copyright. Are you supposed to put the SAP line on > these new files also? > I usually place SAP copyrights in addition to Oracle ones on those files fully created by me. > > http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- > metaspace-cpp/webrev.02/webrev/src/hotspot/share/ > memory/metaspace/occupancyMap.cpp.html > > 3 * Copyright (c) 2018, SAP. > > > Is this SAP copyright correct? I thought there was more on the line. > Maybe not. > I'll double check on Tuesday with my coworkers. jdk-submit seems to be dead currently anyway, and I want to put the fix one last time thru the checks. > > For the _ changes and copyright change, I don't need to see another > webrev. This refactoring and the use of namespace metaspace is really > nice. It does give an appreciation for how complicated the memory > management for metaspace became since its start as: > > class Metaspace : public Arena {}. > > hah! yes indeed :) But as I mentioned before, my suspicion is that were we to recreate it from scratch it would soon grow in complexity until something close to the current form. Maybe not, but I would not be surprised. > Thanks! > Sure. Thanks for reviewing! ..Thomas > > Coleen > > > > On 5/19/18 12:55 AM, Thomas St?fe wrote: > > Hi, Coleen and Axel, > > thanks a lot for the reviews! I will be very happy once this is removed > from my patch queue, since merging the patch is becoming a pain. > > New webrev. > > As suggested, I removed the "internals" inner namespace, which leaves us > with "metaspace::". Nothing else changed. Still fine? > > Webrevs: > Incremental: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- > metaspace-cpp/webrev.01-to-02/webrev/ > Full: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- > metaspace-cpp/webrev.02/webrev/ > > Thanks, Thomas > > > On Sat, May 19, 2018 at 12:41 AM, wrote: > >> >> >> On 5/17/18 6:15 AM, Siebenborn, Axel wrote: >> >>> Hi Thomas, >>> The change looks good . >>> I should have read your whole mail before starting looking at the >>> changes, as you explain what you have done and what you didn't do ??. >>> Concerning the decision using ::metaspace vs. ::metaspace::internal: >>> I would use ::metaspace for the internal classes and leave the public >>> classes in the global namespace. >>> This is more consistent to hotspot code where namespaces are rarely used >>> (if this counts as an argument). >>> I would change that in this change, as having having >>> ::metaspace::internal and nothing in ::metaspace seems to be the worst >>> alternative. >>> >> >> Yes, I agree with this. One namespace to encapsulate things like >> ChunkManager and SpaceManager and these things is good, without adding the >> another nested "internals" namespace. That's too much for us :) >> >> Coleen >> >> >>> Altogether, this is really a good refactoring. >>> >>> Cheers, >>> Axel >>> >>> -----Original Message----- >>>> From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- >>>> bounces at openjdk.java.net] On Behalf Of Thomas St?fe >>>> Sent: Montag, 14. Mai 2018 15:34 >>>> To: Hotspot dev runtime >>>> Subject: Re: RFR(L): 8176808: Split up metaspace.cpp >>>> >>>> Ping... >>>> >>>> Is there anything I can do to make review easier? This is really >>>> mostly code shuffling around (out of metaspace.cpp into new files), so >>>> with a bit of effort I could cook up some diffs for those parts. >>>> >>>> ..Thomas >>>> >>>> On Fri, May 11, 2018 at 8:31 AM, Thomas St?fe >>>> wrote: >>>> >>>>> All builds are fixed, jdk-submit tests ran through sucessfully. >>>>> >>>>> Latest webrev: >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace- >>>>> >>>> cpp/webrev.01/webrev/ >>>> >>>>> Only difference to the first webrev is the placement of a single >>>>> semicolon in occupancyMap.hpp, to satisfy the Solaris compiler. >>>>> >>>>> Thanks, Thomas >>>>> >>>>> >>>>> >>>>> On Wed, May 9, 2018 at 5:08 PM, Thomas St?fe >>>>> >>>> wrote: >>>> >>>>> Hi all, >>>>>> >>>>>> This was a long time coming. Lets cut up this beast :) >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8176808 >>>>>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- >>>>>> >>>>> metaspace-cpp/webrev.00/webrev/ >>>> >>>>> This change splits up metaspace.cpp into multiple parts. Note that I >>>>>> did not change much code (exceptions see below in details) - so this >>>>>> is mostly moving code around. >>>>>> >>>>>> This change follows the scheme tentatively outlined in JDK-8201572: >>>>>> >>>>>> - metaspace internal classes go into the sub directory >>>>>> share/memory/metaspace. Ideally, those should never be accessed from >>>>>> outside, apart from other metaspace classes and metaspace tests. They >>>>>> also go into the namespace ::metaspace::internals. >>>>>> >>>>>> - metaspace external classes (MetaspaceUtils, ClassLoaderMetaspace, >>>>>> etc) remain inside share/memory and, for now, remain in the global >>>>>> namespace. >>>>>> >>>>>> Note: the original idea was to move metaspace external classes to >>>>>> namespace ::metaspace. But I shied away from that, since that would >>>>>> mean fixing up all usages of these classes either with metaspace:: >>>>>> scope or with using declarations. I had to make a cut at some point. >>>>>> >>>>>> So here is something to decide: >>>>>> - should we then get rid of the ::metaspace::internals namespace, move >>>>>> metaspace-internal classes to the enclosing ::metaspace namespace and >>>>>> leave external classes in the global namespace ? >>>>>> - or should we follow through (maybe in a future patch): internal >>>>>> classes in ::metaspace::internal, and move external classes to >>>>>> ::metaspace ? >>>>>> >>>>>> Some more details: >>>>>> >>>>>> - the following classes moved out of metaspace.cpp into namespace >>>>>> "metaspace::internal" and the metaspace sub directory: >>>>>> MetaDebug, ChunkManager, SpaceManager, BlockFreeList and >>>>>> >>>>> SmallBlocks, >>>> >>>>> OccupancyMap, VirtualSpaceNode and VirtualSpaceList, the >>>>>> PrintCLDMetaspaceInfoClosure. >>>>>> >>>>>> - I also moved metachunk.hpp to internals, since class Metachunk is >>>>>> not used externally. What is used externally is Metablock though, so I >>>>>> split up metachunk.hpp into metabase.hpp, metablock.hpp and >>>>>> metachunk.hpp, holding Metabase, Metablock and Metachunk, >>>>>> respectively. >>>>>> >>>>>> - Now we see some ugly seams: metaspace.hpp is supposed to be the >>>>>> outside API contract, however, it contains implementation details >>>>>> which should not be there and which manifest now by having to use the >>>>>> metaspace::internals namespace here and there. If we care, we could >>>>>> solve this with a bit redesigning. >>>>>> >>>>>> - The ChunkManager gtest and the SpaceManager gtest had been >>>>>> implemented inside metaspace.cpp, since it was not possible to access >>>>>> the internal classes those tests needed in any other way. Since that >>>>>> is now possible, I moved the coding out to gtest-land and made them >>>>>> real gtests (exchanging the asserts for real gtest ASSERTS, among >>>>>> other things). >>>>>> Note that there are some tests left in metaspace.cpp >>>>>> (TestMetaspaceUtilsTest, TestVirtualSpaceNodeTest) - those were no >>>>>> gtests-in-disguise but were part of -XX:+ExecuteInternalVMTests. I >>>>>> guess these tests precede the gtest framework? I leave it up to others >>>>>> to decide what to do with them and to potentially move them out of >>>>>> metaspace.cpp. >>>>>> Side note: -XX:+ExecuteInternalVMTests seems to have bitrotted, see >>>>>> https://bugs.openjdk.java.net/browse/JDK-8202848. Does this get >>>>>> tested >>>>>> regularly? >>>>>> >>>>>> - metaspace.cpp is quite a bit smaller now - from ~5000 loc down to >>>>>> ~1700 loc. Arguably, one could split out more and clean up more, but I >>>>>> think this is a good start. >>>>>> >>>>>> --- >>>>>> >>>>>> I built locally on linux (release, fastdebug with and without pch, >>>>>> zero) and windows (64, 32bit). jdk-submit tests ran through with a >>>>>> build error on solaris sparc - I currently try to reproduce that build >>>>>> error with our very slow solaris machine. >>>>>> >>>>>> Thanks, Thomas >>>>>> >>>>> >> > > From thomas.stuefe at gmail.com Sun May 20 06:49:57 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sun, 20 May 2018 08:49:57 +0200 Subject: RFR(xs): 8203455: jcmd: VM.metaspace: print loader name for anonymous CLDs. Message-ID: Hi all, may I please have reviews for this small addition. Bug: https://bugs.openjdk.java.net/browse/JDK-8203455 Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.00/webrev/ VM.metaspace show-loaders can be used to display loaders (well, really CLD instances). For anonymous CLDs, only "anonymous" is shown. It would be helpful to show to which loader these CLD are assigned. Before: "272: ClassLoaderData 0x00007f5ba0538f10 for anonymous class" http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-before.txt After patch: "268: CLD 0x00007ff0c45738f0 for , loaded by app, instance of jdk.internal.loader.ClassLoaders$AppClassLoader" http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-after.txt -- Notes: this patch has a bit more lines than I liked because I did not find a singly utility function in ClassloaderData which fit my purpose. I wanted to see name and class of the associated loader for both normal and unloading case. ClassloaderData::loader_name() has a number of shortcomings, I opened https://bugs.openjdk.java.net/browse/JDK-8203456 as a follow up rfe. Thank you, Thomas From thomas.stuefe at gmail.com Sun May 20 06:58:38 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sun, 20 May 2018 08:58:38 +0200 Subject: RFR(s): 8203219: VM.metaspace jcmd should optionally show loaded classes for loaders Message-ID: Hi all, can I have reviews for this small addition to the VM.metaspace jcmd, which adds the ability to list classes loaded by each class loader? Thank you! bug: https://bugs.openjdk.java.net/browse/JDK-8203219 webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/webrev.00/webrev/index.html Example output: http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/example.txt Kind Regards, Thomas From david.holmes at oracle.com Mon May 21 01:38:05 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 21 May 2018 11:38:05 +1000 Subject: RFR (XS) 8202360: [TESTBUG] runtime/LoadClass/TestResize.java needs to print output when it fails In-Reply-To: References: Message-ID: Hi Gerard, On 19/05/2018 4:54 AM, Gerard Ziemski wrote: > hi David, > >> On May 18, 2018, at 12:59 AM, David Holmes wrote: >> >> Hi Gerard, >> >> On 18/05/2018 2:21 AM, Gerard Ziemski wrote: >>> hi David, >>> Thank you for the review! >>>> On May 16, 2018, at 4:54 PM, David Holmes wrote: >>>> >>>> Hi Gerard, >>>> >>>> On 17/05/2018 3:47 AM, Gerard Ziemski wrote: >>>>> Hi all, >>>>> Please review this small enhancement where we print out additional information (i.e. output from PrintSystemDictionaryAtExit), when we detect the failure. >>>>> https://bugs.openjdk.java.net/browse/JDK-8202360 >>>>> http://cr.openjdk.java.net/~gziemski/8202360_rev1/ >>>> >>>> I don't understand what your fix is doing - sorry. >>> The fix takes all the available output bytes from the process (using Scanner to grab all tokens at once) and prints all the output from the process (i.e. PrintSystemDictionaryAtExit, and there is a lot of it) at failure. >> >> Okay but that's the rest of the output that hasn't yet been processed - correct? So the line of output that caused the failure won't be there as it's already been grabbed by the scanner and processed. > > Good catch, I changed the fix to make sure we print all the output. > > >>>> >>>> What I wanted to see was the complete history of lines of the form >>>> >>>> "Java dictionary (table_size=107, classes=6)" >>>> >>>> so that we can see how the table size and number of classes have changed leading up to the failure. But I realize now that we already get that from the: >>>> >>>> System.out.println("PASS table_size:"+table_size+", classes:"+classes+" OK"); >>>> >>>> All that is missing is the printing of the table_size and classes when we throw the exception. That can be trivially fixed: >>>> >>>> if (loadFactor > MAX_LOAD_FACTOR) { >>>> + System.out.println("FAIL table_size:"+table_size+", classes:"+classes); >>>> throw new RuntimeException("Load factor too high, expected MAX "+MAX_LOAD_FACTOR+", got "+loadFactor); >>>> } else { >>>> System.out.println("PASS table_size:"+table_size+", classes:"+classes+" OK"); >>>> } >>> OK, so you determined that only that one piece of info was missing in the end in your case, but maybe the next person might want to see everything that PrintSystemDictionaryAtExit provides? >>> As long as we?re touching the test - should we then stick with the solution that prints out everything from PrintSystemDictionaryAtExit? >> >> I'm not sure how anything that comes later in PrintSystemDictionaryAtExit would help with a failure that happened earlier, but if you see value it then that's fine. However I think you need my suggestion as well to get the information that actually caused the failure. > > We now print all the output from PrintSystemDictionaryAtExit, but So you'll print all the good reports for the "Java dictionary ..." lines then if there is a failure print the entire output. That's okay - though as I mention in the bug report it does tend to hit the jtreg output buffer limit. > also following your suggestion we summarize the failure details in > the exception itself like: > > java.lang.RuntimeException: Load factor too high, expected MAX 5.0, got 467.30841121495325 [table size 107, number of clases 50002] Okay - that helps with the immediate problem I had. I also now realize that the failure in this test is showing is that the SD did not in fact re-size as expected! >> Some sample output in the bug report would be useful. > > I attached a sample output to the issue as requested. Thanks. > > https://bugs.openjdk.java.net/browse/JDK-8202360 > http://cr.openjdk.java.net/~gziemski/8202360_rev2 A few suggestions: 73 String string_out = ""; 74 while (line != null) { 75 string_out += line+"\n"; As this is not in fact a constant I would suggest using a (large-size) StringBuilder directly and avoid all the implicit StringBuilder and String usage that the '+' operator will hide. The variable can just be called 'output'. 88 System.out.println(string_out); Please precede with a comment // We've hit an error so print all of the output parsed so far, and // then the remaining output. 93 throw new RuntimeException("Load factor too high, expected MAX "+MAX_LOAD_FACTOR+", got "+loadFactor+" [table size "+table_size+", number of clases "+classes+"]"); Please split this line up to make it much shorter. I'm tempted to ask you to fix this missing spaces around the '+' operator but that bad-style is prevalent throughout this test. (This violates a number of Java style rules.) Don't forget to add second copyright year. Thanks, David > cheers > From david.holmes at oracle.com Mon May 21 02:05:20 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 21 May 2018 12:05:20 +1000 Subject: RFR 8203381 Replace InstanceKlass::allocate_instance_handle with JavaCalls::construct_new_instance In-Reply-To: <2a449ddb-4512-193f-d6d1-152989b8526c@oracle.com> References: <2a449ddb-4512-193f-d6d1-152989b8526c@oracle.com> Message-ID: <46dd5e52-99c9-9458-1859-5c39d806274b@oracle.com> Hi Ioi, On 19/05/2018 2:34 AM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8203381 > http://cr.openjdk.java.net/~iklam/jdk11/8203381-construct-new-instance.v01/ > > This patch removes a lot of the boilerplate code for allocating a > Java object and calling its constructor. Hopefully the code is cleaner > and easier to read. That's a great cleanup! Why didn't we do this years ago! :) Only nit is that now call_special is replaced with construct_new_instance the argument line indentation is no longer correct. Great comments on the Thread construction process too! Hope it didn't take you too long to figure out why you couldn't make the replacement in those cases! Thanks, David > Also: > > Added assert(InstanceKlass::is_initialized(), ...) in case where > we cannot call JavaCalls::construct_new_instance for thread objects. > > Replaced unnecessary SystemDictionary::resolve_or_null() calls with > SystemDictionary::XXX_klass(). > > Tested with hs-tier[1,2,3,4,5] > > Thanks > - Ioi > > > > > From david.holmes at oracle.com Mon May 21 02:11:13 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 21 May 2018 12:11:13 +1000 Subject: RFR(L): 8176808: Split up metaspace.cpp In-Reply-To: References: <32826e8139614cd68f0b4548ca8c044c@sap.com> <0da5bc3d-3f3a-dd09-c635-fdc67ed7ecaa@oracle.com> <3c69f828-d3b1-00e8-bc37-6c2796474ee1@oracle.com> Message-ID: <6f75acff-6389-bf49-df4d-4aa3660638f1@oracle.com> On 20/05/2018 1:51 AM, Thomas St?fe wrote: > Hi Coleen, > > On Sat, May 19, 2018 at 3:20 PM, wrote: > >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- >> metaspace-cpp/webrev.02/webrev/src/hotspot/share/memory/metaspace/ >> metaspaceCommon.hpp.udiff.html >> >> Not your change but metaspaceCommon.hpp has these with a trailing _ ? >> >> #ifndef SHARE_MEMORY_METASPACE_METASPACECOMMON_HPP_ >> #define SHARE_MEMORY_METASPACE_METASPACECOMMON_HPP_ >> >> The other new files have it this too. I don't think we have this anywhere >> else and it looks strange. I don't like the _. >> >> > Sure, I remove it. These lines were autogenerated by CDT. Maybe I should > return to good old vi :) > > >> http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- >> metaspace-cpp/webrev.02/webrev/src/hotspot/share/memory/metaspace/ >> virtualSpaceList.cpp.html >> >> This is missing a Copyright. Are you supposed to put the SAP line on >> these new files also? >> > > I usually place SAP copyrights in addition to Oracle ones on those files > fully created by me. > > >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- >> metaspace-cpp/webrev.02/webrev/src/hotspot/share/ >> memory/metaspace/occupancyMap.cpp.html >> >> 3 * Copyright (c) 2018, SAP. >> >> >> Is this SAP copyright correct? I thought there was more on the line. >> Maybe not. >> > > I'll double check on Tuesday with my coworkers. jdk-submit seems to be dead > currently anyway, and I want to put the fix one last time thru the checks. The format seems to be e.g.: * Copyright (c) 2018 SAP SE. All rights reserved. Thanks, David > >> >> For the _ changes and copyright change, I don't need to see another >> webrev. This refactoring and the use of namespace metaspace is really >> nice. It does give an appreciation for how complicated the memory >> management for metaspace became since its start as: >> >> class Metaspace : public Arena {}. >> >> > hah! yes indeed :) > > But as I mentioned before, my suspicion is that were we to recreate it from > scratch it would soon grow in complexity until something close to the > current form. Maybe not, but I would not be surprised. > > >> Thanks! >> > > Sure. Thanks for reviewing! > > ..Thomas > > >> >> Coleen >> >> >> >> On 5/19/18 12:55 AM, Thomas St?fe wrote: >> >> Hi, Coleen and Axel, >> >> thanks a lot for the reviews! I will be very happy once this is removed >> from my patch queue, since merging the patch is becoming a pain. >> >> New webrev. >> >> As suggested, I removed the "internals" inner namespace, which leaves us >> with "metaspace::". Nothing else changed. Still fine? >> >> Webrevs: >> Incremental: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- >> metaspace-cpp/webrev.01-to-02/webrev/ >> Full: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- >> metaspace-cpp/webrev.02/webrev/ >> >> Thanks, Thomas >> >> >> On Sat, May 19, 2018 at 12:41 AM, wrote: >> >>> >>> >>> On 5/17/18 6:15 AM, Siebenborn, Axel wrote: >>> >>>> Hi Thomas, >>>> The change looks good . >>>> I should have read your whole mail before starting looking at the >>>> changes, as you explain what you have done and what you didn't do ??. >>>> Concerning the decision using ::metaspace vs. ::metaspace::internal: >>>> I would use ::metaspace for the internal classes and leave the public >>>> classes in the global namespace. >>>> This is more consistent to hotspot code where namespaces are rarely used >>>> (if this counts as an argument). >>>> I would change that in this change, as having having >>>> ::metaspace::internal and nothing in ::metaspace seems to be the worst >>>> alternative. >>>> >>> >>> Yes, I agree with this. One namespace to encapsulate things like >>> ChunkManager and SpaceManager and these things is good, without adding the >>> another nested "internals" namespace. That's too much for us :) >>> >>> Coleen >>> >>> >>>> Altogether, this is really a good refactoring. >>>> >>>> Cheers, >>>> Axel >>>> >>>> -----Original Message----- >>>>> From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- >>>>> bounces at openjdk.java.net] On Behalf Of Thomas St?fe >>>>> Sent: Montag, 14. Mai 2018 15:34 >>>>> To: Hotspot dev runtime >>>>> Subject: Re: RFR(L): 8176808: Split up metaspace.cpp >>>>> >>>>> Ping... >>>>> >>>>> Is there anything I can do to make review easier? This is really >>>>> mostly code shuffling around (out of metaspace.cpp into new files), so >>>>> with a bit of effort I could cook up some diffs for those parts. >>>>> >>>>> ..Thomas >>>>> >>>>> On Fri, May 11, 2018 at 8:31 AM, Thomas St?fe >>>>> wrote: >>>>> >>>>>> All builds are fixed, jdk-submit tests ran through sucessfully. >>>>>> >>>>>> Latest webrev: >>>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace- >>>>>> >>>>> cpp/webrev.01/webrev/ >>>>> >>>>>> Only difference to the first webrev is the placement of a single >>>>>> semicolon in occupancyMap.hpp, to satisfy the Solaris compiler. >>>>>> >>>>>> Thanks, Thomas >>>>>> >>>>>> >>>>>> >>>>>> On Wed, May 9, 2018 at 5:08 PM, Thomas St?fe >>>>>> >>>>> wrote: >>>>> >>>>>> Hi all, >>>>>>> >>>>>>> This was a long time coming. Lets cut up this beast :) >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8176808 >>>>>>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- >>>>>>> >>>>>> metaspace-cpp/webrev.00/webrev/ >>>>> >>>>>> This change splits up metaspace.cpp into multiple parts. Note that I >>>>>>> did not change much code (exceptions see below in details) - so this >>>>>>> is mostly moving code around. >>>>>>> >>>>>>> This change follows the scheme tentatively outlined in JDK-8201572: >>>>>>> >>>>>>> - metaspace internal classes go into the sub directory >>>>>>> share/memory/metaspace. Ideally, those should never be accessed from >>>>>>> outside, apart from other metaspace classes and metaspace tests. They >>>>>>> also go into the namespace ::metaspace::internals. >>>>>>> >>>>>>> - metaspace external classes (MetaspaceUtils, ClassLoaderMetaspace, >>>>>>> etc) remain inside share/memory and, for now, remain in the global >>>>>>> namespace. >>>>>>> >>>>>>> Note: the original idea was to move metaspace external classes to >>>>>>> namespace ::metaspace. But I shied away from that, since that would >>>>>>> mean fixing up all usages of these classes either with metaspace:: >>>>>>> scope or with using declarations. I had to make a cut at some point. >>>>>>> >>>>>>> So here is something to decide: >>>>>>> - should we then get rid of the ::metaspace::internals namespace, move >>>>>>> metaspace-internal classes to the enclosing ::metaspace namespace and >>>>>>> leave external classes in the global namespace ? >>>>>>> - or should we follow through (maybe in a future patch): internal >>>>>>> classes in ::metaspace::internal, and move external classes to >>>>>>> ::metaspace ? >>>>>>> >>>>>>> Some more details: >>>>>>> >>>>>>> - the following classes moved out of metaspace.cpp into namespace >>>>>>> "metaspace::internal" and the metaspace sub directory: >>>>>>> MetaDebug, ChunkManager, SpaceManager, BlockFreeList and >>>>>>> >>>>>> SmallBlocks, >>>>> >>>>>> OccupancyMap, VirtualSpaceNode and VirtualSpaceList, the >>>>>>> PrintCLDMetaspaceInfoClosure. >>>>>>> >>>>>>> - I also moved metachunk.hpp to internals, since class Metachunk is >>>>>>> not used externally. What is used externally is Metablock though, so I >>>>>>> split up metachunk.hpp into metabase.hpp, metablock.hpp and >>>>>>> metachunk.hpp, holding Metabase, Metablock and Metachunk, >>>>>>> respectively. >>>>>>> >>>>>>> - Now we see some ugly seams: metaspace.hpp is supposed to be the >>>>>>> outside API contract, however, it contains implementation details >>>>>>> which should not be there and which manifest now by having to use the >>>>>>> metaspace::internals namespace here and there. If we care, we could >>>>>>> solve this with a bit redesigning. >>>>>>> >>>>>>> - The ChunkManager gtest and the SpaceManager gtest had been >>>>>>> implemented inside metaspace.cpp, since it was not possible to access >>>>>>> the internal classes those tests needed in any other way. Since that >>>>>>> is now possible, I moved the coding out to gtest-land and made them >>>>>>> real gtests (exchanging the asserts for real gtest ASSERTS, among >>>>>>> other things). >>>>>>> Note that there are some tests left in metaspace.cpp >>>>>>> (TestMetaspaceUtilsTest, TestVirtualSpaceNodeTest) - those were no >>>>>>> gtests-in-disguise but were part of -XX:+ExecuteInternalVMTests. I >>>>>>> guess these tests precede the gtest framework? I leave it up to others >>>>>>> to decide what to do with them and to potentially move them out of >>>>>>> metaspace.cpp. >>>>>>> Side note: -XX:+ExecuteInternalVMTests seems to have bitrotted, see >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8202848. Does this get >>>>>>> tested >>>>>>> regularly? >>>>>>> >>>>>>> - metaspace.cpp is quite a bit smaller now - from ~5000 loc down to >>>>>>> ~1700 loc. Arguably, one could split out more and clean up more, but I >>>>>>> think this is a good start. >>>>>>> >>>>>>> --- >>>>>>> >>>>>>> I built locally on linux (release, fastdebug with and without pch, >>>>>>> zero) and windows (64, 32bit). jdk-submit tests ran through with a >>>>>>> build error on solaris sparc - I currently try to reproduce that build >>>>>>> error with our very slow solaris machine. >>>>>>> >>>>>>> Thanks, Thomas >>>>>>> >>>>>> >>> >> >> From david.holmes at oracle.com Mon May 21 02:23:18 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 21 May 2018 12:23:18 +1000 Subject: RFR(xs): 8203455: jcmd: VM.metaspace: print loader name for anonymous CLDs. In-Reply-To: References: Message-ID: Hi Thomas, There's been a bit of discussion lately about how to "describe" classloaders: https://bugs.openjdk.java.net/browse/JDK-8202605 I'd rather not see another chunk of code added that 8202605 will have to fix up. Can you live with the less-than-ideal output of the existing function, under the expectation it will be updated to do a better job in the future? Thanks, David On 20/05/2018 4:49 PM, Thomas St?fe wrote: > Hi all, > > may I please have reviews for this small addition. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203455 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.00/webrev/ > > VM.metaspace show-loaders can be used to display loaders (well, really > CLD instances). > > For anonymous CLDs, only "anonymous" is shown. It would be helpful to > show to which loader these CLD are assigned. > > Before: > > "272: ClassLoaderData 0x00007f5ba0538f10 for anonymous class" > > http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-before.txt > > After patch: > > "268: CLD 0x00007ff0c45738f0 for , loaded by app, > instance of jdk.internal.loader.ClassLoaders$AppClassLoader" > > http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-after.txt > > -- > Notes: this patch has a bit more lines than I liked because I did not > find a singly utility function in ClassloaderData which fit my > purpose. I wanted to see name and class of the associated loader for > both normal and unloading case. ClassloaderData::loader_name() has a > number of shortcomings, I opened > https://bugs.openjdk.java.net/browse/JDK-8203456 as a follow up rfe. > > Thank you, > > Thomas > From kim.barrett at oracle.com Mon May 21 03:52:12 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 21 May 2018 05:52:12 +0200 Subject: RFR 8203183: vmTestbase/vm/mlvm/anonloader/stress/oome/heap/Test.java times out In-Reply-To: <24278661-d13e-770c-a8f9-a010b27c8db2@oracle.com> References: <24278661-d13e-770c-a8f9-a010b27c8db2@oracle.com> Message-ID: > On May 18, 2018, at 5:15 PM, Harold David Seigel wrote: > > Hi, > > Please review this fix for JDK-8203183. The fix is to run the test with -Xmx128m to prevent the test from timing out. The fix was tested on Linux X64, Mac OS X, Windows, and Solaris. > > Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8203183/webrev/index.html > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8203183 > > Thanks, Harold Looks good. From thomas.stuefe at gmail.com Mon May 21 06:46:41 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 21 May 2018 08:46:41 +0200 Subject: RFR(xs): 8203455: jcmd: VM.metaspace: print loader name for anonymous CLDs. In-Reply-To: References: Message-ID: Hi David, On Mon, May 21, 2018 at 4:23 AM, David Holmes wrote: > Hi Thomas, > > There's been a bit of discussion lately about how to "describe" > classloaders: > > https://bugs.openjdk.java.net/browse/JDK-8202605 > > I'd rather not see another chunk of code added that 8202605 will have to fix > up. Can you live with the less-than-ideal output of the existing function, > under the expectation it will be updated to do a better job in the future? > 8206205 does not alleviate the need for this change, since currently we do not print any loader information for anonymous classes. Originally I planned to fix up this patch with a follow up patch later once those utility functions are in place in ClassloaderData. But yes, I can save us some review work by waiting a bit and then doing only one change. Lets hope it does not take too long, I have more changes coming up which print class loader names. So I retract this RFR for now and will repost another patch once 8206205 is done. Thanks, Thomas > Thanks, > David > > > On 20/05/2018 4:49 PM, Thomas St?fe wrote: >> >> Hi all, >> >> may I please have reviews for this small addition. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203455 >> Webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.00/webrev/ >> >> VM.metaspace show-loaders can be used to display loaders (well, really >> CLD instances). >> >> For anonymous CLDs, only "anonymous" is shown. It would be helpful to >> show to which loader these CLD are assigned. >> >> Before: >> >> "272: ClassLoaderData 0x00007f5ba0538f10 for anonymous class" >> >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-before.txt >> >> After patch: >> >> "268: CLD 0x00007ff0c45738f0 for , loaded by app, >> instance of jdk.internal.loader.ClassLoaders$AppClassLoader" >> >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-after.txt >> >> -- >> Notes: this patch has a bit more lines than I liked because I did not >> find a singly utility function in ClassloaderData which fit my >> purpose. I wanted to see name and class of the associated loader for >> both normal and unloading case. ClassloaderData::loader_name() has a >> number of shortcomings, I opened >> https://bugs.openjdk.java.net/browse/JDK-8203456 as a follow up rfe. >> >> Thank you, >> >> Thomas >> > From david.holmes at oracle.com Mon May 21 06:50:18 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 21 May 2018 16:50:18 +1000 Subject: RFR(xs): 8203455: jcmd: VM.metaspace: print loader name for anonymous CLDs. In-Reply-To: References: Message-ID: <68b2d89f-fbe1-01ef-22b1-0a8179dfa5e8@oracle.com> On 21/05/2018 4:46 PM, Thomas St?fe wrote: > Hi David, > > On Mon, May 21, 2018 at 4:23 AM, David Holmes wrote: >> Hi Thomas, >> >> There's been a bit of discussion lately about how to "describe" >> classloaders: >> >> https://bugs.openjdk.java.net/browse/JDK-8202605 >> >> I'd rather not see another chunk of code added that 8202605 will have to fix >> up. Can you live with the less-than-ideal output of the existing function, >> under the expectation it will be updated to do a better job in the future? >> > > 8206205 does not alleviate the need for this change, since currently > we do not print any loader information for anonymous classes. Sorry I was unclear. I meant for this patch to just add e.g. use of loader_name(), even if not ideal, and allow 8202605 to clean up the actual text later. David > Originally I planned to fix up this patch with a follow up patch later > once those utility functions are in place in ClassloaderData. > > But yes, I can save us some review work by waiting a bit and then > doing only one change. Lets hope it does not take too long, I have > more changes coming up which print class loader names. > > So I retract this RFR for now and will repost another patch once > 8206205 is done. > > Thanks, Thomas > >> Thanks, >> David >> >> >> On 20/05/2018 4:49 PM, Thomas St?fe wrote: >>> >>> Hi all, >>> >>> may I please have reviews for this small addition. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203455 >>> Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.00/webrev/ >>> >>> VM.metaspace show-loaders can be used to display loaders (well, really >>> CLD instances). >>> >>> For anonymous CLDs, only "anonymous" is shown. It would be helpful to >>> show to which loader these CLD are assigned. >>> >>> Before: >>> >>> "272: ClassLoaderData 0x00007f5ba0538f10 for anonymous class" >>> >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-before.txt >>> >>> After patch: >>> >>> "268: CLD 0x00007ff0c45738f0 for , loaded by app, >>> instance of jdk.internal.loader.ClassLoaders$AppClassLoader" >>> >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-after.txt >>> >>> -- >>> Notes: this patch has a bit more lines than I liked because I did not >>> find a singly utility function in ClassloaderData which fit my >>> purpose. I wanted to see name and class of the associated loader for >>> both normal and unloading case. ClassloaderData::loader_name() has a >>> number of shortcomings, I opened >>> https://bugs.openjdk.java.net/browse/JDK-8203456 as a follow up rfe. >>> >>> Thank you, >>> >>> Thomas >>> >> From thomas.stuefe at gmail.com Mon May 21 06:59:31 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 21 May 2018 08:59:31 +0200 Subject: RFR(xs): 8203455: jcmd: VM.metaspace: print loader name for anonymous CLDs. In-Reply-To: <68b2d89f-fbe1-01ef-22b1-0a8179dfa5e8@oracle.com> References: <68b2d89f-fbe1-01ef-22b1-0a8179dfa5e8@oracle.com> Message-ID: On Mon, May 21, 2018 at 8:50 AM, David Holmes wrote: > On 21/05/2018 4:46 PM, Thomas St?fe wrote: >> >> Hi David, >> >> On Mon, May 21, 2018 at 4:23 AM, David Holmes >> wrote: >>> >>> Hi Thomas, >>> >>> There's been a bit of discussion lately about how to "describe" >>> classloaders: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8202605 >>> >>> I'd rather not see another chunk of code added that 8202605 will have to >>> fix >>> up. Can you live with the less-than-ideal output of the existing >>> function, >>> under the expectation it will be updated to do a better job in the >>> future? >>> >> >> 8206205 does not alleviate the need for this change, since currently >> we do not print any loader information for anonymous classes. > > > Sorry I was unclear. I meant for this patch to just add e.g. use of > loader_name(), even if not ideal, and allow 8202605 to clean up the actual > text later. > Ah, okay. Lets see how discussion under 8202605 goes - I just added a comment. I really want to print out loader name *and* class name, not one or the other. Thomas > David > >> Originally I planned to fix up this patch with a follow up patch later >> once those utility functions are in place in ClassloaderData. >> >> But yes, I can save us some review work by waiting a bit and then >> doing only one change. Lets hope it does not take too long, I have >> more changes coming up which print class loader names. >> >> So I retract this RFR for now and will repost another patch once >> 8206205 is done. >> >> Thanks, Thomas >> >>> Thanks, >>> David >>> >>> >>> On 20/05/2018 4:49 PM, Thomas St?fe wrote: >>>> >>>> >>>> Hi all, >>>> >>>> may I please have reviews for this small addition. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203455 >>>> Webrev: >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.00/webrev/ >>>> >>>> VM.metaspace show-loaders can be used to display loaders (well, really >>>> CLD instances). >>>> >>>> For anonymous CLDs, only "anonymous" is shown. It would be helpful to >>>> show to which loader these CLD are assigned. >>>> >>>> Before: >>>> >>>> "272: ClassLoaderData 0x00007f5ba0538f10 for anonymous class" >>>> >>>> >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-before.txt >>>> >>>> After patch: >>>> >>>> "268: CLD 0x00007ff0c45738f0 for , loaded by app, >>>> instance of jdk.internal.loader.ClassLoaders$AppClassLoader" >>>> >>>> >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-after.txt >>>> >>>> -- >>>> Notes: this patch has a bit more lines than I liked because I did not >>>> find a singly utility function in ClassloaderData which fit my >>>> purpose. I wanted to see name and class of the associated loader for >>>> both normal and unloading case. ClassloaderData::loader_name() has a >>>> number of shortcomings, I opened >>>> https://bugs.openjdk.java.net/browse/JDK-8203456 as a follow up rfe. >>>> >>>> Thank you, >>>> >>>> Thomas >>>> >>> > From thomas.stuefe at gmail.com Mon May 21 07:42:55 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 21 May 2018 09:42:55 +0200 Subject: RFR(xs): 8203455: jcmd: VM.metaspace: print loader name for anonymous CLDs. In-Reply-To: References: Message-ID: Hi all, second attempt, after discussing things with David: http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.01/webrev/ The patch is much simpler now. I use ClassLoaderData::class_loader_name() and ::class_loader_class() which should always work and be in accordance with planned work in https://bugs.openjdk.java.net/browse/JDK-8202605. Thanks, Thomas On Sun, May 20, 2018 at 8:49 AM, Thomas St?fe wrote: > Hi all, > > may I please have reviews for this small addition. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203455 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.00/webrev/ > > VM.metaspace show-loaders can be used to display loaders (well, really > CLD instances). > > For anonymous CLDs, only "anonymous" is shown. It would be helpful to > show to which loader these CLD are assigned. > > Before: > > "272: ClassLoaderData 0x00007f5ba0538f10 for anonymous class" > > http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-before.txt > > After patch: > > "268: CLD 0x00007ff0c45738f0 for , loaded by app, > instance of jdk.internal.loader.ClassLoaders$AppClassLoader" > > http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-after.txt > > -- > Notes: this patch has a bit more lines than I liked because I did not > find a singly utility function in ClassloaderData which fit my > purpose. I wanted to see name and class of the associated loader for > both normal and unloading case. ClassloaderData::loader_name() has a > number of shortcomings, I opened > https://bugs.openjdk.java.net/browse/JDK-8203456 as a follow up rfe. > > Thank you, > > Thomas From thomas.stuefe at gmail.com Mon May 21 09:36:09 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 21 May 2018 11:36:09 +0200 Subject: RFR(L): 8176808: Split up metaspace.cpp In-Reply-To: <6f75acff-6389-bf49-df4d-4aa3660638f1@oracle.com> References: <32826e8139614cd68f0b4548ca8c044c@sap.com> <0da5bc3d-3f3a-dd09-c635-fdc67ed7ecaa@oracle.com> <3c69f828-d3b1-00e8-bc37-6c2796474ee1@oracle.com> <6f75acff-6389-bf49-df4d-4aa3660638f1@oracle.com> Message-ID: I pushed, and before that I changed the include guard names and the copyright headers as proposed. Thanks a lot, Thomas On Mon, May 21, 2018 at 4:11 AM, David Holmes wrote: > On 20/05/2018 1:51 AM, Thomas St?fe wrote: > >> Hi Coleen, >> >> On Sat, May 19, 2018 at 3:20 PM, wrote: >> >> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- >>> metaspace-cpp/webrev.02/webrev/src/hotspot/share/memory/metaspace/ >>> metaspaceCommon.hpp.udiff.html >>> >>> Not your change but metaspaceCommon.hpp has these with a trailing _ ? >>> >>> #ifndef SHARE_MEMORY_METASPACE_METASPACECOMMON_HPP_ >>> #define SHARE_MEMORY_METASPACE_METASPACECOMMON_HPP_ >>> >>> The other new files have it this too. I don't think we have this >>> anywhere >>> else and it looks strange. I don't like the _. >>> >>> >>> Sure, I remove it. These lines were autogenerated by CDT. Maybe I should >> return to good old vi :) >> >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- >>> metaspace-cpp/webrev.02/webrev/src/hotspot/share/memory/metaspace/ >>> virtualSpaceList.cpp.html >>> >>> This is missing a Copyright. Are you supposed to put the SAP line on >>> these new files also? >>> >>> >> I usually place SAP copyrights in addition to Oracle ones on those files >> fully created by me. >> >> >> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- >>> metaspace-cpp/webrev.02/webrev/src/hotspot/share/ >>> memory/metaspace/occupancyMap.cpp.html >>> >>> 3 * Copyright (c) 2018, SAP. >>> >>> >>> Is this SAP copyright correct? I thought there was more on the line. >>> Maybe not. >>> >>> >> I'll double check on Tuesday with my coworkers. jdk-submit seems to be >> dead >> currently anyway, and I want to put the fix one last time thru the checks. >> > > The format seems to be e.g.: > > * Copyright (c) 2018 SAP SE. All rights reserved. > > Thanks, > David > > > >> >>> For the _ changes and copyright change, I don't need to see another >>> webrev. This refactoring and the use of namespace metaspace is really >>> nice. It does give an appreciation for how complicated the memory >>> management for metaspace became since its start as: >>> >>> class Metaspace : public Arena {}. >>> >>> >>> hah! yes indeed :) >> >> But as I mentioned before, my suspicion is that were we to recreate it >> from >> scratch it would soon grow in complexity until something close to the >> current form. Maybe not, but I would not be surprised. >> >> >> Thanks! >>> >>> >> Sure. Thanks for reviewing! >> >> ..Thomas >> >> >> >>> Coleen >>> >>> >>> >>> On 5/19/18 12:55 AM, Thomas St?fe wrote: >>> >>> Hi, Coleen and Axel, >>> >>> thanks a lot for the reviews! I will be very happy once this is removed >>> from my patch queue, since merging the patch is becoming a pain. >>> >>> New webrev. >>> >>> As suggested, I removed the "internals" inner namespace, which leaves us >>> with "metaspace::". Nothing else changed. Still fine? >>> >>> Webrevs: >>> Incremental: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- >>> metaspace-cpp/webrev.01-to-02/webrev/ >>> Full: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- >>> metaspace-cpp/webrev.02/webrev/ >>> >>> Thanks, Thomas >>> >>> >>> On Sat, May 19, 2018 at 12:41 AM, wrote: >>> >>> >>>> >>>> On 5/17/18 6:15 AM, Siebenborn, Axel wrote: >>>> >>>> Hi Thomas, >>>>> The change looks good . >>>>> I should have read your whole mail before starting looking at the >>>>> changes, as you explain what you have done and what you didn't do ??. >>>>> Concerning the decision using ::metaspace vs. ::metaspace::internal: >>>>> I would use ::metaspace for the internal classes and leave the public >>>>> classes in the global namespace. >>>>> This is more consistent to hotspot code where namespaces are rarely >>>>> used >>>>> (if this counts as an argument). >>>>> I would change that in this change, as having having >>>>> ::metaspace::internal and nothing in ::metaspace seems to be the worst >>>>> alternative. >>>>> >>>>> >>>> Yes, I agree with this. One namespace to encapsulate things like >>>> ChunkManager and SpaceManager and these things is good, without adding >>>> the >>>> another nested "internals" namespace. That's too much for us :) >>>> >>>> Coleen >>>> >>>> >>>> Altogether, this is really a good refactoring. >>>>> >>>>> Cheers, >>>>> Axel >>>>> >>>>> -----Original Message----- >>>>> >>>>>> From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- >>>>>> bounces at openjdk.java.net] On Behalf Of Thomas St?fe >>>>>> Sent: Montag, 14. Mai 2018 15:34 >>>>>> To: Hotspot dev runtime >>>>>> Subject: Re: RFR(L): 8176808: Split up metaspace.cpp >>>>>> >>>>>> Ping... >>>>>> >>>>>> Is there anything I can do to make review easier? This is really >>>>>> mostly code shuffling around (out of metaspace.cpp into new files), so >>>>>> with a bit of effort I could cook up some diffs for those parts. >>>>>> >>>>>> ..Thomas >>>>>> >>>>>> On Fri, May 11, 2018 at 8:31 AM, Thomas St?fe < >>>>>> thomas.stuefe at gmail.com> >>>>>> wrote: >>>>>> >>>>>> All builds are fixed, jdk-submit tests ran through sucessfully. >>>>>>> >>>>>>> Latest webrev: >>>>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split-metaspace- >>>>>>> >>>>>>> cpp/webrev.01/webrev/ >>>>>> >>>>>> Only difference to the first webrev is the placement of a single >>>>>>> semicolon in occupancyMap.hpp, to satisfy the Solaris compiler. >>>>>>> >>>>>>> Thanks, Thomas >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, May 9, 2018 at 5:08 PM, Thomas St?fe >>>>>>> >>>>>>> wrote: >>>>>> >>>>>> Hi all, >>>>>>> >>>>>>>> >>>>>>>> This was a long time coming. Lets cut up this beast :) >>>>>>>> >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8176808 >>>>>>>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8176808-split- >>>>>>>> >>>>>>>> metaspace-cpp/webrev.00/webrev/ >>>>>>> >>>>>> >>>>>> This change splits up metaspace.cpp into multiple parts. Note that I >>>>>>> >>>>>>>> did not change much code (exceptions see below in details) - so this >>>>>>>> is mostly moving code around. >>>>>>>> >>>>>>>> This change follows the scheme tentatively outlined in JDK-8201572: >>>>>>>> >>>>>>>> - metaspace internal classes go into the sub directory >>>>>>>> share/memory/metaspace. Ideally, those should never be accessed from >>>>>>>> outside, apart from other metaspace classes and metaspace tests. >>>>>>>> They >>>>>>>> also go into the namespace ::metaspace::internals. >>>>>>>> >>>>>>>> - metaspace external classes (MetaspaceUtils, ClassLoaderMetaspace, >>>>>>>> etc) remain inside share/memory and, for now, remain in the global >>>>>>>> namespace. >>>>>>>> >>>>>>>> Note: the original idea was to move metaspace external classes to >>>>>>>> namespace ::metaspace. But I shied away from that, since that would >>>>>>>> mean fixing up all usages of these classes either with metaspace:: >>>>>>>> scope or with using declarations. I had to make a cut at some point. >>>>>>>> >>>>>>>> So here is something to decide: >>>>>>>> - should we then get rid of the ::metaspace::internals namespace, >>>>>>>> move >>>>>>>> metaspace-internal classes to the enclosing ::metaspace namespace >>>>>>>> and >>>>>>>> leave external classes in the global namespace ? >>>>>>>> - or should we follow through (maybe in a future patch): internal >>>>>>>> classes in ::metaspace::internal, and move external classes to >>>>>>>> ::metaspace ? >>>>>>>> >>>>>>>> Some more details: >>>>>>>> >>>>>>>> - the following classes moved out of metaspace.cpp into namespace >>>>>>>> "metaspace::internal" and the metaspace sub directory: >>>>>>>> MetaDebug, ChunkManager, SpaceManager, BlockFreeList and >>>>>>>> >>>>>>>> SmallBlocks, >>>>>>> >>>>>> >>>>>> OccupancyMap, VirtualSpaceNode and VirtualSpaceList, the >>>>>>> >>>>>>>> PrintCLDMetaspaceInfoClosure. >>>>>>>> >>>>>>>> - I also moved metachunk.hpp to internals, since class Metachunk is >>>>>>>> not used externally. What is used externally is Metablock though, >>>>>>>> so I >>>>>>>> split up metachunk.hpp into metabase.hpp, metablock.hpp and >>>>>>>> metachunk.hpp, holding Metabase, Metablock and Metachunk, >>>>>>>> respectively. >>>>>>>> >>>>>>>> - Now we see some ugly seams: metaspace.hpp is supposed to be the >>>>>>>> outside API contract, however, it contains implementation details >>>>>>>> which should not be there and which manifest now by having to use >>>>>>>> the >>>>>>>> metaspace::internals namespace here and there. If we care, we could >>>>>>>> solve this with a bit redesigning. >>>>>>>> >>>>>>>> - The ChunkManager gtest and the SpaceManager gtest had been >>>>>>>> implemented inside metaspace.cpp, since it was not possible to >>>>>>>> access >>>>>>>> the internal classes those tests needed in any other way. Since that >>>>>>>> is now possible, I moved the coding out to gtest-land and made them >>>>>>>> real gtests (exchanging the asserts for real gtest ASSERTS, among >>>>>>>> other things). >>>>>>>> Note that there are some tests left in metaspace.cpp >>>>>>>> (TestMetaspaceUtilsTest, TestVirtualSpaceNodeTest) - those were no >>>>>>>> gtests-in-disguise but were part of -XX:+ExecuteInternalVMTests. I >>>>>>>> guess these tests precede the gtest framework? I leave it up to >>>>>>>> others >>>>>>>> to decide what to do with them and to potentially move them out of >>>>>>>> metaspace.cpp. >>>>>>>> Side note: -XX:+ExecuteInternalVMTests seems to have bitrotted, see >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8202848. Does this get >>>>>>>> tested >>>>>>>> regularly? >>>>>>>> >>>>>>>> - metaspace.cpp is quite a bit smaller now - from ~5000 loc down to >>>>>>>> ~1700 loc. Arguably, one could split out more and clean up more, >>>>>>>> but I >>>>>>>> think this is a good start. >>>>>>>> >>>>>>>> --- >>>>>>>> >>>>>>>> I built locally on linux (release, fastdebug with and without pch, >>>>>>>> zero) and windows (64, 32bit). jdk-submit tests ran through with a >>>>>>>> build error on solaris sparc - I currently try to reproduce that >>>>>>>> build >>>>>>>> error with our very slow solaris machine. >>>>>>>> >>>>>>>> Thanks, Thomas >>>>>>>> >>>>>>>> >>>>>>> >>>> >>> >>> From dms at samersoff.net Mon May 21 13:44:06 2018 From: dms at samersoff.net (Dmitry Samersoff) Date: Mon, 21 May 2018 16:44:06 +0300 Subject: RFR(S): JDK-8203481 Incorrect constraint for unextended_sp in frame:safe_for_sender Message-ID: <930b4686-dcf6-5f01-f492-4e65a25145c3@samersoff.net> Hello Everybody, Please review small fix http://cr.openjdk.java.net/~dsamersoff/JDK-8203481/webrev.01/ CR: https://bugs.openjdk.java.net/browse/JDK-8203481 Testing: jfr tests that depends to safe_for_sender functionality ./jdk/jdk/jfr/api/consumer/TestRecordedFullStackTrace.java ./jdk/jdk/jfr/event/profiling/TestFullStackTrace.java fails on AARCH64. These tests passed after the fix. -- Dmitry Samersoff http://devnull.samersoff.net * There will come soft rains ... From lois.foltan at oracle.com Mon May 21 14:03:27 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Mon, 21 May 2018 10:03:27 -0400 Subject: RFR(S) 8193332 MetaspaceShared::check_shared_class_loader_type is not used during archive creation In-Reply-To: <4c9edcca-6c94-4933-326b-b9cd88d7033f@oracle.com> References: <4c9edcca-6c94-4933-326b-b9cd88d7033f@oracle.com> Message-ID: <04098a6b-49f5-c4fe-5324-bd49dc36cfb3@oracle.com> On 5/17/2018 9:13 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8193332 > http://cr.openjdk.java.net/~iklam/jdk11/8193332-check_shared_class_loader_type.v01/ > > > Summary: > > When we restructured the AppCDS code, the call to > MetaspaceShared::check_shared_class_loader_type > was somehow left out. I have restored it, and actually found a minor > bug related to modules > and -Xbootclasspath/a: > > In classLoader.cpp, at dump time we used to load a class from > -Xbootclasspath/a even if > this class belongs to a named packaged, which means it would not be > loaded by the JVM at runtime. > Such dump-time loading seems unnecessary so I removed it, and fixed a > related test case > to reflect the latest behavior. > > Tested with hs-tier1,2. > > Thanks > - Ioi Looks good.? Can you either remove or improve upon the DumpSharedSpaces comment in lines 1413-1424 of classLoader.cpp to reflect the changed behavior? Thanks, Lois From daniel.daugherty at oracle.com Mon May 21 14:28:56 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 21 May 2018 10:28:56 -0400 Subject: RFR(S): JDK-8203481 Incorrect constraint for unextended_sp in frame:safe_for_sender In-Reply-To: <930b4686-dcf6-5f01-f492-4e65a25145c3@samersoff.net> References: <930b4686-dcf6-5f01-f492-4e65a25145c3@samersoff.net> Message-ID: <3421e284-fda3-ff82-29cd-70e4c4e0c8c0@oracle.com> Hi Dmitry, I think something else must be going wrong here. The unextended SP is typically documented like this: ? // This is the sp before any possible extension (adapter/locals). ? intptr_t* unextended_sp = interpreter_frame_sender_sp(); and like this: ? // stack frames shouldn't be much larger than max_stack elements ? // this test requires the use of unextended_sp which is the sp as seen by ? // the current frame, and not sp which is the "raw" pc which could point ? // further because of local variables of the callee method inserted after ? // method arguments ? if (fp() - unextended_sp() > 1024 + m->max_stack()*Interpreter::stackElementSize) { ??? return false; ? } So I think this existing comment and assertion are correct: ??? L72: ? // unextended sp must be within the stack and above or equal sp ??? L73: ? bool unextended_sp_safe = (unextended_sp < thread->stack_base()) && ??? L74: ??????????????????????????? (unextended_sp >= sp); Also, your proposed fix only changed this for two platforms. The same logic exists on 'arm' and 'sparc' also. Dan On 5/21/18 9:44 AM, Dmitry Samersoff wrote: > Hello Everybody, > > Please review small fix > > http://cr.openjdk.java.net/~dsamersoff/JDK-8203481/webrev.01/ > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8203481 > > Testing: > > jfr tests that depends to safe_for_sender functionality > > ./jdk/jdk/jfr/api/consumer/TestRecordedFullStackTrace.java > ./jdk/jdk/jfr/event/profiling/TestFullStackTrace.java > > fails on AARCH64. > > These tests passed after the fix. > > From lois.foltan at oracle.com Mon May 21 14:37:36 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Mon, 21 May 2018 10:37:36 -0400 Subject: RFR 8203381 Replace InstanceKlass::allocate_instance_handle with JavaCalls::construct_new_instance In-Reply-To: <2a449ddb-4512-193f-d6d1-152989b8526c@oracle.com> References: <2a449ddb-4512-193f-d6d1-152989b8526c@oracle.com> Message-ID: <5fb49873-88ad-3443-dea7-d09ea4186815@oracle.com> On 5/18/2018 12:34 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8203381 > http://cr.openjdk.java.net/~iklam/jdk11/8203381-construct-new-instance.v01/ > > > This patch removes a lot of the boilerplate code for allocating a > Java object and calling its constructor. Hopefully the code is cleaner > and easier to read. > > Also: > > Added assert(InstanceKlass::is_initialized(), ...) in case where > we cannot call JavaCalls::construct_new_instance for thread objects. > > Replaced unnecessary SystemDictionary::resolve_or_null() calls with > SystemDictionary::XXX_klass(). > > Tested with hs-tier[1,2,3,4,5] > > Thanks > - Ioi > Great cleanup, looks good! Lois From mikhailo.seledtsov at oracle.com Mon May 21 15:52:59 2018 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Mon, 21 May 2018 08:52:59 -0700 Subject: RFR(M): 8199257: [TESTBUG] Open source VM testbase metaspace tests In-Reply-To: References: Message-ID: <5B02EB5B.3010805@oracle.com> Ping... On 5/8/18, 10:38 AM, mikhailo wrote: > http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html > > Please review this change open sourcing VM metaspace tests. These > tests have been used internally for a while, and are now being open > sourced. Since this is not an creation of new tests, we would like to > keep the changes during this review to an absolute required minimum. > If you have any feedback on improvements of these tests, please file > RFE(s) that will be addressed later in order of priority. > > > Here is what was done for this change: > 1. Moved the tests to OpenJDK repository to the specified directory > location and structure. > 3. Updated Copyright statements accordingly. > 4. Updated "@library" statements accordingly. > 5. Updated TEST.groups and ProblemList.txt > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199257 > Webrev: > http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html > > Testing: > 1. Ran the following tests on open-only repository and build, > using "make run-test" (Linux-x64) > vmTestbase_vm_metaspace, vmTestbase_largepages > All PASS > > 2. Automated multip-platform test system (usual 4 platforms): > - vmTestbase_vm_metaspace, vmTestbase_largepages > - hs-tier{1,2,3} > All PASS > > Thank you, > Misha > From ioi.lam at oracle.com Mon May 21 16:55:28 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 21 May 2018 09:55:28 -0700 Subject: RFR(S) 8193332 MetaspaceShared::check_shared_class_loader_type is not used during archive creation In-Reply-To: <04098a6b-49f5-c4fe-5324-bd49dc36cfb3@oracle.com> References: <4c9edcca-6c94-4933-326b-b9cd88d7033f@oracle.com> <04098a6b-49f5-c4fe-5324-bd49dc36cfb3@oracle.com> Message-ID: <98a35985-328e-f128-9f20-19b882f54bbe@oracle.com> On 5/21/18 7:03 AM, Lois Foltan wrote: > On 5/17/2018 9:13 PM, Ioi Lam wrote: > >> https://bugs.openjdk.java.net/browse/JDK-8193332 >> http://cr.openjdk.java.net/~iklam/jdk11/8193332-check_shared_class_loader_type.v01/ >> >> >> Summary: >> >> When we restructured the AppCDS code, the call to >> MetaspaceShared::check_shared_class_loader_type >> was somehow left out. I have restored it, and actually found a minor >> bug related to modules >> and -Xbootclasspath/a: >> >> In classLoader.cpp, at dump time we used to load a class from >> -Xbootclasspath/a even if >> this class belongs to a named packaged, which means it would not be >> loaded by the JVM at runtime. >> Such dump-time loading seems unnecessary so I removed it, and fixed a >> related test case >> to reflect the latest behavior. >> >> Tested with hs-tier1,2. >> >> Thanks >> - Ioi > Looks good.? Can you either remove or improve upon the > DumpSharedSpaces comment in lines 1413-1424 of classLoader.cpp to > reflect the changed behavior? > Thanks, > Lois > Hi Lois, Thanks for the review. I've changed the comments from: ? // If DumpSharedSpaces is true boot loader visibility boundaries are set to: ? //?? - [jimage] + [_first_append_entry to _last_append_entry] (all path entries). ? // ? // If search_append_only is true, boot loader visibility boundaries are ? // set to be _first_append_entry to the end. This includes: ? //?? [-Xbootclasspath/a]; [jvmti appended entries] ? // ? // If both DumpSharedSpaces and search_append_only are false, boot loader ? // visibility boundaries are set to be the --patch-module entries plus the base piece. ? // This would include: ? // [--patch-module==()*]; [jimage | exploded module build] to ? // If search_append_only is true, boot loader visibility boundaries are ? // set to be _first_append_entry to the end. This includes: ? //?? [-Xbootclasspath/a]; [jvmti appended entries] ? // ? // If search_append_only is false, boot loader visibility boundaries are ? // set to be the --patch-module entries plus the base piece. This includes: ? // [--patch-module==()*]; [jimage | exploded module build] Thanks - Ioi From jiangli.zhou at oracle.com Mon May 21 17:12:29 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Mon, 21 May 2018 10:12:29 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: <9A7035A6-892E-45E6-8F1A-65D6A344D78C@oracle.com> References: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> <5AF62823.7050804@oracle.com> <70F6AEB0-9103-4899-87D9-DC845BF66C51@oracle.com> <4CBA8376-54A2-4863-8732-B16E1AE9C38A@oracle.com> <5C996E41-CDA0-4EDA-B289-CDD7C38151B9@oracle.com> <6b94fc12-61ae-e328-59b1-40096544cbc1@oracle.com> <9A7035A6-892E-45E6-8F1A-65D6A344D78C@oracle.com> Message-ID: <58606140-A712-406B-A4D9-59D44017477D@oracle.com> After discussion with Ioi and Calvin, we all agreed the usages of os::file_name_strcmp() in SharedPathsMiscInfo::check() should be fixed as part of this change. Calvin (thanks!) also found an unlikely but possible edge case (-Xshare:dump -Xbootclasspath/a:foo.jar; -Xshare:on -Xbootclasspath/a:foo.jar123) that we need to handle. Added the test case in BootClassPathMismatch.java. Here is the updated webrev: http://cr.openjdk.java.net/%7Ejiangli/8199807/webrev.03/ Thanks, Jiangli > On May 17, 2018, at 11:55 AM, Jiangli Zhou wrote: > > Thanks, Ioi! Will incorporate your suggestions below before integration. > > Jiangli > >> On May 17, 2018, at 11:52 AM, Ioi Lam wrote: >> >> Hi Jiangli, >> >> The changes look good. Just some minor nits. No need for new webrev if you decide to make these changes. >> >> // The first entry in boot path is the modules_image (guaranteed by >> >> // ClassLoader::setup_boot_search_path()). Skip the first entry. The >> // path of the runtime modules_image may be different from the dump >> // time path (e.g. the JDK image is copied to a different location >> // after generating the shared archive), which is acceptable. For most >> // common cases, the dump time boot path might contain modules_image only. >> >> Perhaps add an assert that #0 of the bootclasspath is the runtime module image? >> >> >> 444 isSigned = stream->check_is_signed(); >> 445 if (isSigned) { >> >> Maybe remove isSigned variable and change to if (stream->check_is_signed()) .... >> >> Thanks >> - Ioi >> >> >> On 5/16/18 8:27 PM, Jiangli Zhou wrote: >>> Here is the updated webrev: >>> http://cr.openjdk.java.net/~jiangli/8199807/webrev.02/ >>> >>> At CDS dump time, when SharedClassPathEntry::init() is called from FileMapInfo::allocate_shared_path_table(), we can identify the modules image entry directly. SharedClassPathEntry::init() now takes an extra bool argument to indicate if the current entry is the modules image. We no longer need to test the entry name using ClassLoader::is_modules_image(), which might be wrong for a shared path entry as Ioi pointed out. >>> >>> I added a ?_type? field in SharedClassPathEntry to record the type of shared path entries at dump time. At runtime, the path entry ?_type? is checked to identify the modules image in SharedClassPathEntry::validate(). SharedClassPathEntry::is_modules_image() no longer need to use ClassLoader::is_modules_image(). >>> >>> Reworked the BOOT_PATH case in SharedPathsMiscInfo::check(). I left os::file_name_strcmp() unchanged since it?s also used by APP_PATH. I'll handle it in a separate REF to do more testing. >>> >>> I also included the MoveJDKTest.java written by Ioi in the updated webrev. In the test, I added a new case for a JAR entry named with Hello.modules. >>> >>> Reran all appcds tests including MoveJDKTest.java locally. All tests passed. Running tiered tests. >>> >>> Thanks, >>> Jiangli >>> >>>> On May 15, 2018, at 6:10 PM, Jiangli Zhou wrote: >>>> >>>> Hi Ioi, >>>> >>>> Thank you very much for the review. Good catches for those edge cases and thanks for the test cases. I will resolves them. >>>> >>>> Thanks! >>>> Jiangli >>>> >>>>> On May 15, 2018, at 4:33 PM, Ioi Lam wrote: >>>>> >>>>> Hi Jiangli, >>>>> >>>>> 245 // We need to validate the runtime modules image file size against the archived >>>>> 246 // size information, obtain the runtime modules image path. The recorded dump >>>>> 247 // time modules image path in the archive may be different from the runtime path >>>>> 248 // if the JDK image has beed moved after generating the archive. >>>>> 249 if (ClassLoader::is_modules_image(name)) { >>>>> 250 name = ClassLoader::get_jrt_entry()->name(); >>>>> 251 } >>>>> >>>>> What happens if someone has a JAR file called foo.modules, and dumped with >>>>> >>>>> java -Xshare:dump -cp foo.modules >>>>> >>>>> >>>>> I think it's better to check that it's #0 in the classpath so we know for sure it's the system modules file. >>>>> >>>>> There's a similar problem here: >>>>> >>>>> 220 if (!ClassLoader::is_modules_image(name)) { >>>>> >>>>> In the following, instead of doing a copy, maybe it's better to add a length argument for os::file_name_strcmp, since sharedPathsMiscInfo.cpp is the only place in the VM that calls this function. >>>>> >>>>> 162 if (relaxed_check) { >>>>> 163 // only check the begining portion of the runtime boot path, up to >>>>> 164 // the length of the dump time boot path >>>>> 165 size_t len = strlen(path); >>>>> 166 runtime_boot_path = NEW_RESOURCE_ARRAY(char, len + 1); >>>>> 167 strncpy(runtime_boot_path, Arguments::get_sysclasspath(), len); >>>>> 168 runtime_boot_path[len] = '\0'; >>>>> 169 } else { >>>>> 170 // check the full runtime boot path >>>>> 171 runtime_boot_path = Arguments::get_sysclasspath(); >>>>> 172 } >>>>> 173 >>>>> 174 // Do a quick check first with a simple >>>>> 175 // strcmp(dump_time_boot_path, runtime_boot_path). If the paths are the >>>>> 176 // same, the check has succeeded. >>>>> 177 if (os::file_name_strcmp(path, runtime_boot_path) == 0) { >>>>> 178 break; // ok >>>>> 179 } >>>>> >>>>> >>>>> Also, the copy could be wrong for the following output: >>>>> >>>>> path = /tmp/foo/modules >>>>> Arguments::get_sysclasspath() = /tmp/foo/modulesxyz:/aaa:/bbb >>>>> >>>>> What is runtime_boot_path supposed to contain here? I am not sure if this code gives the intended output or not, but it's not easy to understand what it actually does by operating on an truncated pathname. >>>>> >>>>> String manipulation code is always hard to read. I would suggest braking it up into smaller functions: >>>>> >>>>> 1. Get the first entry of dump time and run time boot path -> d0 and r0 >>>>> -> ignore d0 >>>>> -> r0 same as ClassLoader::get_jrt_entry()->name(); >>>>> >>>>> 2. Get the remaining part of the dump time and run time boot path -> d_remain, r_remain >>>>> -> relaxed_check: d_remain must be a prefix of r_remain >>>>> -> !relaxed_check: They must be identical. >>>>> >>>>> >>>>> By the way, I think the test case in JDK-8202935 should be reviewed together inside this RFR, since the test validates the feature implemented here. I don't think we need a separate bug ID. Otherwise it will be hard to track the test cases. >>>>> >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> On 5/13/18 5:31 PM, Jiangli Zhou wrote: >>>>>> Here is the updated webrev: >>>>>> http://cr.openjdk.java.net/~jiangli/8199807/webrev.01/ >>>>>> >>>>>> Thanks, >>>>>> Jiangli >>>>>> >>>>>>> On May 11, 2018, at 6:07 PM, Jiangli Zhou wrote: >>>>>>> >>>>>>> Hi Calvin, >>>>>>> >>>>>>> Thanks for the review. >>>>>>> >>>>>>>> On May 11, 2018, at 4:32 PM, Calvin Cheung wrote: >>>>>>>> >>>>>>>> Hi Jiangli, >>>>>>>> >>>>>>>> Thanks for doing this useful change. >>>>>>>> >>>>>>>> I have a few minor comments: >>>>>>>> >>>>>>>> 1) sharedPathsMiscInfo.cpp >>>>>>>> >>>>>>>> 199 dp ++; >>>>>>>> >>>>>>>> Should the above be similar to line 194 as follows? >>>>>>>> dp += path_sep_len; >>>>>>> Good catch. Will fix. >>>>>>> >>>>>>>> 2) filemap.cpp >>>>>>>> >>>>>>>> 244 if (ClassLoader::is_modules_image(name)) { >>>>>>>> >>>>>>>> I think the above could be: >>>>>>>> if (is_modules_image()) { >>>>>>>> >>>>>>> Let?s keep it this way as is_modules_image() is a wrapper of 'ClassLoader::is_modules_image(name())?. We already have the ?name? in this case. >>>>>>> >>>>>>>> The is_modules_image() is a member function of SharedClassPathEntry. >>>>>>>> >>>>>>>> Btw, why is it necessary to get the runtime modules image path again in lines 244 - 246? >>>>>>> As the runtime modules image path could be different from the dump time if the JDK image is moved/copied after archive generation, we need to make sure to use the correct file for file size check. The recored path in the archive is the dump time modules image path, which may no longer be existing at runtime. I will add some comments if that?s helpful. >>>>>>> >>>>>>>> 3) BootClassPathMismatch.java >>>>>>>> >>>>>>>> Could you move line 103 to right above line 107 (opts.addPrefix(?))? >>>>>>>> Because the archive dumping part of the test doesn?t use appJar. >>>>>>> Sure. >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> Jiangli >>>>>>> >>>>>>>> thanks, >>>>>>>> Calvin >>>>>>>> >>>>>>>> On 5/11/18, 3:15 PM, Jiangli Zhou wrote: >>>>>>>>> I?ve withdrawn my original weberv and removed the build change. Here is the updated webrev that only addresses 8199807. >>>>>>>>> >>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.00/ >>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Jiangli >>>>>>>>> >>>>>>>>>> On May 11, 2018, at 2:44 PM, Jiangli Zhou wrote: >>>>>>>>>> >>>>>>>>>> Hi Erik, >>>>>>>>>> >>>>>>>>>> Thank you so much for investigating a proper fix for the vm_version.o issue. I will withdraw the build change from my original webrev. >>>>>>>>>> >>>>>>>>>> Thanks again for taking over the bug! >>>>>>>>>> Jiangli >>>>>>>>>> >>>>>>>>>>> On May 11, 2018, at 2:33 PM, Erik Joelsson wrote: >>>>>>>>>>> >>>>>>>>>>> Received: from [10.132.185.167] (/10.132.185.167) >>>>>>>>>>> by default (Oracle Beehive Gateway v4.0) >>>>>>>>>>> with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 >>>>>>>>>>> Subject: Re: RFR: 8199807& 8202738: AppCDS performs overly restrictive path >>>>>>>>>>> matching check >>>>>>>>>>> To: Jiangli Zhou, >>>>>>>>>>> "hotspot-runtime-dev at openjdk.java.net runtime" >>>>>>>>>>> , >>>>>>>>>>> build-dev >>>>>>>>>>> References: >>>>>>>>>>> From: Erik Joelsson >>>>>>>>>>> Organization: Oracle Corporation >>>>>>>>>>> Message-ID: >>>>>>>>>>> Date: Fri, 11 May 2018 14:33:20 -0700 >>>>>>>>>>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) >>>>>>>>>>> Gecko/20100101 Thunderbird/52.7.0 >>>>>>>>>>> MIME-Version: 1.0 >>>>>>>>>>> In-Reply-To: >>>>>>>>>>> Content-Type: text/plain; charset=utf-8; format=flowed >>>>>>>>>>> Content-Transfer-Encoding: 8bit >>>>>>>>>>> Content-Language: en-US >>>>>>>>>>> >>>>>>>>>>> Hello, >>>>>>>>>>> >>>>>>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>>>>>> >>>>>>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>>>>>> >>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>>>>>> >>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>>>>>> >>>>>>>>>>> /Erik >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>>>>>> >>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>>>>>> >>>>>>>>>>>> The webrev includes the following three main parts: >>>>>>>>>>>> >>>>>>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>>>>>> >>>>>>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>>>>>> >>>>>>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>>>>>> >>>>>>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Jiangli >>>>>>>>>>>> >>>>>>>>>>> Hello, >>>>>>>>>>> >>>>>>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>>>>>> >>>>>>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>>>>>> >>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>>>>>> >>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>>>>>> >>>>>>>>>>> /Erik >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>>>>>> >>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>>>>>> >>>>>>>>>>>> The webrev includes the following three main parts: >>>>>>>>>>>> >>>>>>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>>>>>> >>>>>>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>>>>>> >>>>>>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>>>>>> >>>>>>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Jiangli >> > From lois.foltan at oracle.com Mon May 21 17:19:54 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Mon, 21 May 2018 13:19:54 -0400 Subject: RFR(S) 8193332 MetaspaceShared::check_shared_class_loader_type is not used during archive creation In-Reply-To: <98a35985-328e-f128-9f20-19b882f54bbe@oracle.com> References: <4c9edcca-6c94-4933-326b-b9cd88d7033f@oracle.com> <04098a6b-49f5-c4fe-5324-bd49dc36cfb3@oracle.com> <98a35985-328e-f128-9f20-19b882f54bbe@oracle.com> Message-ID: <5a4e87af-494b-7810-c79c-9728521c55f0@oracle.com> On 5/21/2018 12:55 PM, Ioi Lam wrote: > > > On 5/21/18 7:03 AM, Lois Foltan wrote: >> On 5/17/2018 9:13 PM, Ioi Lam wrote: >> >>> https://bugs.openjdk.java.net/browse/JDK-8193332 >>> http://cr.openjdk.java.net/~iklam/jdk11/8193332-check_shared_class_loader_type.v01/ >>> >>> >>> Summary: >>> >>> When we restructured the AppCDS code, the call to >>> MetaspaceShared::check_shared_class_loader_type >>> was somehow left out. I have restored it, and actually found a minor >>> bug related to modules >>> and -Xbootclasspath/a: >>> >>> In classLoader.cpp, at dump time we used to load a class from >>> -Xbootclasspath/a even if >>> this class belongs to a named packaged, which means it would not be >>> loaded by the JVM at runtime. >>> Such dump-time loading seems unnecessary so I removed it, and fixed >>> a related test case >>> to reflect the latest behavior. >>> >>> Tested with hs-tier1,2. >>> >>> Thanks >>> - Ioi >> Looks good.? Can you either remove or improve upon the >> DumpSharedSpaces comment in lines 1413-1424 of classLoader.cpp to >> reflect the changed behavior? >> Thanks, >> Lois >> > Hi Lois, > > Thanks for the review. I've changed the comments from: > > ? // If DumpSharedSpaces is true boot loader visibility boundaries are > set to: > ? //?? - [jimage] + [_first_append_entry to _last_append_entry] (all > path entries). > ? // > ? // If search_append_only is true, boot loader visibility boundaries are > ? // set to be _first_append_entry to the end. This includes: > ? //?? [-Xbootclasspath/a]; [jvmti appended entries] > ? // > ? // If both DumpSharedSpaces and search_append_only are false, boot > loader > ? // visibility boundaries are set to be the --patch-module entries > plus the base piece. > ? // This would include: > ? // [--patch-module==()*]; [jimage | > exploded module build] > > to > > ? // If search_append_only is true, boot loader visibility boundaries are > ? // set to be _first_append_entry to the end. This includes: > ? //?? [-Xbootclasspath/a]; [jvmti appended entries] > ? // > ? // If search_append_only is false, boot loader visibility boundaries > are > ? // set to be the --patch-module entries plus the base piece. This > includes: > ? // [--patch-module==()*]; [jimage | > exploded module build] > > Thanks > - Ioi > Looks good, thanks! Lois From ioi.lam at oracle.com Mon May 21 17:40:26 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 21 May 2018 10:40:26 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: <58606140-A712-406B-A4D9-59D44017477D@oracle.com> References: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> <5AF62823.7050804@oracle.com> <70F6AEB0-9103-4899-87D9-DC845BF66C51@oracle.com> <4CBA8376-54A2-4863-8732-B16E1AE9C38A@oracle.com> <5C996E41-CDA0-4EDA-B289-CDD7C38151B9@oracle.com> <6b94fc12-61ae-e328-59b1-40096544cbc1@oracle.com> <9A7035A6-892E-45E6-8F1A-65D6A344D78C@oracle.com> <58606140-A712-406B-A4D9-59D44017477D@oracle.com> Message-ID: <6544016c-6252-8fd0-5a86-2900368f7221@oracle.com> Looks good. Thanks for doing this Jiangli! - Ioi On 5/21/18 10:12 AM, Jiangli Zhou wrote: > After discussion with Ioi and Calvin, we all agreed the usages of os::file_name_strcmp() in SharedPathsMiscInfo::check() should be fixed as part of this change. Calvin (thanks!) also found an unlikely but possible edge case (-Xshare:dump -Xbootclasspath/a:foo.jar; -Xshare:on -Xbootclasspath/a:foo.jar123) that we need to handle. Added the test case in BootClassPathMismatch.java. > > Here is the updated webrev: > http://cr.openjdk.java.net/%7Ejiangli/8199807/webrev.03/ > > Thanks, > Jiangli > >> On May 17, 2018, at 11:55 AM, Jiangli Zhou wrote: >> >> Thanks, Ioi! Will incorporate your suggestions below before integration. >> >> Jiangli >> >>> On May 17, 2018, at 11:52 AM, Ioi Lam wrote: >>> >>> Hi Jiangli, >>> >>> The changes look good. Just some minor nits. No need for new webrev if you decide to make these changes. >>> >>> // The first entry in boot path is the modules_image (guaranteed by >>> >>> // ClassLoader::setup_boot_search_path()). Skip the first entry. The >>> // path of the runtime modules_image may be different from the dump >>> // time path (e.g. the JDK image is copied to a different location >>> // after generating the shared archive), which is acceptable. For most >>> // common cases, the dump time boot path might contain modules_image only. >>> >>> Perhaps add an assert that #0 of the bootclasspath is the runtime module image? >>> >>> >>> 444 isSigned = stream->check_is_signed(); >>> 445 if (isSigned) { >>> >>> Maybe remove isSigned variable and change to if (stream->check_is_signed()) .... >>> >>> Thanks >>> - Ioi >>> >>> >>> On 5/16/18 8:27 PM, Jiangli Zhou wrote: >>>> Here is the updated webrev: >>>> http://cr.openjdk.java.net/~jiangli/8199807/webrev.02/ >>>> >>>> At CDS dump time, when SharedClassPathEntry::init() is called from FileMapInfo::allocate_shared_path_table(), we can identify the modules image entry directly. SharedClassPathEntry::init() now takes an extra bool argument to indicate if the current entry is the modules image. We no longer need to test the entry name using ClassLoader::is_modules_image(), which might be wrong for a shared path entry as Ioi pointed out. >>>> >>>> I added a ?_type? field in SharedClassPathEntry to record the type of shared path entries at dump time. At runtime, the path entry ?_type? is checked to identify the modules image in SharedClassPathEntry::validate(). SharedClassPathEntry::is_modules_image() no longer need to use ClassLoader::is_modules_image(). >>>> >>>> Reworked the BOOT_PATH case in SharedPathsMiscInfo::check(). I left os::file_name_strcmp() unchanged since it?s also used by APP_PATH. I'll handle it in a separate REF to do more testing. >>>> >>>> I also included the MoveJDKTest.java written by Ioi in the updated webrev. In the test, I added a new case for a JAR entry named with Hello.modules. >>>> >>>> Reran all appcds tests including MoveJDKTest.java locally. All tests passed. Running tiered tests. >>>> >>>> Thanks, >>>> Jiangli >>>> >>>>> On May 15, 2018, at 6:10 PM, Jiangli Zhou wrote: >>>>> >>>>> Hi Ioi, >>>>> >>>>> Thank you very much for the review. Good catches for those edge cases and thanks for the test cases. I will resolves them. >>>>> >>>>> Thanks! >>>>> Jiangli >>>>> >>>>>> On May 15, 2018, at 4:33 PM, Ioi Lam wrote: >>>>>> >>>>>> Hi Jiangli, >>>>>> >>>>>> 245 // We need to validate the runtime modules image file size against the archived >>>>>> 246 // size information, obtain the runtime modules image path. The recorded dump >>>>>> 247 // time modules image path in the archive may be different from the runtime path >>>>>> 248 // if the JDK image has beed moved after generating the archive. >>>>>> 249 if (ClassLoader::is_modules_image(name)) { >>>>>> 250 name = ClassLoader::get_jrt_entry()->name(); >>>>>> 251 } >>>>>> >>>>>> What happens if someone has a JAR file called foo.modules, and dumped with >>>>>> >>>>>> java -Xshare:dump -cp foo.modules >>>>>> >>>>>> >>>>>> I think it's better to check that it's #0 in the classpath so we know for sure it's the system modules file. >>>>>> >>>>>> There's a similar problem here: >>>>>> >>>>>> 220 if (!ClassLoader::is_modules_image(name)) { >>>>>> >>>>>> In the following, instead of doing a copy, maybe it's better to add a length argument for os::file_name_strcmp, since sharedPathsMiscInfo.cpp is the only place in the VM that calls this function. >>>>>> >>>>>> 162 if (relaxed_check) { >>>>>> 163 // only check the begining portion of the runtime boot path, up to >>>>>> 164 // the length of the dump time boot path >>>>>> 165 size_t len = strlen(path); >>>>>> 166 runtime_boot_path = NEW_RESOURCE_ARRAY(char, len + 1); >>>>>> 167 strncpy(runtime_boot_path, Arguments::get_sysclasspath(), len); >>>>>> 168 runtime_boot_path[len] = '\0'; >>>>>> 169 } else { >>>>>> 170 // check the full runtime boot path >>>>>> 171 runtime_boot_path = Arguments::get_sysclasspath(); >>>>>> 172 } >>>>>> 173 >>>>>> 174 // Do a quick check first with a simple >>>>>> 175 // strcmp(dump_time_boot_path, runtime_boot_path). If the paths are the >>>>>> 176 // same, the check has succeeded. >>>>>> 177 if (os::file_name_strcmp(path, runtime_boot_path) == 0) { >>>>>> 178 break; // ok >>>>>> 179 } >>>>>> >>>>>> >>>>>> Also, the copy could be wrong for the following output: >>>>>> >>>>>> path = /tmp/foo/modules >>>>>> Arguments::get_sysclasspath() = /tmp/foo/modulesxyz:/aaa:/bbb >>>>>> >>>>>> What is runtime_boot_path supposed to contain here? I am not sure if this code gives the intended output or not, but it's not easy to understand what it actually does by operating on an truncated pathname. >>>>>> >>>>>> String manipulation code is always hard to read. I would suggest braking it up into smaller functions: >>>>>> >>>>>> 1. Get the first entry of dump time and run time boot path -> d0 and r0 >>>>>> -> ignore d0 >>>>>> -> r0 same as ClassLoader::get_jrt_entry()->name(); >>>>>> >>>>>> 2. Get the remaining part of the dump time and run time boot path -> d_remain, r_remain >>>>>> -> relaxed_check: d_remain must be a prefix of r_remain >>>>>> -> !relaxed_check: They must be identical. >>>>>> >>>>>> >>>>>> By the way, I think the test case in JDK-8202935 should be reviewed together inside this RFR, since the test validates the feature implemented here. I don't think we need a separate bug ID. Otherwise it will be hard to track the test cases. >>>>>> >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>>>> On 5/13/18 5:31 PM, Jiangli Zhou wrote: >>>>>>> Here is the updated webrev: >>>>>>> http://cr.openjdk.java.net/~jiangli/8199807/webrev.01/ >>>>>>> >>>>>>> Thanks, >>>>>>> Jiangli >>>>>>> >>>>>>>> On May 11, 2018, at 6:07 PM, Jiangli Zhou wrote: >>>>>>>> >>>>>>>> Hi Calvin, >>>>>>>> >>>>>>>> Thanks for the review. >>>>>>>> >>>>>>>>> On May 11, 2018, at 4:32 PM, Calvin Cheung wrote: >>>>>>>>> >>>>>>>>> Hi Jiangli, >>>>>>>>> >>>>>>>>> Thanks for doing this useful change. >>>>>>>>> >>>>>>>>> I have a few minor comments: >>>>>>>>> >>>>>>>>> 1) sharedPathsMiscInfo.cpp >>>>>>>>> >>>>>>>>> 199 dp ++; >>>>>>>>> >>>>>>>>> Should the above be similar to line 194 as follows? >>>>>>>>> dp += path_sep_len; >>>>>>>> Good catch. Will fix. >>>>>>>> >>>>>>>>> 2) filemap.cpp >>>>>>>>> >>>>>>>>> 244 if (ClassLoader::is_modules_image(name)) { >>>>>>>>> >>>>>>>>> I think the above could be: >>>>>>>>> if (is_modules_image()) { >>>>>>>>> >>>>>>>> Let?s keep it this way as is_modules_image() is a wrapper of 'ClassLoader::is_modules_image(name())?. We already have the ?name? in this case. >>>>>>>> >>>>>>>>> The is_modules_image() is a member function of SharedClassPathEntry. >>>>>>>>> >>>>>>>>> Btw, why is it necessary to get the runtime modules image path again in lines 244 - 246? >>>>>>>> As the runtime modules image path could be different from the dump time if the JDK image is moved/copied after archive generation, we need to make sure to use the correct file for file size check. The recored path in the archive is the dump time modules image path, which may no longer be existing at runtime. I will add some comments if that?s helpful. >>>>>>>> >>>>>>>>> 3) BootClassPathMismatch.java >>>>>>>>> >>>>>>>>> Could you move line 103 to right above line 107 (opts.addPrefix(?))? >>>>>>>>> Because the archive dumping part of the test doesn?t use appJar. >>>>>>>> Sure. >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> Jiangli >>>>>>>> >>>>>>>>> thanks, >>>>>>>>> Calvin >>>>>>>>> >>>>>>>>> On 5/11/18, 3:15 PM, Jiangli Zhou wrote: >>>>>>>>>> I?ve withdrawn my original weberv and removed the build change. Here is the updated webrev that only addresses 8199807. >>>>>>>>>> >>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.00/ >>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Jiangli >>>>>>>>>> >>>>>>>>>>> On May 11, 2018, at 2:44 PM, Jiangli Zhou wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Erik, >>>>>>>>>>> >>>>>>>>>>> Thank you so much for investigating a proper fix for the vm_version.o issue. I will withdraw the build change from my original webrev. >>>>>>>>>>> >>>>>>>>>>> Thanks again for taking over the bug! >>>>>>>>>>> Jiangli >>>>>>>>>>> >>>>>>>>>>>> On May 11, 2018, at 2:33 PM, Erik Joelsson wrote: >>>>>>>>>>>> >>>>>>>>>>>> Received: from [10.132.185.167] (/10.132.185.167) >>>>>>>>>>>> by default (Oracle Beehive Gateway v4.0) >>>>>>>>>>>> with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 >>>>>>>>>>>> Subject: Re: RFR: 8199807& 8202738: AppCDS performs overly restrictive path >>>>>>>>>>>> matching check >>>>>>>>>>>> To: Jiangli Zhou, >>>>>>>>>>>> "hotspot-runtime-dev at openjdk.java.net runtime" >>>>>>>>>>>> , >>>>>>>>>>>> build-dev >>>>>>>>>>>> References: >>>>>>>>>>>> From: Erik Joelsson >>>>>>>>>>>> Organization: Oracle Corporation >>>>>>>>>>>> Message-ID: >>>>>>>>>>>> Date: Fri, 11 May 2018 14:33:20 -0700 >>>>>>>>>>>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) >>>>>>>>>>>> Gecko/20100101 Thunderbird/52.7.0 >>>>>>>>>>>> MIME-Version: 1.0 >>>>>>>>>>>> In-Reply-To: >>>>>>>>>>>> Content-Type: text/plain; charset=utf-8; format=flowed >>>>>>>>>>>> Content-Transfer-Encoding: 8bit >>>>>>>>>>>> Content-Language: en-US >>>>>>>>>>>> >>>>>>>>>>>> Hello, >>>>>>>>>>>> >>>>>>>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>>>>>>> >>>>>>>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>>>>>>> >>>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>>>>>>> >>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>>>>>>> >>>>>>>>>>>> /Erik >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>>>>>>> >>>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>>>>>>> >>>>>>>>>>>>> The webrev includes the following three main parts: >>>>>>>>>>>>> >>>>>>>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>>>>>>> >>>>>>>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>>>>>>> >>>>>>>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>>>>>>> >>>>>>>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Jiangli >>>>>>>>>>>>> >>>>>>>>>>>> Hello, >>>>>>>>>>>> >>>>>>>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>>>>>>> >>>>>>>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>>>>>>> >>>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>>>>>>> >>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>>>>>>> >>>>>>>>>>>> /Erik >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>>>>>>> >>>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>>>>>>> >>>>>>>>>>>>> The webrev includes the following three main parts: >>>>>>>>>>>>> >>>>>>>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>>>>>>> >>>>>>>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>>>>>>> >>>>>>>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>>>>>>> >>>>>>>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Jiangli From jiangli.zhou at oracle.com Mon May 21 17:41:01 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Mon, 21 May 2018 10:41:01 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: <6544016c-6252-8fd0-5a86-2900368f7221@oracle.com> References: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> <5AF62823.7050804@oracle.com> <70F6AEB0-9103-4899-87D9-DC845BF66C51@oracle.com> <4CBA8376-54A2-4863-8732-B16E1AE9C38A@oracle.com> <5C996E41-CDA0-4EDA-B289-CDD7C38151B9@oracle.com> <6b94fc12-61ae-e328-59b1-40096544cbc1@oracle.com> <9A7035A6-892E-45E6-8F1A-65D6A344D78C@oracle.com> <58606140-A712-406B-A4D9-59D44017477D@oracle.com> <6544016c-6252-8fd0-5a86-2900368f7221@oracle.com> Message-ID: <96C69C43-2988-475F-AE6C-92CB138D93A1@oracle.com> Thanks, Ioi! Jiangli > On May 21, 2018, at 10:40 AM, Ioi Lam wrote: > > Looks good. Thanks for doing this Jiangli! > > - Ioi > > > On 5/21/18 10:12 AM, Jiangli Zhou wrote: >> After discussion with Ioi and Calvin, we all agreed the usages of os::file_name_strcmp() in SharedPathsMiscInfo::check() should be fixed as part of this change. Calvin (thanks!) also found an unlikely but possible edge case (-Xshare:dump -Xbootclasspath/a:foo.jar; -Xshare:on -Xbootclasspath/a:foo.jar123) that we need to handle. Added the test case in BootClassPathMismatch.java. >> >> Here is the updated webrev: >> http://cr.openjdk.java.net/%7Ejiangli/8199807/webrev.03/ >> >> Thanks, >> Jiangli >> >>> On May 17, 2018, at 11:55 AM, Jiangli Zhou wrote: >>> >>> Thanks, Ioi! Will incorporate your suggestions below before integration. >>> >>> Jiangli >>> >>>> On May 17, 2018, at 11:52 AM, Ioi Lam wrote: >>>> >>>> Hi Jiangli, >>>> >>>> The changes look good. Just some minor nits. No need for new webrev if you decide to make these changes. >>>> >>>> // The first entry in boot path is the modules_image (guaranteed by >>>> >>>> // ClassLoader::setup_boot_search_path()). Skip the first entry. The >>>> // path of the runtime modules_image may be different from the dump >>>> // time path (e.g. the JDK image is copied to a different location >>>> // after generating the shared archive), which is acceptable. For most >>>> // common cases, the dump time boot path might contain modules_image only. >>>> >>>> Perhaps add an assert that #0 of the bootclasspath is the runtime module image? >>>> >>>> >>>> 444 isSigned = stream->check_is_signed(); >>>> 445 if (isSigned) { >>>> >>>> Maybe remove isSigned variable and change to if (stream->check_is_signed()) .... >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> On 5/16/18 8:27 PM, Jiangli Zhou wrote: >>>>> Here is the updated webrev: >>>>> http://cr.openjdk.java.net/~jiangli/8199807/webrev.02/ >>>>> >>>>> At CDS dump time, when SharedClassPathEntry::init() is called from FileMapInfo::allocate_shared_path_table(), we can identify the modules image entry directly. SharedClassPathEntry::init() now takes an extra bool argument to indicate if the current entry is the modules image. We no longer need to test the entry name using ClassLoader::is_modules_image(), which might be wrong for a shared path entry as Ioi pointed out. >>>>> >>>>> I added a ?_type? field in SharedClassPathEntry to record the type of shared path entries at dump time. At runtime, the path entry ?_type? is checked to identify the modules image in SharedClassPathEntry::validate(). SharedClassPathEntry::is_modules_image() no longer need to use ClassLoader::is_modules_image(). >>>>> >>>>> Reworked the BOOT_PATH case in SharedPathsMiscInfo::check(). I left os::file_name_strcmp() unchanged since it?s also used by APP_PATH. I'll handle it in a separate REF to do more testing. >>>>> >>>>> I also included the MoveJDKTest.java written by Ioi in the updated webrev. In the test, I added a new case for a JAR entry named with Hello.modules. >>>>> >>>>> Reran all appcds tests including MoveJDKTest.java locally. All tests passed. Running tiered tests. >>>>> >>>>> Thanks, >>>>> Jiangli >>>>> >>>>>> On May 15, 2018, at 6:10 PM, Jiangli Zhou wrote: >>>>>> >>>>>> Hi Ioi, >>>>>> >>>>>> Thank you very much for the review. Good catches for those edge cases and thanks for the test cases. I will resolves them. >>>>>> >>>>>> Thanks! >>>>>> Jiangli >>>>>> >>>>>>> On May 15, 2018, at 4:33 PM, Ioi Lam wrote: >>>>>>> >>>>>>> Hi Jiangli, >>>>>>> >>>>>>> 245 // We need to validate the runtime modules image file size against the archived >>>>>>> 246 // size information, obtain the runtime modules image path. The recorded dump >>>>>>> 247 // time modules image path in the archive may be different from the runtime path >>>>>>> 248 // if the JDK image has beed moved after generating the archive. >>>>>>> 249 if (ClassLoader::is_modules_image(name)) { >>>>>>> 250 name = ClassLoader::get_jrt_entry()->name(); >>>>>>> 251 } >>>>>>> >>>>>>> What happens if someone has a JAR file called foo.modules, and dumped with >>>>>>> >>>>>>> java -Xshare:dump -cp foo.modules >>>>>>> >>>>>>> >>>>>>> I think it's better to check that it's #0 in the classpath so we know for sure it's the system modules file. >>>>>>> >>>>>>> There's a similar problem here: >>>>>>> >>>>>>> 220 if (!ClassLoader::is_modules_image(name)) { >>>>>>> >>>>>>> In the following, instead of doing a copy, maybe it's better to add a length argument for os::file_name_strcmp, since sharedPathsMiscInfo.cpp is the only place in the VM that calls this function. >>>>>>> >>>>>>> 162 if (relaxed_check) { >>>>>>> 163 // only check the begining portion of the runtime boot path, up to >>>>>>> 164 // the length of the dump time boot path >>>>>>> 165 size_t len = strlen(path); >>>>>>> 166 runtime_boot_path = NEW_RESOURCE_ARRAY(char, len + 1); >>>>>>> 167 strncpy(runtime_boot_path, Arguments::get_sysclasspath(), len); >>>>>>> 168 runtime_boot_path[len] = '\0'; >>>>>>> 169 } else { >>>>>>> 170 // check the full runtime boot path >>>>>>> 171 runtime_boot_path = Arguments::get_sysclasspath(); >>>>>>> 172 } >>>>>>> 173 >>>>>>> 174 // Do a quick check first with a simple >>>>>>> 175 // strcmp(dump_time_boot_path, runtime_boot_path). If the paths are the >>>>>>> 176 // same, the check has succeeded. >>>>>>> 177 if (os::file_name_strcmp(path, runtime_boot_path) == 0) { >>>>>>> 178 break; // ok >>>>>>> 179 } >>>>>>> >>>>>>> >>>>>>> Also, the copy could be wrong for the following output: >>>>>>> >>>>>>> path = /tmp/foo/modules >>>>>>> Arguments::get_sysclasspath() = /tmp/foo/modulesxyz:/aaa:/bbb >>>>>>> >>>>>>> What is runtime_boot_path supposed to contain here? I am not sure if this code gives the intended output or not, but it's not easy to understand what it actually does by operating on an truncated pathname. >>>>>>> >>>>>>> String manipulation code is always hard to read. I would suggest braking it up into smaller functions: >>>>>>> >>>>>>> 1. Get the first entry of dump time and run time boot path -> d0 and r0 >>>>>>> -> ignore d0 >>>>>>> -> r0 same as ClassLoader::get_jrt_entry()->name(); >>>>>>> >>>>>>> 2. Get the remaining part of the dump time and run time boot path -> d_remain, r_remain >>>>>>> -> relaxed_check: d_remain must be a prefix of r_remain >>>>>>> -> !relaxed_check: They must be identical. >>>>>>> >>>>>>> >>>>>>> By the way, I think the test case in JDK-8202935 should be reviewed together inside this RFR, since the test validates the feature implemented here. I don't think we need a separate bug ID. Otherwise it will be hard to track the test cases. >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> On 5/13/18 5:31 PM, Jiangli Zhou wrote: >>>>>>>> Here is the updated webrev: >>>>>>>> http://cr.openjdk.java.net/~jiangli/8199807/webrev.01/ >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jiangli >>>>>>>> >>>>>>>>> On May 11, 2018, at 6:07 PM, Jiangli Zhou wrote: >>>>>>>>> >>>>>>>>> Hi Calvin, >>>>>>>>> >>>>>>>>> Thanks for the review. >>>>>>>>> >>>>>>>>>> On May 11, 2018, at 4:32 PM, Calvin Cheung wrote: >>>>>>>>>> >>>>>>>>>> Hi Jiangli, >>>>>>>>>> >>>>>>>>>> Thanks for doing this useful change. >>>>>>>>>> >>>>>>>>>> I have a few minor comments: >>>>>>>>>> >>>>>>>>>> 1) sharedPathsMiscInfo.cpp >>>>>>>>>> >>>>>>>>>> 199 dp ++; >>>>>>>>>> >>>>>>>>>> Should the above be similar to line 194 as follows? >>>>>>>>>> dp += path_sep_len; >>>>>>>>> Good catch. Will fix. >>>>>>>>> >>>>>>>>>> 2) filemap.cpp >>>>>>>>>> >>>>>>>>>> 244 if (ClassLoader::is_modules_image(name)) { >>>>>>>>>> >>>>>>>>>> I think the above could be: >>>>>>>>>> if (is_modules_image()) { >>>>>>>>>> >>>>>>>>> Let?s keep it this way as is_modules_image() is a wrapper of 'ClassLoader::is_modules_image(name())?. We already have the ?name? in this case. >>>>>>>>> >>>>>>>>>> The is_modules_image() is a member function of SharedClassPathEntry. >>>>>>>>>> >>>>>>>>>> Btw, why is it necessary to get the runtime modules image path again in lines 244 - 246? >>>>>>>>> As the runtime modules image path could be different from the dump time if the JDK image is moved/copied after archive generation, we need to make sure to use the correct file for file size check. The recored path in the archive is the dump time modules image path, which may no longer be existing at runtime. I will add some comments if that?s helpful. >>>>>>>>> >>>>>>>>>> 3) BootClassPathMismatch.java >>>>>>>>>> >>>>>>>>>> Could you move line 103 to right above line 107 (opts.addPrefix(?))? >>>>>>>>>> Because the archive dumping part of the test doesn?t use appJar. >>>>>>>>> Sure. >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> >>>>>>>>> Jiangli >>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> Calvin >>>>>>>>>> >>>>>>>>>> On 5/11/18, 3:15 PM, Jiangli Zhou wrote: >>>>>>>>>>> I?ve withdrawn my original weberv and removed the build change. Here is the updated webrev that only addresses 8199807. >>>>>>>>>>> >>>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.00/ >>>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Jiangli >>>>>>>>>>> >>>>>>>>>>>> On May 11, 2018, at 2:44 PM, Jiangli Zhou wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Erik, >>>>>>>>>>>> >>>>>>>>>>>> Thank you so much for investigating a proper fix for the vm_version.o issue. I will withdraw the build change from my original webrev. >>>>>>>>>>>> >>>>>>>>>>>> Thanks again for taking over the bug! >>>>>>>>>>>> Jiangli >>>>>>>>>>>> >>>>>>>>>>>>> On May 11, 2018, at 2:33 PM, Erik Joelsson wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Received: from [10.132.185.167] (/10.132.185.167) >>>>>>>>>>>>> by default (Oracle Beehive Gateway v4.0) >>>>>>>>>>>>> with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 >>>>>>>>>>>>> Subject: Re: RFR: 8199807& 8202738: AppCDS performs overly restrictive path >>>>>>>>>>>>> matching check >>>>>>>>>>>>> To: Jiangli Zhou, >>>>>>>>>>>>> "hotspot-runtime-dev at openjdk.java.net runtime" >>>>>>>>>>>>> , >>>>>>>>>>>>> build-dev >>>>>>>>>>>>> References: >>>>>>>>>>>>> From: Erik Joelsson >>>>>>>>>>>>> Organization: Oracle Corporation >>>>>>>>>>>>> Message-ID: >>>>>>>>>>>>> Date: Fri, 11 May 2018 14:33:20 -0700 >>>>>>>>>>>>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) >>>>>>>>>>>>> Gecko/20100101 Thunderbird/52.7.0 >>>>>>>>>>>>> MIME-Version: 1.0 >>>>>>>>>>>>> In-Reply-To: >>>>>>>>>>>>> Content-Type: text/plain; charset=utf-8; format=flowed >>>>>>>>>>>>> Content-Transfer-Encoding: 8bit >>>>>>>>>>>>> Content-Language: en-US >>>>>>>>>>>>> >>>>>>>>>>>>> Hello, >>>>>>>>>>>>> >>>>>>>>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>>>>>>>> >>>>>>>>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>>>>>>>> >>>>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>>>>>>>> >>>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>>>>>>>> >>>>>>>>>>>>> /Erik >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>>>>>>>> >>>>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>>>>>>>> >>>>>>>>>>>>>> The webrev includes the following three main parts: >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>>>>>>>> >>>>>>>>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>>>>>>>> >>>>>>>>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Jiangli >>>>>>>>>>>>>> >>>>>>>>>>>>> Hello, >>>>>>>>>>>>> >>>>>>>>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>>>>>>>> >>>>>>>>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>>>>>>>> >>>>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>>>>>>>> >>>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>>>>>>>> >>>>>>>>>>>>> /Erik >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>>>>>>>> >>>>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>>>>>>>> >>>>>>>>>>>>>> The webrev includes the following three main parts: >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>>>>>>>> >>>>>>>>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>>>>>>>> >>>>>>>>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Jiangli > From calvin.cheung at oracle.com Mon May 21 17:52:03 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Mon, 21 May 2018 10:52:03 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: <58606140-A712-406B-A4D9-59D44017477D@oracle.com> References: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> <5AF62823.7050804@oracle.com> <70F6AEB0-9103-4899-87D9-DC845BF66C51@oracle.com> <4CBA8376-54A2-4863-8732-B16E1AE9C38A@oracle.com> <5C996E41-CDA0-4EDA-B289-CDD7C38151B9@oracle.com> <6b94fc12-61ae-e328-59b1-40096544cbc1@oracle.com> <9A7035A6-892E-45E6-8F1A-65D6A344D78C@oracle.com> <58606140-A712-406B-A4D9-59D44017477D@oracle.com> Message-ID: <5B030743.4010105@oracle.com> Hi Jiangli, The latest changes look good. Thanks for adding the test scenarios. thanks, Calvin On 5/21/18, 10:12 AM, Jiangli Zhou wrote: > After discussion with Ioi and Calvin, we all agreed the usages of os::file_name_strcmp() in SharedPathsMiscInfo::check() should be fixed as part of this change. Calvin (thanks!) also found an unlikely but possible edge case (-Xshare:dump -Xbootclasspath/a:foo.jar; -Xshare:on -Xbootclasspath/a:foo.jar123) that we need to handle. Added the test case in BootClassPathMismatch.java. > > Here is the updated webrev: > http://cr.openjdk.java.net/%7Ejiangli/8199807/webrev.03/ > > Thanks, > Jiangli > >> On May 17, 2018, at 11:55 AM, Jiangli Zhou wrote: >> >> Thanks, Ioi! Will incorporate your suggestions below before integration. >> >> Jiangli >> >>> On May 17, 2018, at 11:52 AM, Ioi Lam wrote: >>> >>> Hi Jiangli, >>> >>> The changes look good. Just some minor nits. No need for new webrev if you decide to make these changes. >>> >>> // The first entry in boot path is the modules_image (guaranteed by >>> >>> // ClassLoader::setup_boot_search_path()). Skip the first entry. The >>> // path of the runtime modules_image may be different from the dump >>> // time path (e.g. the JDK image is copied to a different location >>> // after generating the shared archive), which is acceptable. For most >>> // common cases, the dump time boot path might contain modules_image only. >>> >>> Perhaps add an assert that #0 of the bootclasspath is the runtime module image? >>> >>> >>> 444 isSigned = stream->check_is_signed(); >>> 445 if (isSigned) { >>> >>> Maybe remove isSigned variable and change to if (stream->check_is_signed()) .... >>> >>> Thanks >>> - Ioi >>> >>> >>> On 5/16/18 8:27 PM, Jiangli Zhou wrote: >>>> Here is the updated webrev: >>>> http://cr.openjdk.java.net/~jiangli/8199807/webrev.02/ >>>> >>>> At CDS dump time, when SharedClassPathEntry::init() is called from FileMapInfo::allocate_shared_path_table(), we can identify the modules image entry directly. SharedClassPathEntry::init() now takes an extra bool argument to indicate if the current entry is the modules image. We no longer need to test the entry name using ClassLoader::is_modules_image(), which might be wrong for a shared path entry as Ioi pointed out. >>>> >>>> I added a ?_type? field in SharedClassPathEntry to record the type of shared path entries at dump time. At runtime, the path entry ?_type? is checked to identify the modules image in SharedClassPathEntry::validate(). SharedClassPathEntry::is_modules_image() no longer need to use ClassLoader::is_modules_image(). >>>> >>>> Reworked the BOOT_PATH case in SharedPathsMiscInfo::check(). I left os::file_name_strcmp() unchanged since it?s also used by APP_PATH. I'll handle it in a separate REF to do more testing. >>>> >>>> I also included the MoveJDKTest.java written by Ioi in the updated webrev. In the test, I added a new case for a JAR entry named with Hello.modules. >>>> >>>> Reran all appcds tests including MoveJDKTest.java locally. All tests passed. Running tiered tests. >>>> >>>> Thanks, >>>> Jiangli >>>> >>>>> On May 15, 2018, at 6:10 PM, Jiangli Zhou wrote: >>>>> >>>>> Hi Ioi, >>>>> >>>>> Thank you very much for the review. Good catches for those edge cases and thanks for the test cases. I will resolves them. >>>>> >>>>> Thanks! >>>>> Jiangli >>>>> >>>>>> On May 15, 2018, at 4:33 PM, Ioi Lam wrote: >>>>>> >>>>>> Hi Jiangli, >>>>>> >>>>>> 245 // We need to validate the runtime modules image file size against the archived >>>>>> 246 // size information, obtain the runtime modules image path. The recorded dump >>>>>> 247 // time modules image path in the archive may be different from the runtime path >>>>>> 248 // if the JDK image has beed moved after generating the archive. >>>>>> 249 if (ClassLoader::is_modules_image(name)) { >>>>>> 250 name = ClassLoader::get_jrt_entry()->name(); >>>>>> 251 } >>>>>> >>>>>> What happens if someone has a JAR file called foo.modules, and dumped with >>>>>> >>>>>> java -Xshare:dump -cp foo.modules >>>>>> >>>>>> >>>>>> I think it's better to check that it's #0 in the classpath so we know for sure it's the system modules file. >>>>>> >>>>>> There's a similar problem here: >>>>>> >>>>>> 220 if (!ClassLoader::is_modules_image(name)) { >>>>>> >>>>>> In the following, instead of doing a copy, maybe it's better to add a length argument for os::file_name_strcmp, since sharedPathsMiscInfo.cpp is the only place in the VM that calls this function. >>>>>> >>>>>> 162 if (relaxed_check) { >>>>>> 163 // only check the begining portion of the runtime boot path, up to >>>>>> 164 // the length of the dump time boot path >>>>>> 165 size_t len = strlen(path); >>>>>> 166 runtime_boot_path = NEW_RESOURCE_ARRAY(char, len + 1); >>>>>> 167 strncpy(runtime_boot_path, Arguments::get_sysclasspath(), len); >>>>>> 168 runtime_boot_path[len] = '\0'; >>>>>> 169 } else { >>>>>> 170 // check the full runtime boot path >>>>>> 171 runtime_boot_path = Arguments::get_sysclasspath(); >>>>>> 172 } >>>>>> 173 >>>>>> 174 // Do a quick check first with a simple >>>>>> 175 // strcmp(dump_time_boot_path, runtime_boot_path). If the paths are the >>>>>> 176 // same, the check has succeeded. >>>>>> 177 if (os::file_name_strcmp(path, runtime_boot_path) == 0) { >>>>>> 178 break; // ok >>>>>> 179 } >>>>>> >>>>>> >>>>>> Also, the copy could be wrong for the following output: >>>>>> >>>>>> path = /tmp/foo/modules >>>>>> Arguments::get_sysclasspath() = /tmp/foo/modulesxyz:/aaa:/bbb >>>>>> >>>>>> What is runtime_boot_path supposed to contain here? I am not sure if this code gives the intended output or not, but it's not easy to understand what it actually does by operating on an truncated pathname. >>>>>> >>>>>> String manipulation code is always hard to read. I would suggest braking it up into smaller functions: >>>>>> >>>>>> 1. Get the first entry of dump time and run time boot path -> d0 and r0 >>>>>> -> ignore d0 >>>>>> -> r0 same as ClassLoader::get_jrt_entry()->name(); >>>>>> >>>>>> 2. Get the remaining part of the dump time and run time boot path -> d_remain, r_remain >>>>>> -> relaxed_check: d_remain must be a prefix of r_remain >>>>>> -> !relaxed_check: They must be identical. >>>>>> >>>>>> >>>>>> By the way, I think the test case in JDK-8202935 should be reviewed together inside this RFR, since the test validates the feature implemented here. I don't think we need a separate bug ID. Otherwise it will be hard to track the test cases. >>>>>> >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>>>> On 5/13/18 5:31 PM, Jiangli Zhou wrote: >>>>>>> Here is the updated webrev: >>>>>>> http://cr.openjdk.java.net/~jiangli/8199807/webrev.01/ >>>>>>> >>>>>>> Thanks, >>>>>>> Jiangli >>>>>>> >>>>>>>> On May 11, 2018, at 6:07 PM, Jiangli Zhou wrote: >>>>>>>> >>>>>>>> Hi Calvin, >>>>>>>> >>>>>>>> Thanks for the review. >>>>>>>> >>>>>>>>> On May 11, 2018, at 4:32 PM, Calvin Cheung wrote: >>>>>>>>> >>>>>>>>> Hi Jiangli, >>>>>>>>> >>>>>>>>> Thanks for doing this useful change. >>>>>>>>> >>>>>>>>> I have a few minor comments: >>>>>>>>> >>>>>>>>> 1) sharedPathsMiscInfo.cpp >>>>>>>>> >>>>>>>>> 199 dp ++; >>>>>>>>> >>>>>>>>> Should the above be similar to line 194 as follows? >>>>>>>>> dp += path_sep_len; >>>>>>>> Good catch. Will fix. >>>>>>>> >>>>>>>>> 2) filemap.cpp >>>>>>>>> >>>>>>>>> 244 if (ClassLoader::is_modules_image(name)) { >>>>>>>>> >>>>>>>>> I think the above could be: >>>>>>>>> if (is_modules_image()) { >>>>>>>>> >>>>>>>> Let?s keep it this way as is_modules_image() is a wrapper of 'ClassLoader::is_modules_image(name())?. We already have the ?name? in this case. >>>>>>>> >>>>>>>>> The is_modules_image() is a member function of SharedClassPathEntry. >>>>>>>>> >>>>>>>>> Btw, why is it necessary to get the runtime modules image path again in lines 244 - 246? >>>>>>>> As the runtime modules image path could be different from the dump time if the JDK image is moved/copied after archive generation, we need to make sure to use the correct file for file size check. The recored path in the archive is the dump time modules image path, which may no longer be existing at runtime. I will add some comments if that?s helpful. >>>>>>>> >>>>>>>>> 3) BootClassPathMismatch.java >>>>>>>>> >>>>>>>>> Could you move line 103 to right above line 107 (opts.addPrefix(?))? >>>>>>>>> Because the archive dumping part of the test doesn?t use appJar. >>>>>>>> Sure. >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> Jiangli >>>>>>>> >>>>>>>>> thanks, >>>>>>>>> Calvin >>>>>>>>> >>>>>>>>> On 5/11/18, 3:15 PM, Jiangli Zhou wrote: >>>>>>>>>> I?ve withdrawn my original weberv and removed the build change. Here is the updated webrev that only addresses 8199807. >>>>>>>>>> >>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.00/ >>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Jiangli >>>>>>>>>> >>>>>>>>>>> On May 11, 2018, at 2:44 PM, Jiangli Zhou wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Erik, >>>>>>>>>>> >>>>>>>>>>> Thank you so much for investigating a proper fix for the vm_version.o issue. I will withdraw the build change from my original webrev. >>>>>>>>>>> >>>>>>>>>>> Thanks again for taking over the bug! >>>>>>>>>>> Jiangli >>>>>>>>>>> >>>>>>>>>>>> On May 11, 2018, at 2:33 PM, Erik Joelsson wrote: >>>>>>>>>>>> >>>>>>>>>>>> Received: from [10.132.185.167] (/10.132.185.167) >>>>>>>>>>>> by default (Oracle Beehive Gateway v4.0) >>>>>>>>>>>> with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 >>>>>>>>>>>> Subject: Re: RFR: 8199807& 8202738: AppCDS performs overly restrictive path >>>>>>>>>>>> matching check >>>>>>>>>>>> To: Jiangli Zhou, >>>>>>>>>>>> "hotspot-runtime-dev at openjdk.java.net runtime" >>>>>>>>>>>> , >>>>>>>>>>>> build-dev >>>>>>>>>>>> References: >>>>>>>>>>>> From: Erik Joelsson >>>>>>>>>>>> Organization: Oracle Corporation >>>>>>>>>>>> Message-ID: >>>>>>>>>>>> Date: Fri, 11 May 2018 14:33:20 -0700 >>>>>>>>>>>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) >>>>>>>>>>>> Gecko/20100101 Thunderbird/52.7.0 >>>>>>>>>>>> MIME-Version: 1.0 >>>>>>>>>>>> In-Reply-To: >>>>>>>>>>>> Content-Type: text/plain; charset=utf-8; format=flowed >>>>>>>>>>>> Content-Transfer-Encoding: 8bit >>>>>>>>>>>> Content-Language: en-US >>>>>>>>>>>> >>>>>>>>>>>> Hello, >>>>>>>>>>>> >>>>>>>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>>>>>>> >>>>>>>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>>>>>>> >>>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>>>>>>> >>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>>>>>>> >>>>>>>>>>>> /Erik >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>>>>>>> >>>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>>>>>>> >>>>>>>>>>>>> The webrev includes the following three main parts: >>>>>>>>>>>>> >>>>>>>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>>>>>>> >>>>>>>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>>>>>>> >>>>>>>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>>>>>>> >>>>>>>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Jiangli >>>>>>>>>>>>> >>>>>>>>>>>> Hello, >>>>>>>>>>>> >>>>>>>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>>>>>>> >>>>>>>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>>>>>>> >>>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>>>>>>> >>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>>>>>>> >>>>>>>>>>>> /Erik >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>>>>>>> >>>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>>>>>>> >>>>>>>>>>>>> The webrev includes the following three main parts: >>>>>>>>>>>>> >>>>>>>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>>>>>>> >>>>>>>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>>>>>>> >>>>>>>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>>>>>>> >>>>>>>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Jiangli From coleen.phillimore at oracle.com Mon May 21 17:59:40 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 21 May 2018 13:59:40 -0400 Subject: RFR (trivial) 8202853: NotifyCount is not initialized Message-ID: <0bf86907-cd4f-1649-87ec-6f44e2edb34f@oracle.com> Removed NotifyCount.? There's no useful logging in this code that could use this field. Built on linux-x64. open webrev at http://cr.openjdk.java.net/~coleenp/8202853.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8202853 Thanks, Coleen From jiangli.zhou at oracle.com Mon May 21 18:20:23 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Mon, 21 May 2018 11:20:23 -0700 Subject: RFR: 8199807 & 8202738: AppCDS performs overly restrictive path matching check In-Reply-To: <5B030743.4010105@oracle.com> References: <2D898ED9-D84F-4EDF-9DF3-88571396223D@oracle.com> <9C60DE17-F4B7-4535-BA53-765E8741CD52@oracle.com> <5AF62823.7050804@oracle.com> <70F6AEB0-9103-4899-87D9-DC845BF66C51@oracle.com> <4CBA8376-54A2-4863-8732-B16E1AE9C38A@oracle.com> <5C996E41-CDA0-4EDA-B289-CDD7C38151B9@oracle.com> <6b94fc12-61ae-e328-59b1-40096544cbc1@oracle.com> <9A7035A6-892E-45E6-8F1A-65D6A344D78C@oracle.com> <58606140-A712-406B-A4D9-59D44017477D@oracle.com> <5B030743.4010105@oracle.com> Message-ID: Thanks, Calvin! Jiangli > On May 21, 2018, at 10:52 AM, Calvin Cheung wrote: > > Hi Jiangli, > > The latest changes look good. > Thanks for adding the test scenarios. > > thanks, > Calvin > > On 5/21/18, 10:12 AM, Jiangli Zhou wrote: >> After discussion with Ioi and Calvin, we all agreed the usages of os::file_name_strcmp() in SharedPathsMiscInfo::check() should be fixed as part of this change. Calvin (thanks!) also found an unlikely but possible edge case (-Xshare:dump -Xbootclasspath/a:foo.jar; -Xshare:on -Xbootclasspath/a:foo.jar123) that we need to handle. Added the test case in BootClassPathMismatch.java. >> >> Here is the updated webrev: >> http://cr.openjdk.java.net/%7Ejiangli/8199807/webrev.03/ >> >> Thanks, >> Jiangli >> >>> On May 17, 2018, at 11:55 AM, Jiangli Zhou wrote: >>> >>> Thanks, Ioi! Will incorporate your suggestions below before integration. >>> >>> Jiangli >>> >>>> On May 17, 2018, at 11:52 AM, Ioi Lam wrote: >>>> >>>> Hi Jiangli, >>>> >>>> The changes look good. Just some minor nits. No need for new webrev if you decide to make these changes. >>>> >>>> // The first entry in boot path is the modules_image (guaranteed by >>>> >>>> // ClassLoader::setup_boot_search_path()). Skip the first entry. The >>>> // path of the runtime modules_image may be different from the dump >>>> // time path (e.g. the JDK image is copied to a different location >>>> // after generating the shared archive), which is acceptable. For most >>>> // common cases, the dump time boot path might contain modules_image only. >>>> >>>> Perhaps add an assert that #0 of the bootclasspath is the runtime module image? >>>> >>>> >>>> 444 isSigned = stream->check_is_signed(); >>>> 445 if (isSigned) { >>>> >>>> Maybe remove isSigned variable and change to if (stream->check_is_signed()) .... >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> On 5/16/18 8:27 PM, Jiangli Zhou wrote: >>>>> Here is the updated webrev: >>>>> http://cr.openjdk.java.net/~jiangli/8199807/webrev.02/ >>>>> >>>>> At CDS dump time, when SharedClassPathEntry::init() is called from FileMapInfo::allocate_shared_path_table(), we can identify the modules image entry directly. SharedClassPathEntry::init() now takes an extra bool argument to indicate if the current entry is the modules image. We no longer need to test the entry name using ClassLoader::is_modules_image(), which might be wrong for a shared path entry as Ioi pointed out. >>>>> >>>>> I added a ?_type? field in SharedClassPathEntry to record the type of shared path entries at dump time. At runtime, the path entry ?_type? is checked to identify the modules image in SharedClassPathEntry::validate(). SharedClassPathEntry::is_modules_image() no longer need to use ClassLoader::is_modules_image(). >>>>> >>>>> Reworked the BOOT_PATH case in SharedPathsMiscInfo::check(). I left os::file_name_strcmp() unchanged since it?s also used by APP_PATH. I'll handle it in a separate REF to do more testing. >>>>> >>>>> I also included the MoveJDKTest.java written by Ioi in the updated webrev. In the test, I added a new case for a JAR entry named with Hello.modules. >>>>> >>>>> Reran all appcds tests including MoveJDKTest.java locally. All tests passed. Running tiered tests. >>>>> >>>>> Thanks, >>>>> Jiangli >>>>> >>>>>> On May 15, 2018, at 6:10 PM, Jiangli Zhou wrote: >>>>>> >>>>>> Hi Ioi, >>>>>> >>>>>> Thank you very much for the review. Good catches for those edge cases and thanks for the test cases. I will resolves them. >>>>>> >>>>>> Thanks! >>>>>> Jiangli >>>>>> >>>>>>> On May 15, 2018, at 4:33 PM, Ioi Lam wrote: >>>>>>> >>>>>>> Hi Jiangli, >>>>>>> >>>>>>> 245 // We need to validate the runtime modules image file size against the archived >>>>>>> 246 // size information, obtain the runtime modules image path. The recorded dump >>>>>>> 247 // time modules image path in the archive may be different from the runtime path >>>>>>> 248 // if the JDK image has beed moved after generating the archive. >>>>>>> 249 if (ClassLoader::is_modules_image(name)) { >>>>>>> 250 name = ClassLoader::get_jrt_entry()->name(); >>>>>>> 251 } >>>>>>> >>>>>>> What happens if someone has a JAR file called foo.modules, and dumped with >>>>>>> >>>>>>> java -Xshare:dump -cp foo.modules >>>>>>> >>>>>>> >>>>>>> I think it's better to check that it's #0 in the classpath so we know for sure it's the system modules file. >>>>>>> >>>>>>> There's a similar problem here: >>>>>>> >>>>>>> 220 if (!ClassLoader::is_modules_image(name)) { >>>>>>> >>>>>>> In the following, instead of doing a copy, maybe it's better to add a length argument for os::file_name_strcmp, since sharedPathsMiscInfo.cpp is the only place in the VM that calls this function. >>>>>>> >>>>>>> 162 if (relaxed_check) { >>>>>>> 163 // only check the begining portion of the runtime boot path, up to >>>>>>> 164 // the length of the dump time boot path >>>>>>> 165 size_t len = strlen(path); >>>>>>> 166 runtime_boot_path = NEW_RESOURCE_ARRAY(char, len + 1); >>>>>>> 167 strncpy(runtime_boot_path, Arguments::get_sysclasspath(), len); >>>>>>> 168 runtime_boot_path[len] = '\0'; >>>>>>> 169 } else { >>>>>>> 170 // check the full runtime boot path >>>>>>> 171 runtime_boot_path = Arguments::get_sysclasspath(); >>>>>>> 172 } >>>>>>> 173 >>>>>>> 174 // Do a quick check first with a simple >>>>>>> 175 // strcmp(dump_time_boot_path, runtime_boot_path). If the paths are the >>>>>>> 176 // same, the check has succeeded. >>>>>>> 177 if (os::file_name_strcmp(path, runtime_boot_path) == 0) { >>>>>>> 178 break; // ok >>>>>>> 179 } >>>>>>> >>>>>>> >>>>>>> Also, the copy could be wrong for the following output: >>>>>>> >>>>>>> path = /tmp/foo/modules >>>>>>> Arguments::get_sysclasspath() = /tmp/foo/modulesxyz:/aaa:/bbb >>>>>>> >>>>>>> What is runtime_boot_path supposed to contain here? I am not sure if this code gives the intended output or not, but it's not easy to understand what it actually does by operating on an truncated pathname. >>>>>>> >>>>>>> String manipulation code is always hard to read. I would suggest braking it up into smaller functions: >>>>>>> >>>>>>> 1. Get the first entry of dump time and run time boot path -> d0 and r0 >>>>>>> -> ignore d0 >>>>>>> -> r0 same as ClassLoader::get_jrt_entry()->name(); >>>>>>> >>>>>>> 2. Get the remaining part of the dump time and run time boot path -> d_remain, r_remain >>>>>>> -> relaxed_check: d_remain must be a prefix of r_remain >>>>>>> -> !relaxed_check: They must be identical. >>>>>>> >>>>>>> >>>>>>> By the way, I think the test case in JDK-8202935 should be reviewed together inside this RFR, since the test validates the feature implemented here. I don't think we need a separate bug ID. Otherwise it will be hard to track the test cases. >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> On 5/13/18 5:31 PM, Jiangli Zhou wrote: >>>>>>>> Here is the updated webrev: >>>>>>>> http://cr.openjdk.java.net/~jiangli/8199807/webrev.01/ >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jiangli >>>>>>>> >>>>>>>>> On May 11, 2018, at 6:07 PM, Jiangli Zhou wrote: >>>>>>>>> >>>>>>>>> Hi Calvin, >>>>>>>>> >>>>>>>>> Thanks for the review. >>>>>>>>> >>>>>>>>>> On May 11, 2018, at 4:32 PM, Calvin Cheung wrote: >>>>>>>>>> >>>>>>>>>> Hi Jiangli, >>>>>>>>>> >>>>>>>>>> Thanks for doing this useful change. >>>>>>>>>> >>>>>>>>>> I have a few minor comments: >>>>>>>>>> >>>>>>>>>> 1) sharedPathsMiscInfo.cpp >>>>>>>>>> >>>>>>>>>> 199 dp ++; >>>>>>>>>> >>>>>>>>>> Should the above be similar to line 194 as follows? >>>>>>>>>> dp += path_sep_len; >>>>>>>>> Good catch. Will fix. >>>>>>>>> >>>>>>>>>> 2) filemap.cpp >>>>>>>>>> >>>>>>>>>> 244 if (ClassLoader::is_modules_image(name)) { >>>>>>>>>> >>>>>>>>>> I think the above could be: >>>>>>>>>> if (is_modules_image()) { >>>>>>>>>> >>>>>>>>> Let?s keep it this way as is_modules_image() is a wrapper of 'ClassLoader::is_modules_image(name())?. We already have the ?name? in this case. >>>>>>>>> >>>>>>>>>> The is_modules_image() is a member function of SharedClassPathEntry. >>>>>>>>>> >>>>>>>>>> Btw, why is it necessary to get the runtime modules image path again in lines 244 - 246? >>>>>>>>> As the runtime modules image path could be different from the dump time if the JDK image is moved/copied after archive generation, we need to make sure to use the correct file for file size check. The recored path in the archive is the dump time modules image path, which may no longer be existing at runtime. I will add some comments if that?s helpful. >>>>>>>>> >>>>>>>>>> 3) BootClassPathMismatch.java >>>>>>>>>> >>>>>>>>>> Could you move line 103 to right above line 107 (opts.addPrefix(?))? >>>>>>>>>> Because the archive dumping part of the test doesn?t use appJar. >>>>>>>>> Sure. >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> >>>>>>>>> Jiangli >>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> Calvin >>>>>>>>>> >>>>>>>>>> On 5/11/18, 3:15 PM, Jiangli Zhou wrote: >>>>>>>>>>> I?ve withdrawn my original weberv and removed the build change. Here is the updated webrev that only addresses 8199807. >>>>>>>>>>> >>>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807/webrev.00/ >>>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Jiangli >>>>>>>>>>> >>>>>>>>>>>> On May 11, 2018, at 2:44 PM, Jiangli Zhou wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Erik, >>>>>>>>>>>> >>>>>>>>>>>> Thank you so much for investigating a proper fix for the vm_version.o issue. I will withdraw the build change from my original webrev. >>>>>>>>>>>> >>>>>>>>>>>> Thanks again for taking over the bug! >>>>>>>>>>>> Jiangli >>>>>>>>>>>> >>>>>>>>>>>>> On May 11, 2018, at 2:33 PM, Erik Joelsson wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Received: from [10.132.185.167] (/10.132.185.167) >>>>>>>>>>>>> by default (Oracle Beehive Gateway v4.0) >>>>>>>>>>>>> with ESMTP ; Fri, 11 May 2018 14:33:20 -0700 >>>>>>>>>>>>> Subject: Re: RFR: 8199807& 8202738: AppCDS performs overly restrictive path >>>>>>>>>>>>> matching check >>>>>>>>>>>>> To: Jiangli Zhou, >>>>>>>>>>>>> "hotspot-runtime-dev at openjdk.java.net runtime" >>>>>>>>>>>>> , >>>>>>>>>>>>> build-dev >>>>>>>>>>>>> References: >>>>>>>>>>>>> From: Erik Joelsson >>>>>>>>>>>>> Organization: Oracle Corporation >>>>>>>>>>>>> Message-ID: >>>>>>>>>>>>> Date: Fri, 11 May 2018 14:33:20 -0700 >>>>>>>>>>>>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) >>>>>>>>>>>>> Gecko/20100101 Thunderbird/52.7.0 >>>>>>>>>>>>> MIME-Version: 1.0 >>>>>>>>>>>>> In-Reply-To: >>>>>>>>>>>>> Content-Type: text/plain; charset=utf-8; format=flowed >>>>>>>>>>>>> Content-Transfer-Encoding: 8bit >>>>>>>>>>>>> Content-Language: en-US >>>>>>>>>>>>> >>>>>>>>>>>>> Hello, >>>>>>>>>>>>> >>>>>>>>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>>>>>>>> >>>>>>>>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>>>>>>>> >>>>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>>>>>>>> >>>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>>>>>>>> >>>>>>>>>>>>> /Erik >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>>>>>>>> >>>>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>>>>>>>> >>>>>>>>>>>>>> The webrev includes the following three main parts: >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>>>>>>>> >>>>>>>>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>>>>>>>> >>>>>>>>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Jiangli >>>>>>>>>>>>>> >>>>>>>>>>>>> Hello, >>>>>>>>>>>>> >>>>>>>>>>>>> For the build change, it's very undesirable to always have to relink libjvm on every incremental build. Such a change cannot be accepted. >>>>>>>>>>>>> >>>>>>>>>>>>> I have a counter suggestion, which is still a bit of a hack, but it will cause vm_version.cpp to be recompiled (almost) every time libjvm.so needs to be relinked. The drawback is that compiling vm_version.cpp is now bound to happen absolutely last and so cannot happen in parallel with other compilations. >>>>>>>>>>>>> >>>>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~erikj/8202738/webrev.01/index.html >>>>>>>>>>>>> >>>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738 >>>>>>>>>>>>> >>>>>>>>>>>>> /Erik >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> On 2018-05-10 16:11, Jiangli Zhou wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please review the following webrev that addresses the issue of copied/moved JDK image after generating a CDS archive. Thanks Karen Kinnear and Alan Baterman for initiating the investigation& discussions in this area (especially the ease of usage). Thanks Ioi for implementing a test case for moved JDK (JDK-8202935). >>>>>>>>>>>>>> >>>>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~jiangli/8199807_8202738/webrev.00/ >>>>>>>>>>>>>> RFE: https://bugs.openjdk.java.net/browse/JDK-8199807?filter=14921 >>>>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202738?filter=14921 >>>>>>>>>>>>>> >>>>>>>>>>>>>> The webrev includes the following three main parts: >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1. Reduced check for JDK ?modules? image file at runtime. The runtime path to the ?modules? image is no longer required to the the same as dump time path. Runtime performs file size check only for the ?modules? image, which must match with the dump time ?modules? size. Invalidation of an outdated archive is achieved by the existing vm_version string check (the archived vm_version string must match with the runtime vm_version string). >>>>>>>>>>>>>> >>>>>>>>>>>>>> 2. Boot path check are now performed based on the content of the archive. Also added a new test case in BootClassPathMismatch.java and add more comments for existing test cases. >>>>>>>>>>>>>> >>>>>>>>>>>>>> 3. Fixed the stale vm_version string issue with incremental build. The issue was discovered during the work of 8199807. CDS uses vm_version string as part of the runtime validation check for archive. A stale vm_version string causes the CDS runtime to mistakenly accept an outdated archive. The fix is to make sure vm_version.o is recompiled properly when the library/vm is rebuilt. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Tested with hs-tier1-4 and jdk-tier1-2. Tested by relocating the JDK image manually after generating an archive. Also tested with Ioi?s test both locally and via Mach5. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Jiangli From jiangli.zhou at oracle.com Mon May 21 18:21:46 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Mon, 21 May 2018 11:21:46 -0700 Subject: RFR (trivial) 8202853: NotifyCount is not initialized In-Reply-To: <0bf86907-cd4f-1649-87ec-6f44e2edb34f@oracle.com> References: <0bf86907-cd4f-1649-87ec-6f44e2edb34f@oracle.com> Message-ID: Looks good and trivial! Thanks, Jiangli > On May 21, 2018, at 10:59 AM, coleen.phillimore at oracle.com wrote: > > Removed NotifyCount. There's no useful logging in this code that could use this field. > > Built on linux-x64. > > open webrev at http://cr.openjdk.java.net/~coleenp/8202853.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8202853 > > Thanks, > Coleen From mikhailo.seledtsov at oracle.com Mon May 21 18:34:59 2018 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Mon, 21 May 2018 11:34:59 -0700 Subject: RFR(L): 8199255: [TESTBUG] Open source VM testbase default methods tests Message-ID: <5B031153.6020408@oracle.com> Please review this change that will open source VM default method tests. These tests have been used internally for a while, and are now being open sourced. Since this is not an creation of new tests, we would like to keep the changes during this review to a minimum required for open sourcing these tests, such as major issues and integration blockers. If you have other feedback regarding improvements to these tests, please file RFE(s) that will be addressed later in order of priority. Here is what was done for this change: 1. Moved the tests to OpenJDK repository to the specified directory location and structure. 3. Updated Copyright statements accordingly. 4. Updated "@library" statements accordingly. 5. Updated TEST.groups and a HotSpot test make file JBS: https://bugs.openjdk.java.net/browse/JDK-8199255 Webrev: http://cr.openjdk.java.net/~mseledtsov/8199255.01/ Testing: 1. Ran the following tests on open-only repository and build, using "make run-test" (Linux-x64) vmTestbase_vm_defmeth All PASS 2. Automated multip-platform test system (usual 4 platforms): - vmTestbase_vm_defmeth - hs-tier{1,2} In progress Thank you, Misha From erik.joelsson at oracle.com Mon May 21 18:43:30 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 21 May 2018 11:43:30 -0700 Subject: RFR(L): 8199255: [TESTBUG] Open source VM testbase default methods tests In-Reply-To: <5B031153.6020408@oracle.com> References: <5B031153.6020408@oracle.com> Message-ID: Build changes look ok. /Erik On 2018-05-21 11:34, Mikhailo Seledtsov wrote: > Please review this change that will open source VM default method tests. > These tests have been used internally for a while, and are now being > open sourced. Since this is not an creation of new tests, we would > like to keep the changes during this review to a minimum required for > open sourcing these tests, such as major issues and integration > blockers. If you have other feedback regarding improvements to these > tests, please file RFE(s) that will be addressed later in order of > priority. > > Here is what was done for this change: > ? 1. Moved the tests to OpenJDK repository to the specified directory > location and structure. > ? 3. Updated Copyright statements accordingly. > ? 4. Updated "@library" statements accordingly. > ? 5. Updated TEST.groups and a HotSpot test make file > > ? JBS:??? https://bugs.openjdk.java.net/browse/JDK-8199255 > ? Webrev: http://cr.openjdk.java.net/~mseledtsov/8199255.01/ > > ? Testing: > ????? 1. Ran the following tests on open-only repository and build, > using "make run-test" (Linux-x64) > ???????? vmTestbase_vm_defmeth > ???????? All PASS > > ????? 2. Automated multip-platform test system (usual 4 platforms): > ???????? - vmTestbase_vm_defmeth > ???????? - hs-tier{1,2} > ???????? In progress > > > Thank you, > Misha > From coleen.phillimore at oracle.com Mon May 21 18:55:52 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 21 May 2018 14:55:52 -0400 Subject: RFR (trivial) 8202853: NotifyCount is not initialized In-Reply-To: References: <0bf86907-cd4f-1649-87ec-6f44e2edb34f@oracle.com> Message-ID: <2be208d9-91ab-a39e-9019-868e732b219e@oracle.com> Thank you! Coleen On 5/21/18 2:21 PM, Jiangli Zhou wrote: > Looks good and trivial! > > Thanks, > Jiangli > >> On May 21, 2018, at 10:59 AM, coleen.phillimore at oracle.com wrote: >> >> Removed NotifyCount. There's no useful logging in this code that could use this field. >> >> Built on linux-x64. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8202853.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8202853 >> >> Thanks, >> Coleen From lois.foltan at oracle.com Mon May 21 19:50:23 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Mon, 21 May 2018 15:50:23 -0400 Subject: RFR (s) JDK-8203435: Circular nested dynamic constant test needed to confirm JVMS resolution behavior Message-ID: <4842faca-0775-f217-a5a0-cdbd9cd90357@oracle.com> Please review this change to add a nested circular dynamic constant test to confirm the JVMS 5.4.3.6 Dynamically-Computed Constant and Call Site Resolution behavior as discussed in http://mail.openjdk.java.net/pipermail/valhalla-spec-observers/2018-February/000518.html. open webrev at: http://cr.openjdk.java.net/~lfoltan/bug_jdk8203435/webrev/ bug link: https://bugs.openjdk.java.net/browse/JDK-8203435 Testing: hs-tier1-2, jdk-tier1-3 in progress Thanks, Lois From gerard.ziemski at oracle.com Mon May 21 20:57:02 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Mon, 21 May 2018 15:57:02 -0500 Subject: RFR (XS) 8202360: [TESTBUG] runtime/LoadClass/TestResize.java needs to print output when it fails In-Reply-To: References: Message-ID: hi David, > On May 20, 2018, at 8:38 PM, David Holmes wrote: > > Hi Gerard, > > On 19/05/2018 4:54 AM, Gerard Ziemski wrote: >> hi David, >>> On May 18, 2018, at 12:59 AM, David Holmes wrote: >>> >>> Hi Gerard, >>> >>> On 18/05/2018 2:21 AM, Gerard Ziemski wrote: >>>> hi David, >>>> Thank you for the review! >>>>> On May 16, 2018, at 4:54 PM, David Holmes wrote: >>>>> >>>>> Hi Gerard, >>>>> >>>>> On 17/05/2018 3:47 AM, Gerard Ziemski wrote: >>>>>> Hi all, >>>>>> Please review this small enhancement where we print out additional information (i.e. output from PrintSystemDictionaryAtExit), when we detect the failure. >>>>>> https://bugs.openjdk.java.net/browse/JDK-8202360 >>>>>> http://cr.openjdk.java.net/~gziemski/8202360_rev1/ >>>>> >>>>> I don't understand what your fix is doing - sorry. >>>> The fix takes all the available output bytes from the process (using Scanner to grab all tokens at once) and prints all the output from the process (i.e. PrintSystemDictionaryAtExit, and there is a lot of it) at failure. >>> >>> Okay but that's the rest of the output that hasn't yet been processed - correct? So the line of output that caused the failure won't be there as it's already been grabbed by the scanner and processed. >> Good catch, I changed the fix to make sure we print all the output. >>>>> >>>>> What I wanted to see was the complete history of lines of the form >>>>> >>>>> "Java dictionary (table_size=107, classes=6)" >>>>> >>>>> so that we can see how the table size and number of classes have changed leading up to the failure. But I realize now that we already get that from the: >>>>> >>>>> System.out.println("PASS table_size:"+table_size+", classes:"+classes+" OK"); >>>>> >>>>> All that is missing is the printing of the table_size and classes when we throw the exception. That can be trivially fixed: >>>>> >>>>> if (loadFactor > MAX_LOAD_FACTOR) { >>>>> + System.out.println("FAIL table_size:"+table_size+", classes:"+classes); >>>>> throw new RuntimeException("Load factor too high, expected MAX "+MAX_LOAD_FACTOR+", got "+loadFactor); >>>>> } else { >>>>> System.out.println("PASS table_size:"+table_size+", classes:"+classes+" OK"); >>>>> } >>>> OK, so you determined that only that one piece of info was missing in the end in your case, but maybe the next person might want to see everything that PrintSystemDictionaryAtExit provides? >>>> As long as we?re touching the test - should we then stick with the solution that prints out everything from PrintSystemDictionaryAtExit? >>> >>> I'm not sure how anything that comes later in PrintSystemDictionaryAtExit would help with a failure that happened earlier, but if you see value it then that's fine. However I think you need my suggestion as well to get the information that actually caused the failure. >> We now print all the output from PrintSystemDictionaryAtExit, but > > So you'll print all the good reports for the "Java dictionary ..." lines then if there is a failure print the entire output. That's okay - though as I mention in the bug report it does tend to hit the jtreg output buffer limit. We only print the dictionary in case of failure, i.e: 87 if (loadFactor > MAX_LOAD_FACTOR) { > >> also following your suggestion we summarize the failure details in >> the exception itself like: >> java.lang.RuntimeException: Load factor too high, expected MAX 5.0, got 467.30841121495325 [table size 107, number of clases 50002] > > Okay - that helps with the immediate problem I had. I also now realize that the failure in this test is showing is that the SD did not in fact re-size as expected! Can you elaborate? Is there a reproducible case I can try to run? > >>> Some sample output in the bug report would be useful. >> I attached a sample output to the issue as requested. > > Thanks. > >> https://bugs.openjdk.java.net/browse/JDK-8202360 >> http://cr.openjdk.java.net/~gziemski/8202360_rev2 > > A few suggestions: > > 73 String string_out = ""; > 74 while (line != null) { > 75 string_out += line+"\n"; > > As this is not in fact a constant I would suggest using a (large-size) StringBuilder directly and avoid all the implicit StringBuilder and String usage that the '+' operator will hide. The variable can just be called 'output?. I have re-done the main logic a bit to simplify it, and not have to do either String and ?+? nor StringBuffer, by taking advantage of the fact that the info we need to parse is on the 2nd output line. > 88 System.out.println(string_out); > > Please precede with a comment > > // We've hit an error so print all of the output parsed so far, and > // then the remaining output. Done. > > 93 throw new RuntimeException("Load factor too high, expected MAX "+MAX_LOAD_FACTOR+", got "+loadFactor+" [table size "+table_size+", number of clases "+classes+"]"); > > > Please split this line up to make it much shorter. I'm tempted to ask you to fix this missing spaces around the '+' operator but that bad-style is prevalent throughout this test. (This violates a number of Java style rules.) Done and done. > > Don't forget to add second copyright year. Done. After all the feedback I like the code much better now. Thank you for thorough review and the detailed feedback. https://bugs.openjdk.java.net/browse/JDK-8202360 http://cr.openjdk.java.net/~gziemski/8202360_rev3 cheers From david.holmes at oracle.com Mon May 21 22:34:19 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 22 May 2018 08:34:19 +1000 Subject: RFR (XS) 8202360: [TESTBUG] runtime/LoadClass/TestResize.java needs to print output when it fails In-Reply-To: References: Message-ID: <18a88525-650a-05bd-a37b-a9ff9162e148@oracle.com> Hi Gerard, On 22/05/2018 6:57 AM, Gerard Ziemski wrote: >> On May 20, 2018, at 8:38 PM, David Holmes wrote: >> Okay - that helps with the immediate problem I had. I also now realize that the failure in this test is showing is that the SD did not in fact re-size as expected! > > Can you elaborate? Is there a reproducible case I can try to run? The test's purpose is to check that the SD resizes, so if it finds a load factor too high (which indicates it didn't resize) then the SD resizing logic has either not worked as expected, or the test is not doing what it thinks! > I have re-done the main logic a bit to simplify it, and not have to do either String and ?+? nor StringBuffer, by taking advantage of the fact that the info we need to parse is on the 2nd output line. I was under the impression that we could see multiple lines of the form: Java dictionary (table_size=40423, classes=50002) as the table resized. If that is not the case then none of the output seems relevant to me except for this one line. ?? Thanks, David ----- > >> 88 System.out.println(string_out); >> >> Please precede with a comment >> >> // We've hit an error so print all of the output parsed so far, and >> // then the remaining output. > > Done. > > >> >> 93 throw new RuntimeException("Load factor too high, expected MAX "+MAX_LOAD_FACTOR+", got "+loadFactor+" [table size "+table_size+", number of clases "+classes+"]"); >> >> >> Please split this line up to make it much shorter. I'm tempted to ask you to fix this missing spaces around the '+' operator but that bad-style is prevalent throughout this test. (This violates a number of Java style rules.) > > Done and done. > > >> >> Don't forget to add second copyright year. > > Done. > > After all the feedback I like the code much better now. Thank you for thorough review and the detailed feedback. > > https://bugs.openjdk.java.net/browse/JDK-8202360 > http://cr.openjdk.java.net/~gziemski/8202360_rev3 > > > cheers > From coleen.phillimore at oracle.com Mon May 21 23:01:12 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 21 May 2018 19:01:12 -0400 Subject: RFR (S) 8202669: Intermittent crash in ClassLoadingService::compute_class_size() Message-ID: Summary: Don't report classes that failed to load when reporting class unloading.? Remove the classes first. Also add a NULL check for transitive_interfaces for safety, but the new tests pass without it and fail without the fix. open webrev at http://cr.openjdk.java.net/~coleenp/8202669.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8202669 Tested with hs-tier1-5. Thanks, Coleen From david.holmes at oracle.com Mon May 21 23:28:03 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 22 May 2018 09:28:03 +1000 Subject: RFR (S) 8202669: Intermittent crash in ClassLoadingService::compute_class_size() In-Reply-To: References: Message-ID: <53a31aec-a99c-67ed-f268-e61322ae2225@oracle.com> Hi Coleen, This looks good. On 22/05/2018 9:01 AM, coleen.phillimore at oracle.com wrote: > Summary: Don't report classes that failed to load when reporting class > unloading.? Remove the classes first. > > Also add a NULL check for transitive_interfaces for safety, but the new > tests pass without it and fail without the fix. > > open webrev at http://cr.openjdk.java.net/~coleenp/8202669.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8202669 src/hotspot/share/classfile/classLoaderData.cpp 590 // after erroreous classes are released. Typo: erroreous -> erroneous --- test/hotspot/jtreg/runtime/BadObjectClass/TestLoadObject.java 56 byte[] buf = InMemoryJavaCompiler.compile("java.lang.Object", source, 57 "--patch-module=java.base"); Indent is wrong on line 57. I don't see why you need the two test programs as TestUnloadClassError.java tests both conditions. No need to see a new webrev. Thanks, David ----- > Tested with hs-tier1-5. > > Thanks, > Coleen From ioi.lam at oracle.com Tue May 22 04:24:32 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 21 May 2018 21:24:32 -0700 Subject: RFR 8203381 Replace InstanceKlass::allocate_instance_handle with JavaCalls::construct_new_instance In-Reply-To: <46dd5e52-99c9-9458-1859-5c39d806274b@oracle.com> References: <2a449ddb-4512-193f-d6d1-152989b8526c@oracle.com> <46dd5e52-99c9-9458-1859-5c39d806274b@oracle.com> Message-ID: <7ff17350-ee74-dc2e-4309-9dd5ec04649d@oracle.com> On 5/20/18 7:05 PM, David Holmes wrote: > Hi Ioi, > > On 19/05/2018 2:34 AM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8203381 >> http://cr.openjdk.java.net/~iklam/jdk11/8203381-construct-new-instance.v01/ >> >> >> This patch removes a lot of the boilerplate code for allocating a >> Java object and calling its constructor. Hopefully the code is cleaner >> and easier to read. > > That's a great cleanup! Why didn't we do this years ago! :) > > Only nit is that now call_special is replaced with > construct_new_instance the argument line indentation is no longer > correct. > Hi David, I noticed that there isn't a uniform style of indentation with passing function parameters. Usually people indent to the open parenthesis, but I am hesitant to do that when the open parenthesis is more than 40 chars from the beginning of a line. Otherwise I'll end up with lots of skinny lines. They would also run past 80-char width when you have "wide" parameters. ? return JavaCalls::construct_new_instance(SystemDictionary::Module_klass(), vmSymbols::java_lang_module_init_signature(), ????????????????????????? loader, module_name, CHECK_NH); So if you don't have a big objection, I'll just keep the indentation as is. Thanks - Ioi > Great comments on the Thread construction process too! Hope it didn't > take you too long to figure out why you couldn't make the replacement > in those cases! > > Thanks, > David > >> Also: >> >> Added assert(InstanceKlass::is_initialized(), ...) in case where >> we cannot call JavaCalls::construct_new_instance for thread objects. >> >> Replaced unnecessary SystemDictionary::resolve_or_null() calls with >> SystemDictionary::XXX_klass(). >> >> Tested with hs-tier[1,2,3,4,5] >> >> Thanks >> - Ioi >> >> >> >> >> From david.holmes at oracle.com Tue May 22 04:54:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 22 May 2018 14:54:12 +1000 Subject: RFR 8203381 Replace InstanceKlass::allocate_instance_handle with JavaCalls::construct_new_instance In-Reply-To: <7ff17350-ee74-dc2e-4309-9dd5ec04649d@oracle.com> References: <2a449ddb-4512-193f-d6d1-152989b8526c@oracle.com> <46dd5e52-99c9-9458-1859-5c39d806274b@oracle.com> <7ff17350-ee74-dc2e-4309-9dd5ec04649d@oracle.com> Message-ID: <0611d0d8-fb99-921e-6ca5-21f265c23547@oracle.com> On 22/05/2018 2:24 PM, Ioi Lam wrote: > On 5/20/18 7:05 PM, David Holmes wrote: >> Hi Ioi, >> >> On 19/05/2018 2:34 AM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8203381 >>> http://cr.openjdk.java.net/~iklam/jdk11/8203381-construct-new-instance.v01/ >>> >>> >>> This patch removes a lot of the boilerplate code for allocating a >>> Java object and calling its constructor. Hopefully the code is cleaner >>> and easier to read. >> >> That's a great cleanup! Why didn't we do this years ago! :) >> >> Only nit is that now call_special is replaced with >> construct_new_instance the argument line indentation is no longer >> correct. >> > Hi David, > > I noticed that there isn't a uniform style of indentation with passing > function parameters. Usually people indent to the open parenthesis, but > I am hesitant to do that when the open parenthesis is more than 40 chars > from the beginning of a line. Otherwise I'll end up with lots of skinny > lines. They would also run past 80-char width when you have "wide" > parameters. Yes it is certainly not a uniform rule and often conflicts with other line length guidelines. But in this case the change from aligned to not-aligned was quite noticeable. > ? return > JavaCalls::construct_new_instance(SystemDictionary::Module_klass(), > vmSymbols::java_lang_module_init_signature(), > ????????????????????????? loader, module_name, CHECK_NH); > > So if you don't have a big objection, I'll just keep the indentation as is. It was only a nit so that is fine. Though I was almost tempted to suggest that you rename construct_new_instance to do_construct, as it is the same length as call_special. ;-) Cheers, David > Thanks > - Ioi >> Great comments on the Thread construction process too! Hope it didn't >> take you too long to figure out why you couldn't make the replacement >> in those cases! >> >> Thanks, >> David >> >>> Also: >>> >>> Added assert(InstanceKlass::is_initialized(), ...) in case where >>> we cannot call JavaCalls::construct_new_instance for thread objects. >>> >>> Replaced unnecessary SystemDictionary::resolve_or_null() calls with >>> SystemDictionary::XXX_klass(). >>> >>> Tested with hs-tier[1,2,3,4,5] >>> >>> Thanks >>> - Ioi >>> >>> >>> >>> >>> > From ioi.lam at oracle.com Tue May 22 05:23:21 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 21 May 2018 22:23:21 -0700 Subject: RFR 8203381 Replace InstanceKlass::allocate_instance_handle with JavaCalls::construct_new_instance In-Reply-To: <0611d0d8-fb99-921e-6ca5-21f265c23547@oracle.com> References: <2a449ddb-4512-193f-d6d1-152989b8526c@oracle.com> <46dd5e52-99c9-9458-1859-5c39d806274b@oracle.com> <7ff17350-ee74-dc2e-4309-9dd5ec04649d@oracle.com> <0611d0d8-fb99-921e-6ca5-21f265c23547@oracle.com> Message-ID: <29295ce6-7544-8d6e-b94d-48f979fa8ee7@oracle.com> On 5/21/18 9:54 PM, David Holmes wrote: > On 22/05/2018 2:24 PM, Ioi Lam wrote: >> On 5/20/18 7:05 PM, David Holmes wrote: >>> Hi Ioi, >>> >>> On 19/05/2018 2:34 AM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8203381 >>>> http://cr.openjdk.java.net/~iklam/jdk11/8203381-construct-new-instance.v01/ >>>> >>>> >>>> This patch removes a lot of the boilerplate code for allocating a >>>> Java object and calling its constructor. Hopefully the code is cleaner >>>> and easier to read. >>> >>> That's a great cleanup! Why didn't we do this years ago! :) >>> >>> Only nit is that now call_special is replaced with >>> construct_new_instance the argument line indentation is no longer >>> correct. >>> >> Hi David, >> >> I noticed that there isn't a uniform style of indentation with >> passing function parameters. Usually people indent to the open >> parenthesis, but I am hesitant to do that when the open parenthesis >> is more than 40 chars from the beginning of a line. Otherwise I'll >> end up with lots of skinny lines. They would also run past 80-char >> width when you have "wide" parameters. > > Yes it is certainly not a uniform rule and often conflicts with other > line length guidelines. But in this case the change from aligned to > not-aligned was quite noticeable. > >> ?? return >> JavaCalls::construct_new_instance(SystemDictionary::Module_klass(), >> vmSymbols::java_lang_module_init_signature(), >> ?????????????????????????? loader, module_name, CHECK_NH); >> >> So if you don't have a big objection, I'll just keep the indentation >> as is. > > It was only a nit so that is fine. > Thanks! > Though I was almost tempted to suggest that you rename > construct_new_instance to do_construct, as it is the same length as > call_special. ;-) > That wouldn't help too much, because call_special is a void function so it gets to start at the front of the line, but construct_new_instance would have an assignment or a "return" in front of it :-( Thanks - Ioi > Cheers, > David > > >> Thanks >> - Ioi >>> Great comments on the Thread construction process too! Hope it didn't > >>> take you too long to figure out why you couldn't make the >>> replacement in those cases! >>> >>> Thanks, >>> David >>> >>>> Also: >>>> >>>> Added assert(InstanceKlass::is_initialized(), ...) in case where >>>> we cannot call JavaCalls::construct_new_instance for thread objects. >>>> >>>> Replaced unnecessary SystemDictionary::resolve_or_null() calls with >>>> SystemDictionary::XXX_klass(). >>>> >>>> Tested with hs-tier[1,2,3,4,5] >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> >>>> >>>> >> From serguei.spitsyn at oracle.com Tue May 22 05:33:00 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 21 May 2018 22:33:00 -0700 Subject: RFR(M) : 8202946 : [TESTBUG] Open source VM testbase OOM tests In-Reply-To: <2DD8F9C6-8471-4BF6-8573-0DA3F2B6C66B@oracle.com> References: <2DD8F9C6-8471-4BF6-8573-0DA3F2B6C66B@oracle.com> Message-ID: <23ea1dbe-f6bc-b76a-237a-ff6fc1a6dc58@oracle.com> Hi Igor, This looks good. This change has to be reviewed by someone from the GC team. Thanks, Serguei On 5/15/18 16:16, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8202946/webrev.00/index.html >> 1619 lines changed: 1619 ins; 0 del; 0 mod; > Hi all, > > could you please review this patch which open sources OOM tests from VM testbase? these tests test OutOfMemoryError throwing in different scenarios. > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8202946 > webrev: http://cr.openjdk.java.net/~iignatyev//8202946/webrev.00/index.html > testing: :vmTestbase_vm_oom test group > > Thanks, > -- Igor From lois.foltan at oracle.com Tue May 22 13:59:15 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Tue, 22 May 2018 09:59:15 -0400 Subject: RFR (S) 8202669: Intermittent crash in ClassLoadingService::compute_class_size() In-Reply-To: References: Message-ID: <83cf9d33-a2d9-5818-e2bf-cdd23b1c6a5f@oracle.com> On 5/21/2018 7:01 PM, coleen.phillimore at oracle.com wrote: > Summary: Don't report classes that failed to load when reporting class > unloading.? Remove the classes first. > > Also add a NULL check for transitive_interfaces for safety, but the > new tests pass without it and fail without the fix. > > open webrev at http://cr.openjdk.java.net/~coleenp/8202669.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8202669 > > Tested with hs-tier1-5. > > Thanks, > Coleen Looks good. A couple of review comments: - classLoaderData.cpp, line #590 "erroreous" or should that be "erroneous"? - can the 2 tests be combined into 1? - use "throw new RuntimeException" instead of "throw new Exception", see line #61 in TestLoadObject.java Thanks, Lois From coleen.phillimore at oracle.com Tue May 22 15:57:04 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 22 May 2018 11:57:04 -0400 Subject: RFR (S) 8202669: Intermittent crash in ClassLoadingService::compute_class_size() In-Reply-To: <53a31aec-a99c-67ed-f268-e61322ae2225@oracle.com> References: <53a31aec-a99c-67ed-f268-e61322ae2225@oracle.com> Message-ID: <0d46965c-41f3-e16a-3d0c-fa44e4c67da5@oracle.com> On 5/21/18 7:28 PM, David Holmes wrote: > Hi Coleen, > > This looks good. > > On 22/05/2018 9:01 AM, coleen.phillimore at oracle.com wrote: >> Summary: Don't report classes that failed to load when reporting >> class unloading.? Remove the classes first. >> >> Also add a NULL check for transitive_interfaces for safety, but the >> new tests pass without it and fail without the fix. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8202669.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8202669 > > src/hotspot/share/classfile/classLoaderData.cpp > > 590?? // after erroreous classes are released. > > Typo: erroreous -> erroneous Fixed. > > --- > > test/hotspot/jtreg/runtime/BadObjectClass/TestLoadObject.java > > 56?????? byte[] buf = InMemoryJavaCompiler.compile("java.lang.Object", > source, > ?57 "--patch-module=java.base"); > > Indent is wrong on line 57. I'll fix the indentation in the TestUnloadClassError.java test. > > I don't see why you need the two test programs as > TestUnloadClassError.java tests both conditions. True, they are the same (except the one does class unloading to show the bug).?? I'll keep TestUnloadClassError.java, and Lois looked over my shoulder so I did it right. Thanks! Coleen > > No need to see a new webrev. > > Thanks, > David > ----- > >> Tested with hs-tier1-5. >> >> Thanks, >> Coleen From coleen.phillimore at oracle.com Tue May 22 16:02:08 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 22 May 2018 12:02:08 -0400 Subject: RFR (S) 8202669: Intermittent crash in ClassLoadingService::compute_class_size() In-Reply-To: <83cf9d33-a2d9-5818-e2bf-cdd23b1c6a5f@oracle.com> References: <83cf9d33-a2d9-5818-e2bf-cdd23b1c6a5f@oracle.com> Message-ID: <7dd5a857-693c-3912-b33e-ab7f727c831c@oracle.com> Thank you for the code review! On 5/22/18 9:59 AM, Lois Foltan wrote: > On 5/21/2018 7:01 PM, coleen.phillimore at oracle.com wrote: > >> Summary: Don't report classes that failed to load when reporting >> class unloading.? Remove the classes first. >> >> Also add a NULL check for transitive_interfaces for safety, but the >> new tests pass without it and fail without the fix. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8202669.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8202669 >> >> Tested with hs-tier1-5. >> >> Thanks, >> Coleen > Looks good. A couple of review comments: > > - classLoaderData.cpp, line #590 "erroreous" or should that be > "erroneous"? fixed. > - can the 2 tests be combined into 1? Yes, I combined them in TestUnloadClassError.java.? Thank you for your help. > - use "throw new RuntimeException" instead of "throw new Exception", > see line #61 in TestLoadObject.java TestUnloadClassError.java throws RuntimeException so that's fixed too. FTR: http://cr.openjdk.java.net/~coleenp/8202669.02/webrev Thank you! Coleen > > Thanks, > Lois From calvin.cheung at oracle.com Tue May 22 16:14:06 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Tue, 22 May 2018 09:14:06 -0700 Subject: RFR (S) 8202669: Intermittent crash in ClassLoadingService::compute_class_size() In-Reply-To: <7dd5a857-693c-3912-b33e-ab7f727c831c@oracle.com> References: <83cf9d33-a2d9-5818-e2bf-cdd23b1c6a5f@oracle.com> <7dd5a857-693c-3912-b33e-ab7f727c831c@oracle.com> Message-ID: <5B0441CE.9090008@oracle.com> Hi Coleen, The changes look good. Thank you for fixing it. thanks, Calvin On 5/22/18, 9:02 AM, coleen.phillimore at oracle.com wrote: > > Thank you for the code review! > > On 5/22/18 9:59 AM, Lois Foltan wrote: >> On 5/21/2018 7:01 PM, coleen.phillimore at oracle.com wrote: >> >>> Summary: Don't report classes that failed to load when reporting >>> class unloading. Remove the classes first. >>> >>> Also add a NULL check for transitive_interfaces for safety, but the >>> new tests pass without it and fail without the fix. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8202669.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8202669 >>> >>> Tested with hs-tier1-5. >>> >>> Thanks, >>> Coleen >> Looks good. A couple of review comments: >> >> - classLoaderData.cpp, line #590 "erroreous" or should that be >> "erroneous"? > fixed. >> - can the 2 tests be combined into 1? > > Yes, I combined them in TestUnloadClassError.java. Thank you for your > help. > >> - use "throw new RuntimeException" instead of "throw new Exception", >> see line #61 in TestLoadObject.java > > TestUnloadClassError.java throws RuntimeException so that's fixed too. > > FTR: http://cr.openjdk.java.net/~coleenp/8202669.02/webrev > > Thank you! > Coleen >> >> Thanks, >> Lois > From coleen.phillimore at oracle.com Tue May 22 17:06:11 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 22 May 2018 13:06:11 -0400 Subject: RFR (S) 8202669: Intermittent crash in ClassLoadingService::compute_class_size() In-Reply-To: <5B0441CE.9090008@oracle.com> References: <83cf9d33-a2d9-5818-e2bf-cdd23b1c6a5f@oracle.com> <7dd5a857-693c-3912-b33e-ab7f727c831c@oracle.com> <5B0441CE.9090008@oracle.com> Message-ID: <8d5094ab-0e0e-2c43-3207-682813162092@oracle.com> Thanks Calvin! Coleen On 5/22/18 12:14 PM, Calvin Cheung wrote: > Hi Coleen, > > The changes look good. Thank you for fixing it. > > thanks, > Calvin > > On 5/22/18, 9:02 AM, coleen.phillimore at oracle.com wrote: >> >> Thank you for the code review! >> >> On 5/22/18 9:59 AM, Lois Foltan wrote: >>> On 5/21/2018 7:01 PM, coleen.phillimore at oracle.com wrote: >>> >>>> Summary: Don't report classes that failed to load when reporting >>>> class unloading.? Remove the classes first. >>>> >>>> Also add a NULL check for transitive_interfaces for safety, but the >>>> new tests pass without it and fail without the fix. >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8202669.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8202669 >>>> >>>> Tested with hs-tier1-5. >>>> >>>> Thanks, >>>> Coleen >>> Looks good. A couple of review comments: >>> >>> - classLoaderData.cpp, line #590 "erroreous" or should that be >>> "erroneous"? >> fixed. >>> - can the 2 tests be combined into 1? >> >> Yes, I combined them in TestUnloadClassError.java.? Thank you for >> your help. >> >>> - use "throw new RuntimeException" instead of "throw new Exception", >>> see line #61 in TestLoadObject.java >> >> TestUnloadClassError.java throws RuntimeException so that's fixed too. >> >> FTR: http://cr.openjdk.java.net/~coleenp/8202669.02/webrev >> >> Thank you! >> Coleen >>> >>> Thanks, >>> Lois >> From lois.foltan at oracle.com Tue May 22 17:06:38 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Tue, 22 May 2018 13:06:38 -0400 Subject: RFR (S) 8202669: Intermittent crash in ClassLoadingService::compute_class_size() In-Reply-To: <7dd5a857-693c-3912-b33e-ab7f727c831c@oracle.com> References: <83cf9d33-a2d9-5818-e2bf-cdd23b1c6a5f@oracle.com> <7dd5a857-693c-3912-b33e-ab7f727c831c@oracle.com> Message-ID: <301d6f71-9902-7d88-a967-0c46c82c4c25@oracle.com> On 5/22/2018 12:02 PM, coleen.phillimore at oracle.com wrote: > > Thank you for the code review! > > On 5/22/18 9:59 AM, Lois Foltan wrote: >> On 5/21/2018 7:01 PM, coleen.phillimore at oracle.com wrote: >> >>> Summary: Don't report classes that failed to load when reporting >>> class unloading.? Remove the classes first. >>> >>> Also add a NULL check for transitive_interfaces for safety, but the >>> new tests pass without it and fail without the fix. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8202669.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8202669 >>> >>> Tested with hs-tier1-5. >>> >>> Thanks, >>> Coleen >> Looks good. A couple of review comments: >> >> - classLoaderData.cpp, line #590 "erroreous" or should that be >> "erroneous"? > fixed. >> - can the 2 tests be combined into 1? > > Yes, I combined them in TestUnloadClassError.java.? Thank you for your > help. > >> - use "throw new RuntimeException" instead of "throw new Exception", >> see line #61 in TestLoadObject.java > > TestUnloadClassError.java throws RuntimeException so that's fixed too. > > FTR: http://cr.openjdk.java.net/~coleenp/8202669.02/webrev Looks good, thanks for making those fixes! Lois > > Thank you! > Coleen >> >> Thanks, >> Lois > From coleen.phillimore at oracle.com Tue May 22 18:15:26 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 22 May 2018 14:15:26 -0400 Subject: RFR(M): 8199257: [TESTBUG] Open source VM testbase metaspace tests In-Reply-To: References: Message-ID: <03365d3f-75e6-77f7-39ad-492751f450d7@oracle.com> Looks good. Thank you for open sourcing these tests! Coleen On 5/8/18 1:38 PM, mikhailo wrote: > http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html > > Please review this change open sourcing VM metaspace tests. These > tests have been used internally for a while, and are now being open > sourced. Since this is not an creation of new tests, we would like to > keep the changes during this review to an absolute required minimum. > If you have any feedback on improvements of these tests, please file > RFE(s) that will be addressed later in order of priority. > > > Here is what was done for this change: > ? 1. Moved the tests to OpenJDK repository to the specified directory > location and structure. > ? 3. Updated Copyright statements accordingly. > ? 4. Updated "@library" statements accordingly. > ? 5. Updated TEST.groups and ProblemList.txt > > ? JBS:??? https://bugs.openjdk.java.net/browse/JDK-8199257 > ? Webrev: > http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html > > ? Testing: > ????? 1. Ran the following tests on open-only repository and build, > using "make run-test" (Linux-x64) > ???????? vmTestbase_vm_metaspace, vmTestbase_largepages > ???????? All PASS > > ????? 2. Automated multip-platform test system (usual 4 platforms): > ???????? - vmTestbase_vm_metaspace, vmTestbase_largepages > ???????? - hs-tier{1,2,3} > ???????? All PASS > > Thank you, > Misha > From thomas.stuefe at gmail.com Tue May 22 18:17:13 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 22 May 2018 20:17:13 +0200 Subject: RFR(M): 8199257: [TESTBUG] Open source VM testbase metaspace tests In-Reply-To: <5B02EB5B.3010805@oracle.com> References: <5B02EB5B.3010805@oracle.com> Message-ID: Hi Mikhailo, looks good to me too. Now its a review :) Best Regards, Thomas On Mon, May 21, 2018 at 5:52 PM, Mikhailo Seledtsov wrote: > Ping... > > > On 5/8/18, 10:38 AM, mikhailo wrote: >> >> http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html >> >> Please review this change open sourcing VM metaspace tests. These tests >> have been used internally for a while, and are now being open sourced. Since >> this is not an creation of new tests, we would like to keep the changes >> during this review to an absolute required minimum. If you have any feedback >> on improvements of these tests, please file RFE(s) that will be addressed >> later in order of priority. >> >> >> Here is what was done for this change: >> 1. Moved the tests to OpenJDK repository to the specified directory >> location and structure. >> 3. Updated Copyright statements accordingly. >> 4. Updated "@library" statements accordingly. >> 5. Updated TEST.groups and ProblemList.txt >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8199257 >> Webrev: >> http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html >> >> Testing: >> 1. Ran the following tests on open-only repository and build, using >> "make run-test" (Linux-x64) >> vmTestbase_vm_metaspace, vmTestbase_largepages >> All PASS >> >> 2. Automated multip-platform test system (usual 4 platforms): >> - vmTestbase_vm_metaspace, vmTestbase_largepages >> - hs-tier{1,2,3} >> All PASS >> >> Thank you, >> Misha >> > From coleen.phillimore at oracle.com Tue May 22 18:22:28 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 22 May 2018 14:22:28 -0400 Subject: RFR(M): 8199257: [TESTBUG] Open source VM testbase metaspace tests In-Reply-To: <5AF4BC29.6080806@oracle.com> References: <5AF4BC29.6080806@oracle.com> Message-ID: <630e2753-5204-3f92-3fff-06bc45194dfb@oracle.com> On 5/10/18 5:39 PM, Mikhailo Seledtsov wrote: > Hi Thomas, > > On 5/9/18, 9:01 AM, Thomas St?fe wrote: >> Not a review, but I ran these tests on my local machine (with my >> current metaspace changes) and they all passed. > Thank you. >> >> Will these tests be somehow merged with the existing metaspace tests >> at runtime/Metaspace? > We can consider this in the future. We can either collect them into > the same test group, or consider migrating the tests into a different > location/directory. Yes, this is a good suggestion for a future RFE (once they are open-sourced). >> >> I think the general question is what is vmTestBase, and what sets the >> tests in there apart from the existing tests outside vmTestBase? > VM Testbase is a collection of tests used for testing HotSpot. There > are several things that set them apart from other JTReg HotSpot tests, > such as certain common conventions (coding conventions, keywords) and > common test libraries. There is a bit more info in this README file: > http://hg.openjdk.java.net/jdk/jdk/file/tip/test/hotspot/jtreg/vmTestbase/README.md > If moved to test/hotspot/jtreg/runtime/metaspace we'd have to be careful that the stress tests aren't run with earlier tiers. thanks, Coleen > > I hope I have answered your questions, > > Thank you, > Misha >> >> Thanks, Thomas >> >> >> On Tue, May 8, 2018 at 7:38 PM, >> mikhailo? wrote: >>> http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html >>> >>> Please review this change open sourcing VM metaspace tests. These >>> tests have >>> been used internally for a while, and are now being open sourced. >>> Since this >>> is not an creation of new tests, we would like to keep the changes >>> during >>> this review to an absolute required minimum. If you have any >>> feedback on >>> improvements of these tests, please file RFE(s) that will be >>> addressed later >>> in order of priority. >>> >>> >>> Here is what was done for this change: >>> ?? 1. Moved the tests to OpenJDK repository to the specified directory >>> location and structure. >>> ?? 3. Updated Copyright statements accordingly. >>> ?? 4. Updated "@library" statements accordingly. >>> ?? 5. Updated TEST.groups and ProblemList.txt >>> >>> ?? JBS:??? https://bugs.openjdk.java.net/browse/JDK-8199257 >>> ?? Webrev: >>> http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html >>> >>> ?? Testing: >>> ?????? 1. Ran the following tests on open-only repository and build, >>> using >>> "make run-test" (Linux-x64) >>> ????????? vmTestbase_vm_metaspace, vmTestbase_largepages >>> ????????? All PASS >>> >>> ?????? 2. Automated multip-platform test system (usual 4 platforms): >>> ????????? - vmTestbase_vm_metaspace, vmTestbase_largepages >>> ????????? - hs-tier{1,2,3} >>> ????????? All PASS >>> >>> Thank you, >>> Misha >>> From calvin.cheung at oracle.com Tue May 22 19:08:44 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Tue, 22 May 2018 12:08:44 -0700 Subject: RFR(S): 8203354: assert in ClassLoader::update_module_path_entry_list() could have incorrect message Message-ID: <5B046ABC.4070405@oracle.com> JBS: https://bugs.openjdk.java.net/browse/JDK-8203354 webrev: http://cr.openjdk.java.net/~ccheung/8203354/webrev.00/ Converting the assert in ClassLoader::update_module_path_entry_list() to a meaningful error message before aborting the CDS dump. With a module path with very long length (> 2K), the error message will be like the following: os::stat error 36 (ENAMETOOLONG). CDS dump aborted. Bad path /scratch/myDir/JTwork/scratch/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Note that tty->print_cr() has a buffer size limit of 2000, so the full path won't be shown entirely. That's why I'd like to put the error code up front before the path. Ran the modified tests on Oracle platforms. thanks, Calvin From markus.gronlund at oracle.com Tue May 22 19:24:00 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Tue, 22 May 2018 12:24:00 -0700 (PDT) Subject: RFR(M): JDK-8203321: Windows: assert(current_query_index < process_query_set->size) failed: invariant Message-ID: <06e925c8-8d77-43c8-9ef2-ad9113034b98@default> Greetings, Kindly asking for reviews for the following change: Bug: https://bugs.openjdk.java.net/browse/JDK-8203321 Webrev: http://cr.openjdk.java.net/~mgronlun/8203321/webrev00/ Summary: For some context about what this is about, please see this (now) relatively old issue: https://bugs.openjdk.java.net/browse/JDK-8019921 The porting work that brought this code from closed to open were optimistic in that the following PDH query, "\Process(java#n)\ID Process", performed relatively stable on Windows 10. An invariant was added in that your ID Process query would never return an index that was lower than the index at construction. During testing, it was discovered that this invariant did not hold, especially when running on Windows Server 2012 R2 and there is a high churn rate with many processes with the same base name ("java#") starting and stopping (stressing PDH list of processes). We have to reinsert back the original code that handled the case where the PDH process list is not stable (that were originally put in place with JDK-8019921). The defensive logic is located at lines 418 - 422. I had to rework some related code to make some room for this as well as to keep track of the previous process index. Thanks Markus From mikhailo.seledtsov at oracle.com Tue May 22 20:07:34 2018 From: mikhailo.seledtsov at oracle.com (mikhailo) Date: Tue, 22 May 2018 13:07:34 -0700 Subject: RFR(M): 8199257: [TESTBUG] Open source VM testbase metaspace tests In-Reply-To: References: <5B02EB5B.3010805@oracle.com> Message-ID: <1c492f28-7040-d250-a6bf-05d47229e1fd@oracle.com> Thank you Thomas, Misha On 05/22/2018 11:17 AM, Thomas St?fe wrote: > Hi Mikhailo, > > looks good to me too. Now its a review :) > > Best Regards, Thomas > > On Mon, May 21, 2018 at 5:52 PM, Mikhailo Seledtsov > wrote: >> Ping... >> >> >> On 5/8/18, 10:38 AM, mikhailo wrote: >>> http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html >>> >>> Please review this change open sourcing VM metaspace tests. These tests >>> have been used internally for a while, and are now being open sourced. Since >>> this is not an creation of new tests, we would like to keep the changes >>> during this review to an absolute required minimum. If you have any feedback >>> on improvements of these tests, please file RFE(s) that will be addressed >>> later in order of priority. >>> >>> >>> Here is what was done for this change: >>> 1. Moved the tests to OpenJDK repository to the specified directory >>> location and structure. >>> 3. Updated Copyright statements accordingly. >>> 4. Updated "@library" statements accordingly. >>> 5. Updated TEST.groups and ProblemList.txt >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8199257 >>> Webrev: >>> http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html >>> >>> Testing: >>> 1. Ran the following tests on open-only repository and build, using >>> "make run-test" (Linux-x64) >>> vmTestbase_vm_metaspace, vmTestbase_largepages >>> All PASS >>> >>> 2. Automated multip-platform test system (usual 4 platforms): >>> - vmTestbase_vm_metaspace, vmTestbase_largepages >>> - hs-tier{1,2,3} >>> All PASS >>> >>> Thank you, >>> Misha >>> From mikhailo.seledtsov at oracle.com Tue May 22 20:08:04 2018 From: mikhailo.seledtsov at oracle.com (mikhailo) Date: Tue, 22 May 2018 13:08:04 -0700 Subject: RFR(M): 8199257: [TESTBUG] Open source VM testbase metaspace tests In-Reply-To: <03365d3f-75e6-77f7-39ad-492751f450d7@oracle.com> References: <03365d3f-75e6-77f7-39ad-492751f450d7@oracle.com> Message-ID: <75945ddc-97ee-8a65-bce9-be945688a540@oracle.com> Thank you for review Coleen, Misha On 05/22/2018 11:15 AM, coleen.phillimore at oracle.com wrote: > > Looks good. Thank you for open sourcing these tests! > Coleen > > On 5/8/18 1:38 PM, mikhailo wrote: >> http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html >> >> Please review this change open sourcing VM metaspace tests. These >> tests have been used internally for a while, and are now being open >> sourced. Since this is not an creation of new tests, we would like to >> keep the changes during this review to an absolute required minimum. >> If you have any feedback on improvements of these tests, please file >> RFE(s) that will be addressed later in order of priority. >> >> >> Here is what was done for this change: >> ? 1. Moved the tests to OpenJDK repository to the specified directory >> location and structure. >> ? 3. Updated Copyright statements accordingly. >> ? 4. Updated "@library" statements accordingly. >> ? 5. Updated TEST.groups and ProblemList.txt >> >> ? JBS:??? https://bugs.openjdk.java.net/browse/JDK-8199257 >> ? Webrev: >> http://cr.openjdk.java.net/~mseledtsov/8199257.01.open/index.html >> >> ? Testing: >> ????? 1. Ran the following tests on open-only repository and build, >> using "make run-test" (Linux-x64) >> ???????? vmTestbase_vm_metaspace, vmTestbase_largepages >> ???????? All PASS >> >> ????? 2. Automated multip-platform test system (usual 4 platforms): >> ???????? - vmTestbase_vm_metaspace, vmTestbase_largepages >> ???????? - hs-tier{1,2,3} >> ???????? All PASS >> >> Thank you, >> Misha >> > From thomas.stuefe at gmail.com Tue May 22 20:36:19 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 22 May 2018 22:36:19 +0200 Subject: RFR(S): 8203354: assert in ClassLoader::update_module_path_entry_list() could have incorrect message In-Reply-To: <5B046ABC.4070405@oracle.com> References: <5B046ABC.4070405@oracle.com> Message-ID: Hi Calvin, some small nits: - could you use vm_exit_during_initialization instead of raw exit? - "Bad path %s" - that is an assumption, stat() may fail for a number of reasons. So I would make the message more neutral. I also always enclose such an output in quotes, to catch if the caller had leading/trailing spaces. So, maybe this: "CDS dump aborted (path was \"%s\")." ? Side note: I looked at the various os::stat implementations and see a number of issues. We copy the input path around, which is on Posix platforms unnecessary since os::native_path() returns just the path; and since we use a fixed sized buffer we artificially limit the path os::stat() recognizes. That is bad. Also, on windows, we return the Win32 error code (GetLastError) as errno, which is just plain wrong. But these are issues beyond your patch. Kind Regards, Thomas On Tue, May 22, 2018 at 9:08 PM, Calvin Cheung wrote: > JBS: https://bugs.openjdk.java.net/browse/JDK-8203354 > > webrev: http://cr.openjdk.java.net/~ccheung/8203354/webrev.00/ > > Converting the assert in ClassLoader::update_module_path_entry_list() to a > meaningful error message before aborting the CDS dump. > > With a module path with very long length (> 2K), the error message will be > like the following: > > os::stat error 36 (ENAMETOOLONG). CDS dump aborted. Bad path > /scratch/myDir/JTwork/scratch/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx > > Note that tty->print_cr() has a buffer size limit of 2000, so the full path > won't be shown entirely. That's why I'd like to put the error code up front > before the path. > > Ran the modified tests on Oracle platforms. > > thanks, > Calvin From paul.sandoz at oracle.com Tue May 22 22:06:08 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 22 May 2018 15:06:08 -0700 Subject: RFR (s) JDK-8203435: Circular nested dynamic constant test needed to confirm JVMS resolution behavior In-Reply-To: <4842faca-0775-f217-a5a0-cdbd9cd90357@oracle.com> References: <4842faca-0775-f217-a5a0-cdbd9cd90357@oracle.com> Message-ID: Looks ok. Note there is a companion test here: http://hg.openjdk.java.net/jdk/jdk/file/2942ae532175/test/jdk/java/lang/invoke/condy/CondyNestedTest.java Paul. > On May 21, 2018, at 12:50 PM, Lois Foltan wrote: > > Please review this change to add a nested circular dynamic constant test to confirm the JVMS 5.4.3.6 Dynamically-Computed Constant and Call Site Resolution behavior as discussed in http://mail.openjdk.java.net/pipermail/valhalla-spec-observers/2018-February/000518.html. > > open webrev at: http://cr.openjdk.java.net/~lfoltan/bug_jdk8203435/webrev/ > bug link: https://bugs.openjdk.java.net/browse/JDK-8203435 > > Testing: hs-tier1-2, jdk-tier1-3 in progress > > Thanks, > Lois From calvin.cheung at oracle.com Tue May 22 22:49:02 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Tue, 22 May 2018 15:49:02 -0700 Subject: RFR(S): 8203354: assert in ClassLoader::update_module_path_entry_list() could have incorrect message In-Reply-To: References: <5B046ABC.4070405@oracle.com> Message-ID: <5B049E5E.8090601@oracle.com> Hi Thomas, Thanks for your quick review. On 5/22/18, 1:36 PM, Thomas St?fe wrote: > Hi Calvin, > > some small nits: > > - could you use vm_exit_during_initialization instead of raw exit? > - "Bad path %s" - that is an assumption, stat() may fail for a number > of reasons. So I would make the message more neutral. I also always > enclose such an output in quotes, to catch if the caller had > leading/trailing spaces. So, maybe this: "CDS dump aborted (path was > \"%s\")." ? I've fixed the above. Updated webrev: http://cr.openjdk.java.net/~ccheung/8203354/webrev.01/ > > Side note: I looked at the various os::stat implementations and see a > number of issues. We copy the input path around, which is on Posix > platforms unnecessary since os::native_path() returns just the path; > and since we use a fixed sized buffer we artificially limit the path > os::stat() recognizes. That is bad. Also, on windows, we return the > Win32 error code (GetLastError) as errno, which is just plain wrong. > > But these are issues beyond your patch. Could you file bugs on the above issues? thanks, Calvin > > Kind Regards, Thomas > > > On Tue, May 22, 2018 at 9:08 PM, Calvin Cheung wrote: >> JBS: https://bugs.openjdk.java.net/browse/JDK-8203354 >> >> webrev: http://cr.openjdk.java.net/~ccheung/8203354/webrev.00/ >> >> Converting the assert in ClassLoader::update_module_path_entry_list() to a >> meaningful error message before aborting the CDS dump. >> >> With a module path with very long length (> 2K), the error message will be >> like the following: >> >> os::stat error 36 (ENAMETOOLONG). CDS dump aborted. Bad path >> /scratch/myDir/JTwork/scratch/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx >> >> Note that tty->print_cr() has a buffer size limit of 2000, so the full path >> won't be shown entirely. That's why I'd like to put the error code up front >> before the path. >> >> Ran the modified tests on Oracle platforms. >> >> thanks, >> Calvin From leonid.mesnik at oracle.com Wed May 23 01:15:41 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Tue, 22 May 2018 18:15:41 -0700 Subject: RFR: 8203491: [TESTBUG] Port heapdump tests into java Message-ID: <650E6684-1996-4230-B9A4-CB4921419847@oracle.com> Hi Could you please review following patch which rewrite vmTestbase/heapdump tests as a jtreg java tests. These heapdump tests were developed as a shell tests which verify heapdump functionality in JDK 5. They work fine but needs some refactoring. webrev: http://cr.openjdk.java.net/~lmesnik/8203491/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8203491 Below is high-level overview of existing tests and changes: 1) Tests vmTestbase/heapdump/OnOOM* verify that VM correctly generates heapdump when OOME in heap or metaspace happens. Also they verify that -XX:HeapDumpPath= works correctly. Re-written as: http://cr.openjdk.java.net/~lmesnik/8203491/webrev.00/test/hotspot/jtreg/runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java.html Test provokes OOME and verifies generated heapdump. Please note that new jtreg test is very similar and use same test patterns as most of ErrorHandling tests in http://hg.openjdk.java.net/jdk/jdk/file/tip/test/hotspot/jtreg/runtime/ErrorHandling The main goal is to verify error handling when OOME is happen so I used the same way to provoke OOME as most of other tests in this directory. New test might takes more then minute to complete test so I removed from tier1. 2) Tests vmTestbase/heapdump/JMap* verify that jmap correctly generates heapdump from live process and core/mdmp file. There are existing tests which verify that jmap could correctly dump heap for live process: http://hg.openjdk.java.net/jdk/jdk/file/f4735ff8d17d/test/hotspot/jtreg/serviceability/sa/TestHeapDumpForInvokeDynamic.java http://hg.openjdk.java.net/jdk/jdk/file/f4735ff8d17d/test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java I believe that they provide the same coverage as heapdump/JMapHeap and heapdump/JMapMetaspace tests and no new tests is needed. The new test which verifies that jmap generate heapdump for corefile is: http://cr.openjdk.java.net/~lmesnik/8203491/webrev.00/test/hotspot/jtreg/serviceability/sa/TestJmapCore.java.html Test provokes OOME to generate corefile (or mdml), generate heapdump with jhsdb jmap and verify it. Previously tests send signals to provoke coredump. Since JDK9 option 'CreateCoredumpOnCrash' could be used to easily dump core on all platforms. Please note that test silently pass if it doesn't find core file. Even if test tries to set ulimit -c unlimited there is no guarantee that core file is generated and could be found. I have verified that test still could reproduce issue https://bugs.openjdk.java.net/browse/JDK-8176557 if default heap size is large enough. However I think that it make a sense to add variant of this test which explicit heap settings as a regression fix for bug JDK-8176557 . Also I haven't managed to reproduce following issues with neither existing or new tests: https://bugs.openjdk.java.net/browse/JDK-8001227 https://bugs.openjdk.java.net/browse/JDK-8023376 https://bugs.openjdk.java.net/browse/JDK-8051445 Corresponding tests were excluded for a long time so there are no any results for them. I also see comments in bugs with unsuccessful attempts to reproduce issues. I think that it makes a sense just to remove these entries from problemlist and file new open bugs if any of failures happen. I quickly looked through all resolved bugs found by heapdump/ tests. Most of them are related with different size limits overflow. It might be make a sense also create test which fill heap with large amount of small objects to possible overflow of sizes of records. The similar functionality was removed by JDK-8144137 . But I would prefer to have separate RFE since it is not supported by existing tests. Please let me know if it is possible to lost test coverage during this conversion and more tests are needed. The more conservative way might be to left existing tests for some time to see if if they still found different bugs. Testing: Run new tests on linux-x64,windows-x64,macos-x64. Run tier1/2. Leonid From thomas.stuefe at gmail.com Wed May 23 05:02:59 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 23 May 2018 07:02:59 +0200 Subject: RFR(S): 8203354: assert in ClassLoader::update_module_path_entry_list() could have incorrect message In-Reply-To: <5B049E5E.8090601@oracle.com> References: <5B046ABC.4070405@oracle.com> <5B049E5E.8090601@oracle.com> Message-ID: Hi Calvin, On Wed, May 23, 2018 at 12:49 AM, Calvin Cheung wrote: > Hi Thomas, > > Thanks for your quick review. > > On 5/22/18, 1:36 PM, Thomas St?fe wrote: >> >> Hi Calvin, >> >> some small nits: >> >> - could you use vm_exit_during_initialization instead of raw exit? >> - "Bad path %s" - that is an assumption, stat() may fail for a number >> of reasons. So I would make the message more neutral. I also always >> enclose such an output in quotes, to catch if the caller had >> leading/trailing spaces. So, maybe this: "CDS dump aborted (path was >> \"%s\")." ? > > I've fixed the above. > Updated webrev: http://cr.openjdk.java.net/~ccheung/8203354/webrev.01/ Looks good now. >> >> >> Side note: I looked at the various os::stat implementations and see a >> number of issues. We copy the input path around, which is on Posix >> platforms unnecessary since os::native_path() returns just the path; >> and since we use a fixed sized buffer we artificially limit the path >> os::stat() recognizes. That is bad. Also, on windows, we return the >> Win32 error code (GetLastError) as errno, which is just plain wrong. >> >> But these are issues beyond your patch. > > Could you file bugs on the above issues? > Done: https://bugs.openjdk.java.net/browse/JDK-8203680 Thanks! ..Thomas > thanks, > Calvin > >> >> Kind Regards, Thomas >> >> >> On Tue, May 22, 2018 at 9:08 PM, Calvin Cheung >> wrote: >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8203354 >>> >>> webrev: http://cr.openjdk.java.net/~ccheung/8203354/webrev.00/ >>> >>> Converting the assert in ClassLoader::update_module_path_entry_list() to >>> a >>> meaningful error message before aborting the CDS dump. >>> >>> With a module path with very long length (> 2K), the error message will >>> be >>> like the following: >>> >>> os::stat error 36 (ENAMETOOLONG). CDS dump aborted. Bad path >>> /scratch/myDir/JTwork/scratch/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx >>> >>> Note that tty->print_cr() has a buffer size limit of 2000, so the full >>> path >>> won't be shown entirely. That's why I'd like to put the error code up >>> front >>> before the path. >>> >>> Ran the modified tests on Oracle platforms. >>> >>> thanks, >>> Calvin From ioi.lam at oracle.com Wed May 23 05:16:35 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 22 May 2018 22:16:35 -0700 Subject: RFR(S): 8203354: assert in ClassLoader::update_module_path_entry_list() could have incorrect message In-Reply-To: <5B049E5E.8090601@oracle.com> References: <5B046ABC.4070405@oracle.com> <5B049E5E.8090601@oracle.com> Message-ID: Hi Calvin, The C code changes look good. However, I am not sure if the test case is correct on all file systems. Does the test assume that the max path length is about 2048? I think some file systems could allow more than 2048 chars. See https://serverfault.com/questions/9546/filename-length-limits-on-linux On my Linux workstation: $ getconf PATH_MAX / 4096 I think it's better to rewrite the test so that it's OK for the dump to succeed. If it fails, it must be "os::stat error", but there's no guarantee that the error code is ENAMETOOLONG. How about this: if (output.getExitValue() != 0) { ??? output.shouldMatch("os::stat error.*CDS dump aborted"); } Thanks - Ioi On 5/22/18 3:49 PM, Calvin Cheung wrote: > Hi Thomas, > > Thanks for your quick review. > > On 5/22/18, 1:36 PM, Thomas St?fe wrote: >> Hi Calvin, >> >> some small nits: >> >> - could you use vm_exit_during_initialization instead of raw exit? >> - "Bad path %s" - that is an assumption, stat() may fail for a number >> of reasons. So I would make the message more neutral. I also always >> enclose such an output in quotes, to catch if the caller had >> leading/trailing spaces. So, maybe this: "CDS dump aborted (path was >> \"%s\")." ? > I've fixed the above. > Updated webrev: http://cr.openjdk.java.net/~ccheung/8203354/webrev.01/ >> >> Side note: I looked at the various os::stat implementations and see a >> number of issues. We copy the input path around, which is on Posix >> platforms unnecessary since os::native_path() returns just the path; >> and since we use a fixed sized buffer we artificially limit the path >> os::stat() recognizes. That is bad. Also, on windows, we return the >> Win32 error code (GetLastError) as errno, which is just plain wrong. >> >> But these are issues beyond your patch. > Could you file bugs on the above issues? > > thanks, > Calvin >> >> Kind Regards, Thomas >> >> >> On Tue, May 22, 2018 at 9:08 PM, Calvin >> Cheung? wrote: >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8203354 >>> >>> webrev: http://cr.openjdk.java.net/~ccheung/8203354/webrev.00/ >>> >>> Converting the assert in >>> ClassLoader::update_module_path_entry_list() to a >>> meaningful error message before aborting the CDS dump. >>> >>> With a module path with very long length (>? 2K), the error message >>> will be >>> like the following: >>> >>> os::stat error 36 (ENAMETOOLONG). CDS dump aborted. Bad path >>> /scratch/myDir/JTwork/scratch/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx >>> >>> >>> Note that tty->print_cr() has a buffer size limit of 2000, so the >>> full path >>> won't be shown entirely. That's why I'd like to put the error code >>> up front >>> before the path. >>> >>> Ran the modified tests on Oracle platforms. >>> >>> thanks, >>> Calvin From dms at samersoff.net Wed May 23 07:16:09 2018 From: dms at samersoff.net (Dmitry Samersoff) Date: Wed, 23 May 2018 10:16:09 +0300 Subject: RFR(S): JDK-8203481 Incorrect constraint for unextended_sp in frame:safe_for_sender In-Reply-To: <3421e284-fda3-ff82-29cd-70e4c4e0c8c0@oracle.com> References: <930b4686-dcf6-5f01-f492-4e65a25145c3@samersoff.net> <3421e284-fda3-ff82-29cd-70e4c4e0c8c0@oracle.com> Message-ID: <95e9c57c-d602-0f7f-98e9-a3efbc31f4be@samersoff.net> Dan, Thank you! I'll re-check what is happening on AArch64 and come back. -Dmitry On 21.05.2018 17:28, Daniel D. Daugherty wrote: > Hi Dmitry, > > I think something else must be going wrong here. The unextended SP > is typically documented like this: > > ? // This is the sp before any possible extension (adapter/locals). > ? intptr_t* unextended_sp = interpreter_frame_sender_sp(); > > and like this: > > ? // stack frames shouldn't be much larger than max_stack elements > ? // this test requires the use of unextended_sp which is the sp as seen by > ? // the current frame, and not sp which is the "raw" pc which could point > ? // further because of local variables of the callee method inserted after > ? // method arguments > ? if (fp() - unextended_sp() > 1024 + > m->max_stack()*Interpreter::stackElementSize) { > ??? return false; > ? } > > So I think this existing comment and assertion are correct: > > ??? L72: ? // unextended sp must be within the stack and above or equal sp > ??? L73: ? bool unextended_sp_safe = (unextended_sp < > thread->stack_base()) && > ??? L74: ??????????????????????????? (unextended_sp >= sp); > > Also, your proposed fix only changed this for two platforms. The same > logic exists on 'arm' and 'sparc' also. > > Dan > > > On 5/21/18 9:44 AM, Dmitry Samersoff wrote: >> Hello Everybody, >> >> Please review small fix >> >> http://cr.openjdk.java.net/~dsamersoff/JDK-8203481/webrev.01/ >> >> CR: >> >> https://bugs.openjdk.java.net/browse/JDK-8203481 >> >> Testing: >> >> jfr tests that depends to safe_for_sender functionality >> >> ./jdk/jdk/jfr/api/consumer/TestRecordedFullStackTrace.java >> ./jdk/jdk/jfr/event/profiling/TestFullStackTrace.java >> >> fails on AARCH64. >> >> These tests passed after the fix. >> >> > -- Dmitry Samersoff http://devnull.samersoff.net * There will come soft rains ... From thomas.stuefe at gmail.com Wed May 23 12:46:13 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 23 May 2018 14:46:13 +0200 Subject: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details Message-ID: Dear all, (not sure if this would be a serviceability or runtime rfe, so sorry for crossposting) may I please have feedback/reviews for this small enhancement. Bug: https://bugs.openjdk.java.net/browse/JDK-8203682 Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.00/webrev/ This adds a new command to jcmd, "VM.classloaders". It complements the existing command "VM.classloader_stats". This command, in its simplest form, prints the class loader tree. In addition to that, it optionally prints out loaded classes (both non-anonymous and anonymous) and various classloader specific information. Examples: http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example.txt http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-classes.txt http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-reflection-and-noinflation.txt Thanks and Best Regards, Thomas From jini.george at oracle.com Wed May 23 15:48:06 2018 From: jini.george at oracle.com (Jini George) Date: Wed, 23 May 2018 21:18:06 +0530 Subject: RFR: 8203491: [TESTBUG] Port heapdump tests into java In-Reply-To: <650E6684-1996-4230-B9A4-CB4921419847@oracle.com> References: <650E6684-1996-4230-B9A4-CB4921419847@oracle.com> Message-ID: <0ab155e3-9e93-11fb-55a7-6f0ac7e41f64@oracle.com> Hi Leonid, I have mostly looked at only the following file: http://cr.openjdk.java.net/~lmesnik/8203491/webrev.00/test/hotspot/jtreg/serviceability/sa/TestJmapCore.java.html * Would you be adding JMapMetaspaceCore testing to this? Other than this, I think the jhsdb heapdump testing from vmTestbase has been covered. * You might want to increase the timeout factor for this test. The test times out on my machine. * You might want to consider removing the corefile and the heapdump files after the test execution (in the cases where the test passes). * The following imports: import jdk.test.lib.classloader.GeneratingClassLoader; import java.util.Arrays; import java.util.stream.Collectors; are not needed. * Nit: // If any arguments are set prints pid so main process coud find corefile coud --> could * "Has not been able to find coredump. Test skipped." One suggestion is to check if /proc/sys/kernel/core_pattern has a line starting with '|' and print a warning that a crash reporting tool might be used (Something like in ClhsdbCDSCore.java). But it is just a suggestion and you are free to ignore it. In due course, we could include this test also as a part of the consolidation of SA's corefile testing effort (https://bugs.openjdk.java.net/browse/JDK-8202297). > I have verified that test still could reproduce issue > https://bugs.openjdk.java.net/browse/JDK-8176557 if default heap size is > large enough. However I think that it make a sense to add variant of > this test which explicit heap settings as a regression fix for bug > JDK-8176557 . > Sounds good to me. > Also I haven't managed to reproduce following issues with neither > existing or new tests: > https://bugs.openjdk.java.net/browse/JDK-8001227 > https://bugs.openjdk.java.net/browse/JDK-8023376 > https://bugs.openjdk.java.net/browse/JDK-8051445 > Corresponding tests were excluded for a long time so there are no any > results for them. I also see comments in bugs with unsuccessful attempts > to reproduce issues. I think that it makes a sense just to remove these > entries from problemlist and file new open bugs if any of failures happen. Sounds good to me. Thanks, Jini (Not a Reviewer). On 5/23/2018 6:45 AM, Leonid Mesnik wrote: > Hi > > Could you please review following patch which rewrite > vmTestbase/heapdump tests as a jtreg java tests. > > These heapdump tests were developed as a shell tests which verify > heapdump functionality in JDK 5. They work fine but needs some refactoring. > > webrev: http://cr.openjdk.java.net/~lmesnik/8203491/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8203491 > > Below is high-level overview of existing tests and changes: > > 1) Tests?vmTestbase/heapdump/OnOOM* verify that VM correctly generates > heapdump when OOME in heap or metaspace happens. Also they verify that > -XX:HeapDumpPath=?works correctly. > Re-written as: > http://cr.openjdk.java.net/~lmesnik/8203491/webrev.00/test/hotspot/jtreg/runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java.html > Test provokes OOME and verifies generated heapdump. > Please note that new jtreg test is very similar and use same test > patterns as most of ErrorHandling tests in > http://hg.openjdk.java.net/jdk/jdk/file/tip/test/hotspot/jtreg/runtime/ErrorHandling > The main goal is to verify error handling when OOME is happen so I used > the same way to provoke OOME as most of other tests in this directory. > New test might takes more then minute to complete test so I removed from > tier1. > > 2) Tests vmTestbase/heapdump/JMap* verify that jmap correctly generates > heapdump from live process and core/mdmp file. > There are existing tests which verify that jmap could correctly dump > heap for live process: > http://hg.openjdk.java.net/jdk/jdk/file/f4735ff8d17d/test/hotspot/jtreg/serviceability/sa/TestHeapDumpForInvokeDynamic.java > http://hg.openjdk.java.net/jdk/jdk/file/f4735ff8d17d/test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java > I believe that they provide the same coverage as heapdump/JMapHeap > and?heapdump/JMapMetaspace tests and no new tests is needed. > > The new test which verifies that jmap generate heapdump for corefile is: > http://cr.openjdk.java.net/~lmesnik/8203491/webrev.00/test/hotspot/jtreg/serviceability/sa/TestJmapCore.java.html > > Test provokes OOME to generate corefile (or mdml), generate heapdump > with jhsdb jmap and verify it. Previously tests send signals to provoke > coredump. Since JDK9 option 'CreateCoredumpOnCrash' could be used to > easily dump core on all platforms. Please note that test silently pass > if it doesn't find core file. Even if test tries to set ulimit -c > unlimited there is no guarantee that core file is generated and could be > found. > I have verified that test ?still could reproduce issue > https://bugs.openjdk.java.net/browse/JDK-8176557?if default heap size is > large enough. However I think that it make a sense to add variant of > this test which explicit heap settings as a regression fix for bug > JDK-8176557 . > > Also I haven't managed to reproduce following issues with neither > existing or new tests: > https://bugs.openjdk.java.net/browse/JDK-8001227 > https://bugs.openjdk.java.net/browse/JDK-8023376 > https://bugs.openjdk.java.net/browse/JDK-8051445 > Corresponding ?tests were excluded for a long time so there are no any > results for them. I also see comments in bugs with unsuccessful attempts > to reproduce issues. ?I think that it makes a sense just to remove these > entries from problemlist and file new open bugs if any of failures happen. > > I quickly looked through all resolved bugs ?found by heapdump/ tests. > Most of them are related with different size limits overflow. ?It might > be make a sense also create test which fill heap with large amount of > small objects to possible overflow of sizes of records. The similar > functionality was removed by JDK-8144137 > . ?But I would prefer > to have separate RFE since it is not supported by existing tests. > > Please let me know if it is possible to lost test coverage during this > conversion and more tests are needed. The more conservative way might be > to left existing tests for some time to see if if they still found > different bugs. > > Testing: > Run new tests on linux-x64,windows-x64,macos-x64. > Run tier1/2. > > > Leonid > > > > > From calvin.cheung at oracle.com Wed May 23 16:57:58 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 23 May 2018 09:57:58 -0700 Subject: RFR(S): 8203354: assert in ClassLoader::update_module_path_entry_list() could have incorrect message In-Reply-To: References: <5B046ABC.4070405@oracle.com> <5B049E5E.8090601@oracle.com> Message-ID: <5B059D96.1060406@oracle.com> Hi Thomas, Thanks for taking another look and filing the bug. thanks, Calvin On 5/22/18, 10:02 PM, Thomas St?fe wrote: > Hi Calvin, > > On Wed, May 23, 2018 at 12:49 AM, Calvin Cheung > wrote: >> Hi Thomas, >> >> Thanks for your quick review. >> >> On 5/22/18, 1:36 PM, Thomas St?fe wrote: >>> Hi Calvin, >>> >>> some small nits: >>> >>> - could you use vm_exit_during_initialization instead of raw exit? >>> - "Bad path %s" - that is an assumption, stat() may fail for a number >>> of reasons. So I would make the message more neutral. I also always >>> enclose such an output in quotes, to catch if the caller had >>> leading/trailing spaces. So, maybe this: "CDS dump aborted (path was >>> \"%s\")." ? >> I've fixed the above. >> Updated webrev: http://cr.openjdk.java.net/~ccheung/8203354/webrev.01/ > Looks good now. > >>> >>> Side note: I looked at the various os::stat implementations and see a >>> number of issues. We copy the input path around, which is on Posix >>> platforms unnecessary since os::native_path() returns just the path; >>> and since we use a fixed sized buffer we artificially limit the path >>> os::stat() recognizes. That is bad. Also, on windows, we return the >>> Win32 error code (GetLastError) as errno, which is just plain wrong. >>> >>> But these are issues beyond your patch. >> Could you file bugs on the above issues? >> > Done: https://bugs.openjdk.java.net/browse/JDK-8203680 > > Thanks! > > ..Thomas > >> thanks, >> Calvin >> >>> Kind Regards, Thomas >>> >>> >>> On Tue, May 22, 2018 at 9:08 PM, Calvin Cheung >>> wrote: >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8203354 >>>> >>>> webrev: http://cr.openjdk.java.net/~ccheung/8203354/webrev.00/ >>>> >>>> Converting the assert in ClassLoader::update_module_path_entry_list() to >>>> a >>>> meaningful error message before aborting the CDS dump. >>>> >>>> With a module path with very long length (> 2K), the error message will >>>> be >>>> like the following: >>>> >>>> os::stat error 36 (ENAMETOOLONG). CDS dump aborted. Bad path >>>> /scratch/myDir/JTwork/scratch/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx >>>> >>>> Note that tty->print_cr() has a buffer size limit of 2000, so the full >>>> path >>>> won't be shown entirely. That's why I'd like to put the error code up >>>> front >>>> before the path. >>>> >>>> Ran the modified tests on Oracle platforms. >>>> >>>> thanks, >>>> Calvin From karen.kinnear at oracle.com Wed May 23 17:31:05 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 23 May 2018 13:31:05 -0400 Subject: RFR (s) JDK-8203435: Circular nested dynamic constant test needed to confirm JVMS resolution behavior In-Reply-To: <4842faca-0775-f217-a5a0-cdbd9cd90357@oracle.com> References: <4842faca-0775-f217-a5a0-cdbd9cd90357@oracle.com> Message-ID: <6EFCB7C3-945B-439D-A8B8-26E1A95299D0@oracle.com> Lois, Test code looks good. Thank you for the comments and clear explanation of expected results. thanks, Karen > On May 21, 2018, at 3:50 PM, Lois Foltan wrote: > > Please review this change to add a nested circular dynamic constant test to confirm the JVMS 5.4.3.6 Dynamically-Computed Constant and Call Site Resolution behavior as discussed in http://mail.openjdk.java.net/pipermail/valhalla-spec-observers/2018-February/000518.html. > > open webrev at: http://cr.openjdk.java.net/~lfoltan/bug_jdk8203435/webrev/ > bug link: https://bugs.openjdk.java.net/browse/JDK-8203435 > > Testing: hs-tier1-2, jdk-tier1-3 in progress > > Thanks, > Lois From calvin.cheung at oracle.com Wed May 23 18:11:07 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 23 May 2018 11:11:07 -0700 Subject: RFR(S): 8203354: assert in ClassLoader::update_module_path_entry_list() could have incorrect message In-Reply-To: References: <5B046ABC.4070405@oracle.com> <5B049E5E.8090601@oracle.com> Message-ID: <5B05AEBA.40407@oracle.com> Hi Ioi, On 5/22/18, 10:16 PM, Ioi Lam wrote: > Hi Calvin, > > The C code changes look good. Thanks for your review. > > However, I am not sure if the test case is correct on all file > systems. Does the test assume that the max path length is about 2048? Yes, it is because the *nix vresions of the os::stat has an artificial limit of 2K: int os::stat(const char *path, struct stat *sbuf) { char pathbuf[MAX_PATH]; if (strlen(path) > MAX_PATH - 1) { errno = ENAMETOOLONG; return -1; } os::native_path(strcpy(pathbuf, path)); return ::stat(pathbuf, sbuf); } > > I think some file systems could allow more than 2048 chars. See > https://serverfault.com/questions/9546/filename-length-limits-on-linux > > On my Linux workstation: > $ getconf PATH_MAX / > 4096 After the fix for JDK-8203680 (filed by Thomas), we should be able to handle path length more than 2K. > > I think it's better to rewrite the test so that it's OK for the dump > to succeed. If it fails, it must be "os::stat error", but there's no > guarantee that the error code is ENAMETOOLONG. How about this: > > if (output.getExitValue() != 0) { > output.shouldMatch("os::stat error.*CDS dump aborted"); > } I think it's a good idea to make the test more flexible. Here's an updated webrev: http://cr.openjdk.java.net/~ccheung/8203354/webrev.02/ thanks, Calvin > > Thanks > - Ioi > > > On 5/22/18 3:49 PM, Calvin Cheung wrote: >> Hi Thomas, >> >> Thanks for your quick review. >> >> On 5/22/18, 1:36 PM, Thomas St?fe wrote: >>> Hi Calvin, >>> >>> some small nits: >>> >>> - could you use vm_exit_during_initialization instead of raw exit? >>> - "Bad path %s" - that is an assumption, stat() may fail for a number >>> of reasons. So I would make the message more neutral. I also always >>> enclose such an output in quotes, to catch if the caller had >>> leading/trailing spaces. So, maybe this: "CDS dump aborted (path was >>> \"%s\")." ? >> I've fixed the above. >> Updated webrev: http://cr.openjdk.java.net/~ccheung/8203354/webrev.01/ >>> >>> Side note: I looked at the various os::stat implementations and see a >>> number of issues. We copy the input path around, which is on Posix >>> platforms unnecessary since os::native_path() returns just the path; >>> and since we use a fixed sized buffer we artificially limit the path >>> os::stat() recognizes. That is bad. Also, on windows, we return the >>> Win32 error code (GetLastError) as errno, which is just plain wrong. >>> >>> But these are issues beyond your patch. >> Could you file bugs on the above issues? >> >> thanks, >> Calvin >>> >>> Kind Regards, Thomas >>> >>> >>> On Tue, May 22, 2018 at 9:08 PM, Calvin >>> Cheung wrote: >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8203354 >>>> >>>> webrev: http://cr.openjdk.java.net/~ccheung/8203354/webrev.00/ >>>> >>>> Converting the assert in >>>> ClassLoader::update_module_path_entry_list() to a >>>> meaningful error message before aborting the CDS dump. >>>> >>>> With a module path with very long length (> 2K), the error message >>>> will be >>>> like the following: >>>> >>>> os::stat error 36 (ENAMETOOLONG). CDS dump aborted. Bad path >>>> /scratch/myDir/JTwork/scratch/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx >>>> >>>> >>>> Note that tty->print_cr() has a buffer size limit of 2000, so the >>>> full path >>>> won't be shown entirely. That's why I'd like to put the error code >>>> up front >>>> before the path. >>>> >>>> Ran the modified tests on Oracle platforms. >>>> >>>> thanks, >>>> Calvin > From leonid.mesnik at oracle.com Wed May 23 18:39:17 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Wed, 23 May 2018 11:39:17 -0700 Subject: RFR: 8203491: [TESTBUG] Port heapdump tests into java In-Reply-To: <0ab155e3-9e93-11fb-55a7-6f0ac7e41f64@oracle.com> References: <650E6684-1996-4230-B9A4-CB4921419847@oracle.com> <0ab155e3-9e93-11fb-55a7-6f0ac7e41f64@oracle.com> Message-ID: <12B5827E-5F4E-4F08-9ED3-01DCC68A7386@oracle.com> Hi Thank you for review and feedback. See my comments inline. I am going to get more feedback and then send new version with fixes for your comments. > On May 23, 2018, at 8:48 AM, Jini George wrote: > > Hi Leonid, > > I have mostly looked at only the following file: > > http://cr.openjdk.java.net/~lmesnik/8203491/webrev.00/test/hotspot/jtreg/serviceability/sa/TestJmapCore.java.html > > * Would you be adding JMapMetaspaceCore testing to this? Other than this, I think the jhsdb heapdump testing from vmTestbase has been covered. > I am not sure that JMapMetaspaceCore provides any additional coverage. The test just fill 64M of metaspace and then send signal to dump core. So I don't see how this test could improve coverage. I think that idea of original test was to fill PermGen like Heap to expand it as much as possible or it was just an analog of test OnOOMToFileMetaspace. While current test just fill highly limited metaspace. So number of classes seems to be not significantly larger then for current TestJmapCore.java test. From my point of view it would be make a sense to generate dump containing a lot of loaded classes or some very large classes. While current test looks pretty similar to existing TestJmapCore.java test. Please let me know if you see the test scenario when such test could be useful. > * You might want to increase the timeout factor for this test. The test times out on my machine. > > I see that test finishes in 1 minute in our lab while. I see that it takes 30 seconds on 2CPU Oracle Linux VM with 2GB java heap. And test just fails with JDK-8176557 when I increase heap. How many time it takes on you machine? The timeoutFactor might be used for untypical environment/command-line options. > * You might want to consider removing the corefile and the heapdump files after the test execution (in the cases where the test passes). > The default jtreg retain policy in make files just removes all files in test directory for passed tests. The jtreg default test policy says "If -retain is not specified, only the files from the last test executed will be retained". So it should be not a problem in most of cases. While there is no way for user to retain core/heapdump files even if user wants to keeps them. However if it is the common rule for sa tests to delete such artifacts then I could remove heap/core dumps. > * The following imports: > import jdk.test.lib.classloader.GeneratingClassLoader; > import java.util.Arrays; > import java.util.stream.Collectors; > > are not needed. > Thanks, will fix it. > * Nit: > // If any arguments are set prints pid so main process coud find corefile > > coud --> could > > * "Has not been able to find coredump. Test skipped." Thanks. > > One suggestion is to check if /proc/sys/kernel/core_pattern has a line starting with '|' and print a warning that a crash reporting tool might be used (Something like in ClhsdbCDSCore.java). But it is just a suggestion and you are free to ignore it. In due course, we could include this test also as a part of the consolidation of SA's corefile testing effort (https://bugs.openjdk.java.net/browse/JDK-8202297). > I would prefer to left this improvement for JDK-8202297. I think good core dump processing/ulimit settings requires more efforts and testing and different version of Linux. (Might be even for Non-Oracle platforms). Also logic in test ClhsdbCDSCore.java is slightly different. It tries to get possible core location from hs_err file and print this hint of core file from hs_err doesn't exists. While printing this hint if core dumps are just completely disabled might just confuse users. Leonid > > > I have verified that test still could reproduce issue > > https://bugs.openjdk.java.net/browse/JDK-8176557 if default heap size is > > large enough. However I think that it make a sense to add variant of > > this test which explicit heap settings as a regression fix for bug > > JDK-8176557 . > > > > Sounds good to me. > > > Also I haven't managed to reproduce following issues with neither > > existing or new tests: > > https://bugs.openjdk.java.net/browse/JDK-8001227 > > https://bugs.openjdk.java.net/browse/JDK-8023376 > > https://bugs.openjdk.java.net/browse/JDK-8051445 > > Corresponding tests were excluded for a long time so there are no any > > results for them. I also see comments in bugs with unsuccessful attempts > > to reproduce issues. I think that it makes a sense just to remove these > > entries from problemlist and file new open bugs if any of failures happen. > > Sounds good to me. > > Thanks, > Jini (Not a Reviewer). > > > On 5/23/2018 6:45 AM, Leonid Mesnik wrote: >> Hi >> Could you please review following patch which rewrite vmTestbase/heapdump tests as a jtreg java tests. >> These heapdump tests were developed as a shell tests which verify heapdump functionality in JDK 5. They work fine but needs some refactoring. >> webrev: http://cr.openjdk.java.net/~lmesnik/8203491/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8203491 >> Below is high-level overview of existing tests and changes: >> 1) Tests vmTestbase/heapdump/OnOOM* verify that VM correctly generates heapdump when OOME in heap or metaspace happens. Also they verify that -XX:HeapDumpPath= works correctly. >> Re-written as: >> http://cr.openjdk.java.net/~lmesnik/8203491/webrev.00/test/hotspot/jtreg/runtime/ErrorHandling/TestHeapDumpOnOutOfMemoryError.java.html >> Test provokes OOME and verifies generated heapdump. >> Please note that new jtreg test is very similar and use same test patterns as most of ErrorHandling tests in http://hg.openjdk.java.net/jdk/jdk/file/tip/test/hotspot/jtreg/runtime/ErrorHandling >> The main goal is to verify error handling when OOME is happen so I used the same way to provoke OOME as most of other tests in this directory. >> New test might takes more then minute to complete test so I removed from tier1. >> 2) Tests vmTestbase/heapdump/JMap* verify that jmap correctly generates heapdump from live process and core/mdmp file. >> There are existing tests which verify that jmap could correctly dump heap for live process: >> http://hg.openjdk.java.net/jdk/jdk/file/f4735ff8d17d/test/hotspot/jtreg/serviceability/sa/TestHeapDumpForInvokeDynamic.java >> http://hg.openjdk.java.net/jdk/jdk/file/f4735ff8d17d/test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >> I believe that they provide the same coverage as heapdump/JMapHeap and heapdump/JMapMetaspace tests and no new tests is needed. >> The new test which verifies that jmap generate heapdump for corefile is: >> http://cr.openjdk.java.net/~lmesnik/8203491/webrev.00/test/hotspot/jtreg/serviceability/sa/TestJmapCore.java.html >> Test provokes OOME to generate corefile (or mdml), generate heapdump with jhsdb jmap and verify it. Previously tests send signals to provoke coredump. Since JDK9 option 'CreateCoredumpOnCrash' could be used to easily dump core on all platforms. Please note that test silently pass if it doesn't find core file. Even if test tries to set ulimit -c unlimited there is no guarantee that core file is generated and could be found. >> I have verified that test still could reproduce issue https://bugs.openjdk.java.net/browse/JDK-8176557 if default heap size is large enough. However I think that it make a sense to add variant of this test which explicit heap settings as a regression fix for bug JDK-8176557 . >> Also I haven't managed to reproduce following issues with neither existing or new tests: >> https://bugs.openjdk.java.net/browse/JDK-8001227 >> https://bugs.openjdk.java.net/browse/JDK-8023376 >> https://bugs.openjdk.java.net/browse/JDK-8051445 >> Corresponding tests were excluded for a long time so there are no any results for them. I also see comments in bugs with unsuccessful attempts to reproduce issues. I think that it makes a sense just to remove these entries from problemlist and file new open bugs if any of failures happen. >> I quickly looked through all resolved bugs found by heapdump/ tests. Most of them are related with different size limits overflow. It might be make a sense also create test which fill heap with large amount of small objects to possible overflow of sizes of records. The similar functionality was removed by JDK-8144137 . But I would prefer to have separate RFE since it is not supported by existing tests. >> Please let me know if it is possible to lost test coverage during this conversion and more tests are needed. The more conservative way might be to left existing tests for some time to see if if they still found different bugs. >> Testing: >> Run new tests on linux-x64,windows-x64,macos-x64. >> Run tier1/2. >> Leonid From gerard.ziemski at oracle.com Wed May 23 19:29:49 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Wed, 23 May 2018 14:29:49 -0500 Subject: RFR (XS) 8202360: [TESTBUG] runtime/LoadClass/TestResize.java needs to print output when it fails In-Reply-To: <18a88525-650a-05bd-a37b-a9ff9162e148@oracle.com> References: <18a88525-650a-05bd-a37b-a9ff9162e148@oracle.com> Message-ID: hi David, > On May 21, 2018, at 5:34 PM, David Holmes wrote: > > Hi Gerard, > > > > On 22/05/2018 6:57 AM, Gerard Ziemski wrote: >>> On May 20, 2018, at 8:38 PM, David Holmes wrote: >>> Okay - that helps with the immediate problem I had. I also now realize that the failure in this test is showing is that the SD did not in fact re-size as expected! >> Can you elaborate? Is there a reproducible case I can try to run? > > The test's purpose is to check that the SD resizes, so if it finds a load factor too high (which indicates it didn't resize) then the SD resizing logic has either not worked as expected, or the test is not doing what it thinks! The resizing is called during a safe point, which we try to trigger by issuing a System.gc() call (in TriggerResize.java). If no safe point takes place, then no resize had chance to occur, and we have test failure, which is probably what you experienced. I changed the test to look for safe points in the output and check the load factor only when resizing had the chance to occur. >> I have re-done the main logic a bit to simplify it, and not have to do either String and ?+? nor StringBuffer, by taking advantage of the fact that the info we need to parse is on the 2nd output line. > > I was under the impression that we could see multiple lines of the form: > > Java dictionary (table_size=40423, classes=50002) > > as the table resized. If that is not the case then none of the output seems relevant to me except for this one line. ?? That print out comes from PrintSystemDictionaryAtExit, which happens only once at the exit, but there is more then one system dictionary, and all of them are checked. https://bugs.openjdk.java.net/browse/JDK-8202360 http://cr.openjdk.java.net/~gziemski/8202360_rev4 cheers From gerard.ziemski at oracle.com Wed May 23 20:25:36 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Wed, 23 May 2018 15:25:36 -0500 Subject: RFR 8133564: Runtime - 2nd followup to Validate JVM Command-Line Flag Arguments Message-ID: <8DD96314-EFB1-4FD7-9E4A-3953241F1752@oracle.com> Hi all, Please review this follow-up fix that cleans up the following: #1 We move the error print method out of jvmFlagRangeList and into jvmFlag, which allows us not to include jvmFlagRangeList.hpp just for the print method in various files #2 We cleanup "utilities/defaultStream.hpp? includes #3 and #4 in the issue are no longer applicable Tested locally with various debug/product, headers/no headers, open/closed variations, as well as Mach5 hs-tier1,tier2,tier3 tests, and Mach5 open/closed variations. https://bugs.openjdk.java.net/browse/JDK-8133564 http://cr.openjdk.java.net/~gziemski/8133564_rev1 cheers From coleen.phillimore at oracle.com Wed May 23 21:29:06 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 23 May 2018 17:29:06 -0400 Subject: RFR 8133564: Runtime - 2nd followup to Validate JVM Command-Line Flag Arguments In-Reply-To: <8DD96314-EFB1-4FD7-9E4A-3953241F1752@oracle.com> References: <8DD96314-EFB1-4FD7-9E4A-3953241F1752@oracle.com> Message-ID: <6684c916-c20b-e860-a3db-c0be6523b411@oracle.com> This looks good apart from fixing the indentation of the arguments for all these changes so that they line up. - CommandLineError::print(verbose, + JVMFlag::printError(verbose, "ParallelGCThreads (" UINT32_FORMAT ") or CMSWorkQueueDrainThreshold (" UINTX_FORMAT ") is too large\n", threads, threshold); Thanks, Coleen On 5/23/18 4:25 PM, Gerard Ziemski wrote: > Hi all, > > Please review this follow-up fix that cleans up the following: > > #1 We move the error print method out of jvmFlagRangeList and into jvmFlag, which allows us not to include jvmFlagRangeList.hpp just for the print method in various files > > #2 We cleanup "utilities/defaultStream.hpp? includes > > #3 and #4 in the issue are no longer applicable > > Tested locally with various debug/product, headers/no headers, open/closed variations, as well as Mach5 hs-tier1,tier2,tier3 tests, and Mach5 open/closed variations. > > > https://bugs.openjdk.java.net/browse/JDK-8133564 > http://cr.openjdk.java.net/~gziemski/8133564_rev1 > > cheers From lois.foltan at oracle.com Wed May 23 21:38:18 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Wed, 23 May 2018 17:38:18 -0400 Subject: RFR (s) JDK-8203435: Circular nested dynamic constant test needed to confirm JVMS resolution behavior In-Reply-To: References: <4842faca-0775-f217-a5a0-cdbd9cd90357@oracle.com> Message-ID: Thanks Paul for the review! Lois On 5/22/2018 6:06 PM, Paul Sandoz wrote: > Looks ok. Note there is a companion test here: > > http://hg.openjdk.java.net/jdk/jdk/file/2942ae532175/test/jdk/java/lang/invoke/condy/CondyNestedTest.java > > Paul. > >> On May 21, 2018, at 12:50 PM, Lois Foltan > > wrote: >> >> Please review this change to add a nested circular dynamic constant >> test to confirm the JVMS 5.4.3.6 Dynamically-Computed Constant and >> Call Site Resolution behavior as discussed in >> http://mail.openjdk.java.net/pipermail/valhalla-spec-observers/2018-February/000518.html. >> >> open webrev at: >> http://cr.openjdk.java.net/~lfoltan/bug_jdk8203435/webrev/ >> >> bug link: https://bugs.openjdk.java.net/browse/JDK-8203435 >> >> Testing: hs-tier1-2, jdk-tier1-3 in progress >> >> Thanks, >> Lois > From lois.foltan at oracle.com Wed May 23 21:38:45 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Wed, 23 May 2018 17:38:45 -0400 Subject: RFR (s) JDK-8203435: Circular nested dynamic constant test needed to confirm JVMS resolution behavior In-Reply-To: <6EFCB7C3-945B-439D-A8B8-26E1A95299D0@oracle.com> References: <4842faca-0775-f217-a5a0-cdbd9cd90357@oracle.com> <6EFCB7C3-945B-439D-A8B8-26E1A95299D0@oracle.com> Message-ID: <7c170d63-d3aa-ed44-2071-2e0969cfef81@oracle.com> Thanks Karen for the review! Lois On 5/23/2018 1:31 PM, Karen Kinnear wrote: > Lois, > > Test code looks good. Thank you for the comments and clear explanation of expected results. > > thanks, > Karen > >> On May 21, 2018, at 3:50 PM, Lois Foltan wrote: >> >> Please review this change to add a nested circular dynamic constant test to confirm the JVMS 5.4.3.6 Dynamically-Computed Constant and Call Site Resolution behavior as discussed in http://mail.openjdk.java.net/pipermail/valhalla-spec-observers/2018-February/000518.html. >> >> open webrev at: http://cr.openjdk.java.net/~lfoltan/bug_jdk8203435/webrev/ >> bug link: https://bugs.openjdk.java.net/browse/JDK-8203435 >> >> Testing: hs-tier1-2, jdk-tier1-3 in progress >> >> Thanks, >> Lois From calvin.cheung at oracle.com Wed May 23 22:15:56 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 23 May 2018 15:15:56 -0700 Subject: RFR(L): 8199255: [TESTBUG] Open source VM testbase default methods tests In-Reply-To: <5B031153.6020408@oracle.com> References: <5B031153.6020408@oracle.com> Message-ID: <5B05E81C.60406@oracle.com> Hi Misha, I've compared the file.list from your closed webrev with the one from this open webrev and didn't see any missing files. Also spot checked a few copyright headers and they look good. Regarding TEST.groups, why was the following removed? 1160 vmTestbase_nsk_stress = \ 1161 vmTestbase/nsk/stress Could you also remove the extra blank line added at line 1273? thanks, Calvin On 5/21/18, 11:34 AM, Mikhailo Seledtsov wrote: > Please review this change that will open source VM default method tests. > These tests have been used internally for a while, and are now being > open sourced. Since this is not an creation of new tests, we would > like to keep the changes during this review to a minimum required for > open sourcing these tests, such as major issues and integration > blockers. If you have other feedback regarding improvements to these > tests, please file RFE(s) that will be addressed later in order of > priority. > > Here is what was done for this change: > 1. Moved the tests to OpenJDK repository to the specified directory > location and structure. > 3. Updated Copyright statements accordingly. > 4. Updated "@library" statements accordingly. > 5. Updated TEST.groups and a HotSpot test make file > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199255 > Webrev: http://cr.openjdk.java.net/~mseledtsov/8199255.01/ > > Testing: > 1. Ran the following tests on open-only repository and build, > using "make run-test" (Linux-x64) > vmTestbase_vm_defmeth > All PASS > > 2. Automated multip-platform test system (usual 4 platforms): > - vmTestbase_vm_defmeth > - hs-tier{1,2} > In progress > > > Thank you, > Misha > From mikhailo.seledtsov at oracle.com Wed May 23 23:13:16 2018 From: mikhailo.seledtsov at oracle.com (mikhailo) Date: Wed, 23 May 2018 16:13:16 -0700 Subject: RFR(L): 8199255: [TESTBUG] Open source VM testbase default methods tests In-Reply-To: <5B05E81C.60406@oracle.com> References: <5B031153.6020408@oracle.com> <5B05E81C.60406@oracle.com> Message-ID: Hi Calvin, ? Thank you for review. I will fix the issue with vmTestbase_nsk_stress (merge issue) and will remove the blank line at line 1273 prior to check in. Misha On 05/23/2018 03:15 PM, Calvin Cheung wrote: > Hi Misha, > > I've compared the file.list from your closed webrev with the one from > this open webrev and didn't see any missing files. > Also spot checked a few copyright headers and they look good. > > Regarding TEST.groups, why was the following removed? > 1160 vmTestbase_nsk_stress = \ > 1161?? vmTestbase/nsk/stress > > Could you also remove the extra blank line added at line 1273? > > thanks, > Calvin > > On 5/21/18, 11:34 AM, Mikhailo Seledtsov wrote: >> Please review this change that will open source VM default method tests. >> These tests have been used internally for a while, and are now being >> open sourced. Since this is not an creation of new tests, we would >> like to keep the changes during this review to a minimum required for >> open sourcing these tests, such as major issues and integration >> blockers. If you have other feedback regarding improvements to these >> tests, please file RFE(s) that will be addressed later in order of >> priority. >> >> Here is what was done for this change: >> ? 1. Moved the tests to OpenJDK repository to the specified directory >> location and structure. >> ? 3. Updated Copyright statements accordingly. >> ? 4. Updated "@library" statements accordingly. >> ? 5. Updated TEST.groups and a HotSpot test make file >> >> ? JBS:??? https://bugs.openjdk.java.net/browse/JDK-8199255 >> ? Webrev: http://cr.openjdk.java.net/~mseledtsov/8199255.01/ >> >> ? Testing: >> ????? 1. Ran the following tests on open-only repository and build, >> using "make run-test" (Linux-x64) >> ???????? vmTestbase_vm_defmeth >> ???????? All PASS >> >> ????? 2. Automated multip-platform test system (usual 4 platforms): >> ???????? - vmTestbase_vm_defmeth >> ???????? - hs-tier{1,2} >> ???????? In progress >> >> >> Thank you, >> Misha >> From igor.ignatyev at oracle.com Wed May 23 23:13:17 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 23 May 2018 16:13:17 -0700 Subject: RFR(L): 8199255: [TESTBUG] Open source VM testbase default methods tests In-Reply-To: References: <5B031153.6020408@oracle.com> <5B05E81C.60406@oracle.com> Message-ID: Hi Misha, looks good to me. -- Igor > On May 23, 2018, at 4:13 PM, mikhailo wrote: > > Hi Calvin, > > Thank you for review. I will fix the issue with vmTestbase_nsk_stress (merge issue) and will remove the blank line at line 1273 prior to check in. > > > Misha > > > On 05/23/2018 03:15 PM, Calvin Cheung wrote: >> Hi Misha, >> >> I've compared the file.list from your closed webrev with the one from this open webrev and didn't see any missing files. >> Also spot checked a few copyright headers and they look good. >> >> Regarding TEST.groups, why was the following removed? >> 1160 vmTestbase_nsk_stress = \ >> 1161 vmTestbase/nsk/stress >> >> Could you also remove the extra blank line added at line 1273? >> >> thanks, >> Calvin >> >> On 5/21/18, 11:34 AM, Mikhailo Seledtsov wrote: >>> Please review this change that will open source VM default method tests. >>> These tests have been used internally for a while, and are now being open sourced. Since this is not an creation of new tests, we would like to keep the changes during this review to a minimum required for open sourcing these tests, such as major issues and integration blockers. If you have other feedback regarding improvements to these tests, please file RFE(s) that will be addressed later in order of priority. >>> >>> Here is what was done for this change: >>> 1. Moved the tests to OpenJDK repository to the specified directory location and structure. >>> 3. Updated Copyright statements accordingly. >>> 4. Updated "@library" statements accordingly. >>> 5. Updated TEST.groups and a HotSpot test make file >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8199255 >>> Webrev: http://cr.openjdk.java.net/~mseledtsov/8199255.01/ >>> >>> Testing: >>> 1. Ran the following tests on open-only repository and build, using "make run-test" (Linux-x64) >>> vmTestbase_vm_defmeth >>> All PASS >>> >>> 2. Automated multip-platform test system (usual 4 platforms): >>> - vmTestbase_vm_defmeth >>> - hs-tier{1,2} >>> In progress >>> >>> >>> Thank you, >>> Misha >>> > From mikhailo.seledtsov at oracle.com Thu May 24 00:05:17 2018 From: mikhailo.seledtsov at oracle.com (mikhailo) Date: Wed, 23 May 2018 17:05:17 -0700 Subject: RFR(L): 8199255: [TESTBUG] Open source VM testbase default methods tests In-Reply-To: References: <5B031153.6020408@oracle.com> <5B05E81C.60406@oracle.com> Message-ID: <430598e1-fb60-eba3-6324-f466a975bcd8@oracle.com> Thank you, Misha On 05/23/2018 04:13 PM, Igor Ignatyev wrote: > Hi Misha, > > looks good to me. > > -- Igor > >> On May 23, 2018, at 4:13 PM, mikhailo wrote: >> >> Hi Calvin, >> >> Thank you for review. I will fix the issue with vmTestbase_nsk_stress (merge issue) and will remove the blank line at line 1273 prior to check in. >> >> >> Misha >> >> >> On 05/23/2018 03:15 PM, Calvin Cheung wrote: >>> Hi Misha, >>> >>> I've compared the file.list from your closed webrev with the one from this open webrev and didn't see any missing files. >>> Also spot checked a few copyright headers and they look good. >>> >>> Regarding TEST.groups, why was the following removed? >>> 1160 vmTestbase_nsk_stress = \ >>> 1161 vmTestbase/nsk/stress >>> >>> Could you also remove the extra blank line added at line 1273? >>> >>> thanks, >>> Calvin >>> >>> On 5/21/18, 11:34 AM, Mikhailo Seledtsov wrote: >>>> Please review this change that will open source VM default method tests. >>>> These tests have been used internally for a while, and are now being open sourced. Since this is not an creation of new tests, we would like to keep the changes during this review to a minimum required for open sourcing these tests, such as major issues and integration blockers. If you have other feedback regarding improvements to these tests, please file RFE(s) that will be addressed later in order of priority. >>>> >>>> Here is what was done for this change: >>>> 1. Moved the tests to OpenJDK repository to the specified directory location and structure. >>>> 3. Updated Copyright statements accordingly. >>>> 4. Updated "@library" statements accordingly. >>>> 5. Updated TEST.groups and a HotSpot test make file >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8199255 >>>> Webrev: http://cr.openjdk.java.net/~mseledtsov/8199255.01/ >>>> >>>> Testing: >>>> 1. Ran the following tests on open-only repository and build, using "make run-test" (Linux-x64) >>>> vmTestbase_vm_defmeth >>>> All PASS >>>> >>>> 2. Automated multip-platform test system (usual 4 platforms): >>>> - vmTestbase_vm_defmeth >>>> - hs-tier{1,2} >>>> In progress >>>> >>>> >>>> Thank you, >>>> Misha >>>> From ioi.lam at oracle.com Thu May 24 00:22:33 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 23 May 2018 17:22:33 -0700 Subject: RFR(S): 8203354: assert in ClassLoader::update_module_path_entry_list() could have incorrect message In-Reply-To: <5B05AEBA.40407@oracle.com> References: <5B046ABC.4070405@oracle.com> <5B049E5E.8090601@oracle.com> <5B05AEBA.40407@oracle.com> Message-ID: <76a400b0-804b-9d11-199e-cf099cc593a9@oracle.com> Looks good. Thanks! - Ioi On 5/23/18 11:11 AM, Calvin Cheung wrote: > Hi Ioi, > > On 5/22/18, 10:16 PM, Ioi Lam wrote: >> Hi Calvin, >> >> The C code changes look good. > Thanks for your review. >> >> However, I am not sure if the test case is correct on all file >> systems. Does the test assume that the max path length is about 2048? > Yes, it is because the *nix vresions of the os::stat has an artificial > limit of 2K: > int os::stat(const char *path, struct stat *sbuf) { > ? char pathbuf[MAX_PATH]; > ? if (strlen(path) > MAX_PATH - 1) { > ??? errno = ENAMETOOLONG; > ??? return -1; > ? } > ? os::native_path(strcpy(pathbuf, path)); > ? return ::stat(pathbuf, sbuf); > } > >> >> I think some file systems could allow more than 2048 chars. See >> https://serverfault.com/questions/9546/filename-length-limits-on-linux >> >> On my Linux workstation: >> $ getconf PATH_MAX / >> 4096 > After the fix for JDK-8203680 (filed by Thomas), we should be able to > handle path length more than 2K. >> >> I think it's better to rewrite the test so that it's OK for the dump >> to succeed. If it fails, it must be "os::stat error", but there's no >> guarantee that the error code is ENAMETOOLONG. How about this: >> >> if (output.getExitValue() != 0) { >> ??? output.shouldMatch("os::stat error.*CDS dump aborted"); >> } > I think it's a good idea to make the test more flexible. Here's an > updated webrev: > ??? http://cr.openjdk.java.net/~ccheung/8203354/webrev.02/ > > thanks, > Calvin >> >> Thanks >> - Ioi >> >> >> On 5/22/18 3:49 PM, Calvin Cheung wrote: >>> Hi Thomas, >>> >>> Thanks for your quick review. >>> >>> On 5/22/18, 1:36 PM, Thomas St?fe wrote: >>>> Hi Calvin, >>>> >>>> some small nits: >>>> >>>> - could you use vm_exit_during_initialization instead of raw exit? >>>> - "Bad path %s" - that is an assumption, stat() may fail for a number >>>> of reasons. So I would make the message more neutral. I also always >>>> enclose such an output in quotes, to catch if the caller had >>>> leading/trailing spaces. So, maybe this: "CDS dump aborted (path was >>>> \"%s\")." ? >>> I've fixed the above. >>> Updated webrev: http://cr.openjdk.java.net/~ccheung/8203354/webrev.01/ >>>> >>>> Side note: I looked at the various os::stat implementations and see a >>>> number of issues. We copy the input path around, which is on Posix >>>> platforms unnecessary since os::native_path() returns just the path; >>>> and since we use a fixed sized buffer we artificially limit the path >>>> os::stat() recognizes. That is bad. Also, on windows, we return the >>>> Win32 error code (GetLastError) as errno, which is just plain wrong. >>>> >>>> But these are issues beyond your patch. >>> Could you file bugs on the above issues? >>> >>> thanks, >>> Calvin >>>> >>>> Kind Regards, Thomas >>>> >>>> >>>> On Tue, May 22, 2018 at 9:08 PM, Calvin >>>> Cheung? wrote: >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8203354 >>>>> >>>>> webrev: http://cr.openjdk.java.net/~ccheung/8203354/webrev.00/ >>>>> >>>>> Converting the assert in >>>>> ClassLoader::update_module_path_entry_list() to a >>>>> meaningful error message before aborting the CDS dump. >>>>> >>>>> With a module path with very long length (>? 2K), the error >>>>> message will be >>>>> like the following: >>>>> >>>>> os::stat error 36 (ENAMETOOLONG). CDS dump aborted. Bad path >>>>> /scratch/myDir/JTwork/scratch/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx >>>>> >>>>> >>>>> Note that tty->print_cr() has a buffer size limit of 2000, so the >>>>> full path >>>>> won't be shown entirely. That's why I'd like to put the error code >>>>> up front >>>>> before the path. >>>>> >>>>> Ran the modified tests on Oracle platforms. >>>>> >>>>> thanks, >>>>> Calvin >> From timberonce at gmail.com Thu May 24 02:55:16 2018 From: timberonce at gmail.com (Mingyu Wu) Date: Thu, 24 May 2018 10:55:16 +0800 Subject: Improve Java serialization with APPCDS (as a fast path) Message-ID: Hi all, APPCDS is a very interesting optimization aiming at reducing memory footprint and class loading overhead for multiple Java processes. In my opinion, it can also be used in other scopes such as serialization. Currently, Java serializer is slow and induces a large footprint (compared to application-level serializer). A major problem is that the serializer should write the description of classes into the serialized bytes, which increases the total memory consumption. On the other hand, application-level (or 3rd-party) serializers like Kryo can reduce the memory footprint by requiring users to assign IDs to certain classes manually. This assignment step should be finished very carefully to avoid inconsistency problem among different JVMs, so application-level serializers are not that easy to use. However, we can actually borrow the idea from application-level serializers with APPCDS (or even CDS) enabling. Consider we already have dumped a class list below: java/lang/Object java/lang/String ...... We can assign IDs directly to those classes according to the order in the class list: java/lang/Object 0 java/lang/String 1 ...... Since multiple JVMs will share the same APPCDS archive correspondence with the class list, those JVMs can directly use IDs to serialize/deserialize the classes stored in the archive. This avoids writing class descriptions into serialization bytes and simplifies the serialization/deserialization phase. Furthermore, it also saves users from manually assigning IDs to classes. Note that APPCDS only provides a fast path for ser/deser. If a class is not on the class list (and the archive), the serializer falls back to class description. However, the fast path can become more efficient with more advanced features, such as supporting custom classloaders. Anyway, I think APPCDS is a good fit to improve Java serialization. I am willing to take suggestions! Mingyu -- Mingyu Wu Institute of Parallel and Distributed Systems School of Software Engineering Shanghai Jiao Tong University From ioi.lam at oracle.com Thu May 24 04:17:00 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 23 May 2018 21:17:00 -0700 Subject: Improve Java serialization with APPCDS (as a fast path) In-Reply-To: References: Message-ID: <4f31909c-e507-87e5-bc08-84cfaaebec8f@oracle.com> Hi Mingyu, I think this is a very interesting idea. Are you thinking about serialize/deserialize with the same JVM on the same host, or different JVMs across the network. Currently different JVM installations can have different CDS archives. You can even use different archives for the same JVM by running with $JAVA_HOME/bin/java -XX:SharedArchiveFile=/tmp/a.jsa $JAVA_HOME/bin/java -XX:SharedArchiveFile=/tmp/b.jsa So there's no guarantee that the same class will have the same ID in these 2 JVM process. Thanks - Ioi On 5/23/18 7:55 PM, Mingyu Wu wrote: > Hi all, > > APPCDS is a very interesting optimization aiming at reducing memory > footprint and class loading overhead for multiple Java processes. In my > opinion, it can also be used in other scopes such as serialization. > > Currently, Java serializer is slow and induces a large footprint (compared > to application-level serializer). A major problem is that the serializer > should write the description of classes into the serialized bytes, which > increases the total memory consumption. > > On the other hand, application-level (or 3rd-party) serializers like Kryo > can reduce the memory footprint by requiring users to assign IDs to certain > classes manually. > This assignment step should be finished very carefully to avoid > inconsistency problem among different JVMs, so application-level > serializers are not that easy to use. > > However, we can actually borrow the idea from application-level serializers > with APPCDS (or even CDS) enabling. Consider we already have dumped a class > list below: > java/lang/Object > java/lang/String > ...... > > We can assign IDs directly to those classes according to the order in the > class list: > java/lang/Object 0 > java/lang/String 1 > ...... > > Since multiple JVMs will share the same APPCDS archive correspondence with > the class list, those JVMs can directly use IDs to serialize/deserialize > the classes stored in the archive. This avoids writing class descriptions > into serialization bytes and simplifies the serialization/deserialization > phase. Furthermore, it also saves users from manually assigning IDs to > classes. > > Note that APPCDS only provides a fast path for ser/deser. If a class is not > on the class list (and the archive), the serializer falls back to class > description. However, the fast path can become more efficient with more > advanced features, such as supporting custom classloaders. > > Anyway, I think APPCDS is a good fit to improve Java serialization. > > I am willing to take suggestions! > > Mingyu From coleen.phillimore at oracle.com Thu May 24 12:27:25 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 24 May 2018 08:27:25 -0400 Subject: RFR(M): JDK-8203321: Windows: assert(current_query_index < process_query_set->size) failed: invariant In-Reply-To: <06e925c8-8d77-43c8-9ef2-ad9113034b98@default> References: <06e925c8-8d77-43c8-9ef2-ad9113034b98@default> Message-ID: Hi, Maybe this should go on serviceability-dev as well. Thanks, Coleen On 5/22/18 3:24 PM, Markus Gronlund wrote: > Greetings, > > Kindly asking for reviews for the following change: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203321 > Webrev: http://cr.openjdk.java.net/~mgronlun/8203321/webrev00/ > > Summary: > > For some context about what this is about, please see this (now) relatively old issue: https://bugs.openjdk.java.net/browse/JDK-8019921 > > The porting work that brought this code from closed to open were optimistic in that the following PDH query, "\Process(java#n)\ID Process", performed relatively stable on Windows 10. > An invariant was added in that your ID Process query would never return an index that was lower than the index at construction. > > During testing, it was discovered that this invariant did not hold, especially when running on Windows Server 2012 R2 and there is a high churn rate with many processes with the same base name ("java#") starting and stopping (stressing PDH list of processes). > > We have to reinsert back the original code that handled the case where the PDH process list is not stable (that were originally put in place with JDK-8019921). > The defensive logic is located at lines 418 - 422. > > I had to rework some related code to make some room for this as well as to keep track of the previous process index. > > Thanks > Markus From timberonce at gmail.com Thu May 24 14:32:33 2018 From: timberonce at gmail.com (Mingyu Wu) Date: Thu, 24 May 2018 22:32:33 +0800 Subject: Improve Java serialization with APPCDS (as a fast path) Message-ID: Hi Ioi, I am targeting on JVMs on both the same machine and network. To solve the problem where different JVMs may use different archives, we can introduce some JVM options. For example, a possible option can be: -XX:FastSerializationFile=/tmp/a.jsa. In this way, the JVM will only use a.jsa for serialization. Besides, the serializer can encode a hash value of the content in a.jsa into serialized bytes. In this way, only JVMs use a.jsa for fast serialization can deserialize successfully. Failure on hash value check will cause a runtime exception. Even if these settings may cause exceptions, I think since users are aware that they are using APPCDS, they can naturally extend the -XX:SharedArchiveFile to -XX:FastSerializationFile (which is much easier than using application-level serializer) to avoid unnecessary exceptions. Besides, since APPCDS is mainly designed to save memory space and class loading time for multiple JVMs, we can reduce the communication traffic among them by the way with fast serialization. In my opinion, this option will not be hard for users to understand and exploit. Thanks - Mingyu On 5/23/18 9:17 PM, Ioi Lam wrote: > Hi Mingyu, > > I think this is a very interesting idea. Are you thinking about > serialize/deserialize with the same JVM on the same host, or different > JVMs across the network. > > Currently different JVM installations can have different CDS archives. > You can even use different archives for the same JVM by running with > > $JAVA_HOME/bin/java -XX:SharedArchiveFile=/tmp/a.jsa > $JAVA_HOME/bin/java -XX:SharedArchiveFile=/tmp/b.jsa > > So there's no guarantee that the same class will have the same ID in > these 2 JVM process. > > Thanks > - Ioi > > > On 5/23/18 7:55 PM, Mingyu Wu wrote: > > Hi all, > > > > APPCDS is a very interesting optimization aiming at reducing memory > > footprint and class loading overhead for multiple Java processes. In my > > opinion, it can also be used in other scopes such as serialization. > > > > Currently, Java serializer is slow and induces a large footprint (compared > > to application-level serializer). A major problem is that the serializer > > should write the description of classes into the serialized bytes, which > > increases the total memory consumption. > > > > On the other hand, application-level (or 3rd-party) serializers like Kryo > > can reduce the memory footprint by requiring users to assign IDs to certain > > classes manually. > > This assignment step should be finished very carefully to avoid > > inconsistency problem among different JVMs, so application-level > > serializers are not that easy to use. > > > > However, we can actually borrow the idea from application-level serializers > > with APPCDS (or even CDS) enabling. Consider we already have dumped a class > > list below: > > java/lang/Object > > java/lang/String > > ...... > > > > We can assign IDs directly to those classes according to the order in the > > class list: > > java/lang/Object 0 > > java/lang/String 1 > > ...... > > > > Since multiple JVMs will share the same APPCDS archive correspondence with > > the class list, those JVMs can directly use IDs to serialize/deserialize > > the classes stored in the archive. This avoids writing class descriptions > > into serialization bytes and simplifies the serialization/deserialization > > phase. Furthermore, it also saves users from manually assigning IDs to > > classes. > > > > Note that APPCDS only provides a fast path for ser/deser. If a class is not > > on the class list (and the archive), the serializer falls back to class > > description. However, the fast path can become more efficient with more > > advanced features, such as supporting custom classloaders. > > > > Anyway, I think APPCDS is a good fit to improve Java serialization. > > > > I am willing to take suggestions! > > > > Mingyu > > > -- Mingyu Wu Institute of Parallel and Distributed Systems School of Software Engineering Shanghai Jiao Tong University From harold.seigel at oracle.com Thu May 24 14:38:18 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Thu, 24 May 2018 10:38:18 -0400 Subject: RFR 8203183: vmTestbase/vm/mlvm/anonloader/stress/oome/heap/Test.java times out In-Reply-To: References: <24278661-d13e-770c-a8f9-a010b27c8db2@oracle.com> Message-ID: Thanks Kim! Harold On 5/20/2018 11:52 PM, Kim Barrett wrote: >> On May 18, 2018, at 5:15 PM, Harold David Seigel wrote: >> >> Hi, >> >> Please review this fix for JDK-8203183. The fix is to run the test with -Xmx128m to prevent the test from timing out. The fix was tested on Linux X64, Mac OS X, Windows, and Solaris. >> >> Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8203183/webrev/index.html >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8203183 >> >> Thanks, Harold > Looks good. > From coleen.phillimore at oracle.com Thu May 24 15:07:21 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 24 May 2018 11:07:21 -0400 Subject: RFR 8203183: vmTestbase/vm/mlvm/anonloader/stress/oome/heap/Test.java times out In-Reply-To: <24278661-d13e-770c-a8f9-a010b27c8db2@oracle.com> References: <24278661-d13e-770c-a8f9-a010b27c8db2@oracle.com> Message-ID: <4b5dd4b1-96bc-3e5e-60d6-f5d43d58b6f2@oracle.com> Looks good! Coleen On 5/18/18 11:15 AM, Harold David Seigel wrote: > Hi, > > Please review this fix for JDK-8203183.? The fix is to run the test > with -Xmx128m to prevent the test from timing out.? The fix was tested > on Linux X64, Mac OS X, Windows, and Solaris. > > Open Webrev: > http://cr.openjdk.java.net/~hseigel/bug_8203183/webrev/index.html > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8203183 > > Thanks, Harold > From harold.seigel at oracle.com Thu May 24 15:09:48 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Thu, 24 May 2018 11:09:48 -0400 Subject: RFR 8203183: vmTestbase/vm/mlvm/anonloader/stress/oome/heap/Test.java times out In-Reply-To: <4b5dd4b1-96bc-3e5e-60d6-f5d43d58b6f2@oracle.com> References: <24278661-d13e-770c-a8f9-a010b27c8db2@oracle.com> <4b5dd4b1-96bc-3e5e-60d6-f5d43d58b6f2@oracle.com> Message-ID: Thanks Coleen! Harold On 5/24/2018 11:07 AM, coleen.phillimore at oracle.com wrote: > Looks good! > Coleen > > On 5/18/18 11:15 AM, Harold David Seigel wrote: >> Hi, >> >> Please review this fix for JDK-8203183.? The fix is to run the test >> with -Xmx128m to prevent the test from timing out.? The fix was >> tested on Linux X64, Mac OS X, Windows, and Solaris. >> >> Open Webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8203183/webrev/index.html >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8203183 >> >> Thanks, Harold >> > From coleen.phillimore at oracle.com Thu May 24 15:36:01 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 24 May 2018 11:36:01 -0400 Subject: RFR (trivial) 8202619: [TESTBUG] open source closed/test/hotspot/jtreg/serviceability/jfr/TestGCOldWithJFR.java Message-ID: <263f6518-3c72-6a9b-009b-bc59b04c68de@oracle.com> Summary: remove the test.? There are more specific tests for the functionality that it was added for. Made sure it wasn't in the ProblemList.txt files. bug link https://bugs.openjdk.java.net/browse/JDK-8202619 local webrev at http://oklahoma.us.oracle.com/~cphillim/webrev/8202619.01/webrev Thanks, Coleen From coleen.phillimore at oracle.com Thu May 24 15:37:17 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 24 May 2018 11:37:17 -0400 Subject: RFR (trivial) 8202619: [TESTBUG] open source closed/test/hotspot/jtreg/serviceability/jfr/TestGCOldWithJFR.java In-Reply-To: <263f6518-3c72-6a9b-009b-bc59b04c68de@oracle.com> References: <263f6518-3c72-6a9b-009b-bc59b04c68de@oracle.com> Message-ID: <284620aa-43a2-6205-0bb6-0d088f6de960@oracle.com> Wrong list, please ignore. On 5/24/18 11:36 AM, coleen.phillimore at oracle.com wrote: > Summary: remove the test.? There are more specific tests for the > functionality that it was added for. > > Made sure it wasn't in the ProblemList.txt files. > > bug link https://bugs.openjdk.java.net/browse/JDK-8202619 > local webrev at > http://oklahoma.us.oracle.com/~cphillim/webrev/8202619.01/webrev > > Thanks, > Coleen From gerard.ziemski at oracle.com Thu May 24 15:50:29 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Thu, 24 May 2018 10:50:29 -0500 Subject: RFR 8133564: Runtime - 2nd followup to Validate JVM Command-Line Flag Arguments In-Reply-To: <6684c916-c20b-e860-a3db-c0be6523b411@oracle.com> References: <8DD96314-EFB1-4FD7-9E4A-3953241F1752@oracle.com> <6684c916-c20b-e860-a3db-c0be6523b411@oracle.com> Message-ID: <95BA4C1C-3752-4991-A696-28BCCBD50A2E@oracle.com> Thank you Coleen for your review. I fixed the indentation as well as the years in copyright header. http://cr.openjdk.java.net/~gziemski/8133564_rev2 https://bugs.openjdk.java.net/browse/JDK-8133564 cheers > On May 23, 2018, at 4:29 PM, coleen.phillimore at oracle.com wrote: > > > This looks good apart from fixing the indentation of the arguments for all these changes so that they line up. > > - CommandLineError::print(verbose, > + JVMFlag::printError(verbose, > "ParallelGCThreads (" UINT32_FORMAT ") or CMSWorkQueueDrainThreshold (" > UINTX_FORMAT ") is too large\n", > threads, threshold); > > > Thanks, > Coleen > > On 5/23/18 4:25 PM, Gerard Ziemski wrote: >> Hi all, >> >> Please review this follow-up fix that cleans up the following: >> >> #1 We move the error print method out of jvmFlagRangeList and into jvmFlag, which allows us not to include jvmFlagRangeList.hpp just for the print method in various files >> >> #2 We cleanup "utilities/defaultStream.hpp? includes >> >> #3 and #4 in the issue are no longer applicable >> >> Tested locally with various debug/product, headers/no headers, open/closed variations, as well as Mach5 hs-tier1,tier2,tier3 tests, and Mach5 open/closed variations. >> >> >> https://bugs.openjdk.java.net/browse/JDK-8133564 >> http://cr.openjdk.java.net/~gziemski/8133564_rev1 >> >> cheers > From coleen.phillimore at oracle.com Thu May 24 15:53:03 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 24 May 2018 11:53:03 -0400 Subject: RFR 8133564: Runtime - 2nd followup to Validate JVM Command-Line Flag Arguments In-Reply-To: <95BA4C1C-3752-4991-A696-28BCCBD50A2E@oracle.com> References: <8DD96314-EFB1-4FD7-9E4A-3953241F1752@oracle.com> <6684c916-c20b-e860-a3db-c0be6523b411@oracle.com> <95BA4C1C-3752-4991-A696-28BCCBD50A2E@oracle.com> Message-ID: Very nice, thanks! Coleen On 5/24/18 11:50 AM, Gerard Ziemski wrote: > Thank you Coleen for your review. > > I fixed the indentation as well as the years in copyright header. > > http://cr.openjdk.java.net/~gziemski/8133564_rev2 > https://bugs.openjdk.java.net/browse/JDK-8133564 > > > cheers > >> On May 23, 2018, at 4:29 PM, coleen.phillimore at oracle.com wrote: >> >> >> This looks good apart from fixing the indentation of the arguments for all these changes so that they line up. >> >> - CommandLineError::print(verbose, >> + JVMFlag::printError(verbose, >> "ParallelGCThreads (" UINT32_FORMAT ") or CMSWorkQueueDrainThreshold (" >> UINTX_FORMAT ") is too large\n", >> threads, threshold); >> >> >> Thanks, >> Coleen >> >> On 5/23/18 4:25 PM, Gerard Ziemski wrote: >>> Hi all, >>> >>> Please review this follow-up fix that cleans up the following: >>> >>> #1 We move the error print method out of jvmFlagRangeList and into jvmFlag, which allows us not to include jvmFlagRangeList.hpp just for the print method in various files >>> >>> #2 We cleanup "utilities/defaultStream.hpp? includes >>> >>> #3 and #4 in the issue are no longer applicable >>> >>> Tested locally with various debug/product, headers/no headers, open/closed variations, as well as Mach5 hs-tier1,tier2,tier3 tests, and Mach5 open/closed variations. >>> >>> >>> https://bugs.openjdk.java.net/browse/JDK-8133564 >>> http://cr.openjdk.java.net/~gziemski/8133564_rev1 >>> >>> cheers From jini.george at oracle.com Thu May 24 16:16:02 2018 From: jini.george at oracle.com (Jini George) Date: Thu, 24 May 2018 21:46:02 +0530 Subject: RFR: 8203491: [TESTBUG] Port heapdump tests into java In-Reply-To: <12B5827E-5F4E-4F08-9ED3-01DCC68A7386@oracle.com> References: <650E6684-1996-4230-B9A4-CB4921419847@oracle.com> <0ab155e3-9e93-11fb-55a7-6f0ac7e41f64@oracle.com> <12B5827E-5F4E-4F08-9ED3-01DCC68A7386@oracle.com> Message-ID: <4ccebda8-f9e8-f1e9-2270-30ee6df8a77d@oracle.com> Hi Leonid, My comments inline. On 5/24/2018 12:09 AM, Leonid Mesnik wrote: > I am not sure that?JMapMetaspaceCore provides any additional coverage. > The test just fill 64M of metaspace and then send signal to dump core. > So I don't see how this test could improve coverage. > I think that idea of original test was to fill PermGen like Heap to > expand it as much as possible or it was just an analog of test > OnOOMToFileMetaspace. While current test just fill highly limited > metaspace. So number of classes seems to be not significantly larger > then for current TestJmapCore.java test. From my point of view it would > be make a sense to generate dump containing a lot of loaded classes or > some very large classes. > While current test looks pretty similar to existing TestJmapCore.java > test. ?Please let me know if you see the test scenario when such test > could be useful. From what I can make out, EatMemory with -metaspace would create a lot of loaded classes with GeneratedClassProducer. And this could provide some good testing for writeClassDumpRecords() of HeapHprofBinWriter. Let me know if you think I have overlooked something. >> * You might want to increase the timeout factor for this test. The >> test times out on my machine. >> >> > I see that test finishes in 1 minute in our lab while. I see that it > takes 30 seconds on 2CPU Oracle Linux VM with 2GB java heap. And test > just fails with JDK-8176557 when I increase heap. > How many time it takes on you machine? The timeoutFactor might be used > for untypical environment/command-line options. It took about 130 secs a couple of times. Don't know if it was an anomaly. >> * You might want to consider removing the corefile and the heapdump >> files after the test execution (in the cases where the test passes). >> > The default jtreg retain policy in make files just removes all files in > test directory for passed tests. The jtreg default test policy says > ?"If -retain is not specified, only the files from the last test > executed will be retained". > ?So it should be not a problem in most of cases. While there is no way > for user to retain core/heapdump files even if user wants to keeps them. Ok. > However if it is the common rule for sa tests to delete such artifacts > then I could remove heap/core dumps. > >> >> One suggestion is to check if /proc/sys/kernel/core_pattern has a line >> starting with '|' and print a warning that a crash reporting tool >> might be used (Something like in ClhsdbCDSCore.java). But it is just a >> suggestion and you are free to ignore it. In due course, we could >> include this test also as a part of the consolidation of SA's corefile >> testing effort (https://bugs.openjdk.java.net/browse/JDK-8202297). >> > I would prefer to left this improvement for?JDK-8202297. I think good > core dump processing/ulimit settings requires more efforts and testing > and different version of Linux. (Might be even for Non-Oracle platforms). > Also logic in test?ClhsdbCDSCore.java is slightly different. It tries to > get possible core location from hs_err file and print this hint of core > file from hs_err doesn't exists. While printing this hint if core dumps > are just completely disabled might just confuse users. > Sounds fine. Thanks, Jini. From lois.foltan at oracle.com Thu May 24 17:35:19 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 24 May 2018 13:35:19 -0400 Subject: RFR (S) JDK-8202758: SIGSEGV calling Class.forName(String,Boolean,ClassLoader) with mocked loader Message-ID: Please review this change to ensure that a given ClassLoader's unnamed Module is a valid instance of java.lang.Module with the JVM. open webrev at http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/ bug link at https://bugs.openjdk.java.net/browse/JDK-8202758 Testing: hs-tier1-3, jdk-tier1-3 in progress Thanks, Lois From coleen.phillimore at oracle.com Thu May 24 17:51:49 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 24 May 2018 13:51:49 -0400 Subject: RFR (trivial) 8203225: Fix of redefining a method that removes 1 or more lambda expressions failed to commit test cases Message-ID: <8399521c-785a-af4e-ee77-b42b19e4ee30@oracle.com> Forgot to add the tests in putback for https://bugs.openjdk.java.net/browse/JDK-8193524. open webrev at http://cr.openjdk.java.net/~coleenp/8203225.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8203225 Reran the tests.? They still pass. thanks, Coleen From lois.foltan at oracle.com Thu May 24 17:54:20 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 24 May 2018 13:54:20 -0400 Subject: RFR (trivial) 8203225: Fix of redefining a method that removes 1 or more lambda expressions failed to commit test cases In-Reply-To: <8399521c-785a-af4e-ee77-b42b19e4ee30@oracle.com> References: <8399521c-785a-af4e-ee77-b42b19e4ee30@oracle.com> Message-ID: <292ff67a-8453-8ffd-5d73-8b8f06740bdc@oracle.com> Looks good. Lois On 5/24/2018 1:51 PM, coleen.phillimore at oracle.com wrote: > Forgot to add the tests in putback for > https://bugs.openjdk.java.net/browse/JDK-8193524. > > open webrev at http://cr.openjdk.java.net/~coleenp/8203225.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8203225 > > Reran the tests.? They still pass. > thanks, > Coleen > > From leonid.mesnik at oracle.com Thu May 24 17:54:37 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Thu, 24 May 2018 10:54:37 -0700 Subject: RFR: 8203491: [TESTBUG] Port heapdump tests into java In-Reply-To: <4ccebda8-f9e8-f1e9-2270-30ee6df8a77d@oracle.com> References: <650E6684-1996-4230-B9A4-CB4921419847@oracle.com> <0ab155e3-9e93-11fb-55a7-6f0ac7e41f64@oracle.com> <12B5827E-5F4E-4F08-9ED3-01DCC68A7386@oracle.com> <4ccebda8-f9e8-f1e9-2270-30ee6df8a77d@oracle.com> Message-ID: <4C06D4F0-8BE8-4AF4-94EE-735E72B8E684@oracle.com> Hi Found new webrev here: http://cr.openjdk.java.net/~lmesnik/8203491/webrev.01/ and inc diff http://cr.openjdk.java.net/~lmesnik/8203491/webrev.01-00/ I don't know if existing 64m is enough to produce a lot of classes. However this size was used in original test so I use same in new test TestJmapCoreMetaspace.java. I fixed comments, import and timeout(set to 240) also. Leonid > On May 24, 2018, at 9:16 AM, Jini George wrote: > > Hi Leonid, > > My comments inline. > > On 5/24/2018 12:09 AM, Leonid Mesnik wrote: > >> I am not sure that JMapMetaspaceCore provides any additional coverage. The test just fill 64M of metaspace and then send signal to dump core. So I don't see how this test could improve coverage. >> I think that idea of original test was to fill PermGen like Heap to expand it as much as possible or it was just an analog of test OnOOMToFileMetaspace. While current test just fill highly limited metaspace. So number of classes seems to be not significantly larger then for current TestJmapCore.java test. From my point of view it would be make a sense to generate dump containing a lot of loaded classes or some very large classes. >> While current test looks pretty similar to existing TestJmapCore.java test. Please let me know if you see the test scenario when such test could be useful. > > From what I can make out, EatMemory with -metaspace would create a lot of loaded classes with GeneratedClassProducer. And this could provide some good testing for writeClassDumpRecords() of HeapHprofBinWriter. Let me know if you think I have overlooked something. > > >>> * You might want to increase the timeout factor for this test. The test times out on my machine. >>> >>> >> I see that test finishes in 1 minute in our lab while. I see that it takes 30 seconds on 2CPU Oracle Linux VM with 2GB java heap. And test just fails with JDK-8176557 when I increase heap. >> How many time it takes on you machine? The timeoutFactor might be used for untypical environment/command-line options. > > It took about 130 secs a couple of times. Don't know if it was an anomaly. > >>> * You might want to consider removing the corefile and the heapdump files after the test execution (in the cases where the test passes). >>> >> The default jtreg retain policy in make files just removes all files in test directory for passed tests. The jtreg default test policy says >> "If -retain is not specified, only the files from the last test executed will be retained". >> So it should be not a problem in most of cases. While there is no way for user to retain core/heapdump files even if user wants to keeps them. > > Ok. > >> However if it is the common rule for sa tests to delete such artifacts then I could remove heap/core dumps. >>> >>> One suggestion is to check if /proc/sys/kernel/core_pattern has a line starting with '|' and print a warning that a crash reporting tool might be used (Something like in ClhsdbCDSCore.java). But it is just a suggestion and you are free to ignore it. In due course, we could include this test also as a part of the consolidation of SA's corefile testing effort (https://bugs.openjdk.java.net/browse/JDK-8202297). >>> >> I would prefer to left this improvement for JDK-8202297. I think good core dump processing/ulimit settings requires more efforts and testing and different version of Linux. (Might be even for Non-Oracle platforms). >> Also logic in test ClhsdbCDSCore.java is slightly different. It tries to get possible core location from hs_err file and print this hint of core file from hs_err doesn't exists. While printing this hint if core dumps are just completely disabled might just confuse users. > Sounds fine. > > Thanks, > Jini. From zgu at redhat.com Thu May 24 17:54:40 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 24 May 2018 13:54:40 -0400 Subject: RFR (trivial) 8203225: Fix of redefining a method that removes 1 or more lambda expressions failed to commit test cases In-Reply-To: <8399521c-785a-af4e-ee77-b42b19e4ee30@oracle.com> References: <8399521c-785a-af4e-ee77-b42b19e4ee30@oracle.com> Message-ID: <9bdd88cc-b852-76c8-6e85-1b119b4bdae7@redhat.com> Good and trivial. Thanks, -Zhengyu On 05/24/2018 01:51 PM, coleen.phillimore at oracle.com wrote: > Forgot to add the tests in putback for > https://bugs.openjdk.java.net/browse/JDK-8193524. > > open webrev at http://cr.openjdk.java.net/~coleenp/8203225.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8203225 > > Reran the tests.? They still pass. > thanks, > Coleen > > From coleen.phillimore at oracle.com Thu May 24 17:57:54 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 24 May 2018 13:57:54 -0400 Subject: RFR (trivial) 8203225: Fix of redefining a method that removes 1 or more lambda expressions failed to commit test cases In-Reply-To: <9bdd88cc-b852-76c8-6e85-1b119b4bdae7@redhat.com> References: <8399521c-785a-af4e-ee77-b42b19e4ee30@oracle.com> <9bdd88cc-b852-76c8-6e85-1b119b4bdae7@redhat.com> Message-ID: <3000989a-80a1-13f0-c6a1-c6c7af78a6f6@oracle.com> Thanks Zhengyu and Lois! Coleen On 5/24/18 1:54 PM, Zhengyu Gu wrote: > Good and trivial. > > Thanks, > > -Zhengyu > > On 05/24/2018 01:51 PM, coleen.phillimore at oracle.com wrote: >> Forgot to add the tests in putback for >> https://bugs.openjdk.java.net/browse/JDK-8193524. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8203225.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8203225 >> >> Reran the tests.? They still pass. >> thanks, >> Coleen >> >> From lois.foltan at oracle.com Thu May 24 17:58:55 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 24 May 2018 13:58:55 -0400 Subject: RFR (s) JDK-8203435: Circular nested dynamic constant test needed to confirm JVMS resolution behavior In-Reply-To: <4842faca-0775-f217-a5a0-cdbd9cd90357@oracle.com> References: <4842faca-0775-f217-a5a0-cdbd9cd90357@oracle.com> Message-ID: <0a13a4e9-1df2-f60e-2b1d-4e87658e57fe@oracle.com> Although I have the required reviews, one off review suggestion has been to move this test to the open/test/jdk/java/lang/invoke/condy directory where more people can benefit.? Here is an updated webrev, test remains the same. http://cr.openjdk.java.net/~lfoltan/bug_jdk8203435.1/webrev/ Thanks, Lois On 5/21/2018 3:50 PM, Lois Foltan wrote: > Please review this change to add a nested circular dynamic constant > test to confirm the JVMS 5.4.3.6 Dynamically-Computed Constant and > Call Site Resolution behavior as discussed in > http://mail.openjdk.java.net/pipermail/valhalla-spec-observers/2018-February/000518.html. > > open webrev at: > http://cr.openjdk.java.net/~lfoltan/bug_jdk8203435/webrev/ > bug link: https://bugs.openjdk.java.net/browse/JDK-8203435 > > Testing: hs-tier1-2, jdk-tier1-3 in progress > > Thanks, > Lois From harold.seigel at oracle.com Thu May 24 18:09:11 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Thu, 24 May 2018 14:09:11 -0400 Subject: RFR (S) JDK-8202758: SIGSEGV calling Class.forName(String,Boolean,ClassLoader) with mocked loader In-Reply-To: References: Message-ID: Hi Lois, This change looks good. Thanks, Harold On 5/24/2018 1:35 PM, Lois Foltan wrote: > Please review this change to ensure that a given ClassLoader's unnamed > Module is a valid instance of java.lang.Module with the JVM. > > open webrev at http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/ > bug link at https://bugs.openjdk.java.net/browse/JDK-8202758 > > Testing: hs-tier1-3, jdk-tier1-3 in progress > > Thanks, > Lois From paul.sandoz at oracle.com Thu May 24 18:36:10 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 24 May 2018 11:36:10 -0700 Subject: RFR (s) JDK-8203435: Circular nested dynamic constant test needed to confirm JVMS resolution behavior In-Reply-To: <0a13a4e9-1df2-f60e-2b1d-4e87658e57fe@oracle.com> References: <4842faca-0775-f217-a5a0-cdbd9cd90357@oracle.com> <0a13a4e9-1df2-f60e-2b1d-4e87658e57fe@oracle.com> Message-ID: +1 Paul. > On May 24, 2018, at 10:58 AM, Lois Foltan wrote: > > Although I have the required reviews, one off review suggestion has been to move this test to the open/test/jdk/java/lang/invoke/condy directory where more people can benefit. Here is an updated webrev, test remains the same. > > http://cr.openjdk.java.net/~lfoltan/bug_jdk8203435.1/webrev/ > > Thanks, > Lois > > On 5/21/2018 3:50 PM, Lois Foltan wrote: >> Please review this change to add a nested circular dynamic constant test to confirm the JVMS 5.4.3.6 Dynamically-Computed Constant and Call Site Resolution behavior as discussed in http://mail.openjdk.java.net/pipermail/valhalla-spec-observers/2018-February/000518.html. >> >> open webrev at: http://cr.openjdk.java.net/~lfoltan/bug_jdk8203435/webrev/ >> bug link: https://bugs.openjdk.java.net/browse/JDK-8203435 >> >> Testing: hs-tier1-2, jdk-tier1-3 in progress >> >> Thanks, >> Lois > From lois.foltan at oracle.com Thu May 24 18:37:09 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 24 May 2018 14:37:09 -0400 Subject: RFR (S) JDK-8202758: SIGSEGV calling Class.forName(String,Boolean,ClassLoader) with mocked loader In-Reply-To: References: Message-ID: <706debaa-b173-b968-8876-6418d6197f18@oracle.com> Thanks Harold! Lois On 5/24/2018 2:09 PM, Harold David Seigel wrote: > Hi Lois, > > This change looks good. > > Thanks, Harold > > On 5/24/2018 1:35 PM, Lois Foltan wrote: >> Please review this change to ensure that a given ClassLoader's >> unnamed Module is a valid instance of java.lang.Module with the JVM. >> >> open webrev at >> http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/ >> bug link at https://bugs.openjdk.java.net/browse/JDK-8202758 >> >> Testing: hs-tier1-3, jdk-tier1-3 in progress >> >> Thanks, >> Lois > From lois.foltan at oracle.com Thu May 24 18:37:41 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 24 May 2018 14:37:41 -0400 Subject: RFR (s) JDK-8203435: Circular nested dynamic constant test needed to confirm JVMS resolution behavior In-Reply-To: References: <4842faca-0775-f217-a5a0-cdbd9cd90357@oracle.com> <0a13a4e9-1df2-f60e-2b1d-4e87658e57fe@oracle.com> Message-ID: Thanks Paul! Lois On 5/24/2018 2:36 PM, Paul Sandoz wrote: > +1 > > Paul. > >> On May 24, 2018, at 10:58 AM, Lois Foltan wrote: >> >> Although I have the required reviews, one off review suggestion has been to move this test to the open/test/jdk/java/lang/invoke/condy directory where more people can benefit. Here is an updated webrev, test remains the same. >> >> http://cr.openjdk.java.net/~lfoltan/bug_jdk8203435.1/webrev/ >> >> Thanks, >> Lois >> >> On 5/21/2018 3:50 PM, Lois Foltan wrote: >>> Please review this change to add a nested circular dynamic constant test to confirm the JVMS 5.4.3.6 Dynamically-Computed Constant and Call Site Resolution behavior as discussed in http://mail.openjdk.java.net/pipermail/valhalla-spec-observers/2018-February/000518.html. >>> >>> open webrev at: http://cr.openjdk.java.net/~lfoltan/bug_jdk8203435/webrev/ >>> bug link: https://bugs.openjdk.java.net/browse/JDK-8203435 >>> >>> Testing: hs-tier1-2, jdk-tier1-3 in progress >>> >>> Thanks, >>> Lois From karen.kinnear at oracle.com Thu May 24 18:37:42 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Thu, 24 May 2018 14:37:42 -0400 Subject: RFR (s) JDK-8203435: Circular nested dynamic constant test needed to confirm JVMS resolution behavior In-Reply-To: References: <4842faca-0775-f217-a5a0-cdbd9cd90357@oracle.com> <0a13a4e9-1df2-f60e-2b1d-4e87658e57fe@oracle.com> Message-ID: <007460D0-83E1-421D-A5C6-6532648DAC5F@oracle.com> Thanks Lois - looks good. thanks, Karen > On May 24, 2018, at 2:36 PM, Paul Sandoz wrote: > > +1 > > Paul. > >> On May 24, 2018, at 10:58 AM, Lois Foltan wrote: >> >> Although I have the required reviews, one off review suggestion has been to move this test to the open/test/jdk/java/lang/invoke/condy directory where more people can benefit. Here is an updated webrev, test remains the same. >> >> http://cr.openjdk.java.net/~lfoltan/bug_jdk8203435.1/webrev/ >> >> Thanks, >> Lois >> >> On 5/21/2018 3:50 PM, Lois Foltan wrote: >>> Please review this change to add a nested circular dynamic constant test to confirm the JVMS 5.4.3.6 Dynamically-Computed Constant and Call Site Resolution behavior as discussed in http://mail.openjdk.java.net/pipermail/valhalla-spec-observers/2018-February/000518.html. >>> >>> open webrev at: http://cr.openjdk.java.net/~lfoltan/bug_jdk8203435/webrev/ >>> bug link: https://bugs.openjdk.java.net/browse/JDK-8203435 >>> >>> Testing: hs-tier1-2, jdk-tier1-3 in progress >>> >>> Thanks, >>> Lois >> > From lois.foltan at oracle.com Thu May 24 18:38:41 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 24 May 2018 14:38:41 -0400 Subject: RFR (s) JDK-8203435: Circular nested dynamic constant test needed to confirm JVMS resolution behavior In-Reply-To: <007460D0-83E1-421D-A5C6-6532648DAC5F@oracle.com> References: <4842faca-0775-f217-a5a0-cdbd9cd90357@oracle.com> <0a13a4e9-1df2-f60e-2b1d-4e87658e57fe@oracle.com> <007460D0-83E1-421D-A5C6-6532648DAC5F@oracle.com> Message-ID: <7df1ca32-5ff5-163d-a420-7d54572b4783@oracle.com> Thank you for taking doing a 2nd review! Lois On 5/24/2018 2:37 PM, Karen Kinnear wrote: > Thanks Lois - looks good. > > thanks, > Karen > >> On May 24, 2018, at 2:36 PM, Paul Sandoz wrote: >> >> +1 >> >> Paul. >> >>> On May 24, 2018, at 10:58 AM, Lois Foltan wrote: >>> >>> Although I have the required reviews, one off review suggestion has been to move this test to the open/test/jdk/java/lang/invoke/condy directory where more people can benefit. Here is an updated webrev, test remains the same. >>> >>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8203435.1/webrev/ >>> >>> Thanks, >>> Lois >>> >>> On 5/21/2018 3:50 PM, Lois Foltan wrote: >>>> Please review this change to add a nested circular dynamic constant test to confirm the JVMS 5.4.3.6 Dynamically-Computed Constant and Call Site Resolution behavior as discussed in http://mail.openjdk.java.net/pipermail/valhalla-spec-observers/2018-February/000518.html. >>>> >>>> open webrev at: http://cr.openjdk.java.net/~lfoltan/bug_jdk8203435/webrev/ >>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8203435 >>>> >>>> Testing: hs-tier1-2, jdk-tier1-3 in progress >>>> >>>> Thanks, >>>> Lois From karen.kinnear at oracle.com Thu May 24 18:42:44 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Thu, 24 May 2018 14:42:44 -0400 Subject: RFR (S) JDK-8202758: SIGSEGV calling Class.forName(String,Boolean,ClassLoader) with mocked loader In-Reply-To: <706debaa-b173-b968-8876-6418d6197f18@oracle.com> References: <706debaa-b173-b968-8876-6418d6197f18@oracle.com> Message-ID: <1E57C142-A625-42C3-B819-7EA2B95DD2FA@oracle.com> Lois, Looks good. thanks, Karen > >> Hi Lois, >> >> This change looks good. >> >> Thanks, Harold >> >> On 5/24/2018 1:35 PM, Lois Foltan wrote: >>> Please review this change to ensure that a given ClassLoader's unnamed Module is a valid instance of java.lang.Module with the JVM. >>> >>> open webrev at http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/ >>> bug link at https://bugs.openjdk.java.net/browse/JDK-8202758 >>> >>> Testing: hs-tier1-3, jdk-tier1-3 in progress >>> >>> Thanks, >>> Lois >> > From lois.foltan at oracle.com Thu May 24 18:51:46 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 24 May 2018 14:51:46 -0400 Subject: RFR (S) JDK-8202758: SIGSEGV calling Class.forName(String,Boolean,ClassLoader) with mocked loader In-Reply-To: <1E57C142-A625-42C3-B819-7EA2B95DD2FA@oracle.com> References: <706debaa-b173-b968-8876-6418d6197f18@oracle.com> <1E57C142-A625-42C3-B819-7EA2B95DD2FA@oracle.com> Message-ID: <9fd9c8a7-5a9d-2e68-b9a8-1affe73e517d@oracle.com> Thank you Karen! Lois On 5/24/2018 2:42 PM, Karen Kinnear wrote: > Lois, > > Looks good. > > thanks, > Karen > >>> Hi Lois, >>> >>> This change looks good. >>> >>> Thanks, Harold >>> >>> On 5/24/2018 1:35 PM, Lois Foltan wrote: >>>> Please review this change to ensure that a given ClassLoader's unnamed Module is a valid instance of java.lang.Module with the JVM. >>>> >>>> open webrev at http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/ >>>> bug link at https://bugs.openjdk.java.net/browse/JDK-8202758 >>>> >>>> Testing: hs-tier1-3, jdk-tier1-3 in progress >>>> >>>> Thanks, >>>> Lois From coleen.phillimore at oracle.com Thu May 24 20:10:07 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 24 May 2018 16:10:07 -0400 Subject: RFR (S) JDK-8202758: SIGSEGV calling Class.forName(String,Boolean,ClassLoader) with mocked loader In-Reply-To: References: Message-ID: <6684ad1d-de96-26cc-74a7-82d831532679@oracle.com> http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/test/hotspot/jtreg/runtime/modules/ClassLoaderNoUnnamedModuleTest.java.html This doesn't seem to use InMemoryJavaCompiler but imports it. http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/src/hotspot/share/classfile/moduleEntry.cpp.udiff.html Are you sure we want to do a guarantee()?? That will look like a VM crash to the user who will report it as a bug, so they and others will still think it's our bug.? Generally when we produce an hs_err_pid.log file, people think it's a bug in the vm (except the OOM variants but sometimes that's debatable too). I took out all the TRAPS to the call stack for where this is called, but you could put them back in and throw an exception instead (NPE) which would give the user his file and line where the error occurred.?? Or else create a variant of vm_exit_during_initialization that doesn't say it's during initialization in order to exit the vm quickly with a message. Thanks, Coleen On 5/24/18 1:35 PM, Lois Foltan wrote: > Please review this change to ensure that a given ClassLoader's unnamed > Module is a valid instance of java.lang.Module with the JVM. > > open webrev at http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/ > bug link at https://bugs.openjdk.java.net/browse/JDK-8202758 > > Testing: hs-tier1-3, jdk-tier1-3 in progress > > Thanks, > Lois From lois.foltan at oracle.com Thu May 24 20:11:50 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 24 May 2018 16:11:50 -0400 Subject: RFR(M): 8199940: Print more information about class loaders in IllegalAccessErrors. In-Reply-To: <515b3476674f45eabcbb7fac5044a459@sap.com> References: <515b3476674f45eabcbb7fac5044a459@sap.com> Message-ID: On 5/16/2018 9:30 AM, Lindenmaier, Goetz wrote: > Hi, > > I finally prepared the webrev for this change. > I simplified the messages wrt. to my first intent that was found too > verbose. > > The IllegalAccessErrors in classFileParser report more verbose > information about the class loaders. > As I understand, the class loaders are the reason for the error here, > so I think information about them is useful. See also the tests 1-3. > I added reporting whether the class is abstract. > > In linkResolver, I just switch to class_loader_and_module_name() > to report a more verbose class name as Lois requested. > I removed mentioning the resolved class in the method case. > I report the modifiers of methods/fields. > But as I understand, even here the class loader can be the reason > of the Error, see tests 6-8. > > Please review. I'm happy to improve the messages further :) > http://cr.openjdk.java.net/~goetz/wr18/8199940-exMsg-IllegalAccess/01/ > > Best regards, > Goetz. Hi Goetz, Thanks for making this change, improving IAEs to include class loader names has been on our radar as well.? Here are my review comments to hopefully move this change forward. - src/hotspot/share/classfile/classFileParser.cpp Based on further discussion in https://bugs.openjdk.java.net/browse/JDK-8199940, I believe you suggested changing the calls from describe_external() to loader_name().? I would support that change or even a change to use class_loader_and_module_name(). - src/hotspot/share/interpreter/linkResolver.cpp I think all the changes in this file look great!? I did write a new test to demonstrate how an IAE looks when a type with correct module readability and package exportability tries to access a private method within another type. The IAE generated will contain the following: java.lang.IllegalAccessError: tried to access private method MySameClassLoader/m2x/p2.c2.method2()V from class MySameClassLoader/m1x/p1.c1 Once your change is in, I will follow up with an RFE to include this test with the other accessibility tests in open/test/hotspot/jtreg/runtime/modules/AccessCheck. Thanks, Lois From lois.foltan at oracle.com Thu May 24 20:31:04 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 24 May 2018 16:31:04 -0400 Subject: RFR(xs): 8203455: jcmd: VM.metaspace: print loader name for anonymous CLDs. In-Reply-To: References: Message-ID: On 5/21/2018 3:42 AM, Thomas St?fe wrote: > Hi all, > > second attempt, after discussing things with David: > > http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.01/webrev/ > > The patch is much simpler now. I use > ClassLoaderData::class_loader_name() and ::class_loader_class() which > should always work and be in accordance with planned work in > https://bugs.openjdk.java.net/browse/JDK-8202605. Hi Thomas, Looks good, thanks for using ClassLoaderData::class_loader_name()! A couple of review comments: - src/hotspot/share/memory/metaspace/printCLDMetaspaceInfoClosure.cpp line #82 - I do not prefer the use of "", but am more in favor of leaving it out and having the message simply read "CLD for instance of ".? Please consider. line #95 - would prefer instead of just .? According to the comment at line #68, I think you intended that anyways. Thanks, Lois > > Thanks, Thomas > > > On Sun, May 20, 2018 at 8:49 AM, Thomas St?fe wrote: >> Hi all, >> >> may I please have reviews for this small addition. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203455 >> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.00/webrev/ >> >> VM.metaspace show-loaders can be used to display loaders (well, really >> CLD instances). >> >> For anonymous CLDs, only "anonymous" is shown. It would be helpful to >> show to which loader these CLD are assigned. >> >> Before: >> >> "272: ClassLoaderData 0x00007f5ba0538f10 for anonymous class" >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-before.txt >> >> After patch: >> >> "268: CLD 0x00007ff0c45738f0 for , loaded by app, >> instance of jdk.internal.loader.ClassLoaders$AppClassLoader" >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-after.txt >> >> -- >> Notes: this patch has a bit more lines than I liked because I did not >> find a singly utility function in ClassloaderData which fit my >> purpose. I wanted to see name and class of the associated loader for >> both normal and unloading case. ClassloaderData::loader_name() has a >> number of shortcomings, I opened >> https://bugs.openjdk.java.net/browse/JDK-8203456 as a follow up rfe. >> >> Thank you, >> >> Thomas From igor.ignatyev at oracle.com Thu May 24 21:08:50 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 24 May 2018 14:08:50 -0700 Subject: RFR: 8203491: [TESTBUG] Port heapdump tests into java In-Reply-To: <4C06D4F0-8BE8-4AF4-94EE-735E72B8E684@oracle.com> References: <650E6684-1996-4230-B9A4-CB4921419847@oracle.com> <0ab155e3-9e93-11fb-55a7-6f0ac7e41f64@oracle.com> <12B5827E-5F4E-4F08-9ED3-01DCC68A7386@oracle.com> <4ccebda8-f9e8-f1e9-2270-30ee6df8a77d@oracle.com> <4C06D4F0-8BE8-4AF4-94EE-735E72B8E684@oracle.com> Message-ID: <9E5BA8C9-5F0A-43C6-934E-F121335BA165@oracle.com> Hi Leonid, I haven't reviewed your change fully, although I noticed that you merged several tests into one -- TestHeapDumpOnOutOfMemoryError, I don't think it's a good idea as we lose atomicity of test results/executions. could you please split TestHeapDumpOnOutOfMemoryError into independent tests? -- Igor > On May 24, 2018, at 10:54 AM, Leonid Mesnik wrote: > > Hi > > Found new webrev here: > http://cr.openjdk.java.net/~lmesnik/8203491/webrev.01/ > and inc diff > http://cr.openjdk.java.net/~lmesnik/8203491/webrev.01-00/ > > I don't know if existing 64m is enough to produce a lot of classes. However this size was used in original test so I use same in new test TestJmapCoreMetaspace.java. > > I fixed comments, import and timeout(set to 240) also. > > Leonid > >> On May 24, 2018, at 9:16 AM, Jini George wrote: >> >> Hi Leonid, >> >> My comments inline. >> >> On 5/24/2018 12:09 AM, Leonid Mesnik wrote: >> >>> I am not sure that JMapMetaspaceCore provides any additional coverage. The test just fill 64M of metaspace and then send signal to dump core. So I don't see how this test could improve coverage. >>> I think that idea of original test was to fill PermGen like Heap to expand it as much as possible or it was just an analog of test OnOOMToFileMetaspace. While current test just fill highly limited metaspace. So number of classes seems to be not significantly larger then for current TestJmapCore.java test. From my point of view it would be make a sense to generate dump containing a lot of loaded classes or some very large classes. >>> While current test looks pretty similar to existing TestJmapCore.java test. Please let me know if you see the test scenario when such test could be useful. >> >> From what I can make out, EatMemory with -metaspace would create a lot of loaded classes with GeneratedClassProducer. And this could provide some good testing for writeClassDumpRecords() of HeapHprofBinWriter. Let me know if you think I have overlooked something. >> >> >>>> * You might want to increase the timeout factor for this test. The test times out on my machine. >>>> >>>> >>> I see that test finishes in 1 minute in our lab while. I see that it takes 30 seconds on 2CPU Oracle Linux VM with 2GB java heap. And test just fails with JDK-8176557 when I increase heap. >>> How many time it takes on you machine? The timeoutFactor might be used for untypical environment/command-line options. >> >> It took about 130 secs a couple of times. Don't know if it was an anomaly. >> >>>> * You might want to consider removing the corefile and the heapdump files after the test execution (in the cases where the test passes). >>>> >>> The default jtreg retain policy in make files just removes all files in test directory for passed tests. The jtreg default test policy says >>> "If -retain is not specified, only the files from the last test executed will be retained". >>> So it should be not a problem in most of cases. While there is no way for user to retain core/heapdump files even if user wants to keeps them. >> >> Ok. >> >>> However if it is the common rule for sa tests to delete such artifacts then I could remove heap/core dumps. >>>> >>>> One suggestion is to check if /proc/sys/kernel/core_pattern has a line starting with '|' and print a warning that a crash reporting tool might be used (Something like in ClhsdbCDSCore.java). But it is just a suggestion and you are free to ignore it. In due course, we could include this test also as a part of the consolidation of SA's corefile testing effort (https://bugs.openjdk.java.net/browse/JDK-8202297). >>>> >>> I would prefer to left this improvement for JDK-8202297. I think good core dump processing/ulimit settings requires more efforts and testing and different version of Linux. (Might be even for Non-Oracle platforms). >>> Also logic in test ClhsdbCDSCore.java is slightly different. It tries to get possible core location from hs_err file and print this hint of core file from hs_err doesn't exists. While printing this hint if core dumps are just completely disabled might just confuse users. >> Sounds fine. >> >> Thanks, >> Jini. > From goetz.lindenmaier at sap.com Thu May 24 21:12:10 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 24 May 2018 21:12:10 +0000 Subject: RFR(M): 8199940: Print more information about class loaders in IllegalAccessErrors. In-Reply-To: References: <515b3476674f45eabcbb7fac5044a459@sap.com> Message-ID: <41e21804690943f2ae49093708166862@sap.com> Hi Lois, I changed the code as agreed: http://cr.openjdk.java.net/~goetz/wr18/8199940-exMsg-IllegalAccess/02/ I'm not really sure this is good: interface IAE1_A { public IAE1_D gen(); } class IAE1_B implements IAE1_A { public IAE1_D gen() { return null; } } A is loaded by "app", B by the custom loader. The message is now: "class test.IAE1_B cannot access its superinterface test.IAE1_A" if there are no names in the loaders. How should anybody know from the message that the classes were loaded by different loaders? It gives no hint at all to the cause of the problem. Or do I misunderstand the test? Best regards, Goetz. > -----Original Message----- > From: Lois Foltan > Sent: Thursday, May 24, 2018 10:12 PM > To: Lindenmaier, Goetz ; hotspot-runtime- > dev at openjdk.java.net > Subject: Re: RFR(M): 8199940: Print more information about class loaders in > IllegalAccessErrors. > > On 5/16/2018 9:30 AM, Lindenmaier, Goetz wrote: > > > Hi, > > > > I finally prepared the webrev for this change. > > I simplified the messages wrt. to my first intent that was found too > > verbose. > > > > The IllegalAccessErrors in classFileParser report more verbose > > information about the class loaders. > > As I understand, the class loaders are the reason for the error here, > > so I think information about them is useful. See also the tests 1-3. > > I added reporting whether the class is abstract. > > > > In linkResolver, I just switch to class_loader_and_module_name() > > to report a more verbose class name as Lois requested. > > I removed mentioning the resolved class in the method case. > > I report the modifiers of methods/fields. > > But as I understand, even here the class loader can be the reason > > of the Error, see tests 6-8. > > > > Please review. I'm happy to improve the messages further :) > > http://cr.openjdk.java.net/~goetz/wr18/8199940-exMsg-IllegalAccess/01/ > > > > Best regards, > > Goetz. > Hi Goetz, > > Thanks for making this change, improving IAEs to include class loader > names has been on our radar as well.? Here are my review comments to > hopefully move this change forward. > > - src/hotspot/share/classfile/classFileParser.cpp > Based on further discussion in > https://bugs.openjdk.java.net/browse/JDK-8199940, I believe you > suggested changing the calls from describe_external() to loader_name(). > I would support that change or even a change to use > class_loader_and_module_name(). > > - src/hotspot/share/interpreter/linkResolver.cpp > I think all the changes in this file look great!? I did write a new test > to demonstrate how an IAE looks when a type with correct module > readability and package exportability tries to access a private method > within another type. The IAE generated will contain the following: > > java.lang.IllegalAccessError: tried to access private method > MySameClassLoader/m2x/p2.c2.method2()V from class > MySameClassLoader/m1x/p1.c1 > > Once your change is in, I will follow up with an RFE to include this > test with the other accessibility tests in > open/test/hotspot/jtreg/runtime/modules/AccessCheck. > > Thanks, > Lois > > > > > From thomas.stuefe at gmail.com Thu May 24 21:25:01 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 24 May 2018 23:25:01 +0200 Subject: RFR(xs): 8203455: jcmd: VM.metaspace: print loader name for anonymous CLDs. In-Reply-To: References: Message-ID: Thanks for the review Lois! See comments inline. On Thu, May 24, 2018 at 10:31 PM, Lois Foltan wrote: > On 5/21/2018 3:42 AM, Thomas St?fe wrote: > >> Hi all, >> >> second attempt, after discussing things with David: >> >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.01/webrev/ >> >> The patch is much simpler now. I use >> ClassLoaderData::class_loader_name() and ::class_loader_class() which >> should always work and be in accordance with planned work in >> https://bugs.openjdk.java.net/browse/JDK-8202605. > > > Hi Thomas, > > Looks good, thanks for using ClassLoaderData::class_loader_name()! A couple > of review comments: > > - src/hotspot/share/memory/metaspace/printCLDMetaspaceInfoClosure.cpp > line #82 - I do not prefer the use of "", but am more in favor of > leaving it out and having the message simply read "CLD for instance of > ". Please consider. Sure - the majority of loaders seem not yet to have names, so this would remove a lot of "". > line #95 - would prefer instead of just . > According to the comment at line #68, I think you intended that anyways. > Fixed. > Thanks, > Lois > New webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.02/webrev/src/hotspot/share/memory/metaspace/printCLDMetaspaceInfoClosure.cpp.udiff.html Example output: anonymous class, loader has name: 5: CLD 0x00007f814c47a7d0 for , loaded by app instance of jdk.internal.loader.ClassLoaders$AppClassLoader anonymous class in 6: CLD 0x00007f814c474ae0 for , loaded by loader has name and class name: 7: CLD 0x00007f814c3afc60 for app instance of jdk.internal.loader.ClassLoaders$AppClassLoader : 8: CLD 0x00007f814c1d8570 for Thank you, Thomas > >> >> Thanks, Thomas >> >> >> On Sun, May 20, 2018 at 8:49 AM, Thomas St?fe >> wrote: >>> >>> Hi all, >>> >>> may I please have reviews for this small addition. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203455 >>> Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.00/webrev/ >>> >>> VM.metaspace show-loaders can be used to display loaders (well, really >>> CLD instances). >>> >>> For anonymous CLDs, only "anonymous" is shown. It would be helpful to >>> show to which loader these CLD are assigned. >>> >>> Before: >>> >>> "272: ClassLoaderData 0x00007f5ba0538f10 for anonymous class" >>> >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-before.txt >>> >>> After patch: >>> >>> "268: CLD 0x00007ff0c45738f0 for , loaded by app, >>> instance of jdk.internal.loader.ClassLoaders$AppClassLoader" >>> >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-after.txt >>> >>> -- >>> Notes: this patch has a bit more lines than I liked because I did not >>> find a singly utility function in ClassloaderData which fit my >>> purpose. I wanted to see name and class of the associated loader for >>> both normal and unloading case. ClassloaderData::loader_name() has a >>> number of shortcomings, I opened >>> https://bugs.openjdk.java.net/browse/JDK-8203456 as a follow up rfe. >>> >>> Thank you, >>> >>> Thomas > > From mandy.chung at oracle.com Thu May 24 22:15:06 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 24 May 2018 15:15:06 -0700 Subject: RFR(M): 8199940: Print more information about class loaders in IllegalAccessErrors. In-Reply-To: <41e21804690943f2ae49093708166862@sap.com> References: <515b3476674f45eabcbb7fac5044a459@sap.com> <41e21804690943f2ae49093708166862@sap.com> Message-ID: <0cd25c87-066f-35f8-51af-126f3976ad88@oracle.com> On 5/24/18 2:12 PM, Lindenmaier, Goetz wrote: > Hi Lois, > > I changed the code as agreed: > http://cr.openjdk.java.net/~goetz/wr18/8199940-exMsg-IllegalAccess/02/ > > I'm not really sure this is good: > > interface IAE1_A { > public IAE1_D gen(); > } > class IAE1_B implements IAE1_A { > public IAE1_D gen() { > return null; > } > } > A is loaded by "app", B by the custom loader. > > The message is now: > "class test.IAE1_B cannot access its superinterface test.IAE1_A" > if there are no names in the loaders. For this IAE, it should include the module in the message as: class test.IAE1_B (in unnamed module @0x3d04a311) cannot access its superinterface test.IAE1_A (in module m1) > How should anybody know from the message that the classes were > loaded by different loaders? It gives no hint at all to the cause of > the problem. The above is one example that does not have the loader name. Each module is defined to one loader so it can derive from the module info. >> java.lang.IllegalAccessError: tried to access private method >> MySameClassLoader/m2x/p2.c2.method2()V from class >> MySameClassLoader/m1x/p1.c1 What about other formats: tried to access private method p2.c2.method2()V (in module m2x) from class p1.c1 (in module m1x) if the loader name is highly desirable (I'm unsure yet): tried to access private method p2.c2.method2()V (in module m2x defined to MySameClassLoader) from class p1.c1 (in module m1x defined to MySameClassLoader) Mandy From david.holmes at oracle.com Thu May 24 22:45:04 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 25 May 2018 08:45:04 +1000 Subject: RFR (S) JDK-8202758: SIGSEGV calling Class.forName(String,Boolean,ClassLoader) with mocked loader In-Reply-To: <6684ad1d-de96-26cc-74a7-82d831532679@oracle.com> References: <6684ad1d-de96-26cc-74a7-82d831532679@oracle.com> Message-ID: Hi Coleen, My 2c On 25/05/2018 6:10 AM, coleen.phillimore at oracle.com wrote: > > http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/test/hotspot/jtreg/runtime/modules/ClassLoaderNoUnnamedModuleTest.java.html > > > This doesn't seem to use InMemoryJavaCompiler but imports it. > > http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/src/hotspot/share/classfile/moduleEntry.cpp.udiff.html > > > Are you sure we want to do a guarantee()?? That will look like a VM > crash to the user who will report it as a bug, so they and others will > still think it's our bug.? Generally when we produce an hs_err_pid.log > file, people think it's a bug in the vm (except the OOM variants but > sometimes that's debatable too). > > I took out all the TRAPS to the call stack for where this is called, but > you could put them back in and throw an exception instead (NPE) which > would give the user his file and line where the error occurred.?? Or > else create a variant of vm_exit_during_initialization that doesn't say > it's during initialization in order to exit the vm quickly with a message. An exception would legitimise this too much. This isn't a simple programming error, but a subversion of the language semantics by skipping the superclass constructor. A vm_exit other than at initialization is not something I'd want to encourage use of. We only have the at-initialization variant to handle the inability to throw exceptions at that time and to avoid ugly crashes. In this case, I think a guarantee failure is warranted - though I wonder whether we can be more specific about the potential cause? I agree this will come back as a bug report against the JVM rather than against Mockito/Objgenesis/whomever. Not sure what we can do there. Cheers, David > Thanks, > Coleen > > > On 5/24/18 1:35 PM, Lois Foltan wrote: >> Please review this change to ensure that a given ClassLoader's unnamed >> Module is a valid instance of java.lang.Module with the JVM. >> >> open webrev at http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/ >> bug link at https://bugs.openjdk.java.net/browse/JDK-8202758 >> >> Testing: hs-tier1-3, jdk-tier1-3 in progress >> >> Thanks, >> Lois > From david.holmes at oracle.com Thu May 24 22:47:42 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 25 May 2018 08:47:42 +1000 Subject: RFR (S) JDK-8202758: SIGSEGV calling Class.forName(String,Boolean,ClassLoader) with mocked loader In-Reply-To: References: Message-ID: <55b3e1d5-a4fc-25ff-3184-874406b5ec34@oracle.com> Hi Lois, This seems okay to me. In the test you should pass the flag that explicitly disables core dumps. And as Coleen said there's an unused import for the InMemoryJavaCompiler Thanks, David On 25/05/2018 3:35 AM, Lois Foltan wrote: > Please review this change to ensure that a given ClassLoader's unnamed > Module is a valid instance of java.lang.Module with the JVM. > > open webrev at http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/ > bug link at https://bugs.openjdk.java.net/browse/JDK-8202758 > > Testing: hs-tier1-3, jdk-tier1-3 in progress > > Thanks, > Lois From leonid.mesnik at oracle.com Thu May 24 23:07:56 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Thu, 24 May 2018 16:07:56 -0700 Subject: RFR: 8203491: [TESTBUG] Port heapdump tests into java In-Reply-To: <9E5BA8C9-5F0A-43C6-934E-F121335BA165@oracle.com> References: <650E6684-1996-4230-B9A4-CB4921419847@oracle.com> <0ab155e3-9e93-11fb-55a7-6f0ac7e41f64@oracle.com> <12B5827E-5F4E-4F08-9ED3-01DCC68A7386@oracle.com> <4ccebda8-f9e8-f1e9-2270-30ee6df8a77d@oracle.com> <4C06D4F0-8BE8-4AF4-94EE-735E72B8E684@oracle.com> <9E5BA8C9-5F0A-43C6-934E-F121335BA165@oracle.com> Message-ID: <7496D39A-4DB5-4E68-AED6-381ECBAEABD9@oracle.com> Hi Thanks for feedback. It is a good idea. Split TestHeapDumpOnOutOfMemoryError into 3 tests. The only test TestHeapDumpOnOutOfMemoryErrorInMetaspace.java requires timeout=240 and excluded from tier1 now. Also I fixed parameters for TestHeapDumpPath test and removed unneeded '@modules java.base/jdk.internal.misc' from all tests. New full webrev: http://cr.openjdk.java.net/~lmesnik/8203491/webrev.02/ inc changes http://cr.openjdk.java.net/~lmesnik/8203491/webrev.02-01/ Leonid > On May 24, 2018, at 2:08 PM, Igor Ignatyev wrote: > > Hi Leonid, > > I haven't reviewed your change fully, although I noticed that you merged several tests into one -- TestHeapDumpOnOutOfMemoryError, I don't think it's a good idea as we lose atomicity of test results/executions. could you please split TestHeapDumpOnOutOfMemoryError into independent tests? > > -- Igor > >> On May 24, 2018, at 10:54 AM, Leonid Mesnik wrote: >> >> Hi >> >> Found new webrev here: >> http://cr.openjdk.java.net/~lmesnik/8203491/webrev.01/ >> and inc diff >> http://cr.openjdk.java.net/~lmesnik/8203491/webrev.01-00/ >> >> I don't know if existing 64m is enough to produce a lot of classes. However this size was used in original test so I use same in new test TestJmapCoreMetaspace.java. >> >> I fixed comments, import and timeout(set to 240) also. >> >> Leonid >> >>> On May 24, 2018, at 9:16 AM, Jini George wrote: >>> >>> Hi Leonid, >>> >>> My comments inline. >>> >>> On 5/24/2018 12:09 AM, Leonid Mesnik wrote: >>> >>>> I am not sure that JMapMetaspaceCore provides any additional coverage. The test just fill 64M of metaspace and then send signal to dump core. So I don't see how this test could improve coverage. >>>> I think that idea of original test was to fill PermGen like Heap to expand it as much as possible or it was just an analog of test OnOOMToFileMetaspace. While current test just fill highly limited metaspace. So number of classes seems to be not significantly larger then for current TestJmapCore.java test. From my point of view it would be make a sense to generate dump containing a lot of loaded classes or some very large classes. >>>> While current test looks pretty similar to existing TestJmapCore.java test. Please let me know if you see the test scenario when such test could be useful. >>> >>> From what I can make out, EatMemory with -metaspace would create a lot of loaded classes with GeneratedClassProducer. And this could provide some good testing for writeClassDumpRecords() of HeapHprofBinWriter. Let me know if you think I have overlooked something. >>> >>> >>>>> * You might want to increase the timeout factor for this test. The test times out on my machine. >>>>> >>>>> >>>> I see that test finishes in 1 minute in our lab while. I see that it takes 30 seconds on 2CPU Oracle Linux VM with 2GB java heap. And test just fails with JDK-8176557 when I increase heap. >>>> How many time it takes on you machine? The timeoutFactor might be used for untypical environment/command-line options. >>> >>> It took about 130 secs a couple of times. Don't know if it was an anomaly. >>> >>>>> * You might want to consider removing the corefile and the heapdump files after the test execution (in the cases where the test passes). >>>>> >>>> The default jtreg retain policy in make files just removes all files in test directory for passed tests. The jtreg default test policy says >>>> "If -retain is not specified, only the files from the last test executed will be retained". >>>> So it should be not a problem in most of cases. While there is no way for user to retain core/heapdump files even if user wants to keeps them. >>> >>> Ok. >>> >>>> However if it is the common rule for sa tests to delete such artifacts then I could remove heap/core dumps. >>>>> >>>>> One suggestion is to check if /proc/sys/kernel/core_pattern has a line starting with '|' and print a warning that a crash reporting tool might be used (Something like in ClhsdbCDSCore.java). But it is just a suggestion and you are free to ignore it. In due course, we could include this test also as a part of the consolidation of SA's corefile testing effort (https://bugs.openjdk.java.net/browse/JDK-8202297). >>>>> >>>> I would prefer to left this improvement for JDK-8202297. I think good core dump processing/ulimit settings requires more efforts and testing and different version of Linux. (Might be even for Non-Oracle platforms). >>>> Also logic in test ClhsdbCDSCore.java is slightly different. It tries to get possible core location from hs_err file and print this hint of core file from hs_err doesn't exists. While printing this hint if core dumps are just completely disabled might just confuse users. >>> Sounds fine. >>> >>> Thanks, >>> Jini. >> > From coleen.phillimore at oracle.com Thu May 24 23:17:35 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 24 May 2018 19:17:35 -0400 Subject: RFR (S) JDK-8202758: SIGSEGV calling Class.forName(String,Boolean,ClassLoader) with mocked loader In-Reply-To: References: <6684ad1d-de96-26cc-74a7-82d831532679@oracle.com> Message-ID: <7acc186e-8d49-751c-63bf-f4ea958a887c@oracle.com> On 5/24/18 6:45 PM, David Holmes wrote: > Hi Coleen, > > My 2c > > On 25/05/2018 6:10 AM, coleen.phillimore at oracle.com wrote: >> >> http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/test/hotspot/jtreg/runtime/modules/ClassLoaderNoUnnamedModuleTest.java.html >> >> >> This doesn't seem to use InMemoryJavaCompiler but imports it. >> >> http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/src/hotspot/share/classfile/moduleEntry.cpp.udiff.html >> >> >> Are you sure we want to do a guarantee()?? That will look like a VM >> crash to the user who will report it as a bug, so they and others >> will still think it's our bug.? Generally when we produce an >> hs_err_pid.log file, people think it's a bug in the vm (except the >> OOM variants but sometimes that's debatable too). >> >> I took out all the TRAPS to the call stack for where this is called, >> but you could put them back in and throw an exception instead (NPE) >> which would give the user his file and line where the error >> occurred.?? Or else create a variant of vm_exit_during_initialization >> that doesn't say it's during initialization in order to exit the vm >> quickly with a message. > > An exception would legitimise this too much. This isn't a simple > programming error, but a subversion of the language semantics by > skipping the superclass constructor. We have constructs that subvert the language and get VerifyError or ClassFormatError.? I don't see the difference. > > A vm_exit other than at initialization is not something I'd want to > encourage use of. We only have the at-initialization variant to handle > the inability to throw exceptions at that time and to avoid ugly crashes. > My second choice would be to add an vm_exit variant that doesn't call VMError::report_and_die and doesn't say that the error happened during initialization.? I'd rather have the user not see an ugly crash as you call it. > In this case, I think a guarantee failure is warranted - though I > wonder whether we can be more specific about the potential cause? I > agree this will come back as a bug report against the JVM rather than > against Mockito/Objgenesis/whomever. Not sure what we can do there. > I would prefer not getting the crashes.? It tooks us several minutes to realize that one of the bugs reported was an instance of this. The guarantee message is a bit useful for finding duplicates but I'd much rather not get the reports. thanks, Coleen > Cheers, > David > >> Thanks, >> Coleen >> >> >> On 5/24/18 1:35 PM, Lois Foltan wrote: >>> Please review this change to ensure that a given ClassLoader's >>> unnamed Module is a valid instance of java.lang.Module with the JVM. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/ >>> bug link at https://bugs.openjdk.java.net/browse/JDK-8202758 >>> >>> Testing: hs-tier1-3, jdk-tier1-3 in progress >>> >>> Thanks, >>> Lois >> From david.holmes at oracle.com Fri May 25 00:27:31 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 25 May 2018 10:27:31 +1000 Subject: RFR (S) JDK-8202758: SIGSEGV calling Class.forName(String,Boolean,ClassLoader) with mocked loader In-Reply-To: <7acc186e-8d49-751c-63bf-f4ea958a887c@oracle.com> References: <6684ad1d-de96-26cc-74a7-82d831532679@oracle.com> <7acc186e-8d49-751c-63bf-f4ea958a887c@oracle.com> Message-ID: <82cfa8ae-3a66-131b-0369-366614927d32@oracle.com> On 25/05/2018 9:17 AM, coleen.phillimore at oracle.com wrote: > On 5/24/18 6:45 PM, David Holmes wrote: >> Hi Coleen, >> >> My 2c >> >> On 25/05/2018 6:10 AM, coleen.phillimore at oracle.com wrote: >>> >>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/test/hotspot/jtreg/runtime/modules/ClassLoaderNoUnnamedModuleTest.java.html >>> >>> >>> This doesn't seem to use InMemoryJavaCompiler but imports it. >>> >>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/src/hotspot/share/classfile/moduleEntry.cpp.udiff.html >>> >>> >>> Are you sure we want to do a guarantee()?? That will look like a VM >>> crash to the user who will report it as a bug, so they and others >>> will still think it's our bug.? Generally when we produce an >>> hs_err_pid.log file, people think it's a bug in the vm (except the >>> OOM variants but sometimes that's debatable too). >>> >>> I took out all the TRAPS to the call stack for where this is called, >>> but you could put them back in and throw an exception instead (NPE) >>> which would give the user his file and line where the error >>> occurred.?? Or else create a variant of vm_exit_during_initialization >>> that doesn't say it's during initialization in order to exit the vm >>> quickly with a message. >> >> An exception would legitimise this too much. This isn't a simple >> programming error, but a subversion of the language semantics by >> skipping the superclass constructor. > > We have constructs that subvert the language and get VerifyError or > ClassFormatError.? I don't see the difference. The difference is that there is a clear specification for correctly formed classfiles that we can check against and throw the appropriate exception. That is not what is happening here. Unfortunately there's nothing in the VM spec that requires the call chain of superclass constructors that the Java language requires - so we can't detect this kind of problem that way. As Alan wrote in the bug report there are numerous system classes that are intimately tied to the VM and which rely on implicit protocols between the VM and the library code. Here the framework has replaced a class and caused it to not invoke its superclass constructor - as the Java language requires - and as a result initialization that the VM depends upon is not performed. There are numerous ways that similar crashes could be provoked. >> >> A vm_exit other than at initialization is not something I'd want to >> encourage use of. We only have the at-initialization variant to handle >> the inability to throw exceptions at that time and to avoid ugly crashes. >> > > My second choice would be to add an vm_exit variant that doesn't call > VMError::report_and_die and doesn't say that the error happened during > initialization.? I'd rather have the user not see an ugly crash as you > call it. I totally oppose any notion that there can be, after initialization, any abnormal termination of the VM other than a crash! If you've subverted bytecode rules or API semantics then you get exceptions. If you load native code, or an agent that stomps on memory, you get a crash. If you inject any other foreign "code" into the VM and it triggers an internal error then you get an internal error - which is abrupt termination with a hs_err log (aka a crash). We've had similar cases where attempts have been made to replace java.lang.Object for example. >> In this case, I think a guarantee failure is warranted - though I >> wonder whether we can be more specific about the potential cause? I >> agree this will come back as a bug report against the JVM rather than >> against Mockito/Objgenesis/whomever. Not sure what we can do there. >> > > I would prefer not getting the crashes.? It tooks us several minutes to > realize that one of the bugs reported was an instance of this. The > guarantee message is a bit useful for finding duplicates but I'd much > rather not get the reports. You think people won't report a sudden termination of the JVM any way?? The end user won't understand this error regardless of how we report it. Thanks, David > thanks, > Coleen >> Cheers, >> David >> >>> Thanks, >>> Coleen >>> >>> >>> On 5/24/18 1:35 PM, Lois Foltan wrote: >>>> Please review this change to ensure that a given ClassLoader's >>>> unnamed Module is a valid instance of java.lang.Module with the JVM. >>>> >>>> open webrev at >>>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/ >>>> bug link at https://bugs.openjdk.java.net/browse/JDK-8202758 >>>> >>>> Testing: hs-tier1-3, jdk-tier1-3 in progress >>>> >>>> Thanks, >>>> Lois >>> > From david.holmes at oracle.com Fri May 25 01:03:59 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 25 May 2018 11:03:59 +1000 Subject: RFR (XS) 8202360: [TESTBUG] runtime/LoadClass/TestResize.java needs to print output when it fails In-Reply-To: References: <18a88525-650a-05bd-a37b-a9ff9162e148@oracle.com> Message-ID: <9006ff6c-c75f-391f-f461-fd842b5bcc9d@oracle.com> Hi Gerard, This has really become far too much work for what the intent of the bug report was. Sorry. On 24/05/2018 5:29 AM, Gerard Ziemski wrote: > hi David, > >> On May 21, 2018, at 5:34 PM, David Holmes wrote: >> >> Hi Gerard, >> >> >> >> On 22/05/2018 6:57 AM, Gerard Ziemski wrote: >>>> On May 20, 2018, at 8:38 PM, David Holmes wrote: >>>> Okay - that helps with the immediate problem I had. I also now realize that the failure in this test is showing is that the SD did not in fact re-size as expected! >>> Can you elaborate? Is there a reproducible case I can try to run? >> >> The test's purpose is to check that the SD resizes, so if it finds a load factor too high (which indicates it didn't resize) then the SD resizing logic has either not worked as expected, or the test is not doing what it thinks! > > The resizing is called during a safe point, which we try to trigger by issuing a System.gc() call (in TriggerResize.java). If no safe point takes place, then no resize had chance to occur, and we have test failure, which is probably what you experienced. > > I changed the test to look for safe points in the output and check the load factor only when resizing had the chance to occur. I'm losing confidence in what this test actually tests. For GC's that ignore explicit GC requests via System.gc() this test is never going to test anything e.g. using default G1 GC. > >>> I have re-done the main logic a bit to simplify it, and not have to do either String and ?+? nor StringBuffer, by taking advantage of the fact that the info we need to parse is on the 2nd output line. >> >> I was under the impression that we could see multiple lines of the form: >> >> Java dictionary (table_size=40423, classes=50002) >> >> as the table resized. If that is not the case then none of the output seems relevant to me except for this one line. ?? > > That print out comes from PrintSystemDictionaryAtExit, which happens only once at the exit, but there is more then one system dictionary, and all of them are checked. But now I understand all that, the dumping of all the output really doesn't help in establishing why the test failed. My original request was for more information to see why the load factor was too high. I thought, incorrectly, that we would see a series of changing size+classes reports until we suddenly hit a bad load factor. But that isn't the case. All that was missing in the end were the actual table_size and classes values that caused the error. Nothing more, nothing less. > https://bugs.openjdk.java.net/browse/JDK-8202360 > http://cr.openjdk.java.net/~gziemski/8202360_rev4 String already has a contains method you don't need to write your own helper. I don't see why you stopped using BufferedReader to read each line, and replaced with Scanner. With the scanner the split into lines: String[] lines = output.split("\n"); should be: String[] lines = output.split("\\\R"); to ensure no issues with platform-specific line terminators. 83 if (line.startsWith("[")) { You presumably only need to know one resize has occurred so this can be guarded with !resize. 99 // We've hit an error, so print all of the output. 100 System.out.println(output); No need to do this. And I'm not even sure what output contains any more. 107 } Once you get here the test is over, one way or another - you saw it resized and you found the Java Dictionary line. So at this point you should terminate the loop and then (if needed) finish processing the output so the target can terminate. Finally if no resize occurred you should print a message that the test trivially passed without any resizing occurring eg: if (!resized) { System.out.println("Test considered PASSED - but no resizing occurred."); } Again I'm sorry this has turned into such an issue. Thanks, David > > cheers > From coleen.phillimore at oracle.com Fri May 25 02:16:10 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 24 May 2018 22:16:10 -0400 Subject: RFR (S) JDK-8202758: SIGSEGV calling Class.forName(String,Boolean,ClassLoader) with mocked loader In-Reply-To: <82cfa8ae-3a66-131b-0369-366614927d32@oracle.com> References: <6684ad1d-de96-26cc-74a7-82d831532679@oracle.com> <7acc186e-8d49-751c-63bf-f4ea958a887c@oracle.com> <82cfa8ae-3a66-131b-0369-366614927d32@oracle.com> Message-ID: <28c3a00e-fea8-ccd0-d8e7-f3da8a7f171a@oracle.com> On 5/24/18 8:27 PM, David Holmes wrote: > On 25/05/2018 9:17 AM, coleen.phillimore at oracle.com wrote: >> On 5/24/18 6:45 PM, David Holmes wrote: >>> Hi Coleen, >>> >>> My 2c >>> >>> On 25/05/2018 6:10 AM, coleen.phillimore at oracle.com wrote: >>>> >>>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/test/hotspot/jtreg/runtime/modules/ClassLoaderNoUnnamedModuleTest.java.html >>>> >>>> >>>> This doesn't seem to use InMemoryJavaCompiler but imports it. >>>> >>>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/src/hotspot/share/classfile/moduleEntry.cpp.udiff.html >>>> >>>> >>>> Are you sure we want to do a guarantee()?? That will look like a VM >>>> crash to the user who will report it as a bug, so they and others >>>> will still think it's our bug.? Generally when we produce an >>>> hs_err_pid.log file, people think it's a bug in the vm (except the >>>> OOM variants but sometimes that's debatable too). >>>> >>>> I took out all the TRAPS to the call stack for where this is >>>> called, but you could put them back in and throw an exception >>>> instead (NPE) which would give the user his file and line where the >>>> error occurred.?? Or else create a variant of >>>> vm_exit_during_initialization that doesn't say it's during >>>> initialization in order to exit the vm quickly with a message. >>> >>> An exception would legitimise this too much. This isn't a simple >>> programming error, but a subversion of the language semantics by >>> skipping the superclass constructor. >> >> We have constructs that subvert the language and get VerifyError or >> ClassFormatError.? I don't see the difference. > > The difference is that there is a clear specification for correctly > formed classfiles that we can check against and throw the appropriate > exception. > > That is not what is happening here. Unfortunately there's nothing in > the VM spec that requires the call chain of superclass constructors > that the Java language requires - so we can't detect this kind of > problem that way. As Alan wrote in the bug report there are numerous > system classes that are intimately tied to the VM and which rely on > implicit protocols between the VM and the library code. Here the > framework has replaced a class and caused it to not invoke its > superclass constructor - as the Java language requires - and as a > result initialization that the VM depends upon is not performed. There > are numerous ways that similar crashes could be provoked. > >>> >>> A vm_exit other than at initialization is not something I'd want to >>> encourage use of. We only have the at-initialization variant to >>> handle the inability to throw exceptions at that time and to avoid >>> ugly crashes. >>> >> >> My second choice would be to add an vm_exit variant that doesn't call >> VMError::report_and_die and doesn't say that the error happened >> during initialization.? I'd rather have the user not see an ugly >> crash as you call it. > > I totally oppose any notion that there can be, after initialization, > any abnormal termination of the VM other than a crash! If you've > subverted bytecode rules or API semantics then you get exceptions. If > you load native code, or an agent that stomps on memory, you get a > crash. If you inject any other foreign "code" into the VM and it > triggers an internal error then you get an internal error - which is > abrupt termination with a hs_err log (aka a crash). > > We've had similar cases where attempts have been made to replace > java.lang.Object for example. In those examples, we call vm_exit_on_initialization though.? See code in javaClasses.cpp. > >>> In this case, I think a guarantee failure is warranted - though I >>> wonder whether we can be more specific about the potential cause? I >>> agree this will come back as a bug report against the JVM rather >>> than against Mockito/Objgenesis/whomever. Not sure what we can do >>> there. >>> >> >> I would prefer not getting the crashes.? It tooks us several minutes >> to realize that one of the bugs reported was an instance of this. The >> guarantee message is a bit useful for finding duplicates but I'd much >> rather not get the reports. > > You think people won't report a sudden termination of the JVM any > way?? The end user won't understand this error regardless of how we > report it. > I'd rather not see an hs_err file and bug report, but I appear to be the minority.?? Lois, can you add a comment to the guarantee so that someone doesn't change it to an assert (or remove it). + // Ensure that the unnamed module was correctly set when + // the class loader was constructed. Guarantee will cause a recognizable crash if the // user code has circumvented calling the ClassLoader constructor (or something like that) and that's his own // problem. + guarantee(java_lang_Module::is_instance(module), err_msg("ClassLoader %s has not been initialized correctly: the unnamed module is not set" class_loader->loader_name())); Might not need err_msg. Thanks, Coleen > Thanks, > David > >> thanks, >> Coleen >>> Cheers, >>> David >>> >>>> Thanks, >>>> Coleen >>>> >>>> >>>> On 5/24/18 1:35 PM, Lois Foltan wrote: >>>>> Please review this change to ensure that a given ClassLoader's >>>>> unnamed Module is a valid instance of java.lang.Module with the JVM. >>>>> >>>>> open webrev at >>>>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/ >>>>> bug link at https://bugs.openjdk.java.net/browse/JDK-8202758 >>>>> >>>>> Testing: hs-tier1-3, jdk-tier1-3 in progress >>>>> >>>>> Thanks, >>>>> Lois >>>> >> From david.holmes at oracle.com Fri May 25 03:01:20 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 25 May 2018 13:01:20 +1000 Subject: RFR(xs): 8203455: jcmd: VM.metaspace: print loader name for anonymous CLDs. In-Reply-To: References: Message-ID: <8446d7bf-83a0-6d76-98f4-4d29c5effc8d@oracle.com> Hi Lois, On 25/05/2018 6:31 AM, Lois Foltan wrote: > On 5/21/2018 3:42 AM, Thomas St?fe wrote: > >> Hi all, >> >> second attempt, after discussing things with David: >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.01/webrev/ >> >> >> The patch is much simpler now. I use >> ClassLoaderData::class_loader_name() and ::class_loader_class() which >> should always work and be in accordance with planned work in >> https://bugs.openjdk.java.net/browse/JDK-8202605. > > Hi Thomas, > > Looks good, thanks for using ClassLoaderData::class_loader_name()! A > couple of review comments: > > - src/hotspot/share/memory/metaspace/printCLDMetaspaceInfoClosure.cpp > line #82 - I do not prefer the use of "", but am more in favor > of leaving it out and having the message simply read "CLD for instance > of ".? Please consider. I thought this style had already been put in place: open/src/hotspot > grepsrc.sh "" ./share/classfile/javaClasses.cpp: name = ""; ./share/classfile/javaClasses.cpp: parentName = ""; ./share/classfile/javaClasses.hpp: // If a classloader has no name, it prints instead. The output ?? > line #95 - would prefer instead of just . "VM anonymous class"? (As opposed to Java language anonymous class.) Thanks, David > According to the comment at line #68, I think you intended that anyways. > > Thanks, > Lois > >> >> Thanks, Thomas >> >> >> On Sun, May 20, 2018 at 8:49 AM, Thomas St?fe >> wrote: >>> Hi all, >>> >>> may I please have reviews for this small addition. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203455 >>> Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.00/webrev/ >>> >>> >>> VM.metaspace show-loaders can be used to display loaders (well, really >>> CLD instances). >>> >>> For anonymous CLDs, only "anonymous" is shown. It would be helpful to >>> show to which loader these CLD are assigned. >>> >>> Before: >>> >>> ? "272: ClassLoaderData 0x00007f5ba0538f10 for anonymous class" >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-before.txt >>> >>> >>> After patch: >>> >>> ? "268: CLD 0x00007ff0c45738f0 for , loaded by app, >>> instance of jdk.internal.loader.ClassLoaders$AppClassLoader" >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-after.txt >>> >>> >>> -- >>> Notes: this patch has a bit more lines than I liked because I did not >>> find a singly utility function in ClassloaderData which fit my >>> purpose. I wanted to see name and class of the associated loader for >>> both normal and unloading case. ClassloaderData::loader_name() has a >>> number of shortcomings, I opened >>> https://bugs.openjdk.java.net/browse/JDK-8203456 as a follow up rfe. >>> >>> Thank you, >>> >>> Thomas > From thomas.stuefe at gmail.com Fri May 25 05:41:13 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 25 May 2018 07:41:13 +0200 Subject: RFR(xs): 8203455: jcmd: VM.metaspace: print loader name for anonymous CLDs. In-Reply-To: <8446d7bf-83a0-6d76-98f4-4d29c5effc8d@oracle.com> References: <8446d7bf-83a0-6d76-98f4-4d29c5effc8d@oracle.com> Message-ID: On Fri, May 25, 2018 at 5:01 AM, David Holmes wrote: > Hi Lois, > > On 25/05/2018 6:31 AM, Lois Foltan wrote: >> >> On 5/21/2018 3:42 AM, Thomas St?fe wrote: >> >>> Hi all, >>> >>> second attempt, after discussing things with David: >>> >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.01/webrev/ >>> >>> The patch is much simpler now. I use >>> ClassLoaderData::class_loader_name() and ::class_loader_class() which >>> should always work and be in accordance with planned work in >>> https://bugs.openjdk.java.net/browse/JDK-8202605. >> >> >> Hi Thomas, >> >> Looks good, thanks for using ClassLoaderData::class_loader_name()! A >> couple of review comments: >> >> - src/hotspot/share/memory/metaspace/printCLDMetaspaceInfoClosure.cpp >> line #82 - I do not prefer the use of "", but am more in favor of >> leaving it out and having the message simply read "CLD for instance of >> ". Please consider. > > > I thought this style had already been put in place: > > open/src/hotspot > grepsrc.sh "" > ./share/classfile/javaClasses.cpp: name = ""; > ./share/classfile/javaClasses.cpp: parentName = ""; > ./share/classfile/javaClasses.hpp: // If a classloader has no name, it > prints instead. The output > > ?? > .... I guess this does not count as review, no? :-) But seriously, I can live with both variants, so please decide among yourself which one you would want to prefer. My only interest is to reduce the number of downstream patches I am juggling with, and if possible in a timely manner (before we reach 11 feature close). Best Regards, Thomas >> line #95 - would prefer instead of just . > > > "VM anonymous class"? (As opposed to Java language anonymous class.) > > Thanks, > David > > >> According to the comment at line #68, I think you intended that anyways. >> >> Thanks, >> Lois >> >>> >>> Thanks, Thomas >>> >>> >>> On Sun, May 20, 2018 at 8:49 AM, Thomas St?fe >>> wrote: >>>> >>>> Hi all, >>>> >>>> may I please have reviews for this small addition. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203455 >>>> Webrev: >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.00/webrev/ >>>> >>>> VM.metaspace show-loaders can be used to display loaders (well, really >>>> CLD instances). >>>> >>>> For anonymous CLDs, only "anonymous" is shown. It would be helpful to >>>> show to which loader these CLD are assigned. >>>> >>>> Before: >>>> >>>> "272: ClassLoaderData 0x00007f5ba0538f10 for anonymous class" >>>> >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-before.txt >>>> >>>> After patch: >>>> >>>> "268: CLD 0x00007ff0c45738f0 for , loaded by app, >>>> instance of jdk.internal.loader.ClassLoaders$AppClassLoader" >>>> >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-after.txt >>>> >>>> -- >>>> Notes: this patch has a bit more lines than I liked because I did not >>>> find a singly utility function in ClassloaderData which fit my >>>> purpose. I wanted to see name and class of the associated loader for >>>> both normal and unloading case. ClassloaderData::loader_name() has a >>>> number of shortcomings, I opened >>>> https://bugs.openjdk.java.net/browse/JDK-8203456 as a follow up rfe. >>>> >>>> Thank you, >>>> >>>> Thomas >> >> > From jini.george at oracle.com Fri May 25 06:31:07 2018 From: jini.george at oracle.com (Jini George) Date: Fri, 25 May 2018 12:01:07 +0530 Subject: RFR: 8203491: [TESTBUG] Port heapdump tests into java In-Reply-To: <7496D39A-4DB5-4E68-AED6-381ECBAEABD9@oracle.com> References: <650E6684-1996-4230-B9A4-CB4921419847@oracle.com> <0ab155e3-9e93-11fb-55a7-6f0ac7e41f64@oracle.com> <12B5827E-5F4E-4F08-9ED3-01DCC68A7386@oracle.com> <4ccebda8-f9e8-f1e9-2270-30ee6df8a77d@oracle.com> <4C06D4F0-8BE8-4AF4-94EE-735E72B8E684@oracle.com> <9E5BA8C9-5F0A-43C6-934E-F121335BA165@oracle.com> <7496D39A-4DB5-4E68-AED6-381ECBAEABD9@oracle.com> Message-ID: <2c881c54-e19b-3a6f-c044-4777d3bbaa45@oracle.com> The SA tests look good to me. One minor comment. public static String HEAP_OOME = "heap"; public static String METASPACE_OOME = "metaspace"; These could be declared as public static final String. Thanks, Jini. On 5/25/2018 4:37 AM, Leonid Mesnik wrote: > Hi > > Thanks for feedback. It is a good idea. > Split TestHeapDumpOnOutOfMemoryError into 3 tests. The only test > TestHeapDumpOnOutOfMemoryErrorInMetaspace.java requires timeout=240 and > excluded from tier1 now. > Also I fixed parameters for TestHeapDumpPath test and removed unneeded > '@modules java.base/jdk.internal.misc' from all tests. > > New full webrev: > http://cr.openjdk.java.net/~lmesnik/8203491/webrev.02/ > inc changes > http://cr.openjdk.java.net/~lmesnik/8203491/webrev.02-01/ > > Leonid > >> On May 24, 2018, at 2:08 PM, Igor Ignatyev > > wrote: >> >> Hi Leonid, >> >> I haven't reviewed your change fully, although I noticed that you >> merged several tests into one -- TestHeapDumpOnOutOfMemoryError, I >> don't think it's a good idea as we lose atomicity of test >> results/executions. could you please split >> TestHeapDumpOnOutOfMemoryError into independent tests? >> >> -- Igor >> >>> On May 24, 2018, at 10:54 AM, Leonid Mesnik >> > wrote: >>> >>> Hi >>> >>> Found new webrev here: >>> http://cr.openjdk.java.net/~lmesnik/8203491/webrev.01/ >>> >>> and inc diff >>> http://cr.openjdk.java.net/~lmesnik/8203491/webrev.01-00/ >>> >>> >>> I don't know if existing 64m is enough to produce a lot of classes. >>> However this size was used in original test so I use same in new test >>> TestJmapCoreMetaspace.java. >>> >>> I fixed comments, import and timeout(set to 240) also. >>> >>> Leonid >>> >>>> On May 24, 2018, at 9:16 AM, Jini George >>> > wrote: >>>> >>>> Hi Leonid, >>>> >>>> My comments inline. >>>> >>>> On 5/24/2018 12:09 AM, Leonid Mesnik wrote: >>>> >>>>> I am not sure that JMapMetaspaceCore provides any additional >>>>> coverage. The test just fill 64M of metaspace and then send signal >>>>> to dump core. So I don't see how this test could improve coverage. >>>>> I think that idea of original test was to fill PermGen like Heap to >>>>> expand it as much as possible or it was just an analog of test >>>>> OnOOMToFileMetaspace. While current test just fill highly limited >>>>> metaspace. So number of classes seems to be not significantly >>>>> larger then for current TestJmapCore.java test. From my point of >>>>> view it would be make a sense to generate dump containing a lot of >>>>> loaded classes or some very large classes. >>>>> While current test looks pretty similar to existing >>>>> TestJmapCore.java test. ?Please let me know if you see the test >>>>> scenario when such test could be useful. >>>> >>>> From what I can make out, EatMemory with -metaspace would create a >>>> lot of loaded classes with GeneratedClassProducer. And this could >>>> provide some good testing for writeClassDumpRecords() of >>>> HeapHprofBinWriter. Let me know if you think I have overlooked >>>> something. >>>> >>>> >>>>>> * You might want to increase the timeout factor for this test. The >>>>>> test times out on my machine. >>>>>> >>>>>> >>>>> I see that test finishes in 1 minute in our lab while. I see that >>>>> it takes 30 seconds on 2CPU Oracle Linux VM with 2GB java heap. And >>>>> test just fails with JDK-8176557 when I increase heap. >>>>> How many time it takes on you machine? The timeoutFactor might be >>>>> used for untypical environment/command-line options. >>>> >>>> It took about 130 secs a couple of times. Don't know if it was an >>>> anomaly. >>>> >>>>>> * You might want to consider removing the corefile and the >>>>>> heapdump files after the test execution (in the cases where the >>>>>> test passes). >>>>>> >>>>> The default jtreg retain policy in make files just removes all >>>>> files in test directory for passed tests. The jtreg default test >>>>> policy says >>>>> "If -retain is not specified, only the files from the last test >>>>> executed will be retained". >>>>> So it should be not a problem in most of cases. While there is no >>>>> way for user to retain core/heapdump files even if user wants to >>>>> keeps them. >>>> >>>> Ok. >>>> >>>>> However if it is the common rule for sa tests to delete such >>>>> artifacts then I could remove heap/core dumps. >>>>>> >>>>>> One suggestion is to check if /proc/sys/kernel/core_pattern has a >>>>>> line starting with '|' and print a warning that a crash reporting >>>>>> tool might be used (Something like in ClhsdbCDSCore.java). But it >>>>>> is just a suggestion and you are free to ignore it. In due course, >>>>>> we could include this test also as a part of the consolidation of >>>>>> SA's corefile testing effort >>>>>> (https://bugs.openjdk.java.net/browse/JDK-8202297). >>>>>> >>>>> I would prefer to left this improvement for JDK-8202297. I think >>>>> good core dump processing/ulimit settings requires more efforts and >>>>> testing and different version of Linux. (Might be even for >>>>> Non-Oracle platforms). >>>>> Also logic in test ClhsdbCDSCore.java is slightly different. It >>>>> tries to get possible core location from hs_err file and print this >>>>> hint of core file from hs_err doesn't exists. While printing this >>>>> hint if core dumps are just completely disabled might just confuse >>>>> users. >>>> Sounds fine. >>>> >>>> Thanks, >>>> Jini. >>> >> > From goetz.lindenmaier at sap.com Fri May 25 06:39:00 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 25 May 2018 06:39:00 +0000 Subject: RFR(M): 8199940: Print more information about class loaders in IllegalAccessErrors. In-Reply-To: <0cd25c87-066f-35f8-51af-126f3976ad88@oracle.com> References: <515b3476674f45eabcbb7fac5044a459@sap.com> <41e21804690943f2ae49093708166862@sap.com> <0cd25c87-066f-35f8-51af-126f3976ad88@oracle.com> Message-ID: Hi Mandy, thanks for looking at my change! > > I changed the code as agreed: > > http://cr.openjdk.java.net/~goetz/wr18/8199940-exMsg-IllegalAccess/02/ > > The message is now: > > "class test.IAE1_B cannot access its superinterface test.IAE1_A" > > if there are no names in the loaders. > > For this IAE, it should include the module in the message as: > class test.IAE1_B (in unnamed module @0x3d04a311) cannot access its > superinterface test.IAE1_A (in module m1) Wouldn't it be better if we extend class_loader_and_module_name() to print module at 0x3d04a311/test.IAE1_B if we pass a flag 'verbose' or the like? I think that is the way Lois would prefer. If the modules differ, we could add some more text: "class module at 0x3d04a311/test.IAE1_B cannot access its superinterface m1/test.IAE1_A because they are in different modules" One still would not know this was caused by using different loaders in the implementation ... > > How should anybody know from the message that the classes were > > loaded by different loaders? It gives no hint at all to the cause of > > the problem. > The above is one example that does not have the loader name. > Each module is defined to one loader so it can derive from > the module info. > > >> java.lang.IllegalAccessError: tried to access private method > >> MySameClassLoader/m2x/p2.c2.method2()V from class > >> MySameClassLoader/m1x/p1.c1 > > What about other formats: > > tried to access private method p2.c2.method2()V (in module m2x) from > class p1.c1 (in module m1x) Hmm, as above, Lois wants me to use class_loader_and_module_name(). Also, in none of my examples, modules have a name or are handled by the code in any way. So it's quite misleading if the modules are reported as issue while it's in first place caused by the class loaders (which might induce modules that then cause the error.) > if the loader name is highly desirable (I'm unsure yet): Yes it is. It can be set by the developer and if he does so it should be printed. The modules here seem to be generated automatically. Maybe they should derive their names from the class loaders if the relation between them is that obvious? > tried to access private method p2.c2.method2()V (in module m2x defined > to MySameClassLoader) from class p1.c1 (in module m1x defined to > MySameClassLoader) > > Mandy From Alan.Bateman at oracle.com Fri May 25 08:05:58 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 25 May 2018 09:05:58 +0100 Subject: RFR (S) JDK-8202758: SIGSEGV calling Class.forName(String,Boolean,ClassLoader) with mocked loader In-Reply-To: References: Message-ID: <0f2e1c4e-579b-3e7e-9af4-0a6478be1f84@oracle.com> On 24/05/2018 18:35, Lois Foltan wrote: > Please review this change to ensure that a given ClassLoader's unnamed > Module is a valid instance of java.lang.Module with the JVM. > > open webrev at http://cr.openjdk.java.net/~lfoltan/bug_jdk8202758/webrev/ > bug link at https://bugs.openjdk.java.net/browse/JDK-8202758 The guarantee is unfortunate but there doesn't seem to be a better option. Hopefully the maintainers of the mocking frameworks will take notice as it is reckless to instantiate core classes like this. One suggestion for the message is to suggest that the unnamed module "is null or not an instance of Module". The mention of null might help developers to get the bug reports to the right place and not assume a JDK bug. -Alan From harold.seigel at oracle.com Fri May 25 13:28:33 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Fri, 25 May 2018 09:28:33 -0400 Subject: RFR 8202913: loader constraint message for fields specifies incorrect referring class Message-ID: Hi, Please review this change to correct and simplify the error message displayed when a loader constraint check fails when trying to access a field. The old message (for this test case): ?loader constraint violation: when resolving field "_field1" the class loader "" (instance of ClassLoaderForChildGrandFoo, child of "app" jdk.internal.loader.ClassLoaders$AppClassLoader) of the referring class, Parent, and the class loader "" (instance of ClassLoaderForParentFoo, child of "app" jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's resolved type, Foo, have different Class objects for that type The new message: loader constraint violation: when resolving field "_field1" of type Foo, the class loader "" (instance of ClassLoaderForChildGrandFoo, child of "app" jdk.internal.loader.ClassLoaders$AppClassLoader) of the current class, Child, and the class loader "" (instance of ClassLoaderForParentFoo, child of "app" jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's defining type, Parent, have different Class objects for type Foo Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8202913/webrev/ JBS Bug:? https://bugs.openjdk.java.net/browse/JDK-8202913 This fix was tested with Mach5 tiers 1 and 2 tests and builds on Linux-X64, Windows, Solaris Sparc, and Mac OS X, with tiers 3-5 tests on Linux-x64, and with JCK-11 Lang and VM tests. Thanks, Harold From karen.kinnear at oracle.com Fri May 25 13:38:38 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Fri, 25 May 2018 09:38:38 -0400 Subject: RFR 8202913: loader constraint message for fields specifies incorrect referring class In-Reply-To: References: Message-ID: Looks good Harold Many thanks, Karen > On May 25, 2018, at 9:28 AM, Harold David Seigel wrote: > > Hi, > > Please review this change to correct and simplify the error message displayed when a loader constraint check fails when trying to access a field. > > The old message (for this test case): > > loader constraint violation: when resolving field "_field1" the > class loader "" (instance of ClassLoaderForChildGrandFoo, > child of "app" jdk.internal.loader.ClassLoaders$AppClassLoader) of > the referring class, Parent, and the class loader "" > (instance of ClassLoaderForParentFoo, child of "app" > jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's > resolved type, Foo, have different Class objects for that type > > The new message: > > loader constraint violation: when resolving field "_field1" of type > Foo, the class loader "" (instance of > ClassLoaderForChildGrandFoo, child of "app" > jdk.internal.loader.ClassLoaders$AppClassLoader) of the current > class, Child, and the class loader "" (instance of > ClassLoaderForParentFoo, child of "app" > jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's > defining type, Parent, have different Class objects for type Foo > > Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8202913/webrev/ > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8202913 > > This fix was tested with Mach5 tiers 1 and 2 tests and builds on Linux-X64, Windows, Solaris Sparc, and Mac OS X, with tiers 3-5 tests on Linux-x64, and with JCK-11 Lang and VM tests. > > Thanks, Harold > From harold.seigel at oracle.com Fri May 25 13:42:27 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Fri, 25 May 2018 09:42:27 -0400 Subject: RFR 8202913: loader constraint message for fields specifies incorrect referring class In-Reply-To: References: Message-ID: <9533fc04-1712-ebac-26b4-4feac9e30535@oracle.com> Thanks Karen! Harold On 5/25/2018 9:38 AM, Karen Kinnear wrote: > Looks good Harold > > Many thanks, > Karen > >> On May 25, 2018, at 9:28 AM, Harold David Seigel wrote: >> >> Hi, >> >> Please review this change to correct and simplify the error message displayed when a loader constraint check fails when trying to access a field. >> >> The old message (for this test case): >> >> loader constraint violation: when resolving field "_field1" the >> class loader "" (instance of ClassLoaderForChildGrandFoo, >> child of "app" jdk.internal.loader.ClassLoaders$AppClassLoader) of >> the referring class, Parent, and the class loader "" >> (instance of ClassLoaderForParentFoo, child of "app" >> jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's >> resolved type, Foo, have different Class objects for that type >> >> The new message: >> >> loader constraint violation: when resolving field "_field1" of type >> Foo, the class loader "" (instance of >> ClassLoaderForChildGrandFoo, child of "app" >> jdk.internal.loader.ClassLoaders$AppClassLoader) of the current >> class, Child, and the class loader "" (instance of >> ClassLoaderForParentFoo, child of "app" >> jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's >> defining type, Parent, have different Class objects for type Foo >> >> Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8202913/webrev/ >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8202913 >> >> This fix was tested with Mach5 tiers 1 and 2 tests and builds on Linux-X64, Windows, Solaris Sparc, and Mac OS X, with tiers 3-5 tests on Linux-x64, and with JCK-11 Lang and VM tests. >> >> Thanks, Harold >> From harold.seigel at oracle.com Fri May 25 13:53:12 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Fri, 25 May 2018 09:53:12 -0400 Subject: RFR 8133564: Runtime - 2nd followup to Validate JVM Command-Line Flag Arguments In-Reply-To: References: <8DD96314-EFB1-4FD7-9E4A-3953241F1752@oracle.com> <6684c916-c20b-e860-a3db-c0be6523b411@oracle.com> <95BA4C1C-3752-4991-A696-28BCCBD50A2E@oracle.com> Message-ID: <6eded9c3-23ad-1712-bc4d-8f3dde2af6cd@oracle.com> The change looks good to me, also. Thanks, Harold On 5/24/2018 11:53 AM, coleen.phillimore at oracle.com wrote: > > Very nice, thanks! > Coleen > > On 5/24/18 11:50 AM, Gerard Ziemski wrote: >> Thank you Coleen for your review. >> >> I fixed the indentation as well as the years in copyright header. >> >> http://cr.openjdk.java.net/~gziemski/8133564_rev2 >> https://bugs.openjdk.java.net/browse/JDK-8133564 >> >> >> cheers >> >>> On May 23, 2018, at 4:29 PM, coleen.phillimore at oracle.com wrote: >>> >>> >>> This looks good apart from fixing the indentation of the arguments >>> for all these changes so that they line up. >>> >>> - CommandLineError::print(verbose, >>> + JVMFlag::printError(verbose, >>> ???????????????????????????? "ParallelGCThreads (" UINT32_FORMAT ") >>> or CMSWorkQueueDrainThreshold (" >>> ???????????????????????????? UINTX_FORMAT ") is too large\n", >>> ???????????????????????????? threads, threshold); >>> >>> >>> Thanks, >>> Coleen >>> >>> On 5/23/18 4:25 PM, Gerard Ziemski wrote: >>>> Hi all, >>>> >>>> Please review this follow-up fix that cleans up the following: >>>> >>>> #1 We move the error print method out of jvmFlagRangeList and into >>>> jvmFlag, which allows us not to include jvmFlagRangeList.hpp just >>>> for the print method in various files >>>> >>>> #2 We cleanup "utilities/defaultStream.hpp? includes >>>> >>>> #3 and #4 in the issue are no longer applicable >>>> >>>> Tested locally with various debug/product, headers/no headers, >>>> open/closed variations, as well as Mach5 hs-tier1,tier2,tier3 >>>> tests, and Mach5 open/closed variations. >>>> >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8133564 >>>> http://cr.openjdk.java.net/~gziemski/8133564_rev1 >>>> >>>> cheers > From goetz.lindenmaier at sap.com Fri May 25 14:17:42 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 25 May 2018 14:17:42 +0000 Subject: RFR 8202913: loader constraint message for fields specifies incorrect referring class In-Reply-To: References: Message-ID: <9a209744d62a4e56a5ec81ddb77afb1f@sap.com> Hi Harold, Thanks for adding this further improvement over my change. > ... AppClassLoader) for the field's > defining type, Parent, have different Class objects for type Foo I think this would read better if you replace 'type' by what is returned by Klass::external_kind(). Actually I would prefer to read ... AppClassLoader) for the class|abstractclass|interface test.Parent defining this field, have different Class objects for type Foo Could you please add the test files into some package so that you can assure class names are printed as test.Parent and not test/Parent? There is external_name() to get the proper names from classes, I don't know for symbols. Related tests are in runtime/LoaderConstraint. Please move your test there. Best regards, Goetz. > -----Original Message----- > From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- > bounces at openjdk.java.net] On Behalf Of Harold David Seigel > Sent: Freitag, 25. Mai 2018 15:29 > To: hotspot-runtime-dev at openjdk.java.net > Subject: RFR 8202913: loader constraint message for fields specifies incorrect > referring class > > Hi, > > Please review this change to correct and simplify the error message > displayed when a loader constraint check fails when trying to access a > field. > > The old message (for this test case): > > ?loader constraint violation: when resolving field "_field1" the > class loader "" (instance of ClassLoaderForChildGrandFoo, > child of "app" jdk.internal.loader.ClassLoaders$AppClassLoader) of > the referring class, Parent, and the class loader "" > (instance of ClassLoaderForParentFoo, child of "app" > jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's > resolved type, Foo, have different Class objects for that type > > The new message: > > loader constraint violation: when resolving field "_field1" of type > Foo, the class loader "" (instance of > ClassLoaderForChildGrandFoo, child of "app" > jdk.internal.loader.ClassLoaders$AppClassLoader) of the current > class, Child, and the class loader "" (instance of > ClassLoaderForParentFoo, child of "app" > jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's > defining type, Parent, have different Class objects for type Foo > > Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8202913/webrev/ > > JBS Bug:? https://bugs.openjdk.java.net/browse/JDK-8202913 > > This fix was tested with Mach5 tiers 1 and 2 tests and builds on > Linux-X64, Windows, Solaris Sparc, and Mac OS X, with tiers 3-5 tests on > Linux-x64, and with JCK-11 Lang and VM tests. > > Thanks, Harold From goetz.lindenmaier at sap.com Fri May 25 14:23:50 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 25 May 2018 14:23:50 +0000 Subject: RFR 8202913: loader constraint message for fields specifies incorrect referring class In-Reply-To: <9a209744d62a4e56a5ec81ddb77afb1f@sap.com> References: <9a209744d62a4e56a5ec81ddb77afb1f@sap.com> Message-ID: <7ed091c7498f44c1b6f2b8a4a5030c38@sap.com> Hi, > I think this would read better if you replace 'type' by what is returned by > Klass::external_kind(). correcting myself, you can just say 'class' ... interfaces don't define fields ... Best regards, Goetz. > -----Original Message----- > From: Lindenmaier, Goetz > Sent: Freitag, 25. Mai 2018 16:18 > To: 'Harold David Seigel' ; hotspot-runtime- > dev at openjdk.java.net > Subject: RE: RFR 8202913: loader constraint message for fields specifies > incorrect referring class > > Hi Harold, > > Thanks for adding this further improvement over my change. > > > ... AppClassLoader) for the field's > > defining type, Parent, have different Class objects for type Foo > I think this would read better if you replace 'type' by what is returned by > Klass::external_kind(). > > Actually I would prefer to read > ... AppClassLoader) for the class|abstractclass|interface test.Parent defining > this field, > have different Class objects for type Foo > > Could you please add the test files into some package so that you > can assure class names are printed as test.Parent and not test/Parent? > There is external_name() to get the proper names from classes, I don't > know for symbols. > Related tests are in runtime/LoaderConstraint. Please move your test > there. > > Best regards, > Goetz. > > > > > > -----Original Message----- > > From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- > > bounces at openjdk.java.net] On Behalf Of Harold David Seigel > > Sent: Freitag, 25. Mai 2018 15:29 > > To: hotspot-runtime-dev at openjdk.java.net > > Subject: RFR 8202913: loader constraint message for fields specifies > incorrect > > referring class > > > > Hi, > > > > Please review this change to correct and simplify the error message > > displayed when a loader constraint check fails when trying to access a > > field. > > > > The old message (for this test case): > > > > ?loader constraint violation: when resolving field "_field1" the > > class loader "" (instance of ClassLoaderForChildGrandFoo, > > child of "app" jdk.internal.loader.ClassLoaders$AppClassLoader) of > > the referring class, Parent, and the class loader "" > > (instance of ClassLoaderForParentFoo, child of "app" > > jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's > > resolved type, Foo, have different Class objects for that type > > > > The new message: > > > > loader constraint violation: when resolving field "_field1" of type > > Foo, the class loader "" (instance of > > ClassLoaderForChildGrandFoo, child of "app" > > jdk.internal.loader.ClassLoaders$AppClassLoader) of the current > > class, Child, and the class loader "" (instance of > > ClassLoaderForParentFoo, child of "app" > > jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's > > defining type, Parent, have different Class objects for type Foo > > > > Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8202913/webrev/ > > > > JBS Bug:? https://bugs.openjdk.java.net/browse/JDK-8202913 > > > > This fix was tested with Mach5 tiers 1 and 2 tests and builds on > > Linux-X64, Windows, Solaris Sparc, and Mac OS X, with tiers 3-5 tests on > > Linux-x64, and with JCK-11 Lang and VM tests. > > > > Thanks, Harold From leonid.mesnik at oracle.com Fri May 25 16:55:46 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Fri, 25 May 2018 09:55:46 -0700 Subject: RFR: 8203491: [TESTBUG] Port heapdump tests into java In-Reply-To: <2c881c54-e19b-3a6f-c044-4777d3bbaa45@oracle.com> References: <650E6684-1996-4230-B9A4-CB4921419847@oracle.com> <0ab155e3-9e93-11fb-55a7-6f0ac7e41f64@oracle.com> <12B5827E-5F4E-4F08-9ED3-01DCC68A7386@oracle.com> <4ccebda8-f9e8-f1e9-2270-30ee6df8a77d@oracle.com> <4C06D4F0-8BE8-4AF4-94EE-735E72B8E684@oracle.com> <9E5BA8C9-5F0A-43C6-934E-F121335BA165@oracle.com> <7496D39A-4DB5-4E68-AED6-381ECBAEABD9@oracle.com> <2c881c54-e19b-3a6f-c044-4777d3bbaa45@oracle.com> Message-ID: <65A5565D-AAF3-44C9-81EC-1BA5A52B5B55@oracle.com> Jini Thank you for review. I added final. Still waiting for review from 'R'eviewer and from runtime. Leonid > On May 24, 2018, at 11:31 PM, Jini George wrote: > > The SA tests look good to me. One minor comment. > > public static String HEAP_OOME = "heap"; > public static String METASPACE_OOME = "metaspace"; > > These could be declared as public static final String. > > Thanks, > Jini. > > > > On 5/25/2018 4:37 AM, Leonid Mesnik wrote: >> Hi >> Thanks for feedback. It is a good idea. >> Split TestHeapDumpOnOutOfMemoryError into 3 tests. The only test TestHeapDumpOnOutOfMemoryErrorInMetaspace.java requires timeout=240 and excluded from tier1 now. >> Also I fixed parameters for TestHeapDumpPath test and removed unneeded '@modules java.base/jdk.internal.misc' from all tests. >> New full webrev: >> http://cr.openjdk.java.net/~lmesnik/8203491/webrev.02/ >> inc changes >> http://cr.openjdk.java.net/~lmesnik/8203491/webrev.02-01/ >> Leonid >>> On May 24, 2018, at 2:08 PM, Igor Ignatyev > wrote: >>> >>> Hi Leonid, >>> >>> I haven't reviewed your change fully, although I noticed that you merged several tests into one -- TestHeapDumpOnOutOfMemoryError, I don't think it's a good idea as we lose atomicity of test results/executions. could you please split TestHeapDumpOnOutOfMemoryError into independent tests? >>> >>> -- Igor >>> >>>> On May 24, 2018, at 10:54 AM, Leonid Mesnik > wrote: >>>> >>>> Hi >>>> >>>> Found new webrev here: >>>> http://cr.openjdk.java.net/~lmesnik/8203491/webrev.01/ >>>> and inc diff >>>> http://cr.openjdk.java.net/~lmesnik/8203491/webrev.01-00/ >>>> >>>> I don't know if existing 64m is enough to produce a lot of classes. However this size was used in original test so I use same in new test TestJmapCoreMetaspace.java. >>>> >>>> I fixed comments, import and timeout(set to 240) also. >>>> >>>> Leonid >>>> >>>>> On May 24, 2018, at 9:16 AM, Jini George > wrote: >>>>> >>>>> Hi Leonid, >>>>> >>>>> My comments inline. >>>>> >>>>> On 5/24/2018 12:09 AM, Leonid Mesnik wrote: >>>>> >>>>>> I am not sure that JMapMetaspaceCore provides any additional coverage. The test just fill 64M of metaspace and then send signal to dump core. So I don't see how this test could improve coverage. >>>>>> I think that idea of original test was to fill PermGen like Heap to expand it as much as possible or it was just an analog of test OnOOMToFileMetaspace. While current test just fill highly limited metaspace. So number of classes seems to be not significantly larger then for current TestJmapCore.java test. From my point of view it would be make a sense to generate dump containing a lot of loaded classes or some very large classes. >>>>>> While current test looks pretty similar to existing TestJmapCore.java test. Please let me know if you see the test scenario when such test could be useful. >>>>> >>>>> From what I can make out, EatMemory with -metaspace would create a lot of loaded classes with GeneratedClassProducer. And this could provide some good testing for writeClassDumpRecords() of HeapHprofBinWriter. Let me know if you think I have overlooked something. >>>>> >>>>> >>>>>>> * You might want to increase the timeout factor for this test. The test times out on my machine. >>>>>>> >>>>>>> >>>>>> I see that test finishes in 1 minute in our lab while. I see that it takes 30 seconds on 2CPU Oracle Linux VM with 2GB java heap. And test just fails with JDK-8176557 when I increase heap. >>>>>> How many time it takes on you machine? The timeoutFactor might be used for untypical environment/command-line options. >>>>> >>>>> It took about 130 secs a couple of times. Don't know if it was an anomaly. >>>>> >>>>>>> * You might want to consider removing the corefile and the heapdump files after the test execution (in the cases where the test passes). >>>>>>> >>>>>> The default jtreg retain policy in make files just removes all files in test directory for passed tests. The jtreg default test policy says >>>>>> "If -retain is not specified, only the files from the last test executed will be retained". >>>>>> So it should be not a problem in most of cases. While there is no way for user to retain core/heapdump files even if user wants to keeps them. >>>>> >>>>> Ok. >>>>> >>>>>> However if it is the common rule for sa tests to delete such artifacts then I could remove heap/core dumps. >>>>>>> >>>>>>> One suggestion is to check if /proc/sys/kernel/core_pattern has a line starting with '|' and print a warning that a crash reporting tool might be used (Something like in ClhsdbCDSCore.java). But it is just a suggestion and you are free to ignore it. In due course, we could include this test also as a part of the consolidation of SA's corefile testing effort (https://bugs.openjdk.java.net/browse/JDK-8202297). >>>>>>> >>>>>> I would prefer to left this improvement for JDK-8202297. I think good core dump processing/ulimit settings requires more efforts and testing and different version of Linux. (Might be even for Non-Oracle platforms). >>>>>> Also logic in test ClhsdbCDSCore.java is slightly different. It tries to get possible core location from hs_err file and print this hint of core file from hs_err doesn't exists. While printing this hint if core dumps are just completely disabled might just confuse users. >>>>> Sounds fine. >>>>> >>>>> Thanks, >>>>> Jini. >>>> >>> From gerard.ziemski at oracle.com Fri May 25 18:23:36 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Fri, 25 May 2018 13:23:36 -0500 Subject: RFR (XS) 8202360: [TESTBUG] runtime/LoadClass/TestResize.java needs to print output when it fails In-Reply-To: <9006ff6c-c75f-391f-f461-fd842b5bcc9d@oracle.com> References: <18a88525-650a-05bd-a37b-a9ff9162e148@oracle.com> <9006ff6c-c75f-391f-f461-fd842b5bcc9d@oracle.com> Message-ID: > On May 24, 2018, at 8:03 PM, David Holmes wrote: > > This has really become far too much work for what the intent of the bug report was. Sorry. > > On 24/05/2018 5:29 AM, Gerard Ziemski wrote: >> hi David, >>> On May 21, 2018, at 5:34 PM, David Holmes wrote: >>> >>> >>> On 22/05/2018 6:57 AM, Gerard Ziemski wrote: >>>>> On May 20, 2018, at 8:38 PM, David Holmes wrote: >>>>> Okay - that helps with the immediate problem I had. I also now realize that the failure in this test is showing is that the SD did not in fact re-size as expected! >>>> Can you elaborate? Is there a reproducible case I can try to run? >>> >>> The test's purpose is to check that the SD resizes, so if it finds a load factor too high (which indicates it didn't resize) then the SD resizing logic has either not worked as expected, or the test is not doing what it thinks! >> The resizing is called during a safe point, which we try to trigger by issuing a System.gc() call (in TriggerResize.java). If no safe point takes place, then no resize had chance to occur, and we have test failure, which is probably what you experienced. >> I changed the test to look for safe points in the output and check the load factor only when resizing had the chance to occur. > > I'm losing confidence in what this test actually tests. For GC's that ignore explicit GC requests via System.gc() this test is never going to test anything e.g. using default G1 GC. The assumption here is that the safe point will be triggered by VM during the test on its own, and it always did in my local testing (but apparently not for you at least once). We don?t rely on System.gc() - it?s just the last resort, which btw seems to be doing the right thing on my Mac. > >>>> I have re-done the main logic a bit to simplify it, and not have to do either String and ?+? nor StringBuffer, by taking advantage of the fact that the info we need to parse is on the 2nd output line. >>> >>> I was under the impression that we could see multiple lines of the form: >>> >>> Java dictionary (table_size=40423, classes=50002) >>> >>> as the table resized. If that is not the case then none of the output seems relevant to me except for this one line. ?? >> That print out comes from PrintSystemDictionaryAtExit, which happens only once at the exit, but there is more then one system dictionary, and all of them are checked. > > But now I understand all that, the dumping of all the output really doesn't help in establishing why the test failed. My original request was for more information to see why the load factor was too high. I thought, incorrectly, that we would see a series of changing size+classes reports until we suddenly hit a bad load factor. But that isn't the case. All that was missing in the end were the actual table_size and classes values that caused the error. Nothing more, nothing less. > >> https://bugs.openjdk.java.net/browse/JDK-8202360 >> http://cr.openjdk.java.net/~gziemski/8202360_rev4 > > String already has a contains method you don't need to write your own helper. String.contains(CharSequence s) API takes CharSequence, not String. > > I don't see why you stopped using BufferedReader to read each line, and replaced with Scanner. Personally, I find it simpler to convert the entire output into String, which then can be tokenized into lines, which then can be looked at one at a time in a simple ?for loop?. This also avoids the issue of remembering to consume all the output to unblock the process. > With the scanner the split into lines: > > String[] lines = output.split("\n"); > > should be: > > String[] lines = output.split("\\\R?); > > > to ensure no issues with platform-specific line terminators. Done. > > 83 if (line.startsWith("[")) { > > You presumably only need to know one resize has occurred so this can be guarded with !resize. Done. > > 99 // We've hit an error, so print all of the output. > 100 System.out.println(output); > > No need to do this. And I'm not even sure what output contains any more. If we ever run into such issue again, it will be helpful to know which dictionary triggered it, and what it contained. Any additional info, such as did the safe point trigger and did it attempt to resize will be helpful as well. > > 107 } > > Once you get here the test is over, one way or another - you saw it resized and you found the Java Dictionary line. So at this point you should terminate the loop and then (if needed) finish processing the output so the target can terminate. There is more than one system dictionary, so we need to keep processing the output to check them all. > > Finally if no resize occurred you should print a message that the test trivially passed without any resizing occurring eg: > > if (!resized) { > System.out.println("Test considered PASSED - but no resizing occurred."); > } Done > > Again I'm sorry this has turned into such an issue. I don?t want anyone else to get tripped off by this test, so getting it right is worth it imho. Also, once finished, it will be a nice test template for anyone else who needs to process the output. Thanks for your patience! https://bugs.openjdk.java.net/browse/JDK-8202360 http://cr.openjdk.java.net/~gziemski/8202360_rev5 cheers From lois.foltan at oracle.com Fri May 25 18:44:46 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Fri, 25 May 2018 14:44:46 -0400 Subject: RFR(xs): 8203455: jcmd: VM.metaspace: print loader name for anonymous CLDs. In-Reply-To: <8446d7bf-83a0-6d76-98f4-4d29c5effc8d@oracle.com> References: <8446d7bf-83a0-6d76-98f4-4d29c5effc8d@oracle.com> Message-ID: <51d56c00-ee17-1738-0999-4ee1f5a32627@oracle.com> On 5/24/2018 11:01 PM, David Holmes wrote: > Hi Lois, > > On 25/05/2018 6:31 AM, Lois Foltan wrote: >> On 5/21/2018 3:42 AM, Thomas St?fe wrote: >> >>> Hi all, >>> >>> second attempt, after discussing things with David: >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.01/webrev/ >>> >>> >>> The patch is much simpler now. I use >>> ClassLoaderData::class_loader_name() and ::class_loader_class() which >>> should always work and be in accordance with planned work in >>> https://bugs.openjdk.java.net/browse/JDK-8202605. >> >> Hi Thomas, >> >> Looks good, thanks for using ClassLoaderData::class_loader_name()! A >> couple of review comments: >> >> - src/hotspot/share/memory/metaspace/printCLDMetaspaceInfoClosure.cpp >> line #82 - I do not prefer the use of "", but am more in >> favor of leaving it out and having the message simply read "CLD for >> instance of ".? Please consider. > > I thought this style had already been put in place: > > open/src/hotspot > grepsrc.sh "" > ./share/classfile/javaClasses.cpp:??? name = ""; > ./share/classfile/javaClasses.cpp:??????? parentName = ""; > ./share/classfile/javaClasses.hpp:? // If a classloader has no name, > it prints instead. The output > > ?? Hi David, After discussing this with Karen on Tuesday, going forward, if the ClassLoader's name has not been set, we would prefer to leave out "" in favor of simply "instance of MyClassLoader" for example.? Hopefully we can renegotiate this change as well for java_lang_ClassLoader::describe_external(). Thanks, Lois > >> line #95 - would prefer instead of just . > > "VM anonymous class"? (As opposed to Java language anonymous class.) > > Thanks, > David > >> According to the comment at line #68, I think you intended that anyways. >> >> Thanks, >> Lois >> >>> >>> Thanks, Thomas >>> >>> >>> On Sun, May 20, 2018 at 8:49 AM, Thomas St?fe >>> wrote: >>>> Hi all, >>>> >>>> may I please have reviews for this small addition. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203455 >>>> Webrev: >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.00/webrev/ >>>> >>>> >>>> VM.metaspace show-loaders can be used to display loaders (well, really >>>> CLD instances). >>>> >>>> For anonymous CLDs, only "anonymous" is shown. It would be helpful to >>>> show to which loader these CLD are assigned. >>>> >>>> Before: >>>> >>>> ? "272: ClassLoaderData 0x00007f5ba0538f10 for anonymous class" >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-before.txt >>>> >>>> >>>> After patch: >>>> >>>> ? "268: CLD 0x00007ff0c45738f0 for , loaded by app, >>>> instance of jdk.internal.loader.ClassLoaders$AppClassLoader" >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-after.txt >>>> >>>> >>>> -- >>>> Notes: this patch has a bit more lines than I liked because I did not >>>> find a singly utility function in ClassloaderData which fit my >>>> purpose. I wanted to see name and class of the associated loader for >>>> both normal and unloading case. ClassloaderData::loader_name() has a >>>> number of shortcomings, I opened >>>> https://bugs.openjdk.java.net/browse/JDK-8203456 as a follow up rfe. >>>> >>>> Thank you, >>>> >>>> Thomas >> From lois.foltan at oracle.com Fri May 25 18:52:30 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Fri, 25 May 2018 14:52:30 -0400 Subject: RFR(xs): 8203455: jcmd: VM.metaspace: print loader name for anonymous CLDs. In-Reply-To: References: Message-ID: On 5/24/2018 5:25 PM, Thomas St?fe wrote: > Thanks for the review Lois! > > See comments inline. > > On Thu, May 24, 2018 at 10:31 PM, Lois Foltan wrote: >> On 5/21/2018 3:42 AM, Thomas St?fe wrote: >> >>> Hi all, >>> >>> second attempt, after discussing things with David: >>> >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.01/webrev/ >>> >>> The patch is much simpler now. I use >>> ClassLoaderData::class_loader_name() and ::class_loader_class() which >>> should always work and be in accordance with planned work in >>> https://bugs.openjdk.java.net/browse/JDK-8202605. >> >> Hi Thomas, >> >> Looks good, thanks for using ClassLoaderData::class_loader_name()! A couple >> of review comments: >> >> - src/hotspot/share/memory/metaspace/printCLDMetaspaceInfoClosure.cpp >> line #82 - I do not prefer the use of "", but am more in favor of >> leaving it out and having the message simply read "CLD for instance of >> ". Please consider. > Sure - the majority of loaders seem not yet to have names, so this > would remove a lot of "". > >> line #95 - would prefer instead of just . >> According to the comment at line #68, I think you intended that anyways. >> > Fixed. > >> Thanks, >> Lois >> > New webrev: > > http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.02/webrev/src/hotspot/share/memory/metaspace/printCLDMetaspaceInfoClosure.cpp.udiff.html Thomas, Looks good, thank you for making those changes! Lois > > Example output: > > anonymous class, loader has name: > 5: CLD 0x00007f814c47a7d0 for , loaded by app > instance of jdk.internal.loader.ClassLoaders$AppClassLoader > > anonymous class in > 6: CLD 0x00007f814c474ae0 for , loaded by > > loader has name and class name: > 7: CLD 0x00007f814c3afc60 for app instance of > jdk.internal.loader.ClassLoaders$AppClassLoader > > : > 8: CLD 0x00007f814c1d8570 for > > Thank you, > > Thomas > >>> Thanks, Thomas >>> >>> >>> On Sun, May 20, 2018 at 8:49 AM, Thomas St?fe >>> wrote: >>>> Hi all, >>>> >>>> may I please have reviews for this small addition. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203455 >>>> Webrev: >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.00/webrev/ >>>> >>>> VM.metaspace show-loaders can be used to display loaders (well, really >>>> CLD instances). >>>> >>>> For anonymous CLDs, only "anonymous" is shown. It would be helpful to >>>> show to which loader these CLD are assigned. >>>> >>>> Before: >>>> >>>> "272: ClassLoaderData 0x00007f5ba0538f10 for anonymous class" >>>> >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-before.txt >>>> >>>> After patch: >>>> >>>> "268: CLD 0x00007ff0c45738f0 for , loaded by app, >>>> instance of jdk.internal.loader.ClassLoaders$AppClassLoader" >>>> >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-after.txt >>>> >>>> -- >>>> Notes: this patch has a bit more lines than I liked because I did not >>>> find a singly utility function in ClassloaderData which fit my >>>> purpose. I wanted to see name and class of the associated loader for >>>> both normal and unloading case. ClassloaderData::loader_name() has a >>>> number of shortcomings, I opened >>>> https://bugs.openjdk.java.net/browse/JDK-8203456 as a follow up rfe. >>>> >>>> Thank you, >>>> >>>> Thomas >> From gerard.ziemski at oracle.com Fri May 25 20:29:52 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Fri, 25 May 2018 15:29:52 -0500 Subject: RFR 8133564: Runtime - 2nd followup to Validate JVM Command-Line Flag Arguments In-Reply-To: <6eded9c3-23ad-1712-bc4d-8f3dde2af6cd@oracle.com> References: <8DD96314-EFB1-4FD7-9E4A-3953241F1752@oracle.com> <6684c916-c20b-e860-a3db-c0be6523b411@oracle.com> <95BA4C1C-3752-4991-A696-28BCCBD50A2E@oracle.com> <6eded9c3-23ad-1712-bc4d-8f3dde2af6cd@oracle.com> Message-ID: <51A91A2E-4752-4469-AA7A-41A047A4A65A@oracle.com> Thank you Harold! > On May 25, 2018, at 8:53 AM, Harold David Seigel wrote: > > The change looks good to me, also. > > Thanks, Harold > > > On 5/24/2018 11:53 AM, coleen.phillimore at oracle.com wrote: >> >> Very nice, thanks! >> Coleen >> >> On 5/24/18 11:50 AM, Gerard Ziemski wrote: >>> Thank you Coleen for your review. >>> >>> I fixed the indentation as well as the years in copyright header. >>> >>> http://cr.openjdk.java.net/~gziemski/8133564_rev2 >>> https://bugs.openjdk.java.net/browse/JDK-8133564 >>> >>> >>> cheers >>> >>>> On May 23, 2018, at 4:29 PM, coleen.phillimore at oracle.com wrote: >>>> >>>> >>>> This looks good apart from fixing the indentation of the arguments for all these changes so that they line up. >>>> >>>> - CommandLineError::print(verbose, >>>> + JVMFlag::printError(verbose, >>>> "ParallelGCThreads (" UINT32_FORMAT ") or CMSWorkQueueDrainThreshold (" >>>> UINTX_FORMAT ") is too large\n", >>>> threads, threshold); >>>> >>>> >>>> Thanks, >>>> Coleen >>>> >>>> On 5/23/18 4:25 PM, Gerard Ziemski wrote: >>>>> Hi all, >>>>> >>>>> Please review this follow-up fix that cleans up the following: >>>>> >>>>> #1 We move the error print method out of jvmFlagRangeList and into jvmFlag, which allows us not to include jvmFlagRangeList.hpp just for the print method in various files >>>>> >>>>> #2 We cleanup "utilities/defaultStream.hpp? includes >>>>> >>>>> #3 and #4 in the issue are no longer applicable >>>>> >>>>> Tested locally with various debug/product, headers/no headers, open/closed variations, as well as Mach5 hs-tier1,tier2,tier3 tests, and Mach5 open/closed variations. >>>>> >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8133564 >>>>> http://cr.openjdk.java.net/~gziemski/8133564_rev1 >>>>> >>>>> cheers >> > From mikhailo.seledtsov at oracle.com Fri May 25 21:26:39 2018 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Fri, 25 May 2018 14:26:39 -0700 Subject: RFR(L): 8199256: [TESTBUG] Open source VM testbase runtime tests Message-ID: <5B087F8F.2010407@oracle.com> Please review this change that will open source VM Testbase Runtime tests. This group includes tests covering a variety of runtime functional areas: arraycopy, barrier, jni, threads, vtable and contented. These tests have been used internally for a while, and are now being open sourced. Since this is not a creation of new tests, we would like to keep the changes during this review to a minimum required for open sourcing these tests, such as major issues and integration blockers. If you have other feedback regarding improvements to these tests, please file RFE(s), which will be addressed later in order of priority. Here is what was done for this change: 1. Moved the tests to OpenJDK repository preserving relative directory structure. 2. Updated Copyright statements accordingly. 3. Updated "@library" statements accordingly. 4. Updated TEST.groups and Problem List 5. Updated HotSpot test make file JBS: https://bugs.openjdk.java.net/browse/JDK-8199256 Webrev: http://cr.openjdk.java.net/~mseledtsov/8199256.01.open/ Testing: 1. Ran the following tests on open-only repository and build, using "make run-test" (Linux-x64) vmTestbase_vm_runtime All PASS 2. Automated multip-platform test system (usual 4 platforms): - vmTestbase_vm_runtime - hs-tier{1,2} In progress Thank you, Misha From erik.joelsson at oracle.com Fri May 25 21:33:47 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 25 May 2018 14:33:47 -0700 Subject: RFR(L): 8199256: [TESTBUG] Open source VM testbase runtime tests In-Reply-To: <5B087F8F.2010407@oracle.com> References: <5B087F8F.2010407@oracle.com> Message-ID: <8ff47df7-9102-c0ba-d0df-75332d571178@oracle.com> Build changes look good. /Erik On 2018-05-25 14:26, Mikhailo Seledtsov wrote: > Please review this change that will open source VM Testbase Runtime > tests. This group includes tests covering a variety of runtime > functional areas: arraycopy, barrier, jni, threads, vtable and contented. > > These tests have been used internally for a while, and are now being > open sourced. Since this is not a creation of new tests, we would like > to keep the changes during this review to a minimum required for open > sourcing these tests, such as major issues and integration blockers. > If you have other feedback regarding improvements to these tests, > please file RFE(s), which will be addressed later in order of priority. > > Here is what was done for this change: > ? 1. Moved the tests to OpenJDK repository preserving relative > directory structure. > ? 2. Updated Copyright statements accordingly. > ? 3. Updated "@library" statements accordingly. > ? 4. Updated TEST.groups and Problem List > ? 5. Updated HotSpot test make file > > ? JBS:??? https://bugs.openjdk.java.net/browse/JDK-8199256 > ? Webrev: http://cr.openjdk.java.net/~mseledtsov/8199256.01.open/ > > ? Testing: > ????? 1. Ran the following tests on open-only repository and build, > using "make run-test" > ???????? (Linux-x64) > ???????? vmTestbase_vm_runtime > ???????? All PASS > > ????? 2. Automated multip-platform test system (usual 4 platforms): > ???????? - vmTestbase_vm_runtime > ???????? - hs-tier{1,2} > ???????? In progress > > Thank you, > Misha From david.holmes at oracle.com Sat May 26 00:02:33 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 26 May 2018 10:02:33 +1000 Subject: RFR (XS) 8202360: [TESTBUG] runtime/LoadClass/TestResize.java needs to print output when it fails In-Reply-To: References: <18a88525-650a-05bd-a37b-a9ff9162e148@oracle.com> <9006ff6c-c75f-391f-f461-fd842b5bcc9d@oracle.com> Message-ID: <8dd50f2c-756a-e538-30eb-c3e25287fba6@oracle.com> On 26/05/2018 4:23 AM, Gerard Ziemski wrote: > String.contains(CharSequence s) API takes CharSequence, not String. String implements CharSequence David From david.holmes at oracle.com Sat May 26 00:40:01 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 26 May 2018 10:40:01 +1000 Subject: RFR 8202913: loader constraint message for fields specifies incorrect referring class In-Reply-To: References: Message-ID: <621e524d-f990-262f-143f-cbc095da4acc@oracle.com> Hi Harold, I don't think this was the minimal fix needed to address the missing reference to the Child class, but these messages are so complicated and unreadable now that I'm beyond trying to argue for their form. I agree with Goetz about moving the tests to where the existing ones are. Otherwise ok. Thanks, David On 25/05/2018 11:28 PM, Harold David Seigel wrote: > Hi, > > Please review this change to correct and simplify the error message > displayed when a loader constraint check fails when trying to access a > field. > > The old message (for this test case): > > ??? ?loader constraint violation: when resolving field "_field1" the > ?? class loader "" (instance of ClassLoaderForChildGrandFoo, > ?? child of "app" jdk.internal.loader.ClassLoaders$AppClassLoader) of > ?? the referring class, Parent, and the class loader "" > ?? (instance of ClassLoaderForParentFoo, child of "app" > ?? jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's > ?? resolved type, Foo, have different Class objects for that type > > The new message: > > ?? loader constraint violation: when resolving field "_field1" of type > ?? Foo, the class loader "" (instance of > ?? ClassLoaderForChildGrandFoo, child of "app" > ?? jdk.internal.loader.ClassLoaders$AppClassLoader) of the current > ?? class, Child, and the class loader "" (instance of > ?? ClassLoaderForParentFoo, child of "app" > ?? jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's > ?? defining type, Parent, have different Class objects for type Foo > > Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8202913/webrev/ > > JBS Bug:? https://bugs.openjdk.java.net/browse/JDK-8202913 > > This fix was tested with Mach5 tiers 1 and 2 tests and builds on > Linux-X64, Windows, Solaris Sparc, and Mac OS X, with tiers 3-5 tests on > Linux-x64, and with JCK-11 Lang and VM tests. > > Thanks, Harold > From david.holmes at oracle.com Sat May 26 00:57:08 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 26 May 2018 10:57:08 +1000 Subject: RFR 8202913: loader constraint message for fields specifies incorrect referring class In-Reply-To: <621e524d-f990-262f-143f-cbc095da4acc@oracle.com> References: <621e524d-f990-262f-143f-cbc095da4acc@oracle.com> Message-ID: On 26/05/2018 10:40 AM, David Holmes wrote: > Hi Harold, > > I don't think this was the minimal fix needed to address the missing > reference to the Child class, No I take that back. I messed up the original message in two ways and this addresses both of them. It's also more consistent with the method version. It also avoids confusing use of "referring class", by which I meant the current class that holds the putfield/getfield, not the REFC type of the putfield/getfield (which I just confused myself with!) Thanks, David ----- > but these messages are so complicated and > unreadable now that I'm beyond trying to argue for their form. > > I agree with Goetz about moving the tests to where the existing ones are. > > Otherwise ok. > > Thanks, > David > > On 25/05/2018 11:28 PM, Harold David Seigel wrote: >> Hi, >> >> Please review this change to correct and simplify the error message >> displayed when a loader constraint check fails when trying to access a >> field. >> >> The old message (for this test case): >> >> ???? ?loader constraint violation: when resolving field "_field1" the >> ??? class loader "" (instance of ClassLoaderForChildGrandFoo, >> ??? child of "app" jdk.internal.loader.ClassLoaders$AppClassLoader) of >> ??? the referring class, Parent, and the class loader "" >> ??? (instance of ClassLoaderForParentFoo, child of "app" >> ??? jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's >> ??? resolved type, Foo, have different Class objects for that type >> >> The new message: >> >> ??? loader constraint violation: when resolving field "_field1" of type >> ??? Foo, the class loader "" (instance of >> ??? ClassLoaderForChildGrandFoo, child of "app" >> ??? jdk.internal.loader.ClassLoaders$AppClassLoader) of the current >> ??? class, Child, and the class loader "" (instance of >> ??? ClassLoaderForParentFoo, child of "app" >> ??? jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's >> ??? defining type, Parent, have different Class objects for type Foo >> >> Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8202913/webrev/ >> >> JBS Bug:? https://bugs.openjdk.java.net/browse/JDK-8202913 >> >> This fix was tested with Mach5 tiers 1 and 2 tests and builds on >> Linux-X64, Windows, Solaris Sparc, and Mac OS X, with tiers 3-5 tests >> on Linux-x64, and with JCK-11 Lang and VM tests. >> >> Thanks, Harold >> From thomas.stuefe at gmail.com Sat May 26 06:19:07 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 26 May 2018 08:19:07 +0200 Subject: RFR(xs): 8203455: jcmd: VM.metaspace: print loader name for anonymous CLDs. In-Reply-To: References: Message-ID: Thanks Lois! Could I have a second reviewer please? Best Regards, Thomas On Fri, May 25, 2018 at 8:52 PM, Lois Foltan wrote: > On 5/24/2018 5:25 PM, Thomas St?fe wrote: > >> Thanks for the review Lois! >> >> See comments inline. >> >> On Thu, May 24, 2018 at 10:31 PM, Lois Foltan >> wrote: >>> >>> On 5/21/2018 3:42 AM, Thomas St?fe wrote: >>> >>>> Hi all, >>>> >>>> second attempt, after discussing things with David: >>>> >>>> >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.01/webrev/ >>>> >>>> The patch is much simpler now. I use >>>> ClassLoaderData::class_loader_name() and ::class_loader_class() which >>>> should always work and be in accordance with planned work in >>>> https://bugs.openjdk.java.net/browse/JDK-8202605. >>> >>> >>> Hi Thomas, >>> >>> Looks good, thanks for using ClassLoaderData::class_loader_name()! A >>> couple >>> of review comments: >>> >>> - src/hotspot/share/memory/metaspace/printCLDMetaspaceInfoClosure.cpp >>> line #82 - I do not prefer the use of "", but am more in favor >>> of >>> leaving it out and having the message simply read "CLD for instance of >>> ". Please consider. >> >> Sure - the majority of loaders seem not yet to have names, so this >> would remove a lot of "". >> >>> line #95 - would prefer instead of just . >>> According to the comment at line #68, I think you intended that anyways. >>> >> Fixed. >> >>> Thanks, >>> Lois >>> >> New webrev: >> >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.02/webrev/src/hotspot/share/memory/metaspace/printCLDMetaspaceInfoClosure.cpp.udiff.html > > > Thomas, > Looks good, thank you for making those changes! > Lois > > >> >> Example output: >> >> anonymous class, loader has name: >> 5: CLD 0x00007f814c47a7d0 for , loaded by app >> instance of jdk.internal.loader.ClassLoaders$AppClassLoader >> >> anonymous class in >> 6: CLD 0x00007f814c474ae0 for , loaded by >> >> loader has name and class name: >> 7: CLD 0x00007f814c3afc60 for app instance of >> jdk.internal.loader.ClassLoaders$AppClassLoader >> >> : >> 8: CLD 0x00007f814c1d8570 for >> >> Thank you, >> >> Thomas >> >>>> Thanks, Thomas >>>> >>>> >>>> On Sun, May 20, 2018 at 8:49 AM, Thomas St?fe >>>> wrote: >>>>> >>>>> Hi all, >>>>> >>>>> may I please have reviews for this small addition. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203455 >>>>> Webrev: >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/webrev.00/webrev/ >>>>> >>>>> VM.metaspace show-loaders can be used to display loaders (well, really >>>>> CLD instances). >>>>> >>>>> For anonymous CLDs, only "anonymous" is shown. It would be helpful to >>>>> show to which loader these CLD are assigned. >>>>> >>>>> Before: >>>>> >>>>> "272: ClassLoaderData 0x00007f5ba0538f10 for anonymous class" >>>>> >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-before.txt >>>>> >>>>> After patch: >>>>> >>>>> "268: CLD 0x00007ff0c45738f0 for , loaded by app, >>>>> instance of jdk.internal.loader.ClassLoaders$AppClassLoader" >>>>> >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace-display-loader-name-for-anonymous-cld/example-after.txt >>>>> >>>>> -- >>>>> Notes: this patch has a bit more lines than I liked because I did not >>>>> find a singly utility function in ClassloaderData which fit my >>>>> purpose. I wanted to see name and class of the associated loader for >>>>> both normal and unloading case. ClassloaderData::loader_name() has a >>>>> number of shortcomings, I opened >>>>> https://bugs.openjdk.java.net/browse/JDK-8203456 as a follow up rfe. >>>>> >>>>> Thank you, >>>>> >>>>> Thomas >>> >>> > From thomas.stuefe at gmail.com Sun May 27 08:26:48 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sun, 27 May 2018 10:26:48 +0200 Subject: RFR(s): 8203219: VM.metaspace jcmd should optionally show loaded classes for loaders In-Reply-To: References: Message-ID: Ping. On Sun, May 20, 2018 at 8:58 AM, Thomas St?fe wrote: > Hi all, > > can I have reviews for this small addition to the VM.metaspace jcmd, > which adds the ability to list classes loaded by each class loader? > > Thank you! > > bug: https://bugs.openjdk.java.net/browse/JDK-8203219 > webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/webrev.00/webrev/index.html > > Example output: > > http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/example.txt > > Kind Regards, Thomas From thomas.stuefe at gmail.com Mon May 28 04:50:34 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 28 May 2018 06:50:34 +0200 Subject: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details In-Reply-To: References: Message-ID: All tests passed on jdk-submit. Anyone interested in a review? More output examples for jcmd VM.classloaders : Spring framework, basic tree: http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_short.txt Spring framework, including all classes: http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_long.txt ... Thomas On Wed, May 23, 2018 at 2:46 PM, Thomas St?fe wrote: > Dear all, > > (not sure if this would be a serviceability or runtime rfe, so sorry > for crossposting) > > may I please have feedback/reviews for this small enhancement. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203682 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.00/webrev/ > > This adds a new command to jcmd, "VM.classloaders". It complements the > existing command "VM.classloader_stats". > > This command, in its simplest form, prints the class loader tree. In > addition to that, it optionally prints out loaded classes (both > non-anonymous and anonymous) and various classloader specific > information. > > Examples: > > http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example.txt > http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-classes.txt > http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-reflection-and-noinflation.txt > > > Thanks and Best Regards, > > Thomas From david.holmes at oracle.com Mon May 28 05:23:47 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 28 May 2018 15:23:47 +1000 Subject: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details In-Reply-To: References: Message-ID: Hi Thomas, I had a look at this and overall seems okay - the output looks good (though I'm not sure how useful the hex values are?). Can't comment too much on the pretty-printing details - the proof is in the output there. (Though have to wonder whether there is any existing tree/graph printing logic somewhere in the OpenJDK code?) Two queries: 1. Have we previously established whether a CSR request is needed for a new Dcmd? (My initial feeling is that it is.) 2. Is ClassLoaderHierarchyVMOperation a safepoint VM-op? I would expect it needs to be to be able to walk the CLD hierarchy, unless that is already guaranteed to be safely walkable. Either way a comment clearly stating that would be useful I think. Related to #2, is it really possible to encounter a CLD in the process of being unloaded? Wouldn't that happen at a safepoint? Thanks, David On 28/05/2018 2:50 PM, Thomas St?fe wrote: > All tests passed on jdk-submit. > > Anyone interested in a review? > > More output examples for jcmd VM.classloaders : > > Spring framework, basic tree: > http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_short.txt > > Spring framework, including all classes: > http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_long.txt > > ... Thomas > > On Wed, May 23, 2018 at 2:46 PM, Thomas St?fe wrote: >> Dear all, >> >> (not sure if this would be a serviceability or runtime rfe, so sorry >> for crossposting) >> >> may I please have feedback/reviews for this small enhancement. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203682 >> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.00/webrev/ >> >> This adds a new command to jcmd, "VM.classloaders". It complements the >> existing command "VM.classloader_stats". >> >> This command, in its simplest form, prints the class loader tree. In >> addition to that, it optionally prints out loaded classes (both >> non-anonymous and anonymous) and various classloader specific >> information. >> >> Examples: >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example.txt >> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-classes.txt >> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-reflection-and-noinflation.txt >> >> >> Thanks and Best Regards, >> >> Thomas From thomas.stuefe at gmail.com Mon May 28 05:59:32 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 28 May 2018 07:59:32 +0200 Subject: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details In-Reply-To: References: Message-ID: Hi David, On Mon, May 28, 2018 at 7:23 AM, David Holmes wrote: > Hi Thomas, > > I had a look at this and overall seems okay - the output looks good thanks! > (though I'm not sure how useful the hex values are?). That is mainly a developer option for us; CLD* and Klass* are useful if one wishes to dig into core files; Loader oop too (I know that one is volatile, which is were the jcmd-bundle-commands-at-safepoint idea comes from. But yes, this is normally too much noise, therefore disabled by default. You need to set "verbose" explicitly to see this. > > Can't comment too much on the pretty-printing details - the proof is in the > output there. (Though have to wonder whether there is any existing > tree/graph printing logic somewhere in the OpenJDK code?) None as good as mine :) Seriously, there is similar but not as evolved printing for class hierarchy. But it does not really print a tree, just a bunch of '|' dividers. If this patch gets in, I would in a follow up patch unify tree printing for these two commands and any other tree-ish structures I find. > > Two queries: > > 1. Have we previously established whether a CSR request is needed for a new > Dcmd? (My initial feeling is that it is.) My feeling is no, since this adds a new command, so there can be no backward compat issues. What is the general policy for new jcmd commands, or for that matter anything new added to the outside facing interface (new options, new Xlog tracing flags, changed output for existing options)? Do these things require CSR? My problem with CSR is that it introduces a bottleneck, since it can only be approved by three very busy people at Oracle - if I understand the process right. Yes, we need a process to agree e.g. on syntax - desperately so, since e.g. sub option syntax in jcmd is a mess - but we seem to be strapped for reviewers even for normal code reviews, so the effect of creating a CSR in my experience is just a stop-of-work. > > 2. Is ClassLoaderHierarchyVMOperation a safepoint VM-op? I would expect it > needs to be to be able to walk the CLD hierarchy, unless that is already > guaranteed to be safely walkable. Either way a comment clearly stating that > would be useful I think. According to Coleen, CLDG can be walked outside a safepoint, but I did not want to risk it so I made it a safepoint operation (like other commands walking the CLDG, e.g. VM.metaspace). > > Related to #2, is it really possible to encounter a CLD in the process of > being unloaded? Wouldn't that happen at a safepoint? Not sure, I am not a GC expert. I see places where this may be called concurrently, e.g. in the process of -XX:+ClassUnloadingWithConcurrentMark? Since a diagnostic command should never endanger the VM it monitors, I coded defensively. > > Thanks, > David > Thank you, Thomas > > On 28/05/2018 2:50 PM, Thomas St?fe wrote: >> >> All tests passed on jdk-submit. >> >> Anyone interested in a review? >> >> More output examples for jcmd VM.classloaders : >> >> Spring framework, basic tree: >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_short.txt >> >> Spring framework, including all classes: >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_long.txt >> >> ... Thomas >> >> On Wed, May 23, 2018 at 2:46 PM, Thomas St?fe >> wrote: >>> >>> Dear all, >>> >>> (not sure if this would be a serviceability or runtime rfe, so sorry >>> for crossposting) >>> >>> may I please have feedback/reviews for this small enhancement. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203682 >>> Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.00/webrev/ >>> >>> This adds a new command to jcmd, "VM.classloaders". It complements the >>> existing command "VM.classloader_stats". >>> >>> This command, in its simplest form, prints the class loader tree. In >>> addition to that, it optionally prints out loaded classes (both >>> non-anonymous and anonymous) and various classloader specific >>> information. >>> >>> Examples: >>> >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example.txt >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-classes.txt >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-reflection-and-noinflation.txt >>> >>> >>> Thanks and Best Regards, >>> >>> Thomas From thomas.stuefe at gmail.com Mon May 28 06:28:58 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 28 May 2018 08:28:58 +0200 Subject: RFR(xxxs, trivial): 8203865: Metaspace cleanup: Remove unused MemRegion in VirtualSpaceNode Message-ID: Hi all, very tiny cleanup. May I get a review (I consider this trivial): Bug: https://bugs.openjdk.java.net/browse/JDK-8203865 Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203865-Remove-unused-MemRegion-in-VirtualSpaceNode/webrev.00/webrev/ VirtualSpaceNode contains: MemRegion _reserved; This member is not used and serves no purpose. It keeps track of the reserved memory location, but that is redundant since the underlying ReservedSpace does the same. It is also nowhere used. So it should be removed. Best Regards, Thomas From david.holmes at oracle.com Mon May 28 06:42:00 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 28 May 2018 16:42:00 +1000 Subject: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details In-Reply-To: References: Message-ID: <16cb0b42-f618-1015-623b-08e4e49e5a69@oracle.com> Hi Thomas, On 28/05/2018 3:59 PM, Thomas St?fe wrote: > Hi David, > > On Mon, May 28, 2018 at 7:23 AM, David Holmes wrote: >> Hi Thomas, >> >> I had a look at this and overall seems okay - the output looks good > > thanks! > >> (though I'm not sure how useful the hex values are?). > > That is mainly a developer option for us; CLD* and Klass* are useful > if one wishes to dig into core files; Loader oop too (I know that one > is volatile, which is were the jcmd-bundle-commands-at-safepoint idea > comes from. > > But yes, this is normally too much noise, therefore disabled by > default. You need to set "verbose" explicitly to see this. Ok. >> >> Can't comment too much on the pretty-printing details - the proof is in the >> output there. (Though have to wonder whether there is any existing >> tree/graph printing logic somewhere in the OpenJDK code?) > > None as good as mine :) Seriously, there is similar but not as evolved > printing for class hierarchy. But it does not really print a tree, > just a bunch of '|' dividers. > > If this patch gets in, I would in a follow up patch unify tree > printing for these two commands and any other tree-ish structures I > find. Sounds good. >> >> Two queries: >> >> 1. Have we previously established whether a CSR request is needed for a new >> Dcmd? (My initial feeling is that it is.) > > My feeling is no, since this adds a new command, so there can be no > backward compat issues. What is the general policy for new jcmd > commands, or for that matter anything new added to the outside facing > interface (new options, new Xlog tracing flags, changed output for > existing options)? Do these things require CSR? Generally yes. https://wiki.openjdk.java.net/display/csr/CSR+FAQs Q: What sort of changes require CSR review? A: Any change to a JDK interface meant to be used outside of the JDK itself requires CSR review. In this context "interface" isn't limited to the Java programing language definition of an interface, but encompasses the broader concept of a protocol between the JDK and users of the JDK. Examples of interfaces by this definition include: Changes to public exported APIs in java.* and javax.* packages. Changes to public and exported APIs in jdk.*packages. New language updates to the Java Programming Language New structures in the Java Virtual Machine Specification Adding or removing a command in $JDK/bin Adding, removing, or changing a command line option Using or defining an environment variable Using or defining a new file format or wire format Changing or defining a new system or security property Interfaces that are experimental or for diagnostic purposes do not need to go through CSR process, but the CSR process may be employed if feedback from the CSR reviewers is desired. --- IIRC (and I was hoping you may have recalled this :) ) last time this was raised it was stated that as jcmd provides diagnostic commands that adding or changing them doesn't need to go through the CSR process. Unfortunately I also found that such commands are documented: https://docs.oracle.com/javase/10/tools/jcmd.htm#GUID-59153599-875E-447D-8D98-0078A5778F05__DIAGNOSTICCOMMANDSFORJCMD-043BDB32 which may have an impact as it may mean we have to do something special to get the documentation updated. :( Not sure. Anyway I think it is okay to proceed without a CSR request at this time, and I/we can check on the documentation issue. > My problem with CSR is that it introduces a bottleneck, since it can > only be approved by three very busy people at Oracle - if I understand > the process right. Yes, we need a process to agree e.g. on syntax - > desperately so, since e.g. sub option syntax in jcmd is a mess - but > we seem to be strapped for reviewers even for normal code reviews, so > the effect of creating a CSR in my experience is just a stop-of-work. First note that a CSR request reviewer does not need to an OpenJDK Reviewer - they simply need to be a competent engineer with an OpenJDK username who basically sanity checks the CSR request to make sure it meets the expected requirements as per the CSR documentation. Second, yes Joe tends to be the final approver, unless he delegates to someone else when he is away. The expectation is that the need for a CSR request is established as one of the first tasks when working on something, so that the request can be put in and processed well ahead of the time you're ready to push. Again as per CSR docs. But please raise any issues you have with the process with the CSR group - as that is why it was created. mailto:csr-discuss at openjdk.java.net >> >> 2. Is ClassLoaderHierarchyVMOperation a safepoint VM-op? I would expect it >> needs to be to be able to walk the CLD hierarchy, unless that is already >> guaranteed to be safely walkable. Either way a comment clearly stating that >> would be useful I think. > > According to Coleen, CLDG can be walked outside a safepoint, but I did > not want to risk it so I made it a safepoint operation (like other > commands walking the CLDG, e.g. VM.metaspace). Okay. Can you document that please. >> >> Related to #2, is it really possible to encounter a CLD in the process of >> being unloaded? Wouldn't that happen at a safepoint? > > Not sure, I am not a GC expert. I see places where this may be called > concurrently, e.g. in the process of > -XX:+ClassUnloadingWithConcurrentMark? > > Since a diagnostic command should never endanger the VM it monitors, I > coded defensively. Okay. Someone else may be able to clarify whether it is indeed possible or not, but defensive is fine by me. Thanks, David >> >> Thanks, >> David >> > > Thank you, > > Thomas > >> >> On 28/05/2018 2:50 PM, Thomas St?fe wrote: >>> >>> All tests passed on jdk-submit. >>> >>> Anyone interested in a review? >>> >>> More output examples for jcmd VM.classloaders : >>> >>> Spring framework, basic tree: >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_short.txt >>> >>> Spring framework, including all classes: >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_long.txt >>> >>> ... Thomas >>> >>> On Wed, May 23, 2018 at 2:46 PM, Thomas St?fe >>> wrote: >>>> >>>> Dear all, >>>> >>>> (not sure if this would be a serviceability or runtime rfe, so sorry >>>> for crossposting) >>>> >>>> may I please have feedback/reviews for this small enhancement. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203682 >>>> Webrev: >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.00/webrev/ >>>> >>>> This adds a new command to jcmd, "VM.classloaders". It complements the >>>> existing command "VM.classloader_stats". >>>> >>>> This command, in its simplest form, prints the class loader tree. In >>>> addition to that, it optionally prints out loaded classes (both >>>> non-anonymous and anonymous) and various classloader specific >>>> information. >>>> >>>> Examples: >>>> >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example.txt >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-classes.txt >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-reflection-and-noinflation.txt >>>> >>>> >>>> Thanks and Best Regards, >>>> >>>> Thomas From kirk at kodewerk.com Mon May 28 06:46:57 2018 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Mon, 28 May 2018 08:46:57 +0200 Subject: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details In-Reply-To: References: <918E7C66-3068-4020-94F4-7351A934B6AE@kodewerk.com> Message-ID: <2D0E8F45-4B27-4AF4-9BA2-575C55E91C2A@kodewerk.com> > On May 28, 2018, at 7:14 AM, Thomas St?fe wrote: > > OID - I was not aware that the hotspot had one :) I guess I sometimes use OID very loosely and interchangeably wth oop.. and I guess I shouldn?t. As you most likely know, in OQL you can run stuff like "select heap.findObject(0x12345ab).parent" so being able to tie in the output from jcmd to a heap dump might be useful. > > AFAIK the only truly unique id of an object is its pointer, the oop > (which is of course only stable at a safepoint, hence my > jcmd-safepoint proposal). My command already prints the oop for the > classloader if sub option "verbose" is given. Sorry, missed this in the output. > > btw thanks for the positive feedback! Your welcome. ? Kirk From david.holmes at oracle.com Mon May 28 06:47:07 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 28 May 2018 16:47:07 +1000 Subject: RFR(xxxs, trivial): 8203865: Metaspace cleanup: Remove unused MemRegion in VirtualSpaceNode In-Reply-To: References: Message-ID: Hi Thomas, On 28/05/2018 4:28 PM, Thomas St?fe wrote: > Hi all, > > very tiny cleanup. > > May I get a review (I consider this trivial): I'd like someone to confirm that the deleted asserts are still logically being checked elsewhere. Otherwise this cleanup seems ok. Thanks, David > Bug: https://bugs.openjdk.java.net/browse/JDK-8203865 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203865-Remove-unused-MemRegion-in-VirtualSpaceNode/webrev.00/webrev/ > > VirtualSpaceNode contains: MemRegion _reserved; > > This member is not used and serves no purpose. It keeps track of the > reserved memory location, but that is redundant since the underlying > ReservedSpace does the same. It is also nowhere used. So it should be > removed. > > Best Regards, Thomas > From ecki at zusammenkunft.net Mon May 28 07:21:16 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Mon, 28 May 2018 07:21:16 +0000 Subject: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details In-Reply-To: References: , Message-ID: Anything which can be done with the DelegatingClassLoaders (either add Info on their target or collabs them to a ?30 x unnamed DelegatingClassloader?. Even in Verbose mode they only reveal their type (constructor, method) but not much more. Gruss Bernd -- http://bernd.eckenfels.net ________________________________ From: serviceability-dev on behalf of Thomas St?fe Sent: Monday, May 28, 2018 6:50:34 AM To: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net; Hotspot dev runtime Subject: Re: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details All tests passed on jdk-submit. Anyone interested in a review? More output examples for jcmd VM.classloaders : Spring framework, basic tree: http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_short.txt Spring framework, including all classes: http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_long.txt ... Thomas On Wed, May 23, 2018 at 2:46 PM, Thomas St?fe wrote: > Dear all, > > (not sure if this would be a serviceability or runtime rfe, so sorry > for crossposting) > > may I please have feedback/reviews for this small enhancement. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203682 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.00/webrev/ > > This adds a new command to jcmd, "VM.classloaders". It complements the > existing command "VM.classloader_stats". > > This command, in its simplest form, prints the class loader tree. In > addition to that, it optionally prints out loaded classes (both > non-anonymous and anonymous) and various classloader specific > information. > > Examples: > > http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example.txt > http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-classes.txt > http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-reflection-and-noinflation.txt > > > Thanks and Best Regards, > > Thomas From thomas.stuefe at gmail.com Mon May 28 09:15:16 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 28 May 2018 11:15:16 +0200 Subject: RFR(xxxs, trivial): 8203865: Metaspace cleanup: Remove unused MemRegion in VirtualSpaceNode In-Reply-To: References: Message-ID: Hi David, Thanks for the quick review! On Mon, May 28, 2018 at 8:47 AM, David Holmes wrote: > Hi Thomas, > > On 28/05/2018 4:28 PM, Thomas St?fe wrote: >> >> Hi all, >> >> very tiny cleanup. >> >> May I get a review (I consider this trivial): > > > I'd like someone to confirm that the deleted asserts are still logically > being checked elsewhere. > set_top((MetaWord*)virtual_space()->low()); - set_reserved(MemRegion((HeapWord*)_rs.base(), - (HeapWord*)(_rs.base() + _rs.size()))); - - assert(reserved()->start() == (HeapWord*) _rs.base(), - "Reserved start was not set properly " PTR_FORMAT - " != " PTR_FORMAT, p2i(reserved()->start()), p2i(_rs.base())); - assert(reserved()->word_size() == _rs.size() / BytesPerWord, - "Reserved size was not set properly " SIZE_FORMAT - " != " SIZE_FORMAT, reserved()->word_size(), - _rs.size() / BytesPerWord); } No need; the asserts check that the MemRegion object is initialized correctly; since it is unused and removed, the asserts can be removed too. Thanks, Thomas > Otherwise this cleanup seems ok. > > Thanks, > David > > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203865 >> Webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/8203865-Remove-unused-MemRegion-in-VirtualSpaceNode/webrev.00/webrev/ >> >> VirtualSpaceNode contains: MemRegion _reserved; >> >> This member is not used and serves no purpose. It keeps track of the >> reserved memory location, but that is redundant since the underlying >> ReservedSpace does the same. It is also nowhere used. So it should be >> removed. >> >> Best Regards, Thomas >> > From thomas.stuefe at gmail.com Mon May 28 09:33:40 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 28 May 2018 11:33:40 +0200 Subject: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details In-Reply-To: References: Message-ID: Hi Bernd, definitely. I have two local patches here and corresponding RFEs open: https://bugs.openjdk.java.net/browse/JDK-8203343 "VM.{metaspace|classloaders} jcmd should show invocation targets for Generated{Method|Constructor}AccessorImpl classes" https://bugs.openjdk.java.net/browse/JDK-8203849 "DelegatingClassLoader name could be used to indicate reflected class" Both complement each other and should make it possible to understand core reflection class loader issues better. Nice idea about that folding, I can add this. I plan to get them upstream for 11, but that may be illusory seeing how slow reviews are currently going. But certainly 12. Gruss, Thomas On Mon, May 28, 2018 at 9:21 AM, Bernd Eckenfels wrote: > Anything which can be done with the DelegatingClassLoaders (either add Info on their target or collabs them to a ?30 x unnamed DelegatingClassloader?. Even in Verbose mode they only reveal their type (constructor, method) but not much more. > > Gruss > Bernd > -- > http://bernd.eckenfels.net > ________________________________ > From: serviceability-dev on behalf of Thomas St?fe > Sent: Monday, May 28, 2018 6:50:34 AM > To: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net; Hotspot dev runtime > Subject: Re: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details > > All tests passed on jdk-submit. > > Anyone interested in a review? > > More output examples for jcmd VM.classloaders : > > Spring framework, basic tree: > http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_short.txt > > Spring framework, including all classes: > http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_long.txt > > ... Thomas > > On Wed, May 23, 2018 at 2:46 PM, Thomas St?fe wrote: >> Dear all, >> >> (not sure if this would be a serviceability or runtime rfe, so sorry >> for crossposting) >> >> may I please have feedback/reviews for this small enhancement. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203682 >> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.00/webrev/ >> >> This adds a new command to jcmd, "VM.classloaders". It complements the >> existing command "VM.classloader_stats". >> >> This command, in its simplest form, prints the class loader tree. In >> addition to that, it optionally prints out loaded classes (both >> non-anonymous and anonymous) and various classloader specific >> information. >> >> Examples: >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example.txt >> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-classes.txt >> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-reflection-and-noinflation.txt >> >> >> Thanks and Best Regards, >> >> Thomas From thomas.stuefe at gmail.com Mon May 28 09:36:43 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 28 May 2018 11:36:43 +0200 Subject: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details In-Reply-To: <2D0E8F45-4B27-4AF4-9BA2-575C55E91C2A@kodewerk.com> References: <918E7C66-3068-4020-94F4-7351A934B6AE@kodewerk.com> <2D0E8F45-4B27-4AF4-9BA2-575C55E91C2A@kodewerk.com> Message-ID: On Mon, May 28, 2018 at 8:46 AM, Kirk Pepperdine wrote: > >> On May 28, 2018, at 7:14 AM, Thomas St?fe wrote: >> >> OID - I was not aware that the hotspot had one :) > > > I guess I sometimes use OID very loosely and interchangeably wth oop.. and I guess I shouldn?t. No problem. I was not sure if you maybe refer to another VM implementation which may have unique object ids. > As you most likely know, in OQL you can run stuff like "select heap.findObject(0x12345ab).parent" so being able to tie in the output from jcmd to a heap dump might be useful. No, I did not know that :) > >> >> AFAIK the only truly unique id of an object is its pointer, the oop >> (which is of course only stable at a safepoint, hence my >> jcmd-safepoint proposal). My command already prints the oop for the >> classloader if sub option "verbose" is given. > > Sorry, missed this in the output. Again, no problem. Thank you for the feedback. ..Thomas > >> >> btw thanks for the positive feedback! > > Your welcome. > > ? Kirk > From thomas.stuefe at gmail.com Mon May 28 09:50:52 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 28 May 2018 11:50:52 +0200 Subject: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details In-Reply-To: <16cb0b42-f618-1015-623b-08e4e49e5a69@oracle.com> References: <16cb0b42-f618-1015-623b-08e4e49e5a69@oracle.com> Message-ID: Hi David On Mon, May 28, 2018 at 8:42 AM, David Holmes wrote: > Hi Thomas, > > On 28/05/2018 3:59 PM, Thomas St?fe wrote: >> >> Hi David, >> >> On Mon, May 28, 2018 at 7:23 AM, David Holmes >> wrote: >>> >>> Hi Thomas, >>> >>> I had a look at this and overall seems okay - the output looks good >> >> >> thanks! >> >>> >>> Two queries: >>> >>> 1. Have we previously established whether a CSR request is needed for a >>> new >>> Dcmd? (My initial feeling is that it is.) >> >> >> My feeling is no, since this adds a new command, so there can be no >> backward compat issues. What is the general policy for new jcmd >> commands, or for that matter anything new added to the outside facing >> interface (new options, new Xlog tracing flags, changed output for >> existing options)? Do these things require CSR? > > > Generally yes. > > https://wiki.openjdk.java.net/display/csr/CSR+FAQs > > Q: What sort of changes require CSR review? > A: Any change to a JDK interface meant to be used outside of the JDK itself > requires CSR review. In this context "interface" isn't limited to the Java > programing language definition of an interface, but encompasses the broader > concept of a protocol between the JDK and users of the JDK. Examples of > interfaces by this definition include: > > Changes to public exported APIs in java.* and javax.* packages. > Changes to public and exported APIs in jdk.*packages. > New language updates to the Java Programming Language > New structures in the Java Virtual Machine Specification > Adding or removing a command in $JDK/bin > Adding, removing, or changing a command line option > Using or defining an environment variable > Using or defining a new file format or wire format > Changing or defining a new system or security property > > Interfaces that are experimental or for diagnostic purposes do not need to > go through CSR process, but the CSR process may be employed if feedback from > the CSR reviewers is desired. > --- > > IIRC (and I was hoping you may have recalled this :) ) last time this was > raised it was stated that as jcmd provides diagnostic commands that adding > or changing them doesn't need to go through the CSR process. > > Unfortunately I also found that such commands are documented: > > https://docs.oracle.com/javase/10/tools/jcmd.htm#GUID-59153599-875E-447D-8D98-0078A5778F05__DIAGNOSTICCOMMANDSFORJCMD-043BDB32 > > which may have an impact as it may mean we have to do something special to > get the documentation updated. :( Not sure. I would expect this to be a task for someone inside Oracle, since these documentations - to my understanding - refer to the Oracle JDK, not OpenJDK. Similar to how Redhat would have to polish up RHEL-relevant documentation once they take over new Fedora features. > > Anyway I think it is okay to proceed without a CSR request at this time, and > I/we can check on the documentation issue. > >> My problem with CSR is that it introduces a bottleneck, since it can >> only be approved by three very busy people at Oracle - if I understand >> the process right. Yes, we need a process to agree e.g. on syntax - >> desperately so, since e.g. sub option syntax in jcmd is a mess - but >> we seem to be strapped for reviewers even for normal code reviews, so >> the effect of creating a CSR in my experience is just a stop-of-work. > > > First note that a CSR request reviewer does not need to an OpenJDK Reviewer > - they simply need to be a competent engineer with an OpenJDK username who > basically sanity checks the CSR request to make sure it meets the expected > requirements as per the CSR documentation. > > Second, yes Joe tends to be the final approver, unless he delegates to > someone else when he is away. > > The expectation is that the need for a CSR request is established as one of > the first tasks when working on something, so that the request can be put in > and processed well ahead of the time you're ready to push. Again as per CSR > docs. In this case this is a hen-and-egg problem. If I assume I do not need a CSR I will only notice that I do once Reviewers tell me in the code review. But yes, there may be cases where I know beforehand that a CSR is required, I'll try to schedule CSR time for that in the future. > > But please raise any issues you have with the process with the CSR group - > as that is why it was created. mailto:csr-discuss at openjdk.java.net > Sure. >>> >>> 2. Is ClassLoaderHierarchyVMOperation a safepoint VM-op? I would expect >>> it >>> needs to be to be able to walk the CLD hierarchy, unless that is already >>> guaranteed to be safely walkable. Either way a comment clearly stating >>> that >>> would be useful I think. >> >> >> According to Coleen, CLDG can be walked outside a safepoint, but I did >> not want to risk it so I made it a safepoint operation (like other >> commands walking the CLDG, e.g. VM.metaspace). > > > Okay. Can you document that please. Will do. >>> >>> >>> Related to #2, is it really possible to encounter a CLD in the process of >>> being unloaded? Wouldn't that happen at a safepoint? >> >> >> Not sure, I am not a GC expert. I see places where this may be called >> concurrently, e.g. in the process of >> -XX:+ClassUnloadingWithConcurrentMark? >> >> Since a diagnostic command should never endanger the VM it monitors, I >> coded defensively. > > > Okay. Someone else may be able to clarify whether it is indeed possible or > not, but defensive is fine by me. > > Thanks, > David > > Thanks, Thomas >>> >>> Thanks, >>> David >>> >> >> Thank you, >> >> Thomas >> >>> >>> On 28/05/2018 2:50 PM, Thomas St?fe wrote: >>>> >>>> >>>> All tests passed on jdk-submit. >>>> >>>> Anyone interested in a review? >>>> >>>> More output examples for jcmd VM.classloaders : >>>> >>>> Spring framework, basic tree: >>>> >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_short.txt >>>> >>>> Spring framework, including all classes: >>>> >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_long.txt >>>> >>>> ... Thomas >>>> >>>> On Wed, May 23, 2018 at 2:46 PM, Thomas St?fe >>>> wrote: >>>>> >>>>> >>>>> Dear all, >>>>> >>>>> (not sure if this would be a serviceability or runtime rfe, so sorry >>>>> for crossposting) >>>>> >>>>> may I please have feedback/reviews for this small enhancement. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203682 >>>>> Webrev: >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.00/webrev/ >>>>> >>>>> This adds a new command to jcmd, "VM.classloaders". It complements the >>>>> existing command "VM.classloader_stats". >>>>> >>>>> This command, in its simplest form, prints the class loader tree. In >>>>> addition to that, it optionally prints out loaded classes (both >>>>> non-anonymous and anonymous) and various classloader specific >>>>> information. >>>>> >>>>> Examples: >>>>> >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example.txt >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-classes.txt >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-reflection-and-noinflation.txt >>>>> >>>>> >>>>> Thanks and Best Regards, >>>>> >>>>> Thomas From goetz.lindenmaier at sap.com Mon May 28 10:40:00 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 28 May 2018 10:40:00 +0000 Subject: RFR (M): 8203881: Print errornous size in NegativeArraySizeException Message-ID: <0366de26b7004c519bd56e07b9186957@sap.com> Hi, This change adds printing the array size in case of negative array size exception. Please review. http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg-NegativeArraySize/01/ Thanks, Goetz. From david.holmes at oracle.com Mon May 28 11:00:14 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 28 May 2018 21:00:14 +1000 Subject: RFR(xxxs, trivial): 8203865: Metaspace cleanup: Remove unused MemRegion in VirtualSpaceNode In-Reply-To: References: Message-ID: On 28/05/2018 7:15 PM, Thomas St?fe wrote: > Hi David, > > Thanks for the quick review! > > On Mon, May 28, 2018 at 8:47 AM, David Holmes wrote: >> Hi Thomas, >> >> On 28/05/2018 4:28 PM, Thomas St?fe wrote: >>> >>> Hi all, >>> >>> very tiny cleanup. >>> >>> May I get a review (I consider this trivial): >> >> >> I'd like someone to confirm that the deleted asserts are still logically >> being checked elsewhere. >> > > set_top((MetaWord*)virtual_space()->low()); > - set_reserved(MemRegion((HeapWord*)_rs.base(), > - (HeapWord*)(_rs.base() + _rs.size()))); > - > - assert(reserved()->start() == (HeapWord*) _rs.base(), > - "Reserved start was not set properly " PTR_FORMAT > - " != " PTR_FORMAT, p2i(reserved()->start()), p2i(_rs.base())); > - assert(reserved()->word_size() == _rs.size() / BytesPerWord, > - "Reserved size was not set properly " SIZE_FORMAT > - " != " SIZE_FORMAT, reserved()->word_size(), > - _rs.size() / BytesPerWord); > } > > No need; the asserts check that the MemRegion object is initialized > correctly; since it is unused and removed, the asserts can be removed > too. Sorry - yes I see that now. Thanks, David > Thanks, Thomas > > >> Otherwise this cleanup seems ok. >> >> Thanks, >> David >> >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203865 >>> Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203865-Remove-unused-MemRegion-in-VirtualSpaceNode/webrev.00/webrev/ >>> >>> VirtualSpaceNode contains: MemRegion _reserved; >>> >>> This member is not used and serves no purpose. It keeps track of the >>> reserved memory location, but that is redundant since the underlying >>> ReservedSpace does the same. It is also nowhere used. So it should be >>> removed. >>> >>> Best Regards, Thomas >>> >> From david.holmes at oracle.com Mon May 28 11:10:18 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 28 May 2018 21:10:18 +1000 Subject: RFR (M): 8203881: Print errornous size in NegativeArraySizeException In-Reply-To: <0366de26b7004c519bd56e07b9186957@sap.com> References: <0366de26b7004c519bd56e07b9186957@sap.com> Message-ID: Hi Goetz, First I have corrected the typo in the synopsis: errornous -> erroneous Second, can you not use the err_msg function instead of the explicit ResourceMark + stringStream code? Thanks, David On 28/05/2018 8:40 PM, Lindenmaier, Goetz wrote: > Hi, > > This change adds printing the array size in case of negative array size exception. > Please review. > http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg-NegativeArraySize/01/ > > Thanks, > Goetz. > From thomas.stuefe at gmail.com Mon May 28 12:05:45 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 28 May 2018 14:05:45 +0200 Subject: RFR(xxxs, trivial): 8203865: Metaspace cleanup: Remove unused MemRegion in VirtualSpaceNode In-Reply-To: References: Message-ID: np thanks for the review. I'll wait my 24 hours and submit then. ..Thomas On Mon, May 28, 2018 at 1:00 PM, David Holmes wrote: > > > On 28/05/2018 7:15 PM, Thomas St?fe wrote: >> >> Hi David, >> >> Thanks for the quick review! >> >> On Mon, May 28, 2018 at 8:47 AM, David Holmes >> wrote: >>> >>> Hi Thomas, >>> >>> On 28/05/2018 4:28 PM, Thomas St?fe wrote: >>>> >>>> >>>> Hi all, >>>> >>>> very tiny cleanup. >>>> >>>> May I get a review (I consider this trivial): >>> >>> >>> >>> I'd like someone to confirm that the deleted asserts are still logically >>> being checked elsewhere. >>> >> >> set_top((MetaWord*)virtual_space()->low()); >> - set_reserved(MemRegion((HeapWord*)_rs.base(), >> - (HeapWord*)(_rs.base() + _rs.size()))); >> - >> - assert(reserved()->start() == (HeapWord*) _rs.base(), >> - "Reserved start was not set properly " PTR_FORMAT >> - " != " PTR_FORMAT, p2i(reserved()->start()), p2i(_rs.base())); >> - assert(reserved()->word_size() == _rs.size() / BytesPerWord, >> - "Reserved size was not set properly " SIZE_FORMAT >> - " != " SIZE_FORMAT, reserved()->word_size(), >> - _rs.size() / BytesPerWord); >> } >> >> No need; the asserts check that the MemRegion object is initialized >> correctly; since it is unused and removed, the asserts can be removed >> too. > > > Sorry - yes I see that now. > > Thanks, > David > > >> Thanks, Thomas >> >> >>> Otherwise this cleanup seems ok. >>> >>> Thanks, >>> David >>> >>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203865 >>>> Webrev: >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203865-Remove-unused-MemRegion-in-VirtualSpaceNode/webrev.00/webrev/ >>>> >>>> VirtualSpaceNode contains: MemRegion _reserved; >>>> >>>> This member is not used and serves no purpose. It keeps track of the >>>> reserved memory location, but that is redundant since the underlying >>>> ReservedSpace does the same. It is also nowhere used. So it should be >>>> removed. >>>> >>>> Best Regards, Thomas >>>> >>> > From goetz.lindenmaier at sap.com Mon May 28 12:10:37 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 28 May 2018 12:10:37 +0000 Subject: RFR (M): 8203881: Print errornous size in NegativeArraySizeException In-Reply-To: References: <0366de26b7004c519bd56e07b9186957@sap.com> Message-ID: Hi David, thanks for having a look! > First I have corrected the typo in the synopsis: errornous -> erroneous Thanks! > Second, can you not use the err_msg function instead of the explicit > ResourceMark + stringStream code? Yes, that's much better. New webrev: http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg-NegativeArraySize/02 Best regards, Goetz. > > Thanks, > David > > On 28/05/2018 8:40 PM, Lindenmaier, Goetz wrote: > > Hi, > > > > This change adds printing the array size in case of negative array size > exception. > > Please review. > > http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg- > NegativeArraySize/01/ > > > > Thanks, > > Goetz. > > From thomas.stuefe at gmail.com Mon May 28 12:16:09 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 28 May 2018 14:16:09 +0200 Subject: RFR (M): 8203881: Print errornous size in NegativeArraySizeException In-Reply-To: References: <0366de26b7004c519bd56e07b9186957@sap.com> Message-ID: On Mon, May 28, 2018 at 2:10 PM, Lindenmaier, Goetz wrote: > Hi David, > > thanks for having a look! > >> First I have corrected the typo in the synopsis: errornous -> erroneous > Thanks! > >> Second, can you not use the err_msg function instead of the explicit >> ResourceMark + stringStream code? > Yes, that's much better. New webrev: > http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg-NegativeArraySize/02 > > Best regards, > Goetz. > Hi Goetz, looks good, but I would probably use a smaller Format Buffer. Default (e.g. err_msg) uses 256 bytes of stack space, seems a bit much for a single number. Leave it up to you if you take the suggestion. Thomas p.s. that's not M :-) > > >> >> Thanks, >> David >> >> On 28/05/2018 8:40 PM, Lindenmaier, Goetz wrote: >> > Hi, >> > >> > This change adds printing the array size in case of negative array size >> exception. >> > Please review. >> > http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg- >> NegativeArraySize/01/ >> > >> > Thanks, >> > Goetz. >> > From zgu at redhat.com Mon May 28 12:38:13 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 28 May 2018 08:38:13 -0400 Subject: RFR(xxxs, trivial): 8203865: Metaspace cleanup: Remove unused MemRegion in VirtualSpaceNode In-Reply-To: References: Message-ID: <4d60b1ab-483e-3c4c-15e5-d3fb5df762c7@redhat.com> Looks good to me. -Zhengyu On 05/28/2018 02:28 AM, Thomas St?fe wrote: > Hi all, > > very tiny cleanup. > > May I get a review (I consider this trivial): > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203865 > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203865-Remove-unused-MemRegion-in-VirtualSpaceNode/webrev.00/webrev/ > > VirtualSpaceNode contains: MemRegion _reserved; > > This member is not used and serves no purpose. It keeps track of the > reserved memory location, but that is redundant since the underlying > ReservedSpace does the same. It is also nowhere used. So it should be > removed. > > Best Regards, Thomas > From thomas.stuefe at gmail.com Mon May 28 12:38:57 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 28 May 2018 14:38:57 +0200 Subject: RFR(xxxs, trivial): 8203865: Metaspace cleanup: Remove unused MemRegion in VirtualSpaceNode In-Reply-To: <4d60b1ab-483e-3c4c-15e5-d3fb5df762c7@redhat.com> References: <4d60b1ab-483e-3c4c-15e5-d3fb5df762c7@redhat.com> Message-ID: Thanks Zhengyu! On Mon, May 28, 2018 at 2:38 PM, Zhengyu Gu wrote: > Looks good to me. > > -Zhengyu > > > On 05/28/2018 02:28 AM, Thomas St?fe wrote: >> >> Hi all, >> >> very tiny cleanup. >> >> May I get a review (I consider this trivial): >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203865 >> Webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/8203865-Remove-unused-MemRegion-in-VirtualSpaceNode/webrev.00/webrev/ >> >> VirtualSpaceNode contains: MemRegion _reserved; >> >> This member is not used and serves no purpose. It keeps track of the >> reserved memory location, but that is redundant since the underlying >> ReservedSpace does the same. It is also nowhere used. So it should be >> removed. >> >> Best Regards, Thomas >> > From david.holmes at oracle.com Mon May 28 13:17:44 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 28 May 2018 23:17:44 +1000 Subject: RFR (M): 8203881: Print errornous size in NegativeArraySizeException In-Reply-To: References: <0366de26b7004c519bd56e07b9186957@sap.com> Message-ID: <8e666f92-83ac-3127-da9a-aa918fc31422@oracle.com> Hi Goetz, On 28/05/2018 10:10 PM, Lindenmaier, Goetz wrote: > Hi David, > > thanks for having a look! > >> First I have corrected the typo in the synopsis: errornous -> erroneous > Thanks! > >> Second, can you not use the err_msg function instead of the explicit >> ResourceMark + stringStream code? > Yes, that's much better. New webrev: > http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg-NegativeArraySize/02 The functional change seems fine. For the test have you checked that all 7 changes are hit by the testcase? I'm curious as to what kinds of array allocations hit the different bits of code. I confess I'm not sure what a typeArray is versus an objArray. I only expected to see 4 cases here: reference or primitive plus single-dim or multi. Plus the two reflection cases. That leaves one I can't account for. :) Possibly overkill to test (implicit) interpreter plus C1 plus C2, given there is no JIT specific code involved - at least I can't see any. Thanks, David > Best regards, > Goetz. > > > >> >> Thanks, >> David >> >> On 28/05/2018 8:40 PM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> This change adds printing the array size in case of negative array size >> exception. >>> Please review. >>> http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg- >> NegativeArraySize/01/ >>> >>> Thanks, >>> Goetz. >>> From robbin.ehn at oracle.com Mon May 28 13:19:50 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 28 May 2018 15:19:50 +0200 Subject: RFR(L): 8195097: Make it possible to process StringTable outside safepoint Message-ID: <0d09f2ad-47b9-2fca-e5ec-17547e923cbb@oracle.com> Hi all, please review. This implements the StringTable with the ConcurrentHashtable for managing the strings using oopStorage for backing the actual oops via WeakHandles. The unlinking and freeing of hashtable nodes is moved outside the safepoint, which means GC only needs to walk the oopStorage, either concurrently or in a safepoint. Walking oopStorage is also faster so there is a good effect on all safepoints visiting the oops. The unlinking and freeing happens during inserts when dead weak oops are encountered in that bucket. In any normal workload the stringtable self-cleans without needing any additional cleaning. Cleaning/unlinking can also be done concurrently via the ServiceThread, it is started when we have a high ?dead factor?. E.g. application have a lot of interned string removes the references and never interns again. The ServiceThread also concurrently grows the table if ?load factor? is high. Both the cleaning and growing take care to not prolonging time to safepoint, at the cost of some speed. Kitchensink24h, multiple tier1-5 with no issue that I can relate to this changeset, various benchmark such as JMH, specJBB2015. Issue: https://bugs.openjdk.java.net/browse/JDK-8195097 Webrev: http://cr.openjdk.java.net/~rehn/8195097/v0/webrev/ Thanks, Robbin From david.holmes at oracle.com Mon May 28 23:05:21 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 29 May 2018 09:05:21 +1000 Subject: RFR (XS) 8202360: [TESTBUG] runtime/LoadClass/TestResize.java needs to print output when it fails In-Reply-To: References: <18a88525-650a-05bd-a37b-a9ff9162e148@oracle.com> <9006ff6c-c75f-391f-f461-fd842b5bcc9d@oracle.com> Message-ID: <6faf0a6e-dd3f-5357-7b90-b3c38ff619e3@oracle.com> On 26/05/2018 4:23 AM, Gerard Ziemski wrote: > >> On May 24, 2018, at 8:03 PM, David Holmes wrote: >> >> This has really become far too much work for what the intent of the bug report was. Sorry. >> >> On 24/05/2018 5:29 AM, Gerard Ziemski wrote: >>> hi David, >>>> On May 21, 2018, at 5:34 PM, David Holmes wrote: >>>> >>>> >>>> On 22/05/2018 6:57 AM, Gerard Ziemski wrote: >>>>>> On May 20, 2018, at 8:38 PM, David Holmes wrote: >>>>>> Okay - that helps with the immediate problem I had. I also now realize that the failure in this test is showing is that the SD did not in fact re-size as expected! >>>>> Can you elaborate? Is there a reproducible case I can try to run? >>>> >>>> The test's purpose is to check that the SD resizes, so if it finds a load factor too high (which indicates it didn't resize) then the SD resizing logic has either not worked as expected, or the test is not doing what it thinks! >>> The resizing is called during a safe point, which we try to trigger by issuing a System.gc() call (in TriggerResize.java). If no safe point takes place, then no resize had chance to occur, and we have test failure, which is probably what you experienced. >>> I changed the test to look for safe points in the output and check the load factor only when resizing had the chance to occur. >> >> I'm losing confidence in what this test actually tests. For GC's that ignore explicit GC requests via System.gc() this test is never going to test anything e.g. using default G1 GC. > > The assumption here is that the safe point will be triggered by VM during the test on its own, and it always did in my local testing (but apparently not for you at least once). We don?t rely on System.gc() - it?s just the last resort, which btw seems to be doing the right thing on my Mac. That's interesting given: product(bool, ExplicitGCInvokesConcurrent, false) unless you're somehow not running G1. > >> >>>>> I have re-done the main logic a bit to simplify it, and not have to do either String and ?+? nor StringBuffer, by taking advantage of the fact that the info we need to parse is on the 2nd output line. >>>> >>>> I was under the impression that we could see multiple lines of the form: >>>> >>>> Java dictionary (table_size=40423, classes=50002) >>>> >>>> as the table resized. If that is not the case then none of the output seems relevant to me except for this one line. ?? >>> That print out comes from PrintSystemDictionaryAtExit, which happens only once at the exit, but there is more then one system dictionary, and all of them are checked. >> >> But now I understand all that, the dumping of all the output really doesn't help in establishing why the test failed. My original request was for more information to see why the load factor was too high. I thought, incorrectly, that we would see a series of changing size+classes reports until we suddenly hit a bad load factor. But that isn't the case. All that was missing in the end were the actual table_size and classes values that caused the error. Nothing more, nothing less. >> >>> https://bugs.openjdk.java.net/browse/JDK-8202360 >>> http://cr.openjdk.java.net/~gziemski/8202360_rev4 >> >> String already has a contains method you don't need to write your own helper. > > String.contains(CharSequence s) API takes CharSequence, not String. A String is a CharSequence (as is a StringBuffer, StringBuilder). > >> >> I don't see why you stopped using BufferedReader to read each line, and replaced with Scanner. > > Personally, I find it simpler to convert the entire output into String, which then can be tokenized into lines, which then can be looked at one at a time in a simple ?for loop?. This also avoids the issue of remembering to consume all the output to unblock the process. Then you could just use OutputAnalyzer which already provides the output as a String that you can either parse or search. > >> With the scanner the split into lines: >> >> String[] lines = output.split("\n"); >> >> should be: >> >> String[] lines = output.split("\\\R?); >> >> >> to ensure no issues with platform-specific line terminators. > > Done. > > >> >> 83 if (line.startsWith("[")) { >> >> You presumably only need to know one resize has occurred so this can be guarded with !resize. > > Done. > > >> >> 99 // We've hit an error, so print all of the output. >> 100 System.out.println(output); >> >> No need to do this. And I'm not even sure what output contains any more. > > If we ever run into such issue again, it will be helpful to know which dictionary triggered it, and what it contained. Any additional info, such as did the safe point trigger and did it attempt to resize will be helpful as well. I'd like to point out that your rev3 only looked at one dictionary - the first one. Since then you've gone back to a version that looks for all the Java Dictionaries (one per classloader I assume). That seems okay though the test is only going to trigger a resize in one specific Java Dictionary not all of them. > >> >> 107 } >> >> Once you get here the test is over, one way or another - you saw it resized and you found the Java Dictionary line. So at this point you should terminate the loop and then (if needed) finish processing the output so the target can terminate. > > There is more than one system dictionary, so we need to keep processing the output to check them all. Ok - see above. > >> >> Finally if no resize occurred you should print a message that the test trivially passed without any resizing occurring eg: >> >> if (!resized) { >> System.out.println("Test considered PASSED - but no resizing occurred."); >> } > > Done > > >> >> Again I'm sorry this has turned into such an issue. > > I don?t want anyone else to get tripped off by this test, so getting it right is worth it imho. Also, once finished, it will be a nice test template for anyone else who needs to process the output. > > Thanks for your patience! > https://bugs.openjdk.java.net/browse/JDK-8202360 > http://cr.openjdk.java.net/~gziemski/8202360_rev5 Okay. Please using String.contains rather than your helper. Otherwise ship it. No need to see a further webrev. Thanks, David ----- > > cheers > > From robbin.ehn at oracle.com Tue May 29 09:13:27 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 29 May 2018 11:13:27 +0200 Subject: RFR(L): 8195097: Make it possible to process StringTable outside safepoint In-Reply-To: <0d09f2ad-47b9-2fca-e5ec-17547e923cbb@oracle.com> References: <0d09f2ad-47b9-2fca-e5ec-17547e923cbb@oracle.com> Message-ID: <700a4c18-06ea-97c4-8be9-59c5f0503f08@oracle.com> Hi, For reference here is the ZGC patch: http://mail.openjdk.java.net/pipermail/zgc-dev/2018-May/000354.html I removed two methods I had left in for an older ZGC patch and fixed some minors in stringTable.hpp. Inc: http://cr.openjdk.java.net/~rehn/8195097/v1/inc/webrev/ Full:http://cr.openjdk.java.net/~rehn/8195097/v1/full/webrev/ /Robbin On 05/28/2018 03:19 PM, Robbin Ehn wrote: > Hi all, please review. > > This implements the StringTable with the ConcurrentHashtable for managing the > strings using oopStorage for backing the actual oops via WeakHandles. > > The unlinking and freeing of hashtable nodes is moved outside the safepoint, > which means GC only needs to walk the oopStorage, either concurrently or in a > safepoint. Walking oopStorage is also faster so there is a good effect on all > safepoints visiting the oops. > > The unlinking and freeing happens during inserts when dead weak oops are > encountered in that bucket. In any normal workload the stringtable self-cleans > without needing any additional cleaning. Cleaning/unlinking can also be done > concurrently via the ServiceThread, it is started when we have a high ?dead > factor?. E.g. application have a lot of interned string removes the references > and never interns again. The ServiceThread also concurrently grows the table if > ?load factor? is high. Both the cleaning and growing take care to not prolonging > time to safepoint, at the cost of some speed. > > Kitchensink24h, multiple tier1-5 with no issue that I can relate to this > changeset, various benchmark such as JMH, specJBB2015. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8195097 > Webrev: http://cr.openjdk.java.net/~rehn/8195097/v0/webrev/ > > Thanks, Robbin From goetz.lindenmaier at sap.com Tue May 29 15:20:36 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 29 May 2018 15:20:36 +0000 Subject: RFR(xs): 8203455: jcmd: VM.metaspace: print loader name for anonymous CLDs. In-Reply-To: References: Message-ID: <13c5b83ecfcc4c6c97572133c9ecfde8@sap.com> Hi Thomas, the change looks good. I don't care about "" etc :) But I think the "for" in this string is superfluous: // Print "CLD for [,] instance of " Isn?t it only needed in the anonymous case? No new webrev needed for removing the "for". Also I would quote the name, else it might seem to be a word in the sentence instead of a name. Imagine you name the classloader 'test', it reads: 'CLD for test, instance of java.lang.ClassLoader' I would prefer CLD "test", instance of java.lang.ClassLoader. But it might be obvious as you print lists of class loaders, so do as you feel in this point. Best regards, Goetz. > -----Original Message----- > From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- > bounces at openjdk.java.net] On Behalf Of Thomas St?fe > Sent: Samstag, 26. Mai 2018 08:19 > To: Lois Foltan > Cc: Hotspot dev runtime > Subject: Re: RFR(xs): 8203455: jcmd: VM.metaspace: print loader name for > anonymous CLDs. > > Thanks Lois! > > Could I have a second reviewer please? > > Best Regards, Thomas > > On Fri, May 25, 2018 at 8:52 PM, Lois Foltan wrote: > > On 5/24/2018 5:25 PM, Thomas St?fe wrote: > > > >> Thanks for the review Lois! > >> > >> See comments inline. > >> > >> On Thu, May 24, 2018 at 10:31 PM, Lois Foltan > >> wrote: > >>> > >>> On 5/21/2018 3:42 AM, Thomas St?fe wrote: > >>> > >>>> Hi all, > >>>> > >>>> second attempt, after discussing things with David: > >>>> > >>>> > >>>> > >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace- > display-loader-name-for-anonymous-cld/webrev.01/webrev/ > >>>> > >>>> The patch is much simpler now. I use > >>>> ClassLoaderData::class_loader_name() and ::class_loader_class() which > >>>> should always work and be in accordance with planned work in > >>>> https://bugs.openjdk.java.net/browse/JDK-8202605. > >>> > >>> > >>> Hi Thomas, > >>> > >>> Looks good, thanks for using ClassLoaderData::class_loader_name()! A > >>> couple > >>> of review comments: > >>> > >>> - > src/hotspot/share/memory/metaspace/printCLDMetaspaceInfoClosure.cpp > >>> line #82 - I do not prefer the use of "", but am more in favor > >>> of > >>> leaving it out and having the message simply read "CLD for instance of > >>> ". Please consider. > >> > >> Sure - the majority of loaders seem not yet to have names, so this > >> would remove a lot of "". > >> > >>> line #95 - would prefer instead of just > . > >>> According to the comment at line #68, I think you intended that > anyways. > >>> > >> Fixed. > >> > >>> Thanks, > >>> Lois > >>> > >> New webrev: > >> > >> > >> http://cr.openjdk.java.net/~stuefe/webrevs/8203455-VM.metaspace- > display-loader-name-for-anonymous- > cld/webrev.02/webrev/src/hotspot/share/memory/metaspace/printCLDMe > taspaceInfoClosure.cpp.udiff.html > > > > > > Thomas, > > Looks good, thank you for making those changes! > > Lois > > > > > >> > >> Example output: > >> > >> anonymous class, loader has name: > >> 5: CLD 0x00007f814c47a7d0 for , loaded by app > >> instance of jdk.internal.loader.ClassLoaders$AppClassLoader > >> > >> anonymous class in > >> 6: CLD 0x00007f814c474ae0 for , loaded by > > >> > >> loader has name and class name: > >> 7: CLD 0x00007f814c3afc60 for app instance of > >> jdk.internal.loader.ClassLoaders$AppClassLoader > >> > >> : > >> 8: CLD 0x00007f814c1d8570 for > >> > >> Thank you, > >> > >> Thomas > >> > >>>> Thanks, Thomas > >>>> > >>>> > >>>> On Sun, May 20, 2018 at 8:49 AM, Thomas St?fe > > >>>> wrote: > >>>>> > >>>>> Hi all, > >>>>> > >>>>> may I please have reviews for this small addition. > >>>>> > >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203455 > >>>>> Webrev: > >>>>> > >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455- > VM.metaspace-display-loader-name-for-anonymous- > cld/webrev.00/webrev/ > >>>>> > >>>>> VM.metaspace show-loaders can be used to display loaders (well, > really > >>>>> CLD instances). > >>>>> > >>>>> For anonymous CLDs, only "anonymous" is shown. It would be helpful > to > >>>>> show to which loader these CLD are assigned. > >>>>> > >>>>> Before: > >>>>> > >>>>> "272: ClassLoaderData 0x00007f5ba0538f10 for anonymous class" > >>>>> > >>>>> > >>>>> > >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455- > VM.metaspace-display-loader-name-for-anonymous-cld/example- > before.txt > >>>>> > >>>>> After patch: > >>>>> > >>>>> "268: CLD 0x00007ff0c45738f0 for , loaded by > app, > >>>>> instance of jdk.internal.loader.ClassLoaders$AppClassLoader" > >>>>> > >>>>> > >>>>> > >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203455- > VM.metaspace-display-loader-name-for-anonymous-cld/example-after.txt > >>>>> > >>>>> -- > >>>>> Notes: this patch has a bit more lines than I liked because I did not > >>>>> find a singly utility function in ClassloaderData which fit my > >>>>> purpose. I wanted to see name and class of the associated loader for > >>>>> both normal and unloading case. ClassloaderData::loader_name() has > a > >>>>> number of shortcomings, I opened > >>>>> https://bugs.openjdk.java.net/browse/JDK-8203456 as a follow up > rfe. > >>>>> > >>>>> Thank you, > >>>>> > >>>>> Thomas > >>> > >>> > > From gerard.ziemski at oracle.com Tue May 29 15:36:46 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Tue, 29 May 2018 10:36:46 -0500 Subject: RFR (XS) 8202360: [TESTBUG] runtime/LoadClass/TestResize.java needs to print output when it fails In-Reply-To: <6faf0a6e-dd3f-5357-7b90-b3c38ff619e3@oracle.com> References: <18a88525-650a-05bd-a37b-a9ff9162e148@oracle.com> <9006ff6c-c75f-391f-f461-fd842b5bcc9d@oracle.com> <6faf0a6e-dd3f-5357-7b90-b3c38ff619e3@oracle.com> Message-ID: <79F13207-21BB-49C9-83D2-91B7DB9B7783@oracle.com> > On May 28, 2018, at 6:05 PM, David Holmes wrote: trimmed? >>> String already has a contains method you don't need to write your own helper. >> String.contains(CharSequence s) API takes CharSequence, not String. > > A String is a CharSequence (as is a StringBuffer, StringBuilder). Learned something new, fixed. >>> >>> I don't see why you stopped using BufferedReader to read each line, and replaced with Scanner. >> Personally, I find it simpler to convert the entire output into String, which then can be tokenized into lines, which then can be looked at one at a time in a simple ?for loop?. This also avoids the issue of remembering to consume all the output to unblock the process. > > Then you could just use OutputAnalyzer which already provides the output as a String that you can either parse or search. Now that you told me this I had to simplify the test by using the OutputAnalyzer. https://bugs.openjdk.java.net/browse/JDK-8202360 http://cr.openjdk.java.net/~gziemski/8202360_rev6 cheers From thomas.stuefe at gmail.com Tue May 29 17:09:02 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 29 May 2018 19:09:02 +0200 Subject: RFR 8133564: Runtime - 2nd followup to Validate JVM Command-Line Flag Arguments In-Reply-To: <51A91A2E-4752-4469-AA7A-41A047A4A65A@oracle.com> References: <8DD96314-EFB1-4FD7-9E4A-3953241F1752@oracle.com> <6684c916-c20b-e860-a3db-c0be6523b411@oracle.com> <95BA4C1C-3752-4991-A696-28BCCBD50A2E@oracle.com> <6eded9c3-23ad-1712-bc4d-8f3dde2af6cd@oracle.com> <51A91A2E-4752-4469-AA7A-41A047A4A65A@oracle.com> Message-ID: Hi guys, this change breaks all our builds. * For target hotspot_variant-server_libjvm_objs_jvmFlagConstraintsCMS.o: /shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/gc/cms/jvmFlagConstraintsCMS.cpp: In function ?JVMFlag::Error ParallelGCThreadsAndCMSWorkQueueDrainThreshold(uint, uintx, bool)?: /shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/gc/cms/jvmFlagConstraintsCMS.cpp:42:43: error: cannot call member function ?void JVMFlag::printError(bool, const char*, ...)? without object threads, threshold); (linux x64) --- a/src/hotspot/share/runtime/flags/jvmFlag.hpp Tue May 29 13:44:44 2018 +0200 +++ b/src/hotspot/share/runtime/flags/jvmFlag.hpp Tue May 29 11:04:56 2018 -0500 @@ -276,6 +276,7 @@ // printRanges will print out flags type, name and range values as expected by -XX:+PrintFlagsRanges static void printFlags(outputStream* out, bool withComments, bool printRanges = false, bool skipDefaults = false); + void printError(bool verbose, const char* msg, ...); static void verify() PRODUCT_RETURN; printError in jvmFlags.hpp must be static. Please fix. Thanks. Best Regards, Thomas On Fri, May 25, 2018 at 10:29 PM, Gerard Ziemski wrote: > Thank you Harold! > >> On May 25, 2018, at 8:53 AM, Harold David Seigel wrote: >> >> The change looks good to me, also. >> >> Thanks, Harold >> >> >> On 5/24/2018 11:53 AM, coleen.phillimore at oracle.com wrote: >>> >>> Very nice, thanks! >>> Coleen >>> >>> On 5/24/18 11:50 AM, Gerard Ziemski wrote: >>>> Thank you Coleen for your review. >>>> >>>> I fixed the indentation as well as the years in copyright header. >>>> >>>> http://cr.openjdk.java.net/~gziemski/8133564_rev2 >>>> https://bugs.openjdk.java.net/browse/JDK-8133564 >>>> >>>> >>>> cheers >>>> >>>>> On May 23, 2018, at 4:29 PM, coleen.phillimore at oracle.com wrote: >>>>> >>>>> >>>>> This looks good apart from fixing the indentation of the arguments for all these changes so that they line up. >>>>> >>>>> - CommandLineError::print(verbose, >>>>> + JVMFlag::printError(verbose, >>>>> "ParallelGCThreads (" UINT32_FORMAT ") or CMSWorkQueueDrainThreshold (" >>>>> UINTX_FORMAT ") is too large\n", >>>>> threads, threshold); >>>>> >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> On 5/23/18 4:25 PM, Gerard Ziemski wrote: >>>>>> Hi all, >>>>>> >>>>>> Please review this follow-up fix that cleans up the following: >>>>>> >>>>>> #1 We move the error print method out of jvmFlagRangeList and into jvmFlag, which allows us not to include jvmFlagRangeList.hpp just for the print method in various files >>>>>> >>>>>> #2 We cleanup "utilities/defaultStream.hpp? includes >>>>>> >>>>>> #3 and #4 in the issue are no longer applicable >>>>>> >>>>>> Tested locally with various debug/product, headers/no headers, open/closed variations, as well as Mach5 hs-tier1,tier2,tier3 tests, and Mach5 open/closed variations. >>>>>> >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8133564 >>>>>> http://cr.openjdk.java.net/~gziemski/8133564_rev1 >>>>>> >>>>>> cheers >>> >> > From gerard.ziemski at oracle.com Tue May 29 17:15:43 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Tue, 29 May 2018 12:15:43 -0500 Subject: RFR 8133564: Runtime - 2nd followup to Validate JVM Command-Line Flag Arguments In-Reply-To: References: <8DD96314-EFB1-4FD7-9E4A-3953241F1752@oracle.com> <6684c916-c20b-e860-a3db-c0be6523b411@oracle.com> <95BA4C1C-3752-4991-A696-28BCCBD50A2E@oracle.com> <6eded9c3-23ad-1712-bc4d-8f3dde2af6cd@oracle.com> <51A91A2E-4752-4469-AA7A-41A047A4A65A@oracle.com> Message-ID: I?m on it. I merged manually and forgot ?static? keyword. cheers > On May 29, 2018, at 12:09 PM, Thomas St?fe wrote: > > Hi guys, > > this change breaks all our builds. > > * For target hotspot_variant-server_libjvm_objs_jvmFlagConstraintsCMS.o: > /shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/gc/cms/jvmFlagConstraintsCMS.cpp: > In function ?JVMFlag::Error > ParallelGCThreadsAndCMSWorkQueueDrainThreshold(uint, uintx, bool)?: > /shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/gc/cms/jvmFlagConstraintsCMS.cpp:42:43: > error: cannot call member function ?void JVMFlag::printError(bool, > const char*, ...)? without object > threads, threshold); > > (linux x64) > > --- a/src/hotspot/share/runtime/flags/jvmFlag.hpp Tue May 29 13:44:44 2018 +0200 > +++ b/src/hotspot/share/runtime/flags/jvmFlag.hpp Tue May 29 11:04:56 2018 -0500 > @@ -276,6 +276,7 @@ > > // printRanges will print out flags type, name and range values as > expected by -XX:+PrintFlagsRanges > static void printFlags(outputStream* out, bool withComments, bool > printRanges = false, bool skipDefaults = false); > + void printError(bool verbose, const char* msg, ...); > > static void verify() PRODUCT_RETURN; > > printError in jvmFlags.hpp must be static. > > Please fix. Thanks. > > Best Regards, Thomas > > > > On Fri, May 25, 2018 at 10:29 PM, Gerard Ziemski > wrote: >> Thank you Harold! >> >>> On May 25, 2018, at 8:53 AM, Harold David Seigel wrote: >>> >>> The change looks good to me, also. >>> >>> Thanks, Harold >>> >>> >>> On 5/24/2018 11:53 AM, coleen.phillimore at oracle.com wrote: >>>> >>>> Very nice, thanks! >>>> Coleen >>>> >>>> On 5/24/18 11:50 AM, Gerard Ziemski wrote: >>>>> Thank you Coleen for your review. >>>>> >>>>> I fixed the indentation as well as the years in copyright header. >>>>> >>>>> http://cr.openjdk.java.net/~gziemski/8133564_rev2 >>>>> https://bugs.openjdk.java.net/browse/JDK-8133564 >>>>> >>>>> >>>>> cheers >>>>> >>>>>> On May 23, 2018, at 4:29 PM, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> >>>>>> This looks good apart from fixing the indentation of the arguments for all these changes so that they line up. >>>>>> >>>>>> - CommandLineError::print(verbose, >>>>>> + JVMFlag::printError(verbose, >>>>>> "ParallelGCThreads (" UINT32_FORMAT ") or CMSWorkQueueDrainThreshold (" >>>>>> UINTX_FORMAT ") is too large\n", >>>>>> threads, threshold); >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>> On 5/23/18 4:25 PM, Gerard Ziemski wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> Please review this follow-up fix that cleans up the following: >>>>>>> >>>>>>> #1 We move the error print method out of jvmFlagRangeList and into jvmFlag, which allows us not to include jvmFlagRangeList.hpp just for the print method in various files >>>>>>> >>>>>>> #2 We cleanup "utilities/defaultStream.hpp? includes >>>>>>> >>>>>>> #3 and #4 in the issue are no longer applicable >>>>>>> >>>>>>> Tested locally with various debug/product, headers/no headers, open/closed variations, as well as Mach5 hs-tier1,tier2,tier3 tests, and Mach5 open/closed variations. >>>>>>> >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8133564 >>>>>>> http://cr.openjdk.java.net/~gziemski/8133564_rev1 >>>>>>> >>>>>>> cheers >>>> >>> >> From thomas.stuefe at gmail.com Tue May 29 17:17:05 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 29 May 2018 19:17:05 +0200 Subject: RFR 8133564: Runtime - 2nd followup to Validate JVM Command-Line Flag Arguments In-Reply-To: References: <8DD96314-EFB1-4FD7-9E4A-3953241F1752@oracle.com> <6684c916-c20b-e860-a3db-c0be6523b411@oracle.com> <95BA4C1C-3752-4991-A696-28BCCBD50A2E@oracle.com> <6eded9c3-23ad-1712-bc4d-8f3dde2af6cd@oracle.com> <51A91A2E-4752-4469-AA7A-41A047A4A65A@oracle.com> Message-ID: Okay thanks! ..Thomas On Tue, May 29, 2018 at 7:15 PM, Gerard Ziemski wrote: > I?m on it. > > I merged manually and forgot ?static? keyword. > > > cheers > >> On May 29, 2018, at 12:09 PM, Thomas St?fe wrote: >> >> Hi guys, >> >> this change breaks all our builds. >> >> * For target hotspot_variant-server_libjvm_objs_jvmFlagConstraintsCMS.o: >> /shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/gc/cms/jvmFlagConstraintsCMS.cpp: >> In function ?JVMFlag::Error >> ParallelGCThreadsAndCMSWorkQueueDrainThreshold(uint, uintx, bool)?: >> /shared/projects/openjdk/jdk-jdk/source/src/hotspot/share/gc/cms/jvmFlagConstraintsCMS.cpp:42:43: >> error: cannot call member function ?void JVMFlag::printError(bool, >> const char*, ...)? without object >> threads, threshold); >> >> (linux x64) >> >> --- a/src/hotspot/share/runtime/flags/jvmFlag.hpp Tue May 29 13:44:44 2018 +0200 >> +++ b/src/hotspot/share/runtime/flags/jvmFlag.hpp Tue May 29 11:04:56 2018 -0500 >> @@ -276,6 +276,7 @@ >> >> // printRanges will print out flags type, name and range values as >> expected by -XX:+PrintFlagsRanges >> static void printFlags(outputStream* out, bool withComments, bool >> printRanges = false, bool skipDefaults = false); >> + void printError(bool verbose, const char* msg, ...); >> >> static void verify() PRODUCT_RETURN; >> >> printError in jvmFlags.hpp must be static. >> >> Please fix. Thanks. >> >> Best Regards, Thomas >> >> >> >> On Fri, May 25, 2018 at 10:29 PM, Gerard Ziemski >> wrote: >>> Thank you Harold! >>> >>>> On May 25, 2018, at 8:53 AM, Harold David Seigel wrote: >>>> >>>> The change looks good to me, also. >>>> >>>> Thanks, Harold >>>> >>>> >>>> On 5/24/2018 11:53 AM, coleen.phillimore at oracle.com wrote: >>>>> >>>>> Very nice, thanks! >>>>> Coleen >>>>> >>>>> On 5/24/18 11:50 AM, Gerard Ziemski wrote: >>>>>> Thank you Coleen for your review. >>>>>> >>>>>> I fixed the indentation as well as the years in copyright header. >>>>>> >>>>>> http://cr.openjdk.java.net/~gziemski/8133564_rev2 >>>>>> https://bugs.openjdk.java.net/browse/JDK-8133564 >>>>>> >>>>>> >>>>>> cheers >>>>>> >>>>>>> On May 23, 2018, at 4:29 PM, coleen.phillimore at oracle.com wrote: >>>>>>> >>>>>>> >>>>>>> This looks good apart from fixing the indentation of the arguments for all these changes so that they line up. >>>>>>> >>>>>>> - CommandLineError::print(verbose, >>>>>>> + JVMFlag::printError(verbose, >>>>>>> "ParallelGCThreads (" UINT32_FORMAT ") or CMSWorkQueueDrainThreshold (" >>>>>>> UINTX_FORMAT ") is too large\n", >>>>>>> threads, threshold); >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> >>>>>>> On 5/23/18 4:25 PM, Gerard Ziemski wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> Please review this follow-up fix that cleans up the following: >>>>>>>> >>>>>>>> #1 We move the error print method out of jvmFlagRangeList and into jvmFlag, which allows us not to include jvmFlagRangeList.hpp just for the print method in various files >>>>>>>> >>>>>>>> #2 We cleanup "utilities/defaultStream.hpp? includes >>>>>>>> >>>>>>>> #3 and #4 in the issue are no longer applicable >>>>>>>> >>>>>>>> Tested locally with various debug/product, headers/no headers, open/closed variations, as well as Mach5 hs-tier1,tier2,tier3 tests, and Mach5 open/closed variations. >>>>>>>> >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8133564 >>>>>>>> http://cr.openjdk.java.net/~gziemski/8133564_rev1 >>>>>>>> >>>>>>>> cheers >>>>> >>>> >>> > From gerard.ziemski at oracle.com Tue May 29 17:51:20 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Tue, 29 May 2018 12:51:20 -0500 Subject: RFR (XS) 8203938: Fix build failures from JDK-8133564 Message-ID: <55187917-B330-4151-9246-3ED0BBCA4704@oracle.com> Hi all, Please review this trivial build fix that adds missing ?static? keyword, which I missed while doing manual merge. I?m sorry about breaking the build. https://bugs.openjdk.java.net/browse/JDK-8203938 http://cr.openjdk.java.net/~gziemski/8203938_rev1 Tested via build locally, broken before the change, builds with the fix. cheers From kim.barrett at oracle.com Tue May 29 17:52:51 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 29 May 2018 13:52:51 -0400 Subject: RFR (XS) 8203938: Fix build failures from JDK-8133564 In-Reply-To: <55187917-B330-4151-9246-3ED0BBCA4704@oracle.com> References: <55187917-B330-4151-9246-3ED0BBCA4704@oracle.com> Message-ID: > On May 29, 2018, at 1:51 PM, Gerard Ziemski wrote: > > Hi all, > > Please review this trivial build fix that adds missing ?static? keyword, which I missed while doing manual merge. > > I?m sorry about breaking the build. > > > https://bugs.openjdk.java.net/browse/JDK-8203938 > http://cr.openjdk.java.net/~gziemski/8203938_rev1 > > Tested via build locally, broken before the change, builds with the fix. > > > cheers Looks good, and trivial. From gerard.ziemski at oracle.com Tue May 29 17:53:43 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Tue, 29 May 2018 12:53:43 -0500 Subject: RFR (XS) 8203938: Fix build failures from JDK-8133564 In-Reply-To: References: <55187917-B330-4151-9246-3ED0BBCA4704@oracle.com> Message-ID: <62903DB3-8E8F-4C7F-8789-2A0426AD0DCE@oracle.com> Thanks Kim! > On May 29, 2018, at 12:52 PM, Kim Barrett wrote: > >> On May 29, 2018, at 1:51 PM, Gerard Ziemski wrote: >> >> Hi all, >> >> Please review this trivial build fix that adds missing ?static? keyword, which I missed while doing manual merge. >> >> I?m sorry about breaking the build. >> >> >> https://bugs.openjdk.java.net/browse/JDK-8203938 >> http://cr.openjdk.java.net/~gziemski/8203938_rev1 >> >> Tested via build locally, broken before the change, builds with the fix. >> >> >> cheers > > Looks good, and trivial. > From thomas.schatzl at oracle.com Tue May 29 17:56:04 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 29 May 2018 19:56:04 +0200 Subject: RFR (XS) 8203938: Fix build failures from JDK-8133564 In-Reply-To: <55187917-B330-4151-9246-3ED0BBCA4704@oracle.com> References: <55187917-B330-4151-9246-3ED0BBCA4704@oracle.com> Message-ID: <1083dfb278689476e2864d7d419a6ec1d5f3b66b.camel@oracle.com> Hi, On Tue, 2018-05-29 at 12:51 -0500, Gerard Ziemski wrote: > Hi all, > > Please review this trivial build fix that adds missing ?static? > keyword, which I missed while doing manual merge. > > I?m sorry about breaking the build. > > > https://bugs.openjdk.java.net/browse/JDK-8203938 > http://cr.openjdk.java.net/~gziemski/8203938_rev1 > > Tested via build locally, broken before the change, builds with the > fix. > looks good. Thomas From gerard.ziemski at oracle.com Tue May 29 17:56:43 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Tue, 29 May 2018 12:56:43 -0500 Subject: RFR (XS) 8203938: Fix build failures from JDK-8133564 In-Reply-To: <1083dfb278689476e2864d7d419a6ec1d5f3b66b.camel@oracle.com> References: <55187917-B330-4151-9246-3ED0BBCA4704@oracle.com> <1083dfb278689476e2864d7d419a6ec1d5f3b66b.camel@oracle.com> Message-ID: Thank you! > On May 29, 2018, at 12:56 PM, Thomas Schatzl wrote: > > Hi, > > On Tue, 2018-05-29 at 12:51 -0500, Gerard Ziemski wrote: >> Hi all, >> >> Please review this trivial build fix that adds missing ?static? >> keyword, which I missed while doing manual merge. >> >> I?m sorry about breaking the build. >> >> >> https://bugs.openjdk.java.net/browse/JDK-8203938 >> http://cr.openjdk.java.net/~gziemski/8203938_rev1 >> >> Tested via build locally, broken before the change, builds with the >> fix. >> > > looks good. > > Thomas From thomas.stuefe at gmail.com Tue May 29 18:06:54 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 29 May 2018 20:06:54 +0200 Subject: RFR (XS) 8203938: Fix build failures from JDK-8133564 In-Reply-To: <55187917-B330-4151-9246-3ED0BBCA4704@oracle.com> References: <55187917-B330-4151-9246-3ED0BBCA4704@oracle.com> Message-ID: Looks good, and thanks for the quick fix! .. Thomas On Tue, May 29, 2018, 19:51 Gerard Ziemski wrote: > Hi all, > > Please review this trivial build fix that adds missing ?static? keyword, > which I missed while doing manual merge. > > I?m sorry about breaking the build. > > > https://bugs.openjdk.java.net/browse/JDK-8203938 > http://cr.openjdk.java.net/~gziemski/8203938_rev1 > > Tested via build locally, broken before the change, builds with the fix. > > > cheers From david.holmes at oracle.com Tue May 29 21:28:31 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 30 May 2018 07:28:31 +1000 Subject: RFR (XS) 8202360: [TESTBUG] runtime/LoadClass/TestResize.java needs to print output when it fails In-Reply-To: <79F13207-21BB-49C9-83D2-91B7DB9B7783@oracle.com> References: <18a88525-650a-05bd-a37b-a9ff9162e148@oracle.com> <9006ff6c-c75f-391f-f461-fd842b5bcc9d@oracle.com> <6faf0a6e-dd3f-5357-7b90-b3c38ff619e3@oracle.com> <79F13207-21BB-49C9-83D2-91B7DB9B7783@oracle.com> Message-ID: On 30/05/2018 1:36 AM, Gerard Ziemski wrote: > >> On May 28, 2018, at 6:05 PM, David Holmes wrote: > > trimmed? > > >>>> String already has a contains method you don't need to write your own helper. >>> String.contains(CharSequence s) API takes CharSequence, not String. >> >> A String is a CharSequence (as is a StringBuffer, StringBuilder). > > Learned something new, fixed. > > >>>> >>>> I don't see why you stopped using BufferedReader to read each line, and replaced with Scanner. >>> Personally, I find it simpler to convert the entire output into String, which then can be tokenized into lines, which then can be looked at one at a time in a simple ?for loop?. This also avoids the issue of remembering to consume all the output to unblock the process. >> >> Then you could just use OutputAnalyzer which already provides the output as a String that you can either parse or search. > > Now that you told me this I had to simplify the test by using the OutputAnalyzer. > > > https://bugs.openjdk.java.net/browse/JDK-8202360 > http://cr.openjdk.java.net/~gziemski/8202360_rev6 :) Okay. Only minor thing is that: 112 analyzer.shouldHaveExitValue(0); should be up at line 72 - otherwise you might report failure after reporting success. No need to see updated webrev. Thanks, David > > cheers > From coleen.phillimore at oracle.com Tue May 29 21:52:34 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 29 May 2018 17:52:34 -0400 Subject: RFR(L): 8195097: Make it possible to process StringTable outside safepoint In-Reply-To: <0d09f2ad-47b9-2fca-e5ec-17547e923cbb@oracle.com> References: <0d09f2ad-47b9-2fca-e5ec-17547e923cbb@oracle.com> Message-ID: Hi Robbin, StringTable::weak_oops_do() functions seem unused. Can you make StringTableDeleteCheck and StringTableDoDelete inherited from StackObj for documentation purposes, and others in this file. I've reviewed this code at intervals in progress, so I think this looks good.? Reviewed by me. thanks, Coleen On 5/28/18 9:19 AM, Robbin Ehn wrote: > Hi all, please review. > > This implements the StringTable with the ConcurrentHashtable for > managing the > strings using oopStorage for backing the actual oops via WeakHandles. > > The unlinking and freeing of hashtable nodes is moved outside the > safepoint, > which means GC only needs to walk the oopStorage, either concurrently > or in a > safepoint. Walking oopStorage is also faster so there is a good effect > on all > safepoints visiting the oops. > > The unlinking and freeing happens during inserts when dead weak oops are > encountered in that bucket. In any normal workload the stringtable > self-cleans > without needing any additional cleaning. Cleaning/unlinking can also > be done > concurrently via the ServiceThread, it is started when we have a high > ?dead > factor?. E.g. application have a lot of interned string removes the > references > and never interns again. The ServiceThread also concurrently grows the > table if > ?load factor? is high. Both the cleaning and growing take care to not > prolonging > time to safepoint, at the cost of some speed. > > Kitchensink24h, multiple tier1-5 with no issue that I can relate to this > changeset, various benchmark such as JMH, specJBB2015. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8195097 > Webrev: http://cr.openjdk.java.net/~rehn/8195097/v0/webrev/ > > Thanks, Robbin From igor.ignatyev at oracle.com Wed May 30 06:38:21 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 29 May 2018 23:38:21 -0700 Subject: RFR(M) : 8202946 : [TESTBUG] Open source VM testbase OOM tests In-Reply-To: <23ea1dbe-f6bc-b76a-237a-ff6fc1a6dc58@oracle.com> References: <2DD8F9C6-8471-4BF6-8573-0DA3F2B6C66B@oracle.com> <23ea1dbe-f6bc-b76a-237a-ff6fc1a6dc58@oracle.com> Message-ID: <95398DD6-5014-443E-A5B5-9834DE21F551@oracle.com> Serguei, thanks for your review! GC team, can someone please review this? Thanks, -- Igor > On May 21, 2018, at 10:33 PM, serguei.spitsyn at oracle.com wrote: > > Hi Igor, > > This looks good. > This change has to be reviewed by someone from the GC team. > > Thanks, > Serguei > > > On 5/15/18 16:16, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8202946/webrev.00/index.html >>> 1619 lines changed: 1619 ins; 0 del; 0 mod; >> Hi all, >> >> could you please review this patch which open sources OOM tests from VM testbase? these tests test OutOfMemoryError throwing in different scenarios. >> >> As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8202946 >> webrev: http://cr.openjdk.java.net/~iignatyev//8202946/webrev.00/index.html >> testing: :vmTestbase_vm_oom test group >> >> Thanks, >> -- Igor > From goetz.lindenmaier at sap.com Wed May 30 09:14:36 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 30 May 2018 09:14:36 +0000 Subject: RFR (M): 8203881: Print errornous size in NegativeArraySizeException In-Reply-To: <8e666f92-83ac-3127-da9a-aa918fc31422@oracle.com> References: <0366de26b7004c519bd56e07b9186957@sap.com> <8e666f92-83ac-3127-da9a-aa918fc31422@oracle.com> Message-ID: <716e41b4591743a5b5bad26ddd084105@sap.com> Hi David, I checked the test, and added cases for multidimensional arrays where the first dimension is 0. A test for arrayKlass.cpp, ArrayKlass::allocate_arrayArray() is missing, though. I don't think this is critical as the change is quite simple. New webrev, (only test changes): http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg-NegativeArraySize/03/ Best regards, Goetz > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Montag, 28. Mai 2018 15:18 > To: Lindenmaier, Goetz ; hotspot-runtime- > dev at openjdk.java.net > Subject: Re: RFR (M): 8203881: Print errornous size in > NegativeArraySizeException > > Hi Goetz, > > On 28/05/2018 10:10 PM, Lindenmaier, Goetz wrote: > > Hi David, > > > > thanks for having a look! > > > >> First I have corrected the typo in the synopsis: errornous -> erroneous > > Thanks! > > > >> Second, can you not use the err_msg function instead of the explicit > >> ResourceMark + stringStream code? > > Yes, that's much better. New webrev: > > http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg- > NegativeArraySize/02 > > The functional change seems fine. > > For the test have you checked that all 7 changes are hit by the > testcase? I'm curious as to what kinds of array allocations hit the > different bits of code. I confess I'm not sure what a typeArray is > versus an objArray. I only expected to see 4 cases here: reference or > primitive plus single-dim or multi. Plus the two reflection cases. That > leaves one I can't account for. :) > > Possibly overkill to test (implicit) interpreter plus C1 plus C2, given > there is no JIT specific code involved - at least I can't see any. > > Thanks, > David > > > Best regards, > > Goetz. > > > > > > > >> > >> Thanks, > >> David > >> > >> On 28/05/2018 8:40 PM, Lindenmaier, Goetz wrote: > >>> Hi, > >>> > >>> This change adds printing the array size in case of negative array size > >> exception. > >>> Please review. > >>> http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg- > >> NegativeArraySize/01/ > >>> > >>> Thanks, > >>> Goetz. > >>> From david.holmes at oracle.com Wed May 30 09:46:54 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 30 May 2018 19:46:54 +1000 Subject: RFR (M): 8203881: Print errornous size in NegativeArraySizeException In-Reply-To: <716e41b4591743a5b5bad26ddd084105@sap.com> References: <0366de26b7004c519bd56e07b9186957@sap.com> <8e666f92-83ac-3127-da9a-aa918fc31422@oracle.com> <716e41b4591743a5b5bad26ddd084105@sap.com> Message-ID: <5a61aa84-024a-6cfc-c4ea-4776ec8b0252@oracle.com> Hi Goetz, On 30/05/2018 7:14 PM, Lindenmaier, Goetz wrote: > Hi David, > > I checked the test, and added cases for multidimensional > arrays where the first dimension is 0. A test for arrayKlass.cpp, > ArrayKlass::allocate_arrayArray() is missing, though. > I don't think this is critical as the change is quite simple. Does make me wonder what code you need to write to exercise it though ?? Maybe it's dead code :) > New webrev, (only test changes): > http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg-NegativeArraySize/03/ 29 * @compile NegativeArraySizeExceptionTest.java 30 * @run main NegativeArraySizeExceptionTest You don't need a separate @compile, the @run will compile the test file. No need to see updated webrev. Thanks, David > Best regards, > Goetz > > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Montag, 28. Mai 2018 15:18 >> To: Lindenmaier, Goetz ; hotspot-runtime- >> dev at openjdk.java.net >> Subject: Re: RFR (M): 8203881: Print errornous size in >> NegativeArraySizeException >> >> Hi Goetz, >> >> On 28/05/2018 10:10 PM, Lindenmaier, Goetz wrote: >>> Hi David, >>> >>> thanks for having a look! >>> >>>> First I have corrected the typo in the synopsis: errornous -> erroneous >>> Thanks! >>> >>>> Second, can you not use the err_msg function instead of the explicit >>>> ResourceMark + stringStream code? >>> Yes, that's much better. New webrev: >>> http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg- >> NegativeArraySize/02 >> >> The functional change seems fine. >> >> For the test have you checked that all 7 changes are hit by the >> testcase? I'm curious as to what kinds of array allocations hit the >> different bits of code. I confess I'm not sure what a typeArray is >> versus an objArray. I only expected to see 4 cases here: reference or >> primitive plus single-dim or multi. Plus the two reflection cases. That >> leaves one I can't account for. :) >> >> Possibly overkill to test (implicit) interpreter plus C1 plus C2, given >> there is no JIT specific code involved - at least I can't see any. >> >> Thanks, >> David >> >>> Best regards, >>> Goetz. >>> >>> >>> >>>> >>>> Thanks, >>>> David >>>> >>>> On 28/05/2018 8:40 PM, Lindenmaier, Goetz wrote: >>>>> Hi, >>>>> >>>>> This change adds printing the array size in case of negative array size >>>> exception. >>>>> Please review. >>>>> http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg- >>>> NegativeArraySize/01/ >>>>> >>>>> Thanks, >>>>> Goetz. >>>>> From thomas.stuefe at gmail.com Wed May 30 09:56:08 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 30 May 2018 11:56:08 +0200 Subject: RFR (M): 8203881: Print errornous size in NegativeArraySizeException In-Reply-To: <716e41b4591743a5b5bad26ddd084105@sap.com> References: <0366de26b7004c519bd56e07b9186957@sap.com> <8e666f92-83ac-3127-da9a-aa918fc31422@oracle.com> <716e41b4591743a5b5bad26ddd084105@sap.com> Message-ID: Good for me too. ..Thomas On Wed 30. May 2018 at 11:14, Lindenmaier, Goetz wrote: > Hi David, > > I checked the test, and added cases for multidimensional > arrays where the first dimension is 0. A test for arrayKlass.cpp, > ArrayKlass::allocate_arrayArray() is missing, though. > I don't think this is critical as the change is quite simple. > > New webrev, (only test changes): > http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg-NegativeArraySize/03/ > > Best regards, > Goetz > > > > -----Original Message----- > > From: David Holmes [mailto:david.holmes at oracle.com] > > Sent: Montag, 28. Mai 2018 15:18 > > To: Lindenmaier, Goetz ; hotspot-runtime- > > dev at openjdk.java.net > > Subject: Re: RFR (M): 8203881: Print errornous size in > > NegativeArraySizeException > > > > Hi Goetz, > > > > On 28/05/2018 10:10 PM, Lindenmaier, Goetz wrote: > > > Hi David, > > > > > > thanks for having a look! > > > > > >> First I have corrected the typo in the synopsis: errornous -> > erroneous > > > Thanks! > > > > > >> Second, can you not use the err_msg function instead of the explicit > > >> ResourceMark + stringStream code? > > > Yes, that's much better. New webrev: > > > http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg- > > NegativeArraySize/02 > > > > The functional change seems fine. > > > > For the test have you checked that all 7 changes are hit by the > > testcase? I'm curious as to what kinds of array allocations hit the > > different bits of code. I confess I'm not sure what a typeArray is > > versus an objArray. I only expected to see 4 cases here: reference or > > primitive plus single-dim or multi. Plus the two reflection cases. That > > leaves one I can't account for. :) > > > > Possibly overkill to test (implicit) interpreter plus C1 plus C2, given > > there is no JIT specific code involved - at least I can't see any. > > > > Thanks, > > David > > > > > Best regards, > > > Goetz. > > > > > > > > > > > >> > > >> Thanks, > > >> David > > >> > > >> On 28/05/2018 8:40 PM, Lindenmaier, Goetz wrote: > > >>> Hi, > > >>> > > >>> This change adds printing the array size in case of negative array > size > > >> exception. > > >>> Please review. > > >>> http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg- > > >> NegativeArraySize/01/ > > >>> > > >>> Thanks, > > >>> Goetz. > > >>> > From goetz.lindenmaier at sap.com Wed May 30 10:00:47 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 30 May 2018 10:00:47 +0000 Subject: RFR (M): 8203881: Print errornous size in NegativeArraySizeException In-Reply-To: References: <0366de26b7004c519bd56e07b9186957@sap.com> <8e666f92-83ac-3127-da9a-aa918fc31422@oracle.com> <716e41b4591743a5b5bad26ddd084105@sap.com> Message-ID: <67c837f652004bf28813a280ca7ca749@sap.com> Hi Thomas, thanks for the review! ... The err_msg was requested by David. It's also used in other exceptions. So this should be fine, but obviously leaves room for improvement. Best regards, Goetz. > -----Original Message----- > From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] > Sent: Mittwoch, 30. Mai 2018 11:56 > To: Lindenmaier, Goetz > Cc: David Holmes ; hotspot-runtime- > dev at openjdk.java.net > Subject: Re: RFR (M): 8203881: Print errornous size in > NegativeArraySizeException > > Good for me too. > > ..Thomas > > On Wed 30. May 2018 at 11:14, Lindenmaier, Goetz > > > wrote: > > > Hi David, > > I checked the test, and added cases for multidimensional > arrays where the first dimension is 0. A test for arrayKlass.cpp, > ArrayKlass::allocate_arrayArray() is missing, though. > I don't think this is critical as the change is quite simple. > > New webrev, (only test changes): > http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg- > NegativeArraySize/03/ > > Best regards, > Goetz > > > > -----Original Message----- > > From: David Holmes [mailto:david.holmes at oracle.com > ] > > Sent: Montag, 28. Mai 2018 15:18 > > To: Lindenmaier, Goetz >; hotspot-runtime- > > dev at openjdk.java.net > > Subject: Re: RFR (M): 8203881: Print errornous size in > > NegativeArraySizeException > > > > Hi Goetz, > > > > On 28/05/2018 10:10 PM, Lindenmaier, Goetz wrote: > > > Hi David, > > > > > > thanks for having a look! > > > > > >> First I have corrected the typo in the synopsis: errornous -> > erroneous > > > Thanks! > > > > > >> Second, can you not use the err_msg function instead of the > explicit > > >> ResourceMark + stringStream code? > > > Yes, that's much better. New webrev: > > > http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg- > > NegativeArraySize/02 > > > > The functional change seems fine. > > > > For the test have you checked that all 7 changes are hit by the > > testcase? I'm curious as to what kinds of array allocations hit the > > different bits of code. I confess I'm not sure what a typeArray is > > versus an objArray. I only expected to see 4 cases here: reference > or > > primitive plus single-dim or multi. Plus the two reflection cases. > That > > leaves one I can't account for. :) > > > > Possibly overkill to test (implicit) interpreter plus C1 plus C2, given > > there is no JIT specific code involved - at least I can't see any. > > > > Thanks, > > David > > > > > Best regards, > > > Goetz. > > > > > > > > > > > >> > > >> Thanks, > > >> David > > >> > > >> On 28/05/2018 8:40 PM, Lindenmaier, Goetz wrote: > > >>> Hi, > > >>> > > >>> This change adds printing the array size in case of negative > array size > > >> exception. > > >>> Please review. > > >>> http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg- > > >> NegativeArraySize/01/ > > >>> > > >>> Thanks, > > >>> Goetz. > > >>> > From goetz.lindenmaier at sap.com Wed May 30 12:13:40 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 30 May 2018 12:13:40 +0000 Subject: RFR (M): 8203881: Print errornous size in NegativeArraySizeException In-Reply-To: <5a61aa84-024a-6cfc-c4ea-4776ec8b0252@oracle.com> References: <0366de26b7004c519bd56e07b9186957@sap.com> <8e666f92-83ac-3127-da9a-aa918fc31422@oracle.com> <716e41b4591743a5b5bad26ddd084105@sap.com> <5a61aa84-024a-6cfc-c4ea-4776ec8b0252@oracle.com> Message-ID: Hi, I figured it ... it's if the inner dimension is omitted: new byte[-1][] Added tests and will push after jdk-submit passed. Best regards, Goetz. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Mittwoch, 30. Mai 2018 11:47 > To: Lindenmaier, Goetz ; hotspot-runtime- > dev at openjdk.java.net > Subject: Re: RFR (M): 8203881: Print errornous size in > NegativeArraySizeException > > Hi Goetz, > > On 30/05/2018 7:14 PM, Lindenmaier, Goetz wrote: > > Hi David, > > > > I checked the test, and added cases for multidimensional > > arrays where the first dimension is 0. A test for arrayKlass.cpp, > > ArrayKlass::allocate_arrayArray() is missing, though. > > I don't think this is critical as the change is quite simple. > > Does make me wonder what code you need to write to exercise it though ?? > Maybe it's dead code :) > > > New webrev, (only test changes): > > http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg- > NegativeArraySize/03/ > > 29 * @compile NegativeArraySizeExceptionTest.java > 30 * @run main NegativeArraySizeExceptionTest > > You don't need a separate @compile, the @run will compile the test file. > > No need to see updated webrev. > > Thanks, > David > > > Best regards, > > Goetz > > > > > >> -----Original Message----- > >> From: David Holmes [mailto:david.holmes at oracle.com] > >> Sent: Montag, 28. Mai 2018 15:18 > >> To: Lindenmaier, Goetz ; hotspot-runtime- > >> dev at openjdk.java.net > >> Subject: Re: RFR (M): 8203881: Print errornous size in > >> NegativeArraySizeException > >> > >> Hi Goetz, > >> > >> On 28/05/2018 10:10 PM, Lindenmaier, Goetz wrote: > >>> Hi David, > >>> > >>> thanks for having a look! > >>> > >>>> First I have corrected the typo in the synopsis: errornous -> erroneous > >>> Thanks! > >>> > >>>> Second, can you not use the err_msg function instead of the explicit > >>>> ResourceMark + stringStream code? > >>> Yes, that's much better. New webrev: > >>> http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg- > >> NegativeArraySize/02 > >> > >> The functional change seems fine. > >> > >> For the test have you checked that all 7 changes are hit by the > >> testcase? I'm curious as to what kinds of array allocations hit the > >> different bits of code. I confess I'm not sure what a typeArray is > >> versus an objArray. I only expected to see 4 cases here: reference or > >> primitive plus single-dim or multi. Plus the two reflection cases. That > >> leaves one I can't account for. :) > >> > >> Possibly overkill to test (implicit) interpreter plus C1 plus C2, given > >> there is no JIT specific code involved - at least I can't see any. > >> > >> Thanks, > >> David > >> > >>> Best regards, > >>> Goetz. > >>> > >>> > >>> > >>>> > >>>> Thanks, > >>>> David > >>>> > >>>> On 28/05/2018 8:40 PM, Lindenmaier, Goetz wrote: > >>>>> Hi, > >>>>> > >>>>> This change adds printing the array size in case of negative array size > >>>> exception. > >>>>> Please review. > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8203881-exMsg- > >>>> NegativeArraySize/01/ > >>>>> > >>>>> Thanks, > >>>>> Goetz. > >>>>> From harold.seigel at oracle.com Wed May 30 12:58:36 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Wed, 30 May 2018 08:58:36 -0400 Subject: RFR 8202913: loader constraint message for fields specifies incorrect referring class In-Reply-To: References: <621e524d-f990-262f-143f-cbc095da4acc@oracle.com> Message-ID: Hi David, Thanks for the review! Harold On 5/25/2018 8:57 PM, David Holmes wrote: > On 26/05/2018 10:40 AM, David Holmes wrote: >> Hi Harold, >> >> I don't think this was the minimal fix needed to address the missing >> reference to the Child class, > > No I take that back. I messed up the original message in two ways and > this addresses both of them. It's also more consistent with the method > version. It also avoids confusing use of "referring class", by which I > meant the current class that holds the putfield/getfield, not the REFC > type of the putfield/getfield (which I just confused myself with!) > > Thanks, > David > ----- > >> but these messages are so complicated and unreadable now that I'm >> beyond trying to argue for their form. >> >> I agree with Goetz about moving the tests to where the existing ones >> are. >> >> Otherwise ok. >> >> Thanks, >> David >> >> On 25/05/2018 11:28 PM, Harold David Seigel wrote: >>> Hi, >>> >>> Please review this change to correct and simplify the error message >>> displayed when a loader constraint check fails when trying to access >>> a field. >>> >>> The old message (for this test case): >>> >>> ???? ?loader constraint violation: when resolving field "_field1" the >>> ??? class loader "" (instance of ClassLoaderForChildGrandFoo, >>> ??? child of "app" jdk.internal.loader.ClassLoaders$AppClassLoader) of >>> ??? the referring class, Parent, and the class loader "" >>> ??? (instance of ClassLoaderForParentFoo, child of "app" >>> ??? jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's >>> ??? resolved type, Foo, have different Class objects for that type >>> >>> The new message: >>> >>> ??? loader constraint violation: when resolving field "_field1" of type >>> ??? Foo, the class loader "" (instance of >>> ??? ClassLoaderForChildGrandFoo, child of "app" >>> ??? jdk.internal.loader.ClassLoaders$AppClassLoader) of the current >>> ??? class, Child, and the class loader "" (instance of >>> ??? ClassLoaderForParentFoo, child of "app" >>> ??? jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's >>> ??? defining type, Parent, have different Class objects for type Foo >>> >>> Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8202913/webrev/ >>> >>> JBS Bug:? https://bugs.openjdk.java.net/browse/JDK-8202913 >>> >>> This fix was tested with Mach5 tiers 1 and 2 tests and builds on >>> Linux-X64, Windows, Solaris Sparc, and Mac OS X, with tiers 3-5 >>> tests on Linux-x64, and with JCK-11 Lang and VM tests. >>> >>> Thanks, Harold >>> From harold.seigel at oracle.com Wed May 30 13:07:12 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Wed, 30 May 2018 09:07:12 -0400 Subject: RFR 8202913: loader constraint message for fields specifies incorrect referring class In-Reply-To: <7ed091c7498f44c1b6f2b8a4a5030c38@sap.com> References: <9a209744d62a4e56a5ec81ddb77afb1f@sap.com> <7ed091c7498f44c1b6f2b8a4a5030c38@sap.com> Message-ID: <56a039e3-5b91-dbea-ecaa-be58b0646823@oracle.com> Hi Goetz, Thanks for reviewing this change! Please review this updated webrev for this bug: http://cr.openjdk.java.net/~hseigel/bug_8202913.2/webrev/index.html The new exception message for the included test looks like this: ?loader constraint violation: when resolving field "_field1" of type pkg.Foo, the class loader "" (instance of pkg.ClassLoaderForChildGrandFoo, child of "app" jdk.internal.loader.ClassLoaders$AppClassLoader) of the current class, pkg.Child, and the class loader "" (instance of pkg.ClassLoaderForParentFoo, child of "app" jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's defining type, pkg.Parent, have different Class objects for type pkg.Foo Please also see comments in-line. On 5/25/2018 10:23 AM, Lindenmaier, Goetz wrote: > Hi, > >> I think this would read better if you replace 'type' by what is returned by >> Klass::external_kind(). > correcting myself, you can just say 'class' ... interfaces don't define fields ... Karen pointed out that interfaces can have public static fields. So, I'll keep it as 'type', not 'class'. > > Best regards, > Goetz. > > >> -----Original Message----- >> From: Lindenmaier, Goetz >> Sent: Freitag, 25. Mai 2018 16:18 >> To: 'Harold David Seigel' ; hotspot-runtime- >> dev at openjdk.java.net >> Subject: RE: RFR 8202913: loader constraint message for fields specifies >> incorrect referring class >> >> Hi Harold, >> >> Thanks for adding this further improvement over my change. >> >>> ... AppClassLoader) for the field's >>> defining type, Parent, have different Class objects for type Foo >> I think this would read better if you replace 'type' by what is returned by >> Klass::external_kind(). >> >> Actually I would prefer to read >> ... AppClassLoader) for the class|abstractclass|interface test.Parent defining >> this field, >> have different Class objects for type Foo The message is already too wordy.? I'd prefer to not add this additional text because it is not related to the reason for the exception. >> >> Could you please add the test files into some package so that you >> can assure class names are printed as test.Parent and not test/Parent? >> There is external_name() to get the proper names from classes, I don't >> know for symbols. >> Related tests are in runtime/LoaderConstraint. Please move your test >> there. Done. Thanks, Harold >> >> Best regards, >> Goetz. >> >> >> >> >>> -----Original Message----- >>> From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- >>> bounces at openjdk.java.net] On Behalf Of Harold David Seigel >>> Sent: Freitag, 25. Mai 2018 15:29 >>> To: hotspot-runtime-dev at openjdk.java.net >>> Subject: RFR 8202913: loader constraint message for fields specifies >> incorrect >>> referring class >>> >>> Hi, >>> >>> Please review this change to correct and simplify the error message >>> displayed when a loader constraint check fails when trying to access a >>> field. >>> >>> The old message (for this test case): >>> >>> ?loader constraint violation: when resolving field "_field1" the >>> class loader "" (instance of ClassLoaderForChildGrandFoo, >>> child of "app" jdk.internal.loader.ClassLoaders$AppClassLoader) of >>> the referring class, Parent, and the class loader "" >>> (instance of ClassLoaderForParentFoo, child of "app" >>> jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's >>> resolved type, Foo, have different Class objects for that type >>> >>> The new message: >>> >>> loader constraint violation: when resolving field "_field1" of type >>> Foo, the class loader "" (instance of >>> ClassLoaderForChildGrandFoo, child of "app" >>> jdk.internal.loader.ClassLoaders$AppClassLoader) of the current >>> class, Child, and the class loader "" (instance of >>> ClassLoaderForParentFoo, child of "app" >>> jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's >>> defining type, Parent, have different Class objects for type Foo >>> >>> Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8202913/webrev/ >>> >>> JBS Bug:? https://bugs.openjdk.java.net/browse/JDK-8202913 >>> >>> This fix was tested with Mach5 tiers 1 and 2 tests and builds on >>> Linux-X64, Windows, Solaris Sparc, and Mac OS X, with tiers 3-5 tests on >>> Linux-x64, and with JCK-11 Lang and VM tests. >>> >>> Thanks, Harold From karen.kinnear at oracle.com Wed May 30 13:17:08 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Wed, 30 May 2018 09:17:08 -0400 Subject: RFR 8202913: loader constraint message for fields specifies incorrect referring class In-Reply-To: <56a039e3-5b91-dbea-ecaa-be58b0646823@oracle.com> References: <9a209744d62a4e56a5ec81ddb77afb1f@sap.com> <7ed091c7498f44c1b6f2b8a4a5030c38@sap.com> <56a039e3-5b91-dbea-ecaa-be58b0646823@oracle.com> Message-ID: <20E4A7E2-B9AB-438D-8497-5A3D480C4B26@oracle.com> Harold, Looks good. thanks, Karen > On May 30, 2018, at 9:07 AM, Harold David Seigel wrote: > > Hi Goetz, > > Thanks for reviewing this change! > > Please review this updated webrev for this bug: > > http://cr.openjdk.java.net/~hseigel/bug_8202913.2/webrev/index.html > > The new exception message for the included test looks like this: > > loader constraint violation: when resolving field "_field1" of > type pkg.Foo, the class loader "" (instance of > pkg.ClassLoaderForChildGrandFoo, child of "app" > jdk.internal.loader.ClassLoaders$AppClassLoader) of the current > class, pkg.Child, and the class loader "" (instance of > pkg.ClassLoaderForParentFoo, child of "app" > jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's > defining type, pkg.Parent, have different Class objects for type pkg.Foo > > Please also see comments in-line. > > > On 5/25/2018 10:23 AM, Lindenmaier, Goetz wrote: >> Hi, >> >>> I think this would read better if you replace 'type' by what is returned by >>> Klass::external_kind(). >> correcting myself, you can just say 'class' ... interfaces don't define fields ... > Karen pointed out that interfaces can have public static fields. So, I'll keep it as 'type', not 'class'. >> >> Best regards, >> Goetz. >> >> >>> -----Original Message----- >>> From: Lindenmaier, Goetz >>> Sent: Freitag, 25. Mai 2018 16:18 >>> To: 'Harold David Seigel' ; hotspot-runtime- >>> dev at openjdk.java.net >>> Subject: RE: RFR 8202913: loader constraint message for fields specifies >>> incorrect referring class >>> >>> Hi Harold, >>> >>> Thanks for adding this further improvement over my change. >>> >>>> ... AppClassLoader) for the field's >>>> defining type, Parent, have different Class objects for type Foo >>> I think this would read better if you replace 'type' by what is returned by >>> Klass::external_kind(). >>> >>> Actually I would prefer to read >>> ... AppClassLoader) for the class|abstractclass|interface test.Parent defining >>> this field, >>> have different Class objects for type Foo > The message is already too wordy. I'd prefer to not add this additional text because it is not related to the reason for the exception. >>> >>> Could you please add the test files into some package so that you >>> can assure class names are printed as test.Parent and not test/Parent? >>> There is external_name() to get the proper names from classes, I don't >>> know for symbols. >>> Related tests are in runtime/LoaderConstraint. Please move your test >>> there. > Done. > > Thanks, Harold >>> >>> Best regards, >>> Goetz. >>> >>> >>> >>> >>>> -----Original Message----- >>>> From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- >>>> bounces at openjdk.java.net] On Behalf Of Harold David Seigel >>>> Sent: Freitag, 25. Mai 2018 15:29 >>>> To: hotspot-runtime-dev at openjdk.java.net >>>> Subject: RFR 8202913: loader constraint message for fields specifies >>> incorrect >>>> referring class >>>> >>>> Hi, >>>> >>>> Please review this change to correct and simplify the error message >>>> displayed when a loader constraint check fails when trying to access a >>>> field. >>>> >>>> The old message (for this test case): >>>> >>>> loader constraint violation: when resolving field "_field1" the >>>> class loader "" (instance of ClassLoaderForChildGrandFoo, >>>> child of "app" jdk.internal.loader.ClassLoaders$AppClassLoader) of >>>> the referring class, Parent, and the class loader "" >>>> (instance of ClassLoaderForParentFoo, child of "app" >>>> jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's >>>> resolved type, Foo, have different Class objects for that type >>>> >>>> The new message: >>>> >>>> loader constraint violation: when resolving field "_field1" of type >>>> Foo, the class loader "" (instance of >>>> ClassLoaderForChildGrandFoo, child of "app" >>>> jdk.internal.loader.ClassLoaders$AppClassLoader) of the current >>>> class, Child, and the class loader "" (instance of >>>> ClassLoaderForParentFoo, child of "app" >>>> jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's >>>> defining type, Parent, have different Class objects for type Foo >>>> >>>> Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8202913/webrev/ >>>> >>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8202913 >>>> >>>> This fix was tested with Mach5 tiers 1 and 2 tests and builds on >>>> Linux-X64, Windows, Solaris Sparc, and Mac OS X, with tiers 3-5 tests on >>>> Linux-x64, and with JCK-11 Lang and VM tests. >>>> >>>> Thanks, Harold From harold.seigel at oracle.com Wed May 30 13:16:54 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Wed, 30 May 2018 09:16:54 -0400 Subject: RFR 8202913: loader constraint message for fields specifies incorrect referring class In-Reply-To: <20E4A7E2-B9AB-438D-8497-5A3D480C4B26@oracle.com> References: <9a209744d62a4e56a5ec81ddb77afb1f@sap.com> <7ed091c7498f44c1b6f2b8a4a5030c38@sap.com> <56a039e3-5b91-dbea-ecaa-be58b0646823@oracle.com> <20E4A7E2-B9AB-438D-8497-5A3D480C4B26@oracle.com> Message-ID: Thanks Karen! Harold On 5/30/2018 9:17 AM, Karen Kinnear wrote: > Harold, > > Looks good. > > thanks, > Karen > >> On May 30, 2018, at 9:07 AM, Harold David Seigel >> > wrote: >> >> Hi Goetz, >> >> Thanks for reviewing this change! >> >> Please review this updated webrev for this bug: >> >> http://cr.openjdk.java.net/~hseigel/bug_8202913.2/webrev/index.html >> >> >> The new exception message for the included test looks like this: >> >> ????loader constraint violation: when resolving field "_field1" of >> ??type pkg.Foo, the class loader "" (instance of >> ??pkg.ClassLoaderForChildGrandFoo, child of "app" >> ??jdk.internal.loader.ClassLoaders$AppClassLoader) of the current >> ??class, pkg.Child, and the class loader "" (instance of >> ??pkg.ClassLoaderForParentFoo, child of "app" >> ??jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's >> ??defining type, pkg.Parent, have different Class objects for type >> pkg.Foo >> >> Please also see comments in-line. >> >> >> On 5/25/2018 10:23 AM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>>> I think this would read better if you replace 'type' by what is >>>> returned by >>>> Klass::external_kind(). >>> correcting myself, you can just say 'class' ... interfaces don't >>> define fields ... >> Karen pointed out that interfaces can have public static fields. So, >> I'll keep it as 'type', not 'class'. >>> >>> Best regards, >>> ??Goetz. >>> >>> >>>> -----Original Message----- >>>> From: Lindenmaier, Goetz >>>> Sent: Freitag, 25. Mai 2018 16:18 >>>> To: 'Harold David Seigel' >>> >; hotspot-runtime- >>>> dev at openjdk.java.net >>>> Subject: RE: RFR 8202913: loader constraint message for fields >>>> specifies >>>> incorrect referring class >>>> >>>> Hi Harold, >>>> >>>> Thanks for adding this further improvement over my change. >>>> >>>>> ... AppClassLoader) for the field's >>>>> ?defining type, Parent, have different Class objects for type Foo >>>> I think this would read better if you replace 'type' by what is >>>> returned by >>>> Klass::external_kind(). >>>> >>>> Actually I would prefer to read >>>> ... AppClassLoader) for the class|abstractclass|interface >>>> test.Parent defining >>>> this field, >>>> have different Class objects for type Foo >> The message is already too wordy.? I'd prefer to not add this >> additional text because it is not related to the reason for the >> exception. >>>> >>>> Could you please add the test files into some package so that you >>>> can assure class names are printed as test.Parent and not test/Parent? >>>> There is external_name() to get the proper names from classes, I don't >>>> know for symbols. >>>> Related tests are in runtime/LoaderConstraint. Please move your test >>>> there. >> Done. >> >> Thanks, Harold >>>> >>>> Best regards, >>>> ??Goetz. >>>> >>>> >>>> >>>> >>>>> -----Original Message----- >>>>> From: hotspot-runtime-dev [mailto:hotspot-runtime-dev- >>>>> bounces at openjdk.java.net ] On >>>>> Behalf Of Harold David Seigel >>>>> Sent: Freitag, 25. Mai 2018 15:29 >>>>> To: hotspot-runtime-dev at openjdk.java.net >>>>> >>>>> Subject: RFR 8202913: loader constraint message for fields specifies >>>> incorrect >>>>> referring class >>>>> >>>>> Hi, >>>>> >>>>> Please review this change to correct and simplify the error message >>>>> displayed when a loader constraint check fails when trying to access a >>>>> field. >>>>> >>>>> The old message (for this test case): >>>>> >>>>> ??????loader constraint violation: when resolving field "_field1" the >>>>> ????class loader "" (instance of ClassLoaderForChildGrandFoo, >>>>> ????child of "app" jdk.internal.loader.ClassLoaders$AppClassLoader) of >>>>> ????the referring class, Parent, and the class loader "" >>>>> ????(instance of ClassLoaderForParentFoo, child of "app" >>>>> ????jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's >>>>> ????resolved type, Foo, have different Class objects for that type >>>>> >>>>> The new message: >>>>> >>>>> ????loader constraint violation: when resolving field "_field1" of >>>>> type >>>>> ????Foo, the class loader "" (instance of >>>>> ????ClassLoaderForChildGrandFoo, child of "app" >>>>> ????jdk.internal.loader.ClassLoaders$AppClassLoader) of the current >>>>> ????class, Child, and the class loader "" (instance of >>>>> ????ClassLoaderForParentFoo, child of "app" >>>>> ????jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's >>>>> ????defining type, Parent, have different Class objects for type Foo >>>>> >>>>> Open Webrev: >>>>> http://cr.openjdk.java.net/~hseigel/bug_8202913/webrev/ >>>>> >>>>> >>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8202913 >>>>> >>>>> This fix was tested with Mach5 tiers 1 and 2 tests and builds on >>>>> Linux-X64, Windows, Solaris Sparc, and Mac OS X, with tiers 3-5 >>>>> tests on >>>>> Linux-x64, and with JCK-11 Lang and VM tests. >>>>> >>>>> Thanks, Harold > From thomas.stuefe at gmail.com Wed May 30 13:57:57 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 30 May 2018 15:57:57 +0200 Subject: RFR(s): 8203219: VM.metaspace jcmd should optionally show loaded classes for loaders In-Reply-To: References: Message-ID: Ping... All tests passed on jdk-submit. Thanks for any reviews. Thomas On Sun, May 20, 2018 at 8:58 AM, Thomas St?fe wrote: > Hi all, > > can I have reviews for this small addition to the VM.metaspace jcmd, > which adds the ability to list classes loaded by each class loader? > > Thank you! > > bug: https://bugs.openjdk.java.net/browse/JDK-8203219 > webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/webrev.00/webrev/index.html > > Example output: > > http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/example.txt > > Kind Regards, Thomas From zgu at redhat.com Wed May 30 14:08:40 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 30 May 2018 10:08:40 -0400 Subject: RFR(s): 8203219: VM.metaspace jcmd should optionally show loaded classes for loaders In-Reply-To: References: Message-ID: <1c7d51c6-77b6-897b-d841-d94815458b9c@redhat.com> Useful information! and looks good to me. Thanks, -Zhengyu On 05/30/2018 09:57 AM, Thomas St?fe wrote: > Ping... > > All tests passed on jdk-submit. > > Thanks for any reviews. > > Thomas > > On Sun, May 20, 2018 at 8:58 AM, Thomas St?fe wrote: >> Hi all, >> >> can I have reviews for this small addition to the VM.metaspace jcmd, >> which adds the ability to list classes loaded by each class loader? >> >> Thank you! >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8203219 >> webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/webrev.00/webrev/index.html >> >> Example output: >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/example.txt >> >> Kind Regards, Thomas From thomas.stuefe at gmail.com Wed May 30 14:38:10 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 30 May 2018 16:38:10 +0200 Subject: RFR(s): 8203219: VM.metaspace jcmd should optionally show loaded classes for loaders In-Reply-To: <1c7d51c6-77b6-897b-d841-d94815458b9c@redhat.com> References: <1c7d51c6-77b6-897b-d841-d94815458b9c@redhat.com> Message-ID: :) Thank as usual! On Wed, May 30, 2018 at 4:08 PM, Zhengyu Gu wrote: > Useful information! and looks good to me. > > Thanks, > > -Zhengyu > > > On 05/30/2018 09:57 AM, Thomas St?fe wrote: >> >> Ping... >> >> All tests passed on jdk-submit. >> >> Thanks for any reviews. >> >> Thomas >> >> On Sun, May 20, 2018 at 8:58 AM, Thomas St?fe >> wrote: >>> >>> Hi all, >>> >>> can I have reviews for this small addition to the VM.metaspace jcmd, >>> which adds the ability to list classes loaded by each class loader? >>> >>> Thank you! >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8203219 >>> webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/webrev.00/webrev/index.html >>> >>> Example output: >>> >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/example.txt >>> >>> Kind Regards, Thomas From coleen.phillimore at oracle.com Wed May 30 14:57:33 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 30 May 2018 10:57:33 -0400 Subject: RFR(s): 8203219: VM.metaspace jcmd should optionally show loaded classes for loaders In-Reply-To: References: Message-ID: This seems okay.? There is already a jcmd VM.systemdictionary that prints all the classes per class loader. Coleen On 5/30/18 9:57 AM, Thomas St?fe wrote: > Ping... > > All tests passed on jdk-submit. > > Thanks for any reviews. > > Thomas > > On Sun, May 20, 2018 at 8:58 AM, Thomas St?fe wrote: >> Hi all, >> >> can I have reviews for this small addition to the VM.metaspace jcmd, >> which adds the ability to list classes loaded by each class loader? >> >> Thank you! >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8203219 >> webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/webrev.00/webrev/index.html >> >> Example output: >> >> http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/example.txt >> >> Kind Regards, Thomas From thomas.stuefe at gmail.com Wed May 30 15:09:52 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 30 May 2018 17:09:52 +0200 Subject: RFR(s): 8203219: VM.metaspace jcmd should optionally show loaded classes for loaders In-Reply-To: References: Message-ID: Thank you Coleen! On Wed, May 30, 2018 at 4:57 PM, wrote: > > This seems okay. There is already a jcmd VM.systemdictionary that prints > all the classes per class loader. > Would you be okay with still putting this improvement in? I think the added benefit for the users of VM.metaspace outweighs the additional cost in code complexity. Thanks, Thomas > Coleen > > > On 5/30/18 9:57 AM, Thomas St?fe wrote: >> >> Ping... >> >> All tests passed on jdk-submit. >> >> Thanks for any reviews. >> >> Thomas >> >> On Sun, May 20, 2018 at 8:58 AM, Thomas St?fe >> wrote: >>> >>> Hi all, >>> >>> can I have reviews for this small addition to the VM.metaspace jcmd, >>> which adds the ability to list classes loaded by each class loader? >>> >>> Thank you! >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8203219 >>> webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/webrev.00/webrev/index.html >>> >>> Example output: >>> >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/example.txt >>> >>> Kind Regards, Thomas > > From coleen.phillimore at oracle.com Wed May 30 15:22:25 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 30 May 2018 11:22:25 -0400 Subject: RFR(s): 8203219: VM.metaspace jcmd should optionally show loaded classes for loaders In-Reply-To: References: Message-ID: <5a49e9cc-88d9-07b6-2eb8-8ed2c9cc7e21@oracle.com> On 5/30/18 11:09 AM, Thomas St?fe wrote: > Thank you Coleen! > > On Wed, May 30, 2018 at 4:57 PM, wrote: >> This seems okay. There is already a jcmd VM.systemdictionary that prints >> all the classes per class loader. >> > Would you be okay with still putting this improvement in? I think the > added benefit for the users of VM.metaspace outweighs the additional > cost in code complexity. Since it's small and sort of different and the output seems compact enough, I think it's ok.?? The system dictionary output shows class loading that has been initiated by each loader as well, so it's a different view of the information. thanks, Coleen > > Thanks, Thomas > >> Coleen >> >> >> On 5/30/18 9:57 AM, Thomas St?fe wrote: >>> Ping... >>> >>> All tests passed on jdk-submit. >>> >>> Thanks for any reviews. >>> >>> Thomas >>> >>> On Sun, May 20, 2018 at 8:58 AM, Thomas St?fe >>> wrote: >>>> Hi all, >>>> >>>> can I have reviews for this small addition to the VM.metaspace jcmd, >>>> which adds the ability to list classes loaded by each class loader? >>>> >>>> Thank you! >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8203219 >>>> webrev: >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/webrev.00/webrev/index.html >>>> >>>> Example output: >>>> >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/example.txt >>>> >>>> Kind Regards, Thomas >> From thomas.stuefe at gmail.com Wed May 30 15:24:51 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 30 May 2018 17:24:51 +0200 Subject: RFR(s): 8203219: VM.metaspace jcmd should optionally show loaded classes for loaders In-Reply-To: <5a49e9cc-88d9-07b6-2eb8-8ed2c9cc7e21@oracle.com> References: <5a49e9cc-88d9-07b6-2eb8-8ed2c9cc7e21@oracle.com> Message-ID: Thank you! Best Regards, Thomas On Wed, May 30, 2018 at 5:22 PM, wrote: > > > On 5/30/18 11:09 AM, Thomas St?fe wrote: >> >> Thank you Coleen! >> >> On Wed, May 30, 2018 at 4:57 PM, wrote: >>> >>> This seems okay. There is already a jcmd VM.systemdictionary that prints >>> all the classes per class loader. >>> >> Would you be okay with still putting this improvement in? I think the >> added benefit for the users of VM.metaspace outweighs the additional >> cost in code complexity. > > > Since it's small and sort of different and the output seems compact enough, > I think it's ok. The system dictionary output shows class loading that has > been initiated by each loader as well, so it's a different view of the > information. > > thanks, > Coleen > > >> >> Thanks, Thomas >> >>> Coleen >>> >>> >>> On 5/30/18 9:57 AM, Thomas St?fe wrote: >>>> >>>> Ping... >>>> >>>> All tests passed on jdk-submit. >>>> >>>> Thanks for any reviews. >>>> >>>> Thomas >>>> >>>> On Sun, May 20, 2018 at 8:58 AM, Thomas St?fe >>>> wrote: >>>>> >>>>> Hi all, >>>>> >>>>> can I have reviews for this small addition to the VM.metaspace jcmd, >>>>> which adds the ability to list classes loaded by each class loader? >>>>> >>>>> Thank you! >>>>> >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8203219 >>>>> webrev: >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/webrev.00/webrev/index.html >>>>> >>>>> Example output: >>>>> >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203219-VM.metaspace-show-loaded-classes/example.txt >>>>> >>>>> Kind Regards, Thomas >>> >>> > From goetz.lindenmaier at sap.com Wed May 30 15:40:46 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 30 May 2018 15:40:46 +0000 Subject: RFR 8202913: loader constraint message for fields specifies incorrect referring class In-Reply-To: <56a039e3-5b91-dbea-ecaa-be58b0646823@oracle.com> References: <9a209744d62a4e56a5ec81ddb77afb1f@sap.com> <7ed091c7498f44c1b6f2b8a4a5030c38@sap.com> <56a039e3-5b91-dbea-ecaa-be58b0646823@oracle.com> Message-ID: <1465e732076042298280084262f2aeba@sap.com> Hi Harold, > ... AppClassLoader) for the class|abstractclass|interface test.Parent defining I meant you should print the proper identifier, not the text with '|' by using "... %s ..." , ..., k->external_kind(), ... As this replaces 'type' by one or two (abstract class) words I don't think it makes the message more wordy, just more precise. See also patch below (just edited, not tested). Feel free to ignore my proposal, and else I don't need a new webrev in case you adapt this. The rest looks good. Great it's all with '.' and you moved the test! Best regards, Goetz. diff -r 6b4a85ad6bea src/hotspot/share/interpreter/linkResolver.cpp --- a/src/hotspot/share/interpreter/linkResolver.cpp Wed May 30 17:32:28 2018 +0200 +++ b/src/hotspot/share/interpreter/linkResolver.cpp Wed May 30 17:37:04 2018 +0200 @@ -690,19 +690,20 @@ const char* msg = "loader constraint violation: when resolving field" " \"%s\" of type %s, the class loader %s of the current class, " "%s, and the class loader %s for the field's defining " - "type, %s, have different Class objects for type %s"; + "%s, %s, have different Class objects for type %s"; const char* field_name = field->as_C_string(); const char* loader1_name = java_lang_ClassLoader::describe_external(ref_loader()); const char* sel = sel_klass->external_name(); + const char* sel_type = sel_klass->external_kind(); const char* loader2_name = java_lang_ClassLoader::describe_external(sel_loader()); const char* failed_type_name = failed_type_symbol->as_klass_external_name(); const char* curr_klass_name = current_klass->external_name(); size_t buflen = strlen(msg) + strlen(field_name) + 2 * strlen(failed_type_name) + strlen(loader1_name) + strlen(curr_klass_name) + - strlen(loader2_name) + strlen(sel) + 1; + strlen(loader2_name) + strlen(sel) + strlen(sel_type) + 1; char* buf = NEW_RESOURCE_ARRAY_IN_THREAD(THREAD, char, buflen); jio_snprintf(buf, buflen, msg, field_name, failed_type_name, loader1_name, - curr_klass_name, loader2_name, sel, failed_type_name); + curr_klass_name, loader2_name, sel_type, sel, failed_type_name); THROW_MSG(vmSymbols::java_lang_LinkageError(), buf); } } > -----Original Message----- > From: Harold David Seigel [mailto:harold.seigel at oracle.com] > Sent: Mittwoch, 30. Mai 2018 15:07 > To: Lindenmaier, Goetz ; hotspot-runtime- > dev at openjdk.java.net > Subject: Re: RFR 8202913: loader constraint message for fields specifies > incorrect referring class > > Hi Goetz, > > Thanks for reviewing this change! > > > Please review this updated webrev for this bug: > > http://cr.openjdk.java.net/~hseigel/bug_8202913.2/webrev/index.h > tml > > The new exception message for the included test looks like this: > > loader constraint violation: when resolving field "_field1" of type > pkg.Foo, the class loader "" (instance of > pkg.ClassLoaderForChildGrandFoo, child of "app" > jdk.internal.loader.ClassLoaders$AppClassLoader) of the current class, > pkg.Child, and the class loader "" (instance of > pkg.ClassLoaderForParentFoo, child of "app" > jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's defining > type, pkg.Parent, have different Class objects for type pkg.Foo > > > Please also see comments in-line. > > > > On 5/25/2018 10:23 AM, Lindenmaier, Goetz wrote: > > > Hi, > > > I think this would read better if you replace 'type' by what is > returned by > Klass::external_kind(). > > correcting myself, you can just say 'class' ... interfaces don't define > fields ... > > Karen pointed out that interfaces can have public static fields. So, I'll keep it > as 'type', not 'class'. > > > > > Best regards, > Goetz. > > > > -----Original Message----- > From: Lindenmaier, Goetz > Sent: Freitag, 25. Mai 2018 16:18 > To: 'Harold David Seigel' > ; hotspot-runtime- > dev at openjdk.java.net > Subject: RE: RFR 8202913: loader constraint message for fields > specifies > incorrect referring class > > Hi Harold, > > Thanks for adding this further improvement over my change. > > > ... AppClassLoader) for the field's > defining type, Parent, have different Class objects for > type Foo > > I think this would read better if you replace 'type' by what is > returned by > Klass::external_kind(). > > Actually I would prefer to read > ... AppClassLoader) for the class|abstractclass|interface > test.Parent defining > this field, > have different Class objects for type Foo > > The message is already too wordy. I'd prefer to not add this additional text > because it is not related to the reason for the exception. > > > > > Could you please add the test files into some package so that > you > can assure class names are printed as test.Parent and not > test/Parent? > There is external_name() to get the proper names from > classes, I don't > know for symbols. > Related tests are in runtime/LoaderConstraint. Please move > your test > there. > > Done. > > Thanks, Harold > > > > > Best regards, > Goetz. > > > > > > -----Original Message----- > From: hotspot-runtime-dev [mailto:hotspot-runtime- > dev- > bounces at openjdk.java.net > ] On Behalf Of Harold David Seigel > Sent: Freitag, 25. Mai 2018 15:29 > To: hotspot-runtime-dev at openjdk.java.net > > Subject: RFR 8202913: loader constraint message for > fields specifies > > incorrect > > referring class > > Hi, > > Please review this change to correct and simplify the > error message > displayed when a loader constraint check fails when > trying to access a > field. > > The old message (for this test case): > > loader constraint violation: when resolving field > "_field1" the > class loader "" (instance of > ClassLoaderForChildGrandFoo, > child of "app" > jdk.internal.loader.ClassLoaders$AppClassLoader) of > the referring class, Parent, and the class loader > "" > (instance of ClassLoaderForParentFoo, child of > "app" > jdk.internal.loader.ClassLoaders$AppClassLoader) > for the field's > resolved type, Foo, have different Class objects for > that type > > The new message: > > loader constraint violation: when resolving field > "_field1" of type > Foo, the class loader "" (instance of > ClassLoaderForChildGrandFoo, child of "app" > jdk.internal.loader.ClassLoaders$AppClassLoader) > of the current > class, Child, and the class loader "" > (instance of > ClassLoaderForParentFoo, child of "app" > jdk.internal.loader.ClassLoaders$AppClassLoader) > for the field's > defining type, Parent, have different Class objects > for type Foo > > Open Webrev: > http://cr.openjdk.java.net/~hseigel/bug_8202913/webrev/ > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK- > 8202913 > > This fix was tested with Mach5 tiers 1 and 2 tests and > builds on > Linux-X64, Windows, Solaris Sparc, and Mac OS X, with > tiers 3-5 tests on > Linux-x64, and with JCK-11 Lang and VM tests. > > Thanks, Harold > > > From jiangli.zhou at Oracle.COM Wed May 30 19:47:55 2018 From: jiangli.zhou at Oracle.COM (Jiangli Zhou) Date: Wed, 30 May 2018 12:47:55 -0700 Subject: RFR(L): 8195097: Make it possible to process StringTable outside safepoint In-Reply-To: <0d09f2ad-47b9-2fca-e5ec-17547e923cbb@oracle.com> References: <0d09f2ad-47b9-2fca-e5ec-17547e923cbb@oracle.com> Message-ID: <14230A21-5B54-451C-884C-E9E922967A25@oracle.com> Hi Robbin, I mainly focused on the archived string part during review. Here are my comments, which are minor issues mostly. - stringTable.hpp Archived string is only supported with INCLUDE_CDS_JAVA_HEAP. Please add NOT_CDS_JAVA_HEAP_RETURN_(NULL) for lookup_shared() and create_archived_string() below so their call sites are handled properly when java heap object archiving is not supported. 153 oop lookup_shared(jchar* name, int len, unsigned int hash); 154 static oop create_archived_string(oop s, Thread* THREAD); - stringTable.cpp How about renaming CopyArchive to CopyToArchive, so it?s more descriptive? Looks like the ?bool? return type is not needed since we always return with true and the result is not checked. How able changing it to return ?void?? 774 bool operator()(WeakHandle* val) { - genCollectedHeap.cpp Based on the assert at line 863, looks like ?par_state_string? is not NULL when 'scope->n_threads() > 1?. Maybe the if condition at line 865 could be simplified to be just ?if (scope->n_threads() > 1)?? 862 // Either we should be single threaded or have a ParState 863 assert((scope->n_threads() <= 1) || par_state_string != NULL, "Parallel but not ParState"); 864 865 if (scope->n_threads() > 1 && par_state_string != NULL) { Thanks, Jiangli > On May 28, 2018, at 6:19 AM, Robbin Ehn wrote: > > Hi all, please review. > > This implements the StringTable with the ConcurrentHashtable for managing the > strings using oopStorage for backing the actual oops via WeakHandles. > > The unlinking and freeing of hashtable nodes is moved outside the safepoint, > which means GC only needs to walk the oopStorage, either concurrently or in a > safepoint. Walking oopStorage is also faster so there is a good effect on all > safepoints visiting the oops. > > The unlinking and freeing happens during inserts when dead weak oops are > encountered in that bucket. In any normal workload the stringtable self-cleans > without needing any additional cleaning. Cleaning/unlinking can also be done > concurrently via the ServiceThread, it is started when we have a high ?dead > factor?. E.g. application have a lot of interned string removes the references > and never interns again. The ServiceThread also concurrently grows the table if > ?load factor? is high. Both the cleaning and growing take care to not prolonging > time to safepoint, at the cost of some speed. > > Kitchensink24h, multiple tier1-5 with no issue that I can relate to this > changeset, various benchmark such as JMH, specJBB2015. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8195097 > Webrev: http://cr.openjdk.java.net/~rehn/8195097/v0/webrev/ > > Thanks, Robbin From robbin.ehn at oracle.com Wed May 30 20:25:47 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 30 May 2018 22:25:47 +0200 Subject: RFR(M): JDK-8203321: Windows: assert(current_query_index < process_query_set->size) failed: invariant In-Reply-To: <06e925c8-8d77-43c8-9ef2-ad9113034b98@default> References: <06e925c8-8d77-43c8-9ef2-ad9113034b98@default> Message-ID: <14f593dc-11e0-770d-4d70-155edfbc3be3@oracle.com> Hi Markus, I'm not familiar with PDH, but given your description and the code looks good. Thanks, Robbin On 2018-05-22 21:24, Markus Gronlund wrote: > Greetings, > > Kindly asking for reviews for the following change: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203321 > Webrev: http://cr.openjdk.java.net/~mgronlun/8203321/webrev00/ > > Summary: > > For some context about what this is about, please see this (now) relatively old issue: https://bugs.openjdk.java.net/browse/JDK-8019921 > > The porting work that brought this code from closed to open were optimistic in that the following PDH query, "\Process(java#n)\ID Process", performed relatively stable on Windows 10. > An invariant was added in that your ID Process query would never return an index that was lower than the index at construction. > > During testing, it was discovered that this invariant did not hold, especially when running on Windows Server 2012 R2 and there is a high churn rate with many processes with the same base name ("java#") starting and stopping (stressing PDH list of processes). > > We have to reinsert back the original code that handled the case where the PDH process list is not stable (that were originally put in place with JDK-8019921). > The defensive logic is located at lines 418 - 422. > > I had to rework some related code to make some room for this as well as to keep track of the previous process index. > > Thanks > Markus > From ioi.lam at oracle.com Wed May 30 23:39:57 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 30 May 2018 16:39:57 -0700 Subject: RFR(XXS): 8188109 JVM should print a warning message that -Xshare:on may cause VM to abort start-up Message-ID: <2b1d7829-0dc1-bf7b-5e2d-426783738c1b@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8188109 http://cr.openjdk.java.net/~iklam/jdk11/8188109-xshare-on-print-warning.v01/ Hi, Please review this one-liner patch. -Xshare:on may cause infrequent/intermittent start-up failure due to the presence of Address Space Layout Randomization (ASLR). This option is intended for testing (the internals of CDS) only and should not be used in production environments. With this patch, the following warning message is printed when -Xshare:on is specified: $ java -Xshare:on -version Java HotSpot(TM) 64-Bit Server VM warning: -Xshare:on is for testing purpose only and may cause JVM start-up failure. Use -Xshare:auto instead. java version "11-internal" 2018-09-25 Java(TM) SE Runtime Environment 18.9 (fastdebug build 11-internal+0-adhoc.iklam.open) Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build 11-internal+0-adhoc.iklam.open, mixed mode, sharing) ?? --- vs --- $ java-Xshare:auto -version java version "11-internal" 2018-09-25 Java(TM) SE Runtime Environment 18.9 (fastdebug build 11-internal+0-adhoc.iklam.open) Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build 11-internal+0-adhoc.iklam.open, mixed mode, sharing) I am testing with HotSpot tiers 1-3 to make sure the tests don't get tripped by this new warning message. Thanks - Ioi From david.holmes at oracle.com Thu May 31 01:47:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 31 May 2018 11:47:23 +1000 Subject: RFR(XXS): 8188109 JVM should print a warning message that -Xshare:on may cause VM to abort start-up In-Reply-To: <2b1d7829-0dc1-bf7b-5e2d-426783738c1b@oracle.com> References: <2b1d7829-0dc1-bf7b-5e2d-426783738c1b@oracle.com> Message-ID: <288dfdce-57a5-e28e-f4e5-6ac96d87bb98@oracle.com> Hi Ioi, Sorry but this troubles me ... On 31/05/2018 9:39 AM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8188109 > http://cr.openjdk.java.net/~iklam/jdk11/8188109-xshare-on-print-warning.v01/ > > > > Hi, > > Please review this one-liner patch. > > -Xshare:on may cause infrequent/intermittent start-up failure due to the > presence of Address Space Layout Randomization (ASLR). This option is > intended for testing (the internals of CDS) only and should not be used > in production environments. > > With this patch, the following warning message is printed when > -Xshare:on is specified: > > $ java -Xshare:on -version > Java HotSpot(TM) 64-Bit Server VM warning: -Xshare:on is for testing > purpose only and may cause JVM start-up failure. Use -Xshare:auto instead. > java version "11-internal" 2018-09-25 > Java(TM) SE Runtime Environment 18.9 (fastdebug build > 11-internal+0-adhoc.iklam.open) > Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build > 11-internal+0-adhoc.iklam.open, mixed mode, sharing) So should this warning only be enabled in product builds? Even then it may be annoying for anyone who runs with -Xshare:on as they've set up CDS as documented [1][2] and they know their environment works ok - now they get a warning. Also I'm unclear how "on" fails due to ASLR but "auto" keeps going? Maybe only if the archive mapping fails and "on" was used then give a warning? Or just improve the message given when the VM aborts? Thanks, David [1] https://docs.oracle.com/javase/10/vm/class-data-sharing.htm#JSJVM-GUID-0260F857-A70E-4399-A1DF-A5766BE33285 [2] https://docs.oracle.com/javase/10/tools/java.htm#JSWOR-GUID-31503FCE-93D0-4175-9B4F-F6A738B2F4C4 > ?? --- vs --- > > $ java-Xshare:auto -version > java version "11-internal" 2018-09-25 > Java(TM) SE Runtime Environment 18.9 (fastdebug build > 11-internal+0-adhoc.iklam.open) > Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build > 11-internal+0-adhoc.iklam.open, mixed mode, sharing) > > I am testing with HotSpot tiers 1-3 to make sure the tests don't get > tripped by this new warning message. > > Thanks > - Ioi From calvin.cheung at oracle.com Thu May 31 03:58:48 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 30 May 2018 20:58:48 -0700 Subject: RFR(xs): 8203960: [TESTBUG] runtime/logging/DefaultMethodsTest.java failed when running in CDS mode Message-ID: <5B0F72F8.6020503@oracle.com> JBS: https://bugs.openjdk.java.net/browse/JDK-8203960 webrev: http://cr.openjdk.java.net/~ccheung/8203960/webrev.00/ If this test is run in CDS mode, most of the system classes will be in the CDS archive and loading of those classes from the archive will bypass the default method processing. The test fails in CDS mode since it expects the trace output from default method processing. A fix is to load an additional class which isn't in any of the modules defined by default. The loading of the additional class will trigger default method processing. Ran the test with and without CDS. I will do a sanity hs-tier1 and hs-tier2 testing run. thanks, Calvin From ioi.lam at oracle.com Thu May 31 04:01:00 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 30 May 2018 21:01:00 -0700 Subject: RFR(XXS): 8188109 JVM should print a warning message that -Xshare:on may cause VM to abort start-up In-Reply-To: <288dfdce-57a5-e28e-f4e5-6ac96d87bb98@oracle.com> References: <2b1d7829-0dc1-bf7b-5e2d-426783738c1b@oracle.com> <288dfdce-57a5-e28e-f4e5-6ac96d87bb98@oracle.com> Message-ID: On 5/30/18 6:47 PM, David Holmes wrote: > Hi Ioi, > > Sorry but this troubles me ... > > On 31/05/2018 9:39 AM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8188109 >> http://cr.openjdk.java.net/~iklam/jdk11/8188109-xshare-on-print-warning.v01/ >> >> >> >> Hi, >> >> Please review this one-liner patch. >> >> -Xshare:on may cause infrequent/intermittent start-up failure due to >> the presence of Address Space Layout Randomization (ASLR). This >> option is intended for testing (the internals of CDS) only and should >> not be used in production environments. >> >> With this patch, the following warning message is printed when >> -Xshare:on is specified: >> >> $ java -Xshare:on -version >> Java HotSpot(TM) 64-Bit Server VM warning: -Xshare:on is for testing >> purpose only and may cause JVM start-up failure. Use -Xshare:auto >> instead. >> java version "11-internal" 2018-09-25 >> Java(TM) SE Runtime Environment 18.9 (fastdebug build >> 11-internal+0-adhoc.iklam.open) >> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) > > So should this warning only be enabled in product builds? > > Even then it may be annoying for anyone who runs with -Xshare:on as > they've set up CDS as documented [1][2] and they know their > environment works ok - now they get a warning. > > Also I'm unclear how "on" fails due to ASLR but "auto" keeps going? > The documentation [1] says: -Xshare:on To enable class data sharing. If class data sharing can't be enabled, print an error message and exit. -Xshare:auto To enable class data sharing by default. Enable class data sharing whenever possible. So if mapping fails due to ASLR, "on" will exit and "auto" will disable CDS and continue . The documentation in [2] is wrong.? It says "Ensure that you have specified the option -Xshare:on or -Xshare:auto.", but -Xshare:on should not be used in production environments. I have filed a doc bug for this (https://bugs.openjdk.java.net/browse/JDK-8204141). The main reason for doing this REF is -- if people have been following [2] and using -Xshare:on, their setup is NOT OK. ASLR may happen just very rarely, but you don't want your program suddenly failing (e.g., if some admin has turned on more aggressive ASLR settings). As more people are moving their Java workload to micro-services type of environments, JVM launches will happen more, and there will be more chances of running into the ASLR issue. Therefore, we should fix the docs, and warn people that they should switch to -Xshare:auto. > Maybe only if the archive mapping fails and "on" was used then give a > warning? Or just improve the message given when the VM aborts? > That's already too late, especially for people running critical services. We want people to see this warning and actively fix their scripts to get rid -Xshare:on. Thanks - Ioi > > Thanks, > David > > [1] > https://docs.oracle.com/javase/10/vm/class-data-sharing.htm#JSJVM-GUID-0260F857-A70E-4399-A1DF-A5766BE33285 > [2] > https://docs.oracle.com/javase/10/tools/java.htm#JSWOR-GUID-31503FCE-93D0-4175-9B4F-F6A738B2F4C4 > >> ??? --- vs --- >> >> $ java-Xshare:auto -version >> java version "11-internal" 2018-09-25 >> Java(TM) SE Runtime Environment 18.9 (fastdebug build >> 11-internal+0-adhoc.iklam.open) >> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) >> >> I am testing with HotSpot tiers 1-3 to make sure the tests don't get >> tripped by this new warning message. >> >> Thanks >> - Ioi From ioi.lam at oracle.com Thu May 31 04:03:18 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 30 May 2018 21:03:18 -0700 Subject: RFR(xs): 8203960: [TESTBUG] runtime/logging/DefaultMethodsTest.java failed when running in CDS mode In-Reply-To: <5B0F72F8.6020503@oracle.com> References: <5B0F72F8.6020503@oracle.com> Message-ID: Hi Calvin, Instead of relying on an internal JDK class to have default method processing, maybe InnerClass should contain some code to ensure that default method processing will always happen? Thanks - Ioi On 5/30/18 8:58 PM, Calvin Cheung wrote: > JBS: https://bugs.openjdk.java.net/browse/JDK-8203960 > > webrev: http://cr.openjdk.java.net/~ccheung/8203960/webrev.00/ > > If this test is run in CDS mode, most of the system classes will be in > the CDS archive and loading of those classes from the archive will > bypass the default method processing. The test fails in CDS mode since > it expects the trace output from default method processing. > A fix is to load an additional class which isn't in any of the modules > defined by default. The loading of the additional class will trigger > default method processing. > > Ran the test with and without CDS. > I will do a sanity hs-tier1 and hs-tier2 testing run. > > thanks, > Calvin From ioi.lam at oracle.com Thu May 31 04:07:55 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 30 May 2018 21:07:55 -0700 Subject: RFR(XXS): 8188109 JVM should print a warning message that -Xshare:on may cause VM to abort start-up In-Reply-To: References: <2b1d7829-0dc1-bf7b-5e2d-426783738c1b@oracle.com> <288dfdce-57a5-e28e-f4e5-6ac96d87bb98@oracle.com> Message-ID: <53ae993e-2eea-523c-fc35-6617f475ca3e@oracle.com> Another option is to change the behavior of -Xshare:on to be the same as -Xshare:auto. CDS is supposed to be a caching optimization. The VM should not fail just because an optimization doesn't pan out. I can file a CSR for this. What do you think? Thanks - Ioi On 5/30/18 9:01 PM, Ioi Lam wrote: > > > On 5/30/18 6:47 PM, David Holmes wrote: >> Hi Ioi, >> >> Sorry but this troubles me ... >> >> On 31/05/2018 9:39 AM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8188109 >>> http://cr.openjdk.java.net/~iklam/jdk11/8188109-xshare-on-print-warning.v01/ >>> >>> >>> >>> Hi, >>> >>> Please review this one-liner patch. >>> >>> -Xshare:on may cause infrequent/intermittent start-up failure due to >>> the presence of Address Space Layout Randomization (ASLR). This >>> option is intended for testing (the internals of CDS) only and >>> should not be used in production environments. >>> >>> With this patch, the following warning message is printed when >>> -Xshare:on is specified: >>> >>> $ java -Xshare:on -version >>> Java HotSpot(TM) 64-Bit Server VM warning: -Xshare:on is for testing >>> purpose only and may cause JVM start-up failure. Use -Xshare:auto >>> instead. >>> java version "11-internal" 2018-09-25 >>> Java(TM) SE Runtime Environment 18.9 (fastdebug build >>> 11-internal+0-adhoc.iklam.open) >>> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >>> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) >> >> So should this warning only be enabled in product builds? >> >> Even then it may be annoying for anyone who runs with -Xshare:on as >> they've set up CDS as documented [1][2] and they know their >> environment works ok - now they get a warning. >> >> Also I'm unclear how "on" fails due to ASLR but "auto" keeps going? >> > The documentation [1] says: > > -Xshare:on > To enable class data sharing. If class data sharing can't be enabled, > print an error message and exit. > > -Xshare:auto > To enable class data sharing by default. Enable class data sharing > whenever possible. > > So if mapping fails due to ASLR, "on" will exit and "auto" will > disable CDS and continue . > > The documentation in [2] is wrong.? It says "Ensure that you have > specified the option -Xshare:on or -Xshare:auto.", but -Xshare:on > should not be used in production environments. I have filed a doc bug > for this (https://bugs.openjdk.java.net/browse/JDK-8204141). > > The main reason for doing this REF is -- if people have been following > [2] and using -Xshare:on, their setup is NOT OK. ASLR may happen just > very rarely, but you don't want your program suddenly failing (e.g., > if some admin has turned on more aggressive ASLR settings). > > As more people are moving their Java workload to micro-services type > of environments, JVM launches will happen more, and there will be more > chances of running into the ASLR issue. Therefore, we should fix the > docs, and warn people that they should switch to -Xshare:auto. > >> Maybe only if the archive mapping fails and "on" was used then give a >> warning? Or just improve the message given when the VM aborts? >> > That's already too late, especially for people running critical services. > > We want people to see this warning and actively fix their scripts to > get rid -Xshare:on. > > Thanks > - Ioi > > >> >> Thanks, >> David >> >> [1] >> https://docs.oracle.com/javase/10/vm/class-data-sharing.htm#JSJVM-GUID-0260F857-A70E-4399-A1DF-A5766BE33285 >> [2] >> https://docs.oracle.com/javase/10/tools/java.htm#JSWOR-GUID-31503FCE-93D0-4175-9B4F-F6A738B2F4C4 >> >>> ??? --- vs --- >>> >>> $ java-Xshare:auto -version >>> java version "11-internal" 2018-09-25 >>> Java(TM) SE Runtime Environment 18.9 (fastdebug build >>> 11-internal+0-adhoc.iklam.open) >>> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >>> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) >>> >>> I am testing with HotSpot tiers 1-3 to make sure the tests don't get >>> tripped by this new warning message. >>> >>> Thanks >>> - Ioi > From david.holmes at oracle.com Thu May 31 04:13:54 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 31 May 2018 14:13:54 +1000 Subject: RFR(XXS): 8188109 JVM should print a warning message that -Xshare:on may cause VM to abort start-up In-Reply-To: References: <2b1d7829-0dc1-bf7b-5e2d-426783738c1b@oracle.com> <288dfdce-57a5-e28e-f4e5-6ac96d87bb98@oracle.com> Message-ID: Hi Ioi, On 31/05/2018 2:01 PM, Ioi Lam wrote: > On 5/30/18 6:47 PM, David Holmes wrote: >> Hi Ioi, >> >> Sorry but this troubles me ... >> >> On 31/05/2018 9:39 AM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8188109 >>> http://cr.openjdk.java.net/~iklam/jdk11/8188109-xshare-on-print-warning.v01/ >>> >>> >>> >>> Hi, >>> >>> Please review this one-liner patch. >>> >>> -Xshare:on may cause infrequent/intermittent start-up failure due to >>> the presence of Address Space Layout Randomization (ASLR). This >>> option is intended for testing (the internals of CDS) only and should >>> not be used in production environments. >>> >>> With this patch, the following warning message is printed when >>> -Xshare:on is specified: >>> >>> $ java -Xshare:on -version >>> Java HotSpot(TM) 64-Bit Server VM warning: -Xshare:on is for testing >>> purpose only and may cause JVM start-up failure. Use -Xshare:auto >>> instead. >>> java version "11-internal" 2018-09-25 >>> Java(TM) SE Runtime Environment 18.9 (fastdebug build >>> 11-internal+0-adhoc.iklam.open) >>> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >>> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) >> >> So should this warning only be enabled in product builds? >> >> Even then it may be annoying for anyone who runs with -Xshare:on as >> they've set up CDS as documented [1][2] and they know their >> environment works ok - now they get a warning. >> >> Also I'm unclear how "on" fails due to ASLR but "auto" keeps going? >> > The documentation [1] says: > > -Xshare:on > To enable class data sharing. If class data sharing can't be enabled, > print an error message and exit. > > -Xshare:auto > To enable class data sharing by default. Enable class data sharing > whenever possible. > > So if mapping fails due to ASLR, "on" will exit and "auto" will disable > CDS and continue . Ah! In the bug you state "-Xshare:auto continue to execute (with CDS enabled)" - so that should be disabled. > The documentation in [2] is wrong.? It says "Ensure that you have > specified the option -Xshare:on or -Xshare:auto.", but -Xshare:on should > not be used in production environments. I have filed a doc bug for this > (https://bugs.openjdk.java.net/browse/JDK-8204141). > > The main reason for doing this REF is -- if people have been following > [2] and using -Xshare:on, their setup is NOT OK. ASLR may happen just > very rarely, but you don't want your program suddenly failing (e.g., if > some admin has turned on more aggressive ASLR settings). But conversely you don't want your application to suddenly and silently stop using CDS because of ASLR and you've masked that by using "auto"! > As more people are moving their Java workload to micro-services type of > environments, JVM launches will happen more, and there will be more > chances of running into the ASLR issue. Therefore, we should fix the > docs, and warn people that they should switch to -Xshare:auto. That's only a partial solution. If ASLR is a problem then that needs to be known and addressed. >> Maybe only if the archive mapping fails and "on" was used then give a >> warning? Or just improve the message given when the VM aborts? >> > That's already too late, especially for people running critical services. > > We want people to see this warning and actively fix their scripts to get > rid -Xshare:on. I think what we want/need people to realize is that ASLR can seriously impact the ability to use CDS and if you are trying to use CDS for startup or footprint reasons then you're going to have a major problem if CDS is silently disabled! I think "on" and "auto" are both just as valid. "on" is for people who need CDS reliably and want to fail fast if it's not working. "auto" is for people who would like CDS but can live without it. David ----- > Thanks > - Ioi > > >> >> Thanks, >> David >> >> [1] >> https://docs.oracle.com/javase/10/vm/class-data-sharing.htm#JSJVM-GUID-0260F857-A70E-4399-A1DF-A5766BE33285 >> >> [2] >> https://docs.oracle.com/javase/10/tools/java.htm#JSWOR-GUID-31503FCE-93D0-4175-9B4F-F6A738B2F4C4 >> >> >>> ??? --- vs --- >>> >>> $ java-Xshare:auto -version >>> java version "11-internal" 2018-09-25 >>> Java(TM) SE Runtime Environment 18.9 (fastdebug build >>> 11-internal+0-adhoc.iklam.open) >>> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >>> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) >>> >>> I am testing with HotSpot tiers 1-3 to make sure the tests don't get >>> tripped by this new warning message. >>> >>> Thanks >>> - Ioi > From david.holmes at oracle.com Thu May 31 04:16:20 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 31 May 2018 14:16:20 +1000 Subject: RFR(xs): 8203960: [TESTBUG] runtime/logging/DefaultMethodsTest.java failed when running in CDS mode In-Reply-To: References: <5B0F72F8.6020503@oracle.com> Message-ID: +1 the test should be self-sufficient and not rely on an unrelated module. Thanks, David On 31/05/2018 2:03 PM, Ioi Lam wrote: > Hi Calvin, > > Instead of relying on an internal JDK class to have default method > processing, maybe InnerClass should contain some code to ensure that > default method processing will always happen? > > Thanks > > - Ioi > > > On 5/30/18 8:58 PM, Calvin Cheung wrote: >> JBS: https://bugs.openjdk.java.net/browse/JDK-8203960 >> >> webrev: http://cr.openjdk.java.net/~ccheung/8203960/webrev.00/ >> >> If this test is run in CDS mode, most of the system classes will be in >> the CDS archive and loading of those classes from the archive will >> bypass the default method processing. The test fails in CDS mode since >> it expects the trace output from default method processing. >> A fix is to load an additional class which isn't in any of the modules >> defined by default. The loading of the additional class will trigger >> default method processing. >> >> Ran the test with and without CDS. >> I will do a sanity hs-tier1 and hs-tier2 testing run. >> >> thanks, >> Calvin > From ioi.lam at oracle.com Thu May 31 04:24:43 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 30 May 2018 21:24:43 -0700 Subject: RFR(XXS): 8188109 JVM should print a warning message that -Xshare:on may cause VM to abort start-up In-Reply-To: References: <2b1d7829-0dc1-bf7b-5e2d-426783738c1b@oracle.com> <288dfdce-57a5-e28e-f4e5-6ac96d87bb98@oracle.com> Message-ID: <5a655814-755c-354c-4e00-815e8cbbc11a@oracle.com> On 5/30/18 9:13 PM, David Holmes wrote: > Hi Ioi, > > On 31/05/2018 2:01 PM, Ioi Lam wrote: >> On 5/30/18 6:47 PM, David Holmes wrote: >>> Hi Ioi, >>> >>> Sorry but this troubles me ... >>> >>> On 31/05/2018 9:39 AM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8188109 >>>> http://cr.openjdk.java.net/~iklam/jdk11/8188109-xshare-on-print-warning.v01/ >>>> >>>> >>>> >>>> Hi, >>>> >>>> Please review this one-liner patch. >>>> >>>> -Xshare:on may cause infrequent/intermittent start-up failure due >>>> to the presence of Address Space Layout Randomization (ASLR). This >>>> option is intended for testing (the internals of CDS) only and >>>> should not be used in production environments. >>>> >>>> With this patch, the following warning message is printed when >>>> -Xshare:on is specified: >>>> >>>> $ java -Xshare:on -version >>>> Java HotSpot(TM) 64-Bit Server VM warning: -Xshare:on is for >>>> testing purpose only and may cause JVM start-up failure. Use >>>> -Xshare:auto instead. >>>> java version "11-internal" 2018-09-25 >>>> Java(TM) SE Runtime Environment 18.9 (fastdebug build >>>> 11-internal+0-adhoc.iklam.open) >>>> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >>>> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) >>> >>> So should this warning only be enabled in product builds? >>> >>> Even then it may be annoying for anyone who runs with -Xshare:on as >>> they've set up CDS as documented [1][2] and they know their >>> environment works ok - now they get a warning. >>> >>> Also I'm unclear how "on" fails due to ASLR but "auto" keeps going? >>> >> The documentation [1] says: >> >> -Xshare:on >> To enable class data sharing. If class data sharing can't be enabled, >> print an error message and exit. >> >> -Xshare:auto >> To enable class data sharing by default. Enable class data sharing >> whenever possible. >> >> So if mapping fails due to ASLR, "on" will exit and "auto" will >> disable CDS and continue . > > Ah! In the bug you state "-Xshare:auto continue to execute (with CDS > enabled)" - so that should be disabled. > >> The documentation in [2] is wrong.? It says "Ensure that you have >> specified the option -Xshare:on or -Xshare:auto.", but -Xshare:on >> should not be used in production environments. I have filed a doc bug >> for this (https://bugs.openjdk.java.net/browse/JDK-8204141). >> >> The main reason for doing this REF is -- if people have been >> following [2] and using -Xshare:on, their setup is NOT OK. ASLR may >> happen just very rarely, but you don't want your program suddenly >> failing (e.g., if some admin has turned on more aggressive ASLR >> settings). > > But conversely you don't want your application to suddenly and > silently stop using CDS because of ASLR and you've masked that by > using "auto"! > >> As more people are moving their Java workload to micro-services type >> of environments, JVM launches will happen more, and there will be >> more chances of running into the ASLR issue. Therefore, we should fix >> the docs, and warn people that they should switch to -Xshare:auto. > > That's only a partial solution. If ASLR is a problem then that needs > to be known and addressed. > >>> Maybe only if the archive mapping fails and "on" was used then give >>> a warning? Or just improve the message given when the VM aborts? >>> >> That's already too late, especially for people running critical >> services. >> >> We want people to see this warning and actively fix their scripts to >> get rid -Xshare:on. > > I think what we want/need people to realize is that ASLR can seriously > impact the ability to use CDS and if you are trying to use CDS for > startup or footprint reasons then you're going to have a major problem > if CDS is silently disabled! > > I think "on" and "auto" are both just as valid. "on" is for people who > need CDS reliably and want to fail fast if it's not working. "auto" is > for people who would like CDS but can live without it. > I think -Xshare:on has the potential to do much more harm than good. People have lived with the fact that optimizations in the JVM are not always deterministic. They want their programs to run regardless. CDS is the only optimization that has an option to say "let the program fail if the optimization is not available". There are diagnostic options to find out if CDS is enabled. If you run with -showversion, it will tell you if sharing is enabled. People don't need their program to die just to find this out. Thanks - Ioi > David > ----- > >> Thanks >> - Ioi >> >> >>> >>> Thanks, >>> David >>> >>> [1] >>> https://docs.oracle.com/javase/10/vm/class-data-sharing.htm#JSJVM-GUID-0260F857-A70E-4399-A1DF-A5766BE33285 >>> >>> [2] >>> https://docs.oracle.com/javase/10/tools/java.htm#JSWOR-GUID-31503FCE-93D0-4175-9B4F-F6A738B2F4C4 >>> >>> >>>> ??? --- vs --- >>>> >>>> $ java-Xshare:auto -version >>>> java version "11-internal" 2018-09-25 >>>> Java(TM) SE Runtime Environment 18.9 (fastdebug build >>>> 11-internal+0-adhoc.iklam.open) >>>> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >>>> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) >>>> >>>> I am testing with HotSpot tiers 1-3 to make sure the tests don't >>>> get tripped by this new warning message. >>>> >>>> Thanks >>>> - Ioi >> From david.holmes at oracle.com Thu May 31 04:53:48 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 31 May 2018 14:53:48 +1000 Subject: RFR(XXS): 8188109 JVM should print a warning message that -Xshare:on may cause VM to abort start-up In-Reply-To: <5a655814-755c-354c-4e00-815e8cbbc11a@oracle.com> References: <2b1d7829-0dc1-bf7b-5e2d-426783738c1b@oracle.com> <288dfdce-57a5-e28e-f4e5-6ac96d87bb98@oracle.com> <5a655814-755c-354c-4e00-815e8cbbc11a@oracle.com> Message-ID: <84d0c341-ed14-16bb-b829-1dac03b828b5@oracle.com> On 31/05/2018 2:24 PM, Ioi Lam wrote: > > > On 5/30/18 9:13 PM, David Holmes wrote: >> Hi Ioi, >> >> On 31/05/2018 2:01 PM, Ioi Lam wrote: >>> On 5/30/18 6:47 PM, David Holmes wrote: >>>> Hi Ioi, >>>> >>>> Sorry but this troubles me ... >>>> >>>> On 31/05/2018 9:39 AM, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8188109 >>>>> http://cr.openjdk.java.net/~iklam/jdk11/8188109-xshare-on-print-warning.v01/ >>>>> >>>>> >>>>> >>>>> Hi, >>>>> >>>>> Please review this one-liner patch. >>>>> >>>>> -Xshare:on may cause infrequent/intermittent start-up failure due >>>>> to the presence of Address Space Layout Randomization (ASLR). This >>>>> option is intended for testing (the internals of CDS) only and >>>>> should not be used in production environments. >>>>> >>>>> With this patch, the following warning message is printed when >>>>> -Xshare:on is specified: >>>>> >>>>> $ java -Xshare:on -version >>>>> Java HotSpot(TM) 64-Bit Server VM warning: -Xshare:on is for >>>>> testing purpose only and may cause JVM start-up failure. Use >>>>> -Xshare:auto instead. >>>>> java version "11-internal" 2018-09-25 >>>>> Java(TM) SE Runtime Environment 18.9 (fastdebug build >>>>> 11-internal+0-adhoc.iklam.open) >>>>> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >>>>> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) >>>> >>>> So should this warning only be enabled in product builds? >>>> >>>> Even then it may be annoying for anyone who runs with -Xshare:on as >>>> they've set up CDS as documented [1][2] and they know their >>>> environment works ok - now they get a warning. >>>> >>>> Also I'm unclear how "on" fails due to ASLR but "auto" keeps going? >>>> >>> The documentation [1] says: >>> >>> -Xshare:on >>> To enable class data sharing. If class data sharing can't be enabled, >>> print an error message and exit. >>> >>> -Xshare:auto >>> To enable class data sharing by default. Enable class data sharing >>> whenever possible. >>> >>> So if mapping fails due to ASLR, "on" will exit and "auto" will >>> disable CDS and continue . >> >> Ah! In the bug you state "-Xshare:auto continue to execute (with CDS >> enabled)" - so that should be disabled. >> >>> The documentation in [2] is wrong.? It says "Ensure that you have >>> specified the option -Xshare:on or -Xshare:auto.", but -Xshare:on >>> should not be used in production environments. I have filed a doc bug >>> for this (https://bugs.openjdk.java.net/browse/JDK-8204141). >>> >>> The main reason for doing this REF is -- if people have been >>> following [2] and using -Xshare:on, their setup is NOT OK. ASLR may >>> happen just very rarely, but you don't want your program suddenly >>> failing (e.g., if some admin has turned on more aggressive ASLR >>> settings). >> >> But conversely you don't want your application to suddenly and >> silently stop using CDS because of ASLR and you've masked that by >> using "auto"! >> >>> As more people are moving their Java workload to micro-services type >>> of environments, JVM launches will happen more, and there will be >>> more chances of running into the ASLR issue. Therefore, we should fix >>> the docs, and warn people that they should switch to -Xshare:auto. >> >> That's only a partial solution. If ASLR is a problem then that needs >> to be known and addressed. >> >>>> Maybe only if the archive mapping fails and "on" was used then give >>>> a warning? Or just improve the message given when the VM aborts? >>>> >>> That's already too late, especially for people running critical >>> services. >>> >>> We want people to see this warning and actively fix their scripts to >>> get rid -Xshare:on. >> >> I think what we want/need people to realize is that ASLR can seriously >> impact the ability to use CDS and if you are trying to use CDS for >> startup or footprint reasons then you're going to have a major problem >> if CDS is silently disabled! >> >> I think "on" and "auto" are both just as valid. "on" is for people who >> need CDS reliably and want to fail fast if it's not working. "auto" is >> for people who would like CDS but can live without it. >> > > I think -Xshare:on has the potential to do much more harm than good. > > People have lived with the fact that optimizations in the JVM are not > always deterministic. They want their programs to run regardless. CDS is > the only optimization that has an option to say "let the program fail if > the optimization is not available". I don't agree with that characterization - I think it oversimplifies the situation. If you want to push this analogy then the right analogy would be allowing "-server" to silently run the interpreter instead because there was some error configuring the JIT! I wonder how many users would be happy with that! But the bulk of optimizations are not things that can fail as such so I don't think the comparison holds. I think this is purely a documentation and education issue. Particularly because this is not something new with JDK 11 - this is an issue that exists with CDS in all releases. So you want to get the message out to all users that "auto" may be preferable to "on". Until something actually fails I doubt anyone would notice your warning anyway. Maybe we need -Xshare:on_and_I_really_mean_it ;-) Cheers, David > There are diagnostic options to find out if CDS is enabled. If you run > with -showversion, it will tell you if sharing is enabled. People don't > need their program to die just to find this out. > > Thanks > - Ioi > > >> David >> ----- >> >>> Thanks >>> - Ioi >>> >>> >>>> >>>> Thanks, >>>> David >>>> >>>> [1] >>>> https://docs.oracle.com/javase/10/vm/class-data-sharing.htm#JSJVM-GUID-0260F857-A70E-4399-A1DF-A5766BE33285 >>>> >>>> [2] >>>> https://docs.oracle.com/javase/10/tools/java.htm#JSWOR-GUID-31503FCE-93D0-4175-9B4F-F6A738B2F4C4 >>>> >>>> >>>>> ??? --- vs --- >>>>> >>>>> $ java-Xshare:auto -version >>>>> java version "11-internal" 2018-09-25 >>>>> Java(TM) SE Runtime Environment 18.9 (fastdebug build >>>>> 11-internal+0-adhoc.iklam.open) >>>>> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >>>>> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) >>>>> >>>>> I am testing with HotSpot tiers 1-3 to make sure the tests don't >>>>> get tripped by this new warning message. >>>>> >>>>> Thanks >>>>> - Ioi >>> > From calvin.cheung at oracle.com Thu May 31 05:04:35 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 30 May 2018 22:04:35 -0700 Subject: RFR(xs): 8203960: [TESTBUG] runtime/logging/DefaultMethodsTest.java failed when running in CDS mode In-Reply-To: References: <5B0F72F8.6020503@oracle.com> Message-ID: <5B0F8263.6020309@oracle.com> Hi Ioi, David, Thanks for your review and suggestion. I've added a simple interface with a default method. The InnerClass implements the interface. Updated webrev: http://cr.openjdk.java.net/~ccheung/8203960/webrev.01/ thanks, Calvin On 5/30/18, 9:16 PM, David Holmes wrote: > +1 the test should be self-sufficient and not rely on an unrelated > module. > > Thanks, > David > > On 31/05/2018 2:03 PM, Ioi Lam wrote: >> Hi Calvin, >> >> Instead of relying on an internal JDK class to have default method >> processing, maybe InnerClass should contain some code to ensure that >> default method processing will always happen? >> >> Thanks >> >> - Ioi >> >> >> On 5/30/18 8:58 PM, Calvin Cheung wrote: >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8203960 >>> >>> webrev: http://cr.openjdk.java.net/~ccheung/8203960/webrev.00/ >>> >>> If this test is run in CDS mode, most of the system classes will be >>> in the CDS archive and loading of those classes from the archive >>> will bypass the default method processing. The test fails in CDS >>> mode since it expects the trace output from default method processing. >>> A fix is to load an additional class which isn't in any of the >>> modules defined by default. The loading of the additional class will >>> trigger default method processing. >>> >>> Ran the test with and without CDS. >>> I will do a sanity hs-tier1 and hs-tier2 testing run. >>> >>> thanks, >>> Calvin >> From david.holmes at oracle.com Thu May 31 05:10:37 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 31 May 2018 15:10:37 +1000 Subject: RFR(xs): 8203960: [TESTBUG] runtime/logging/DefaultMethodsTest.java failed when running in CDS mode In-Reply-To: <5B0F8263.6020309@oracle.com> References: <5B0F72F8.6020503@oracle.com> <5B0F8263.6020309@oracle.com> Message-ID: <850818c2-a1a7-b503-dbb3-2b6c332432c3@oracle.com> Looks good. Thanks, David On 31/05/2018 3:04 PM, Calvin Cheung wrote: > Hi Ioi, David, > > Thanks for your review and suggestion. > > I've added a simple interface with a default method. The InnerClass > implements the interface. > > Updated webrev: > ??? http://cr.openjdk.java.net/~ccheung/8203960/webrev.01/ > > thanks, > Calvin > > > On 5/30/18, 9:16 PM, David Holmes wrote: >> +1 the test should be self-sufficient and not rely on an unrelated >> module. >> >> Thanks, >> David >> >> On 31/05/2018 2:03 PM, Ioi Lam wrote: >>> Hi Calvin, >>> >>> Instead of relying on an internal JDK class to have default method >>> processing, maybe InnerClass should contain some code to ensure that >>> default method processing will always happen? >>> >>> Thanks >>> >>> - Ioi >>> >>> >>> On 5/30/18 8:58 PM, Calvin Cheung wrote: >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8203960 >>>> >>>> webrev: http://cr.openjdk.java.net/~ccheung/8203960/webrev.00/ >>>> >>>> If this test is run in CDS mode, most of the system classes will be >>>> in the CDS archive and loading of those classes from the archive >>>> will bypass the default method processing. The test fails in CDS >>>> mode since it expects the trace output from default method processing. >>>> A fix is to load an additional class which isn't in any of the >>>> modules defined by default. The loading of the additional class will >>>> trigger default method processing. >>>> >>>> Ran the test with and without CDS. >>>> I will do a sanity hs-tier1 and hs-tier2 testing run. >>>> >>>> thanks, >>>> Calvin >>> From thomas.stuefe at gmail.com Thu May 31 05:20:31 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 31 May 2018 07:20:31 +0200 Subject: RFR(XXS): 8188109 JVM should print a warning message that -Xshare:on may cause VM to abort start-up In-Reply-To: <84d0c341-ed14-16bb-b829-1dac03b828b5@oracle.com> References: <2b1d7829-0dc1-bf7b-5e2d-426783738c1b@oracle.com> <288dfdce-57a5-e28e-f4e5-6ac96d87bb98@oracle.com> <5a655814-755c-354c-4e00-815e8cbbc11a@oracle.com> <84d0c341-ed14-16bb-b829-1dac03b828b5@oracle.com> Message-ID: > Maybe we need -Xshare:on_and_I_really_mean_it ;-) > > Cheers, > David How about -Xshare:on, -Xshare:off and an additional diagnostic flag, -XX:+ExitOnCDSFail. Gives you the same width of options but since diagnostic flags have to be explicitly enabled should make most people aware of the problem. ..Thomas > From calvin.cheung at oracle.com Thu May 31 05:25:10 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 30 May 2018 22:25:10 -0700 Subject: RFR(xs): 8203960: [TESTBUG] runtime/logging/DefaultMethodsTest.java failed when running in CDS mode In-Reply-To: <850818c2-a1a7-b503-dbb3-2b6c332432c3@oracle.com> References: <5B0F72F8.6020503@oracle.com> <5B0F8263.6020309@oracle.com> <850818c2-a1a7-b503-dbb3-2b6c332432c3@oracle.com> Message-ID: <5B0F8736.4040100@oracle.com> Thanks David. Calvin On 5/30/18, 10:10 PM, David Holmes wrote: > Looks good. > > Thanks, > David > > On 31/05/2018 3:04 PM, Calvin Cheung wrote: >> Hi Ioi, David, >> >> Thanks for your review and suggestion. >> >> I've added a simple interface with a default method. The InnerClass >> implements the interface. >> >> Updated webrev: >> http://cr.openjdk.java.net/~ccheung/8203960/webrev.01/ >> >> thanks, >> Calvin >> >> >> On 5/30/18, 9:16 PM, David Holmes wrote: >>> +1 the test should be self-sufficient and not rely on an unrelated >>> module. >>> >>> Thanks, >>> David >>> >>> On 31/05/2018 2:03 PM, Ioi Lam wrote: >>>> Hi Calvin, >>>> >>>> Instead of relying on an internal JDK class to have default method >>>> processing, maybe InnerClass should contain some code to ensure >>>> that default method processing will always happen? >>>> >>>> Thanks >>>> >>>> - Ioi >>>> >>>> >>>> On 5/30/18 8:58 PM, Calvin Cheung wrote: >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8203960 >>>>> >>>>> webrev: http://cr.openjdk.java.net/~ccheung/8203960/webrev.00/ >>>>> >>>>> If this test is run in CDS mode, most of the system classes will >>>>> be in the CDS archive and loading of those classes from the >>>>> archive will bypass the default method processing. The test fails >>>>> in CDS mode since it expects the trace output from default method >>>>> processing. >>>>> A fix is to load an additional class which isn't in any of the >>>>> modules defined by default. The loading of the additional class >>>>> will trigger default method processing. >>>>> >>>>> Ran the test with and without CDS. >>>>> I will do a sanity hs-tier1 and hs-tier2 testing run. >>>>> >>>>> thanks, >>>>> Calvin >>>> From ioi.lam at oracle.com Thu May 31 05:33:04 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 30 May 2018 22:33:04 -0700 Subject: RFR(XXS): 8188109 JVM should print a warning message that -Xshare:on may cause VM to abort start-up In-Reply-To: References: <2b1d7829-0dc1-bf7b-5e2d-426783738c1b@oracle.com> <288dfdce-57a5-e28e-f4e5-6ac96d87bb98@oracle.com> <5a655814-755c-354c-4e00-815e8cbbc11a@oracle.com> <84d0c341-ed14-16bb-b829-1dac03b828b5@oracle.com> Message-ID: <7dd90aed-6f4e-2a87-ac65-703032095257@oracle.com> There is already a flag: -XX:+RequireSharedSpaces, which is turned on when you specify -Xshare:on. I think a better way to make things safer, and still allows a way to revert to the old behavior, is to: [1] make -Xshare:on the same as -Xshare:auto [2] make RequireSharedSpaces a diagnostic flag as Thomas suggested, so people can really shoot themselves in the foot if they want to, but at least they know they are doing that. Thanks - Ioi On 5/30/18 10:20 PM, Thomas St?fe wrote: > > >> Maybe we need -Xshare:on_and_I_really_mean_it ;-) >> >> Cheers, >> David > How about -Xshare:on, -Xshare:off and an additional diagnostic flag, > -XX:+ExitOnCDSFail. Gives you the same width of options but since > diagnostic flags have to be explicitly enabled should make most people > aware of the problem. > > ..Thomas > From ioi.lam at oracle.com Thu May 31 05:50:41 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 30 May 2018 22:50:41 -0700 Subject: RFR(XXS): 8188109 JVM should print a warning message that -Xshare:on may cause VM to abort start-up In-Reply-To: <84d0c341-ed14-16bb-b829-1dac03b828b5@oracle.com> References: <2b1d7829-0dc1-bf7b-5e2d-426783738c1b@oracle.com> <288dfdce-57a5-e28e-f4e5-6ac96d87bb98@oracle.com> <5a655814-755c-354c-4e00-815e8cbbc11a@oracle.com> <84d0c341-ed14-16bb-b829-1dac03b828b5@oracle.com> Message-ID: <0b49b7b4-12fc-5f71-284a-67f962b9fcb9@oracle.com> On 5/30/18 9:53 PM, David Holmes wrote: > On 31/05/2018 2:24 PM, Ioi Lam wrote: >> >> >> On 5/30/18 9:13 PM, David Holmes wrote: >>> Hi Ioi, >>> >>> On 31/05/2018 2:01 PM, Ioi Lam wrote: >>>> On 5/30/18 6:47 PM, David Holmes wrote: >>>>> Hi Ioi, >>>>> >>>>> Sorry but this troubles me ... >>>>> >>>>> On 31/05/2018 9:39 AM, Ioi Lam wrote: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8188109 >>>>>> http://cr.openjdk.java.net/~iklam/jdk11/8188109-xshare-on-print-warning.v01/ >>>>>> >>>>>> >>>>>> >>>>>> Hi, >>>>>> >>>>>> Please review this one-liner patch. >>>>>> >>>>>> -Xshare:on may cause infrequent/intermittent start-up failure due >>>>>> to the presence of Address Space Layout Randomization (ASLR). >>>>>> This option is intended for testing (the internals of CDS) only >>>>>> and should not be used in production environments. >>>>>> >>>>>> With this patch, the following warning message is printed when >>>>>> -Xshare:on is specified: >>>>>> >>>>>> $ java -Xshare:on -version >>>>>> Java HotSpot(TM) 64-Bit Server VM warning: -Xshare:on is for >>>>>> testing purpose only and may cause JVM start-up failure. Use >>>>>> -Xshare:auto instead. >>>>>> java version "11-internal" 2018-09-25 >>>>>> Java(TM) SE Runtime Environment 18.9 (fastdebug build >>>>>> 11-internal+0-adhoc.iklam.open) >>>>>> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >>>>>> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) >>>>> >>>>> So should this warning only be enabled in product builds? >>>>> >>>>> Even then it may be annoying for anyone who runs with -Xshare:on >>>>> as they've set up CDS as documented [1][2] and they know their >>>>> environment works ok - now they get a warning. >>>>> >>>>> Also I'm unclear how "on" fails due to ASLR but "auto" keeps going? >>>>> >>>> The documentation [1] says: >>>> >>>> -Xshare:on >>>> To enable class data sharing. If class data sharing can't be >>>> enabled, print an error message and exit. >>>> >>>> -Xshare:auto >>>> To enable class data sharing by default. Enable class data sharing >>>> whenever possible. >>>> >>>> So if mapping fails due to ASLR, "on" will exit and "auto" will >>>> disable CDS and continue . >>> >>> Ah! In the bug you state "-Xshare:auto continue to execute (with CDS >>> enabled)" - so that should be disabled. >>> >>>> The documentation in [2] is wrong.? It says "Ensure that you have >>>> specified the option -Xshare:on or -Xshare:auto.", but -Xshare:on >>>> should not be used in production environments. I have filed a doc >>>> bug for this (https://bugs.openjdk.java.net/browse/JDK-8204141). >>>> >>>> The main reason for doing this REF is -- if people have been >>>> following [2] and using -Xshare:on, their setup is NOT OK. ASLR may >>>> happen just very rarely, but you don't want your program suddenly >>>> failing (e.g., if some admin has turned on more aggressive ASLR >>>> settings). >>> >>> But conversely you don't want your application to suddenly and >>> silently stop using CDS because of ASLR and you've masked that by >>> using "auto"! >>> >>>> As more people are moving their Java workload to micro-services >>>> type of environments, JVM launches will happen more, and there will >>>> be more chances of running into the ASLR issue. Therefore, we >>>> should fix the docs, and warn people that they should switch to >>>> -Xshare:auto. >>> >>> That's only a partial solution. If ASLR is a problem then that needs >>> to be known and addressed. >>> >>>>> Maybe only if the archive mapping fails and "on" was used then >>>>> give a warning? Or just improve the message given when the VM aborts? >>>>> >>>> That's already too late, especially for people running critical >>>> services. >>>> >>>> We want people to see this warning and actively fix their scripts >>>> to get rid -Xshare:on. >>> >>> I think what we want/need people to realize is that ASLR can >>> seriously impact the ability to use CDS and if you are trying to use >>> CDS for startup or footprint reasons then you're going to have a >>> major problem if CDS is silently disabled! >>> >>> I think "on" and "auto" are both just as valid. "on" is for people >>> who need CDS reliably and want to fail fast if it's not working. >>> "auto" is for people who would like CDS but can live without it. >>> >> >> I think -Xshare:on has the potential to do much more harm than good. >> >> People have lived with the fact that optimizations in the JVM are not >> always deterministic. They want their programs to run regardless. CDS >> is the only optimization that has an option to say "let the program >> fail if the optimization is not available". > > I don't agree with that characterization - I think it oversimplifies > the situation. If you want to push this analogy then the right analogy > would be allowing "-server" to silently run the interpreter instead > because there was some error configuring the JIT! I wonder how many > users would be happy with that! But the bulk of optimizations are not > things that can fail as such so I don't think the comparison holds. > > I think this is purely a documentation and education issue. > Particularly because this is not something new with JDK 11 - this is > an issue that exists with CDS in all releases. So you want to get the > message out to all users that "auto" may be preferable to "on". > > Until something actually fails I doubt anyone would notice your > warning anyway. > > Maybe we need -Xshare:on_and_I_really_mean_it ;-) > -Xshare:on was an ill-conceived and dangerous option. It was designed when ASLR wasn't common. With ASLR more in common use, and short-lived JVMs becoming more common, the danger is getting bigger and bigger. Just because a bad option has existed for a long time does not mean we should not fix it. The only reason for this option to exist is for diagnostic purposes. It can check for * Did my archive fail to map due to ASLR? * Did I specify a bad path to the archive? * Did I specify a bad archive file (e.g., one created by a different JDK version)? So Thomas's suggestion of providing this functionality as a diagnostic flag makes sense. In any case, I think this particular REF is not a good way of handling the problem, so I am withdrawing it. I'll file a separate CSR to actually change how -Xshare:on works, after more discussion on how best to change it. Thanks - Ioi > Cheers, > David > >> There are diagnostic options to find out if CDS is enabled. If you >> run with -showversion, it will tell you if sharing is enabled. People >> don't need their program to die just to find this out. >> >> Thanks >> - Ioi >> >> >>> David >>> ----- >>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> [1] >>>>> https://docs.oracle.com/javase/10/vm/class-data-sharing.htm#JSJVM-GUID-0260F857-A70E-4399-A1DF-A5766BE33285 >>>>> >>>>> [2] >>>>> https://docs.oracle.com/javase/10/tools/java.htm#JSWOR-GUID-31503FCE-93D0-4175-9B4F-F6A738B2F4C4 >>>>> >>>>> >>>>>> ??? --- vs --- >>>>>> >>>>>> $ java-Xshare:auto -version >>>>>> java version "11-internal" 2018-09-25 >>>>>> Java(TM) SE Runtime Environment 18.9 (fastdebug build >>>>>> 11-internal+0-adhoc.iklam.open) >>>>>> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >>>>>> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) >>>>>> >>>>>> I am testing with HotSpot tiers 1-3 to make sure the tests don't >>>>>> get tripped by this new warning message. >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>> >> From david.holmes at oracle.com Thu May 31 05:55:26 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 31 May 2018 15:55:26 +1000 Subject: RFR (S) 8204055: SIGSEGV in java -XX: Message-ID: <72335f6e-3856-ad1f-565e-8b11def70213@oracle.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8204055 webrev: http://cr.openjdk.java.net/~dholmes/8204055/webrev/ The SEGV was introduced with the fuzzy matching flag logic refactoring in JDK-8198554. In: double StringUtils::similarity(const char* str1, size_t len1, const char* str2, size_t len2) { size_t total = len1 + len2; size_t hit = 0; for (size_t i = 0; i < len1 - 1; i++) { for (size_t j = 0; j < len2 - 1; j++) { if ((str1[i] == str2[j]) && (str1[i+1] == str2[j+1])) { ++hit; break; } } } If len2 is zero (which it is in this case) we have passed it as an unsigned size_t, so len2-1 gives a massive positive value and so we enter the loop and try to access str2[n] for some n>0 and we get a SEGV. The original code had: - for (int j = 0; j < (int) len2 -1; ++j) { so the huge positive value reverted to a small negative value and we don't enter the loop. The fix applied is to check explicitly for lengths of zero. Added missing testcases to: test/hotspot/gtest/logging/test_logConfiguration.cpp test/hotspot/jtreg/runtime/CommandLine/UnrecognizedVMOption.java verified they both crash before the fix. Testing (in progress): tier1,2,3 per mach5 CI Thanks, David From ioi.lam at oracle.com Thu May 31 05:55:33 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 30 May 2018 22:55:33 -0700 Subject: RFR(xs): 8203960: [TESTBUG] runtime/logging/DefaultMethodsTest.java failed when running in CDS mode In-Reply-To: <5B0F8263.6020309@oracle.com> References: <5B0F72F8.6020503@oracle.com> <5B0F8263.6020309@oracle.com> Message-ID: <1c5eff9c-0592-8ffd-9927-ebdbf1c08e42@oracle.com> Looks good. Thanks! - Ioi On 5/30/18 10:04 PM, Calvin Cheung wrote: > Hi Ioi, David, > > Thanks for your review and suggestion. > > I've added a simple interface with a default method. The InnerClass > implements the interface. > > Updated webrev: > ??? http://cr.openjdk.java.net/~ccheung/8203960/webrev.01/ > > thanks, > Calvin > > > On 5/30/18, 9:16 PM, David Holmes wrote: >> +1 the test should be self-sufficient and not rely on an unrelated >> module. >> >> Thanks, >> David >> >> On 31/05/2018 2:03 PM, Ioi Lam wrote: >>> Hi Calvin, >>> >>> Instead of relying on an internal JDK class to have default method >>> processing, maybe InnerClass should contain some code to ensure that >>> default method processing will always happen? >>> >>> Thanks >>> >>> - Ioi >>> >>> >>> On 5/30/18 8:58 PM, Calvin Cheung wrote: >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8203960 >>>> >>>> webrev: http://cr.openjdk.java.net/~ccheung/8203960/webrev.00/ >>>> >>>> If this test is run in CDS mode, most of the system classes will be >>>> in the CDS archive and loading of those classes from the archive >>>> will bypass the default method processing. The test fails in CDS >>>> mode since it expects the trace output from default method processing. >>>> A fix is to load an additional class which isn't in any of the >>>> modules defined by default. The loading of the additional class >>>> will trigger default method processing. >>>> >>>> Ran the test with and without CDS. >>>> I will do a sanity hs-tier1 and hs-tier2 testing run. >>>> >>>> thanks, >>>> Calvin >>> From thomas.stuefe at gmail.com Thu May 31 05:57:54 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 31 May 2018 07:57:54 +0200 Subject: RFR(XXS): 8188109 JVM should print a warning message that -Xshare:on may cause VM to abort start-up In-Reply-To: <0b49b7b4-12fc-5f71-284a-67f962b9fcb9@oracle.com> References: <2b1d7829-0dc1-bf7b-5e2d-426783738c1b@oracle.com> <288dfdce-57a5-e28e-f4e5-6ac96d87bb98@oracle.com> <5a655814-755c-354c-4e00-815e8cbbc11a@oracle.com> <84d0c341-ed14-16bb-b829-1dac03b828b5@oracle.com> <0b49b7b4-12fc-5f71-284a-67f962b9fcb9@oracle.com> Message-ID: On Thu, May 31, 2018 at 7:50 AM, Ioi Lam wrote: > > > On 5/30/18 9:53 PM, David Holmes wrote: >> >> On 31/05/2018 2:24 PM, Ioi Lam wrote: >>> >>> >>> >>> On 5/30/18 9:13 PM, David Holmes wrote: >>>> >>>> Hi Ioi, >>>> >>>> On 31/05/2018 2:01 PM, Ioi Lam wrote: >>>>> >>>>> On 5/30/18 6:47 PM, David Holmes wrote: >>>>>> >>>>>> Hi Ioi, >>>>>> >>>>>> Sorry but this troubles me ... >>>>>> >>>>>> On 31/05/2018 9:39 AM, Ioi Lam wrote: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8188109 >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk11/8188109-xshare-on-print-warning.v01/ >>>>>>> >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Please review this one-liner patch. >>>>>>> >>>>>>> -Xshare:on may cause infrequent/intermittent start-up failure due to >>>>>>> the presence of Address Space Layout Randomization (ASLR). This option is >>>>>>> intended for testing (the internals of CDS) only and should not be used in >>>>>>> production environments. >>>>>>> >>>>>>> With this patch, the following warning message is printed when >>>>>>> -Xshare:on is specified: >>>>>>> >>>>>>> $ java -Xshare:on -version >>>>>>> Java HotSpot(TM) 64-Bit Server VM warning: -Xshare:on is for testing >>>>>>> purpose only and may cause JVM start-up failure. Use -Xshare:auto instead. >>>>>>> java version "11-internal" 2018-09-25 >>>>>>> Java(TM) SE Runtime Environment 18.9 (fastdebug build >>>>>>> 11-internal+0-adhoc.iklam.open) >>>>>>> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >>>>>>> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) >>>>>> >>>>>> >>>>>> So should this warning only be enabled in product builds? >>>>>> >>>>>> Even then it may be annoying for anyone who runs with -Xshare:on as >>>>>> they've set up CDS as documented [1][2] and they know their environment >>>>>> works ok - now they get a warning. >>>>>> >>>>>> Also I'm unclear how "on" fails due to ASLR but "auto" keeps going? >>>>>> >>>>> The documentation [1] says: >>>>> >>>>> -Xshare:on >>>>> To enable class data sharing. If class data sharing can't be enabled, >>>>> print an error message and exit. >>>>> >>>>> -Xshare:auto >>>>> To enable class data sharing by default. Enable class data sharing >>>>> whenever possible. >>>>> >>>>> So if mapping fails due to ASLR, "on" will exit and "auto" will disable >>>>> CDS and continue . >>>> >>>> >>>> Ah! In the bug you state "-Xshare:auto continue to execute (with CDS >>>> enabled)" - so that should be disabled. >>>> >>>>> The documentation in [2] is wrong. It says "Ensure that you have >>>>> specified the option -Xshare:on or -Xshare:auto.", but -Xshare:on should not >>>>> be used in production environments. I have filed a doc bug for this >>>>> (https://bugs.openjdk.java.net/browse/JDK-8204141). >>>>> >>>>> The main reason for doing this REF is -- if people have been following >>>>> [2] and using -Xshare:on, their setup is NOT OK. ASLR may happen just very >>>>> rarely, but you don't want your program suddenly failing (e.g., if some >>>>> admin has turned on more aggressive ASLR settings). >>>> >>>> >>>> But conversely you don't want your application to suddenly and silently >>>> stop using CDS because of ASLR and you've masked that by using "auto"! >>>> >>>>> As more people are moving their Java workload to micro-services type of >>>>> environments, JVM launches will happen more, and there will be more chances >>>>> of running into the ASLR issue. Therefore, we should fix the docs, and warn >>>>> people that they should switch to -Xshare:auto. >>>> >>>> >>>> That's only a partial solution. If ASLR is a problem then that needs to >>>> be known and addressed. >>>> >>>>>> Maybe only if the archive mapping fails and "on" was used then give a >>>>>> warning? Or just improve the message given when the VM aborts? >>>>>> >>>>> That's already too late, especially for people running critical >>>>> services. >>>>> >>>>> We want people to see this warning and actively fix their scripts to >>>>> get rid -Xshare:on. >>>> >>>> >>>> I think what we want/need people to realize is that ASLR can seriously >>>> impact the ability to use CDS and if you are trying to use CDS for startup >>>> or footprint reasons then you're going to have a major problem if CDS is >>>> silently disabled! >>>> >>>> I think "on" and "auto" are both just as valid. "on" is for people who >>>> need CDS reliably and want to fail fast if it's not working. "auto" is for >>>> people who would like CDS but can live without it. >>>> >>> >>> I think -Xshare:on has the potential to do much more harm than good. >>> >>> People have lived with the fact that optimizations in the JVM are not >>> always deterministic. They want their programs to run regardless. CDS is the >>> only optimization that has an option to say "let the program fail if the >>> optimization is not available". >> >> >> I don't agree with that characterization - I think it oversimplifies the >> situation. If you want to push this analogy then the right analogy would be >> allowing "-server" to silently run the interpreter instead because there was >> some error configuring the JIT! I wonder how many users would be happy with >> that! But the bulk of optimizations are not things that can fail as such so >> I don't think the comparison holds. >> >> I think this is purely a documentation and education issue. Particularly >> because this is not something new with JDK 11 - this is an issue that exists >> with CDS in all releases. So you want to get the message out to all users >> that "auto" may be preferable to "on". >> >> Until something actually fails I doubt anyone would notice your warning >> anyway. >> >> Maybe we need -Xshare:on_and_I_really_mean_it ;-) >> > > -Xshare:on was an ill-conceived and dangerous option. It was designed when > ASLR wasn't common. With ASLR more in common use, and short-lived JVMs > becoming more common, the danger is getting bigger and bigger. > Just out of curiosity (I never payed close attention to CDS), how do you communicate the mapping address the first process establishes to the subsequent processes attaching? Or do you just have a fixed well known address value baked into the jvm? I just try to understand how probable a failed mapping could be in a 64bit address space. > Just because a bad option has existed for a long time does not mean we > should not fix it. > > The only reason for this option to exist is for diagnostic purposes. It can > check for > > * Did my archive fail to map due to ASLR? > * Did I specify a bad path to the archive? > * Did I specify a bad archive file (e.g., one created by a different > JDK version)? > > So Thomas's suggestion of providing this functionality as a diagnostic flag > makes sense. > > In any case, I think this particular REF is not a good way of handling the > problem, so I am withdrawing it. > > I'll file a separate CSR to actually change how -Xshare:on works, after more > discussion on how best to change it. I agree with all your points. ..Thomas > > Thanks > - Ioi > > >> Cheers, >> David >> >>> There are diagnostic options to find out if CDS is enabled. If you run >>> with -showversion, it will tell you if sharing is enabled. People don't need >>> their program to die just to find this out. >>> >>> Thanks >>> - Ioi >>> >>> >>>> David >>>> ----- >>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>> [1] >>>>>> https://docs.oracle.com/javase/10/vm/class-data-sharing.htm#JSJVM-GUID-0260F857-A70E-4399-A1DF-A5766BE33285 >>>>>> [2] >>>>>> https://docs.oracle.com/javase/10/tools/java.htm#JSWOR-GUID-31503FCE-93D0-4175-9B4F-F6A738B2F4C4 >>>>>> >>>>>>> --- vs --- >>>>>>> >>>>>>> $ java-Xshare:auto -version >>>>>>> java version "11-internal" 2018-09-25 >>>>>>> Java(TM) SE Runtime Environment 18.9 (fastdebug build >>>>>>> 11-internal+0-adhoc.iklam.open) >>>>>>> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >>>>>>> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) >>>>>>> >>>>>>> I am testing with HotSpot tiers 1-3 to make sure the tests don't get >>>>>>> tripped by this new warning message. >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>> >>>>> >>> > From ioi.lam at oracle.com Thu May 31 05:58:19 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 30 May 2018 22:58:19 -0700 Subject: RFR (S) 8204055: SIGSEGV in java -XX: In-Reply-To: <72335f6e-3856-ad1f-565e-8b11def70213@oracle.com> References: <72335f6e-3856-ad1f-565e-8b11def70213@oracle.com> Message-ID: Looks good! Thanks David. - Ioi On 5/30/18 10:55 PM, David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8204055 > webrev: http://cr.openjdk.java.net/~dholmes/8204055/webrev/ > > > The SEGV was introduced with the fuzzy matching flag logic refactoring > in JDK-8198554. In: > > double StringUtils::similarity(const char* str1, size_t len1, const > ?????????????????????????????? char* str2, size_t len2) { > ? size_t total = len1 + len2; > > ? size_t hit = 0; > ? for (size_t i = 0; i < len1 - 1; i++) { > ??? for (size_t j = 0; j < len2 - 1; j++) { > ????? if ((str1[i] == str2[j]) && (str1[i+1] == str2[j+1])) { > ??????? ++hit; > ??????? break; > ????? } > ??? } > ? } > > If len2 is zero (which it is in this case) we have passed it as an > unsigned size_t, so len2-1 gives a massive positive value and so we > enter the loop and try to access str2[n] for some n>0 and we get a SEGV. > > The original code had: > > - for (int j = 0; j < (int) len2 -1; ++j) { > > so the huge positive value reverted to a small negative value and we > don't enter the loop. > > The fix applied is to check explicitly for lengths of zero. > > Added missing testcases to: > > test/hotspot/gtest/logging/test_logConfiguration.cpp > test/hotspot/jtreg/runtime/CommandLine/UnrecognizedVMOption.java > > verified they both crash before the fix. > > Testing (in progress): tier1,2,3 per mach5 CI > > Thanks, > David From ioi.lam at oracle.com Thu May 31 06:09:25 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 30 May 2018 23:09:25 -0700 Subject: RFR(XXS): 8188109 JVM should print a warning message that -Xshare:on may cause VM to abort start-up In-Reply-To: References: <2b1d7829-0dc1-bf7b-5e2d-426783738c1b@oracle.com> <288dfdce-57a5-e28e-f4e5-6ac96d87bb98@oracle.com> <5a655814-755c-354c-4e00-815e8cbbc11a@oracle.com> <84d0c341-ed14-16bb-b829-1dac03b828b5@oracle.com> <0b49b7b4-12fc-5f71-284a-67f962b9fcb9@oracle.com> Message-ID: On 5/30/18 10:57 PM, Thomas St?fe wrote: > On Thu, May 31, 2018 at 7:50 AM, Ioi Lam wrote: >> >> On 5/30/18 9:53 PM, David Holmes wrote: >>> On 31/05/2018 2:24 PM, Ioi Lam wrote: >>>> >>>> >>>> On 5/30/18 9:13 PM, David Holmes wrote: >>>>> Hi Ioi, >>>>> >>>>> On 31/05/2018 2:01 PM, Ioi Lam wrote: >>>>>> On 5/30/18 6:47 PM, David Holmes wrote: >>>>>>> Hi Ioi, >>>>>>> >>>>>>> Sorry but this troubles me ... >>>>>>> >>>>>>> On 31/05/2018 9:39 AM, Ioi Lam wrote: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8188109 >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk11/8188109-xshare-on-print-warning.v01/ >>>>>>>> >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> Please review this one-liner patch. >>>>>>>> >>>>>>>> -Xshare:on may cause infrequent/intermittent start-up failure due to >>>>>>>> the presence of Address Space Layout Randomization (ASLR). This option is >>>>>>>> intended for testing (the internals of CDS) only and should not be used in >>>>>>>> production environments. >>>>>>>> >>>>>>>> With this patch, the following warning message is printed when >>>>>>>> -Xshare:on is specified: >>>>>>>> >>>>>>>> $ java -Xshare:on -version >>>>>>>> Java HotSpot(TM) 64-Bit Server VM warning: -Xshare:on is for testing >>>>>>>> purpose only and may cause JVM start-up failure. Use -Xshare:auto instead. >>>>>>>> java version "11-internal" 2018-09-25 >>>>>>>> Java(TM) SE Runtime Environment 18.9 (fastdebug build >>>>>>>> 11-internal+0-adhoc.iklam.open) >>>>>>>> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >>>>>>>> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) >>>>>>> >>>>>>> So should this warning only be enabled in product builds? >>>>>>> >>>>>>> Even then it may be annoying for anyone who runs with -Xshare:on as >>>>>>> they've set up CDS as documented [1][2] and they know their environment >>>>>>> works ok - now they get a warning. >>>>>>> >>>>>>> Also I'm unclear how "on" fails due to ASLR but "auto" keeps going? >>>>>>> >>>>>> The documentation [1] says: >>>>>> >>>>>> -Xshare:on >>>>>> To enable class data sharing. If class data sharing can't be enabled, >>>>>> print an error message and exit. >>>>>> >>>>>> -Xshare:auto >>>>>> To enable class data sharing by default. Enable class data sharing >>>>>> whenever possible. >>>>>> >>>>>> So if mapping fails due to ASLR, "on" will exit and "auto" will disable >>>>>> CDS and continue . >>>>> >>>>> Ah! In the bug you state "-Xshare:auto continue to execute (with CDS >>>>> enabled)" - so that should be disabled. >>>>> >>>>>> The documentation in [2] is wrong. It says "Ensure that you have >>>>>> specified the option -Xshare:on or -Xshare:auto.", but -Xshare:on should not >>>>>> be used in production environments. I have filed a doc bug for this >>>>>> (https://bugs.openjdk.java.net/browse/JDK-8204141). >>>>>> >>>>>> The main reason for doing this REF is -- if people have been following >>>>>> [2] and using -Xshare:on, their setup is NOT OK. ASLR may happen just very >>>>>> rarely, but you don't want your program suddenly failing (e.g., if some >>>>>> admin has turned on more aggressive ASLR settings). >>>>> >>>>> But conversely you don't want your application to suddenly and silently >>>>> stop using CDS because of ASLR and you've masked that by using "auto"! >>>>> >>>>>> As more people are moving their Java workload to micro-services type of >>>>>> environments, JVM launches will happen more, and there will be more chances >>>>>> of running into the ASLR issue. Therefore, we should fix the docs, and warn >>>>>> people that they should switch to -Xshare:auto. >>>>> >>>>> That's only a partial solution. If ASLR is a problem then that needs to >>>>> be known and addressed. >>>>> >>>>>>> Maybe only if the archive mapping fails and "on" was used then give a >>>>>>> warning? Or just improve the message given when the VM aborts? >>>>>>> >>>>>> That's already too late, especially for people running critical >>>>>> services. >>>>>> >>>>>> We want people to see this warning and actively fix their scripts to >>>>>> get rid -Xshare:on. >>>>> >>>>> I think what we want/need people to realize is that ASLR can seriously >>>>> impact the ability to use CDS and if you are trying to use CDS for startup >>>>> or footprint reasons then you're going to have a major problem if CDS is >>>>> silently disabled! >>>>> >>>>> I think "on" and "auto" are both just as valid. "on" is for people who >>>>> need CDS reliably and want to fail fast if it's not working. "auto" is for >>>>> people who would like CDS but can live without it. >>>>> >>>> I think -Xshare:on has the potential to do much more harm than good. >>>> >>>> People have lived with the fact that optimizations in the JVM are not >>>> always deterministic. They want their programs to run regardless. CDS is the >>>> only optimization that has an option to say "let the program fail if the >>>> optimization is not available". >>> >>> I don't agree with that characterization - I think it oversimplifies the >>> situation. If you want to push this analogy then the right analogy would be >>> allowing "-server" to silently run the interpreter instead because there was >>> some error configuring the JIT! I wonder how many users would be happy with >>> that! But the bulk of optimizations are not things that can fail as such so >>> I don't think the comparison holds. >>> >>> I think this is purely a documentation and education issue. Particularly >>> because this is not something new with JDK 11 - this is an issue that exists >>> with CDS in all releases. So you want to get the message out to all users >>> that "auto" may be preferable to "on". >>> >>> Until something actually fails I doubt anyone would notice your warning >>> anyway. >>> >>> Maybe we need -Xshare:on_and_I_really_mean_it ;-) >>> >> -Xshare:on was an ill-conceived and dangerous option. It was designed when >> ASLR wasn't common. With ASLR more in common use, and short-lived JVMs >> becoming more common, the danger is getting bigger and bigger. >> > Just out of curiosity (I never payed close attention to CDS), how do > you communicate the mapping address the first process establishes to > the subsequent processes attaching? Or do you just have a fixed well > known address value baked into the jvm? > > I just try to understand how probable a failed mapping could be in a > 64bit address space. The CDS archive is created at a fixed address. The default value is in the SharedBaseAddress option (0x800000000 on 64-bit). This is usually a good range on Linux, as ASLR (usually) does not place shared library segments in this range. However, we have analyzed our distributed test runs and found a small percentage (<5%?? can't remember the exact number) of cases where the mapping would fail. So overall you get the benefit of CDS, but not every single time. We have been thinking about making the CDS archive relocatable, but we're not there yet :-( In the short term, we can relocate by patching all the pointers. We just added the ability to iterate over all metaspace pointers in JDK 10 (see metaspaceClosure.hpp) so we can map to an alternative address and relocate. We can probably do part of the relocation incrementally as the classes are being loaded. In the long term, we probably should make the metadata position independent, so it can be mapped at any address. Not quite sure how to do that yet ... Thanks - Ioi >> Just because a bad option has existed for a long time does not mean we >> should not fix it. >> >> The only reason for this option to exist is for diagnostic purposes. It can >> check for >> >> * Did my archive fail to map due to ASLR? >> * Did I specify a bad path to the archive? >> * Did I specify a bad archive file (e.g., one created by a different >> JDK version)? >> >> So Thomas's suggestion of providing this functionality as a diagnostic flag >> makes sense. >> >> In any case, I think this particular REF is not a good way of handling the >> problem, so I am withdrawing it. >> >> I'll file a separate CSR to actually change how -Xshare:on works, after more >> discussion on how best to change it. > I agree with all your points. > > ..Thomas > >> Thanks >> - Ioi >> >> >>> Cheers, >>> David >>> >>>> There are diagnostic options to find out if CDS is enabled. If you run >>>> with -showversion, it will tell you if sharing is enabled. People don't need >>>> their program to die just to find this out. >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>>> David >>>>> ----- >>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>> [1] >>>>>>> https://docs.oracle.com/javase/10/vm/class-data-sharing.htm#JSJVM-GUID-0260F857-A70E-4399-A1DF-A5766BE33285 >>>>>>> [2] >>>>>>> https://docs.oracle.com/javase/10/tools/java.htm#JSWOR-GUID-31503FCE-93D0-4175-9B4F-F6A738B2F4C4 >>>>>>> >>>>>>>> --- vs --- >>>>>>>> >>>>>>>> $ java-Xshare:auto -version >>>>>>>> java version "11-internal" 2018-09-25 >>>>>>>> Java(TM) SE Runtime Environment 18.9 (fastdebug build >>>>>>>> 11-internal+0-adhoc.iklam.open) >>>>>>>> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >>>>>>>> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) >>>>>>>> >>>>>>>> I am testing with HotSpot tiers 1-3 to make sure the tests don't get >>>>>>>> tripped by this new warning message. >>>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>> From thomas.stuefe at gmail.com Thu May 31 06:16:56 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 31 May 2018 08:16:56 +0200 Subject: RFR (S) 8204055: SIGSEGV in java -XX: In-Reply-To: <72335f6e-3856-ad1f-565e-8b11def70213@oracle.com> References: <72335f6e-3856-ad1f-565e-8b11def70213@oracle.com> Message-ID: Looks good. It raises the philosophical question though whether two zero length strings are completely alike or unlike each other :) ..Thomas On Thu, May 31, 2018 at 7:55 AM, David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8204055 > webrev: http://cr.openjdk.java.net/~dholmes/8204055/webrev/ > > > The SEGV was introduced with the fuzzy matching flag logic refactoring in > JDK-8198554. In: > > double StringUtils::similarity(const char* str1, size_t len1, const > char* str2, size_t len2) { > size_t total = len1 + len2; > > size_t hit = 0; > for (size_t i = 0; i < len1 - 1; i++) { > for (size_t j = 0; j < len2 - 1; j++) { > if ((str1[i] == str2[j]) && (str1[i+1] == str2[j+1])) { > ++hit; > break; > } > } > } > > If len2 is zero (which it is in this case) we have passed it as an unsigned > size_t, so len2-1 gives a massive positive value and so we enter the loop > and try to access str2[n] for some n>0 and we get a SEGV. > > The original code had: > > - for (int j = 0; j < (int) len2 -1; ++j) { > > so the huge positive value reverted to a small negative value and we don't > enter the loop. > > The fix applied is to check explicitly for lengths of zero. > > Added missing testcases to: > > test/hotspot/gtest/logging/test_logConfiguration.cpp > test/hotspot/jtreg/runtime/CommandLine/UnrecognizedVMOption.java > > verified they both crash before the fix. > > Testing (in progress): tier1,2,3 per mach5 CI > > Thanks, > David From thomas.stuefe at gmail.com Thu May 31 06:33:05 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 31 May 2018 08:33:05 +0200 Subject: RFR(XXS): 8188109 JVM should print a warning message that -Xshare:on may cause VM to abort start-up In-Reply-To: References: <2b1d7829-0dc1-bf7b-5e2d-426783738c1b@oracle.com> <288dfdce-57a5-e28e-f4e5-6ac96d87bb98@oracle.com> <5a655814-755c-354c-4e00-815e8cbbc11a@oracle.com> <84d0c341-ed14-16bb-b829-1dac03b828b5@oracle.com> <0b49b7b4-12fc-5f71-284a-67f962b9fcb9@oracle.com> Message-ID: On Thu, May 31, 2018 at 8:09 AM, Ioi Lam wrote: > > > On 5/30/18 10:57 PM, Thomas St?fe wrote: >> >> On Thu, May 31, 2018 at 7:50 AM, Ioi Lam wrote: >>> >>> >>> On 5/30/18 9:53 PM, David Holmes wrote: >>>> >>>> On 31/05/2018 2:24 PM, Ioi Lam wrote: >>>>> >>>>> >>>>> >>>>> On 5/30/18 9:13 PM, David Holmes wrote: >>>>>> >>>>>> Hi Ioi, >>>>>> >>>>>> On 31/05/2018 2:01 PM, Ioi Lam wrote: >>>>>>> >>>>>>> On 5/30/18 6:47 PM, David Holmes wrote: >>>>>>>> >>>>>>>> Hi Ioi, >>>>>>>> >>>>>>>> Sorry but this troubles me ... >>>>>>>> >>>>>>>> On 31/05/2018 9:39 AM, Ioi Lam wrote: >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8188109 >>>>>>>>> >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk11/8188109-xshare-on-print-warning.v01/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Please review this one-liner patch. >>>>>>>>> >>>>>>>>> -Xshare:on may cause infrequent/intermittent start-up failure due >>>>>>>>> to >>>>>>>>> the presence of Address Space Layout Randomization (ASLR). This >>>>>>>>> option is >>>>>>>>> intended for testing (the internals of CDS) only and should not be >>>>>>>>> used in >>>>>>>>> production environments. >>>>>>>>> >>>>>>>>> With this patch, the following warning message is printed when >>>>>>>>> -Xshare:on is specified: >>>>>>>>> >>>>>>>>> $ java -Xshare:on -version >>>>>>>>> Java HotSpot(TM) 64-Bit Server VM warning: -Xshare:on is for >>>>>>>>> testing >>>>>>>>> purpose only and may cause JVM start-up failure. Use -Xshare:auto >>>>>>>>> instead. >>>>>>>>> java version "11-internal" 2018-09-25 >>>>>>>>> Java(TM) SE Runtime Environment 18.9 (fastdebug build >>>>>>>>> 11-internal+0-adhoc.iklam.open) >>>>>>>>> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >>>>>>>>> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) >>>>>>>> >>>>>>>> >>>>>>>> So should this warning only be enabled in product builds? >>>>>>>> >>>>>>>> Even then it may be annoying for anyone who runs with -Xshare:on as >>>>>>>> they've set up CDS as documented [1][2] and they know their >>>>>>>> environment >>>>>>>> works ok - now they get a warning. >>>>>>>> >>>>>>>> Also I'm unclear how "on" fails due to ASLR but "auto" keeps going? >>>>>>>> >>>>>>> The documentation [1] says: >>>>>>> >>>>>>> -Xshare:on >>>>>>> To enable class data sharing. If class data sharing can't be enabled, >>>>>>> print an error message and exit. >>>>>>> >>>>>>> -Xshare:auto >>>>>>> To enable class data sharing by default. Enable class data sharing >>>>>>> whenever possible. >>>>>>> >>>>>>> So if mapping fails due to ASLR, "on" will exit and "auto" will >>>>>>> disable >>>>>>> CDS and continue . >>>>>> >>>>>> >>>>>> Ah! In the bug you state "-Xshare:auto continue to execute (with CDS >>>>>> enabled)" - so that should be disabled. >>>>>> >>>>>>> The documentation in [2] is wrong. It says "Ensure that you have >>>>>>> specified the option -Xshare:on or -Xshare:auto.", but -Xshare:on >>>>>>> should not >>>>>>> be used in production environments. I have filed a doc bug for this >>>>>>> (https://bugs.openjdk.java.net/browse/JDK-8204141). >>>>>>> >>>>>>> The main reason for doing this REF is -- if people have been >>>>>>> following >>>>>>> [2] and using -Xshare:on, their setup is NOT OK. ASLR may happen just >>>>>>> very >>>>>>> rarely, but you don't want your program suddenly failing (e.g., if >>>>>>> some >>>>>>> admin has turned on more aggressive ASLR settings). >>>>>> >>>>>> >>>>>> But conversely you don't want your application to suddenly and >>>>>> silently >>>>>> stop using CDS because of ASLR and you've masked that by using "auto"! >>>>>> >>>>>>> As more people are moving their Java workload to micro-services type >>>>>>> of >>>>>>> environments, JVM launches will happen more, and there will be more >>>>>>> chances >>>>>>> of running into the ASLR issue. Therefore, we should fix the docs, >>>>>>> and warn >>>>>>> people that they should switch to -Xshare:auto. >>>>>> >>>>>> >>>>>> That's only a partial solution. If ASLR is a problem then that needs >>>>>> to >>>>>> be known and addressed. >>>>>> >>>>>>>> Maybe only if the archive mapping fails and "on" was used then give >>>>>>>> a >>>>>>>> warning? Or just improve the message given when the VM aborts? >>>>>>>> >>>>>>> That's already too late, especially for people running critical >>>>>>> services. >>>>>>> >>>>>>> We want people to see this warning and actively fix their scripts to >>>>>>> get rid -Xshare:on. >>>>>> >>>>>> >>>>>> I think what we want/need people to realize is that ASLR can seriously >>>>>> impact the ability to use CDS and if you are trying to use CDS for >>>>>> startup >>>>>> or footprint reasons then you're going to have a major problem if CDS >>>>>> is >>>>>> silently disabled! >>>>>> >>>>>> I think "on" and "auto" are both just as valid. "on" is for people who >>>>>> need CDS reliably and want to fail fast if it's not working. "auto" is >>>>>> for >>>>>> people who would like CDS but can live without it. >>>>>> >>>>> I think -Xshare:on has the potential to do much more harm than good. >>>>> >>>>> People have lived with the fact that optimizations in the JVM are not >>>>> always deterministic. They want their programs to run regardless. CDS >>>>> is the >>>>> only optimization that has an option to say "let the program fail if >>>>> the >>>>> optimization is not available". >>>> >>>> >>>> I don't agree with that characterization - I think it oversimplifies the >>>> situation. If you want to push this analogy then the right analogy would >>>> be >>>> allowing "-server" to silently run the interpreter instead because there >>>> was >>>> some error configuring the JIT! I wonder how many users would be happy >>>> with >>>> that! But the bulk of optimizations are not things that can fail as such >>>> so >>>> I don't think the comparison holds. >>>> >>>> I think this is purely a documentation and education issue. Particularly >>>> because this is not something new with JDK 11 - this is an issue that >>>> exists >>>> with CDS in all releases. So you want to get the message out to all >>>> users >>>> that "auto" may be preferable to "on". >>>> >>>> Until something actually fails I doubt anyone would notice your warning >>>> anyway. >>>> >>>> Maybe we need -Xshare:on_and_I_really_mean_it ;-) >>>> >>> -Xshare:on was an ill-conceived and dangerous option. It was designed >>> when >>> ASLR wasn't common. With ASLR more in common use, and short-lived JVMs >>> becoming more common, the danger is getting bigger and bigger. >>> >> Just out of curiosity (I never payed close attention to CDS), how do >> you communicate the mapping address the first process establishes to >> the subsequent processes attaching? Or do you just have a fixed well >> known address value baked into the jvm? >> >> I just try to understand how probable a failed mapping could be in a >> 64bit address space. > > > The CDS archive is created at a fixed address. The default value is in the > SharedBaseAddress option (0x800000000 on 64-bit). This is usually a good > range on Linux, as ASLR (usually) does not place shared library segments in > this range. > > However, we have analyzed our distributed test runs and found a small > percentage (<5%?? can't remember the exact number) of cases where the > mapping would fail. So overall you get the benefit of CDS, but not every > single time. > > We have been thinking about making the CDS archive relocatable, but we're > not there yet :-( > > In the short term, we can relocate by patching all the pointers. We just > added the ability to iterate over all metaspace pointers in JDK 10 (see > metaspaceClosure.hpp) so we can map to an alternative address and relocate. > We can probably do part of the relocation incrementally as the classes are > being loaded. > Okay... so, am I understanding this right, you fix up pointers in the java heap or in other non-shared process local memory sections, pointing to the metaspace? E.g. the object Klass* pointers? What do you do about metaspace internal pointers, e.g. pointers from one Metachunk to another? You cannot fix them, right, since you share the memory with other processes which may have mapped it at different bases? > In the long term, we probably should make the metadata position independent, > so it can be mapped at any address. Not quite sure how to do that yet ... > This sounds (to my very uninformed mind) more promising - by using indexes instead of pointers into the metaspace. E.g. to get a Klass*, add base + index. Like we do already with compressed class pointers. For many things one could even use 32bit indices, at least for Klass* pointers, since the compressed class space cannot exceed 32bit anyway. I recently though about a similar thing myself, at a much much smaller scale, I wondered whether it would be worth to replace all linking pointers in Metachunk with 32bit indices (or even 16bit) indices to shrink the Metachunk header. One may have to get rid of the notion of linking Metaspace nodes together (VirtualSpaceList) in favour of one continuous memory block - but that may have other advantages too (make metaspace coding simpler, allow for page-wise de-commit to shrink process memory, reduce number of memory mappings per process...) Thanks Thomas > > Thanks > - Ioi > >>> Just because a bad option has existed for a long time does not mean we >>> should not fix it. >>> >>> The only reason for this option to exist is for diagnostic purposes. It >>> can >>> check for >>> >>> * Did my archive fail to map due to ASLR? >>> * Did I specify a bad path to the archive? >>> * Did I specify a bad archive file (e.g., one created by a different >>> JDK version)? >>> >>> So Thomas's suggestion of providing this functionality as a diagnostic >>> flag >>> makes sense. >>> >>> In any case, I think this particular REF is not a good way of handling >>> the >>> problem, so I am withdrawing it. >>> >>> I'll file a separate CSR to actually change how -Xshare:on works, after >>> more >>> discussion on how best to change it. >> >> I agree with all your points. >> >> ..Thomas >> >>> Thanks >>> - Ioi >>> >>> >>>> Cheers, >>>> David >>>> >>>>> There are diagnostic options to find out if CDS is enabled. If you run >>>>> with -showversion, it will tell you if sharing is enabled. People don't >>>>> need >>>>> their program to die just to find this out. >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>> [1] >>>>>>>> >>>>>>>> https://docs.oracle.com/javase/10/vm/class-data-sharing.htm#JSJVM-GUID-0260F857-A70E-4399-A1DF-A5766BE33285 >>>>>>>> [2] >>>>>>>> >>>>>>>> https://docs.oracle.com/javase/10/tools/java.htm#JSWOR-GUID-31503FCE-93D0-4175-9B4F-F6A738B2F4C4 >>>>>>>> >>>>>>>>> --- vs --- >>>>>>>>> >>>>>>>>> $ java-Xshare:auto -version >>>>>>>>> java version "11-internal" 2018-09-25 >>>>>>>>> Java(TM) SE Runtime Environment 18.9 (fastdebug build >>>>>>>>> 11-internal+0-adhoc.iklam.open) >>>>>>>>> Java HotSpot(TM) 64-Bit Server VM 18.9 (fastdebug build >>>>>>>>> 11-internal+0-adhoc.iklam.open, mixed mode, sharing) >>>>>>>>> >>>>>>>>> I am testing with HotSpot tiers 1-3 to make sure the tests don't >>>>>>>>> get >>>>>>>>> tripped by this new warning message. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> - Ioi >>>>>>> >>>>>>> > From calvin.cheung at oracle.com Thu May 31 06:34:21 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 30 May 2018 23:34:21 -0700 Subject: RFR (S) 8204055: SIGSEGV in java -XX: In-Reply-To: <72335f6e-3856-ad1f-565e-8b11def70213@oracle.com> References: <72335f6e-3856-ad1f-565e-8b11def70213@oracle.com> Message-ID: <5B0F976D.1000507@oracle.com> Hi David, Looks good. Just a small nit in stringUtils.cpp for which I don't need to see another webrev. 52 // filter out zero-length strings else we will underflow on len-1 below 53 if (len1 == 0 || len2 == 0) { 54 return 0.0; 55 } How about moving the above right before the calculation of "total" (line 49)? thanks, Calvin On 5/30/18, 10:55 PM, David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8204055 > webrev: http://cr.openjdk.java.net/~dholmes/8204055/webrev/ > > > The SEGV was introduced with the fuzzy matching flag logic refactoring > in JDK-8198554. In: > > double StringUtils::similarity(const char* str1, size_t len1, const > char* str2, size_t len2) { > size_t total = len1 + len2; > > size_t hit = 0; > for (size_t i = 0; i < len1 - 1; i++) { > for (size_t j = 0; j < len2 - 1; j++) { > if ((str1[i] == str2[j]) && (str1[i+1] == str2[j+1])) { > ++hit; > break; > } > } > } > > If len2 is zero (which it is in this case) we have passed it as an > unsigned size_t, so len2-1 gives a massive positive value and so we > enter the loop and try to access str2[n] for some n>0 and we get a SEGV. > > The original code had: > > - for (int j = 0; j < (int) len2 -1; ++j) { > > so the huge positive value reverted to a small negative value and we > don't enter the loop. > > The fix applied is to check explicitly for lengths of zero. > > Added missing testcases to: > > test/hotspot/gtest/logging/test_logConfiguration.cpp > test/hotspot/jtreg/runtime/CommandLine/UnrecognizedVMOption.java > > verified they both crash before the fix. > > Testing (in progress): tier1,2,3 per mach5 CI > > Thanks, > David From calvin.cheung at oracle.com Thu May 31 06:35:12 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 30 May 2018 23:35:12 -0700 Subject: RFR(xs): 8203960: [TESTBUG] runtime/logging/DefaultMethodsTest.java failed when running in CDS mode In-Reply-To: <1c5eff9c-0592-8ffd-9927-ebdbf1c08e42@oracle.com> References: <5B0F72F8.6020503@oracle.com> <5B0F8263.6020309@oracle.com> <1c5eff9c-0592-8ffd-9927-ebdbf1c08e42@oracle.com> Message-ID: <5B0F97A0.4010100@oracle.com> Thanks Ioi. Calvin On 5/30/18, 10:55 PM, Ioi Lam wrote: > Looks good. Thanks! > > - Ioi > > > On 5/30/18 10:04 PM, Calvin Cheung wrote: >> Hi Ioi, David, >> >> Thanks for your review and suggestion. >> >> I've added a simple interface with a default method. The InnerClass >> implements the interface. >> >> Updated webrev: >> http://cr.openjdk.java.net/~ccheung/8203960/webrev.01/ >> >> thanks, >> Calvin >> >> >> On 5/30/18, 9:16 PM, David Holmes wrote: >>> +1 the test should be self-sufficient and not rely on an unrelated >>> module. >>> >>> Thanks, >>> David >>> >>> On 31/05/2018 2:03 PM, Ioi Lam wrote: >>>> Hi Calvin, >>>> >>>> Instead of relying on an internal JDK class to have default method >>>> processing, maybe InnerClass should contain some code to ensure >>>> that default method processing will always happen? >>>> >>>> Thanks >>>> >>>> - Ioi >>>> >>>> >>>> On 5/30/18 8:58 PM, Calvin Cheung wrote: >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8203960 >>>>> >>>>> webrev: http://cr.openjdk.java.net/~ccheung/8203960/webrev.00/ >>>>> >>>>> If this test is run in CDS mode, most of the system classes will >>>>> be in the CDS archive and loading of those classes from the >>>>> archive will bypass the default method processing. The test fails >>>>> in CDS mode since it expects the trace output from default method >>>>> processing. >>>>> A fix is to load an additional class which isn't in any of the >>>>> modules defined by default. The loading of the additional class >>>>> will trigger default method processing. >>>>> >>>>> Ran the test with and without CDS. >>>>> I will do a sanity hs-tier1 and hs-tier2 testing run. >>>>> >>>>> thanks, >>>>> Calvin >>>> > From david.holmes at oracle.com Thu May 31 07:10:03 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 31 May 2018 17:10:03 +1000 Subject: RFR (S) 8204055: SIGSEGV in java -XX: In-Reply-To: References: <72335f6e-3856-ad1f-565e-8b11def70213@oracle.com> Message-ID: <109a2f10-353b-19c8-011b-0132e21a3e7a@oracle.com> Thanks Ioi! David On 31/05/2018 3:58 PM, Ioi Lam wrote: > Looks good! Thanks David. > > - Ioi > > > On 5/30/18 10:55 PM, David Holmes wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204055 >> webrev: http://cr.openjdk.java.net/~dholmes/8204055/webrev/ >> >> >> The SEGV was introduced with the fuzzy matching flag logic refactoring >> in JDK-8198554. In: >> >> double StringUtils::similarity(const char* str1, size_t len1, const >> ?????????????????????????????? char* str2, size_t len2) { >> ? size_t total = len1 + len2; >> >> ? size_t hit = 0; >> ? for (size_t i = 0; i < len1 - 1; i++) { >> ??? for (size_t j = 0; j < len2 - 1; j++) { >> ????? if ((str1[i] == str2[j]) && (str1[i+1] == str2[j+1])) { >> ??????? ++hit; >> ??????? break; >> ????? } >> ??? } >> ? } >> >> If len2 is zero (which it is in this case) we have passed it as an >> unsigned size_t, so len2-1 gives a massive positive value and so we >> enter the loop and try to access str2[n] for some n>0 and we get a SEGV. >> >> The original code had: >> >> - for (int j = 0; j < (int) len2 -1; ++j) { >> >> so the huge positive value reverted to a small negative value and we >> don't enter the loop. >> >> The fix applied is to check explicitly for lengths of zero. >> >> Added missing testcases to: >> >> test/hotspot/gtest/logging/test_logConfiguration.cpp >> test/hotspot/jtreg/runtime/CommandLine/UnrecognizedVMOption.java >> >> verified they both crash before the fix. >> >> Testing (in progress): tier1,2,3 per mach5 CI >> >> Thanks, >> David > From david.holmes at oracle.com Thu May 31 07:11:29 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 31 May 2018 17:11:29 +1000 Subject: RFR (S) 8204055: SIGSEGV in java -XX: In-Reply-To: References: <72335f6e-3856-ad1f-565e-8b11def70213@oracle.com> Message-ID: <22e22560-441f-7fa5-e652-0cfa9e9b7aea@oracle.com> On 31/05/2018 4:16 PM, Thomas St?fe wrote: > Looks good. Thanks Thomas! > It raises the philosophical question though whether two zero length > strings are completely alike or unlike each other :) Hmm maybe I should return 1 in that case ;-) David > ..Thomas > > On Thu, May 31, 2018 at 7:55 AM, David Holmes wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204055 >> webrev: http://cr.openjdk.java.net/~dholmes/8204055/webrev/ >> >> >> The SEGV was introduced with the fuzzy matching flag logic refactoring in >> JDK-8198554. In: >> >> double StringUtils::similarity(const char* str1, size_t len1, const >> char* str2, size_t len2) { >> size_t total = len1 + len2; >> >> size_t hit = 0; >> for (size_t i = 0; i < len1 - 1; i++) { >> for (size_t j = 0; j < len2 - 1; j++) { >> if ((str1[i] == str2[j]) && (str1[i+1] == str2[j+1])) { >> ++hit; >> break; >> } >> } >> } >> >> If len2 is zero (which it is in this case) we have passed it as an unsigned >> size_t, so len2-1 gives a massive positive value and so we enter the loop >> and try to access str2[n] for some n>0 and we get a SEGV. >> >> The original code had: >> >> - for (int j = 0; j < (int) len2 -1; ++j) { >> >> so the huge positive value reverted to a small negative value and we don't >> enter the loop. >> >> The fix applied is to check explicitly for lengths of zero. >> >> Added missing testcases to: >> >> test/hotspot/gtest/logging/test_logConfiguration.cpp >> test/hotspot/jtreg/runtime/CommandLine/UnrecognizedVMOption.java >> >> verified they both crash before the fix. >> >> Testing (in progress): tier1,2,3 per mach5 CI >> >> Thanks, >> David From david.holmes at oracle.com Thu May 31 07:13:04 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 31 May 2018 17:13:04 +1000 Subject: RFR (S) 8204055: SIGSEGV in java -XX: In-Reply-To: <5B0F976D.1000507@oracle.com> References: <72335f6e-3856-ad1f-565e-8b11def70213@oracle.com> <5B0F976D.1000507@oracle.com> Message-ID: <75b6999c-ac6c-33e8-d9a2-495b8d2e5e32@oracle.com> On 31/05/2018 4:34 PM, Calvin Cheung wrote: > Hi David, > > Looks good. Thanks Calvin. > Just a small nit in stringUtils.cpp for which I don't need to see > another webrev. > > ? 52?? // filter out zero-length strings else we will underflow on > len-1 below > ? 53?? if (len1 == 0 || len2 == 0) { > ? 54???? return 0.0; > ? 55?? } > > How about moving the above right before the calculation of "total" (line > 49)? Yes I can do that. Cheers, David > thanks, > Calvin > > On 5/30/18, 10:55 PM, David Holmes wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8204055 >> webrev: http://cr.openjdk.java.net/~dholmes/8204055/webrev/ >> >> >> The SEGV was introduced with the fuzzy matching flag logic refactoring >> in JDK-8198554. In: >> >> double StringUtils::similarity(const char* str1, size_t len1, const >> ?????????????????????????????? char* str2, size_t len2) { >> ? size_t total = len1 + len2; >> >> ? size_t hit = 0; >> ? for (size_t i = 0; i < len1 - 1; i++) { >> ??? for (size_t j = 0; j < len2 - 1; j++) { >> ????? if ((str1[i] == str2[j]) && (str1[i+1] == str2[j+1])) { >> ??????? ++hit; >> ??????? break; >> ????? } >> ??? } >> ? } >> >> If len2 is zero (which it is in this case) we have passed it as an >> unsigned size_t, so len2-1 gives a massive positive value and so we >> enter the loop and try to access str2[n] for some n>0 and we get a SEGV. >> >> The original code had: >> >> - for (int j = 0; j < (int) len2 -1; ++j) { >> >> so the huge positive value reverted to a small negative value and we >> don't enter the loop. >> >> The fix applied is to check explicitly for lengths of zero. >> >> Added missing testcases to: >> >> test/hotspot/gtest/logging/test_logConfiguration.cpp >> test/hotspot/jtreg/runtime/CommandLine/UnrecognizedVMOption.java >> >> verified they both crash before the fix. >> >> Testing (in progress): tier1,2,3 per mach5 CI >> >> Thanks, >> David From stefan.karlsson at oracle.com Thu May 31 11:09:29 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 13:09:29 +0200 Subject: RFR: 8204160: BiasedLockingTest needs -XX:+UseBiasedLocking Message-ID: <9d26c61a-9bed-9ddb-cbc6-8d7f5580cb7c@oracle.com> Hi all, Please review this trivial patch to add -XX:+UseBiasLocking to the BiasedLockingTest.java test. http://cr.openjdk.java.net/~stefank/8204160/webrev.01/ This is needed because some GCs (ZGC) turn off biased locking by default. Thanks, StefanK From erik.osterlund at oracle.com Thu May 31 11:23:41 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 31 May 2018 13:23:41 +0200 Subject: RFR: 8204160: BiasedLockingTest needs -XX:+UseBiasedLocking In-Reply-To: <9d26c61a-9bed-9ddb-cbc6-8d7f5580cb7c@oracle.com> References: <9d26c61a-9bed-9ddb-cbc6-8d7f5580cb7c@oracle.com> Message-ID: <5B0FDB3D.9070705@oracle.com> Hi Stefan, Looks good and trivial. Thanks, /Erik On 2018-05-31 13:09, Stefan Karlsson wrote: > Hi all, > > Please review this trivial patch to add -XX:+UseBiasLocking to the > BiasedLockingTest.java test. > > http://cr.openjdk.java.net/~stefank/8204160/webrev.01/ > > This is needed because some GCs (ZGC) turn off biased locking by default. > > Thanks, > StefanK From markus.gronlund at oracle.com Thu May 31 11:31:21 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Thu, 31 May 2018 11:31:21 +0000 (UTC) Subject: RFR(M): JDK-8203321: Windows: assert(current_query_index < process_query_set->size) failed: invariant In-Reply-To: <14f593dc-11e0-770d-4d70-155edfbc3be3@oracle.com> References: <06e925c8-8d77-43c8-9ef2-ad9113034b98@default> <14f593dc-11e0-770d-4d70-155edfbc3be3@oracle.com> Message-ID: Thank you Robbin. Markus -----Original Message----- From: Robbin Ehn Sent: den 30 maj 2018 22:26 To: Markus Gronlund ; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(M): JDK-8203321: Windows: assert(current_query_index < process_query_set->size) failed: invariant Hi Markus, I'm not familiar with PDH, but given your description and the code looks good. Thanks, Robbin On 2018-05-22 21:24, Markus Gronlund wrote: > Greetings, > > Kindly asking for reviews for the following change: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203321 > Webrev: http://cr.openjdk.java.net/~mgronlun/8203321/webrev00/ > > Summary: > > For some context about what this is about, please see this (now) relatively old issue: https://bugs.openjdk.java.net/browse/JDK-8019921 > > The porting work that brought this code from closed to open were optimistic in that the following PDH query, "\Process(java#n)\ID Process", performed relatively stable on Windows 10. > An invariant was added in that your ID Process query would never return an index that was lower than the index at construction. > > During testing, it was discovered that this invariant did not hold, especially when running on Windows Server 2012 R2 and there is a high churn rate with many processes with the same base name ("java#") starting and stopping (stressing PDH list of processes). > > We have to reinsert back the original code that handled the case where the PDH process list is not stable (that were originally put in place with JDK-8019921). > The defensive logic is located at lines 418 - 422. > > I had to rework some related code to make some room for this as well as to keep track of the previous process index. > > Thanks > Markus > From thomas.stuefe at gmail.com Thu May 31 11:36:58 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 31 May 2018 13:36:58 +0200 Subject: RFR(xxxs): 8204164: OOM-only logging in Metaspace Message-ID: Hi all, very tiny improvement. The intent is to be able to restrict metaspace logging to OOM situations. CR: OOM-only logging in Metaspace Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8204164-oom-only-logging-in-metaspace/webrev.00/webrev/ -- On a related note, would anyone be offended if I were to remove the "freelist" logging tag from the metaspace coding? I am not sure that is useful in any way. Thanks, Thomas From thomas.stuefe at gmail.com Thu May 31 11:37:46 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 31 May 2018 13:37:46 +0200 Subject: RFR(xxxs): 8204164: OOM-only logging in Metaspace In-Reply-To: References: Message-ID: Wrong JBS link. Here is the correct one: https://bugs.openjdk.java.net/browse/JDK-8204164 ..Thomas On Thu, May 31, 2018 at 1:36 PM, Thomas St?fe wrote: > Hi all, > > very tiny improvement. The intent is to be able to restrict metaspace > logging to OOM situations. > > CR: OOM-only logging in Metaspace > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8204164-oom-only-logging-in-metaspace/webrev.00/webrev/ > > -- > > On a related note, would anyone be offended if I were to remove the > "freelist" logging tag from the metaspace coding? I am not sure that > is useful in any way. > > Thanks, Thomas From stefan.karlsson at oracle.com Thu May 31 11:54:55 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 13:54:55 +0200 Subject: RFR: 8204160: BiasedLockingTest needs -XX:+UseBiasedLocking In-Reply-To: <5B0FDB3D.9070705@oracle.com> References: <9d26c61a-9bed-9ddb-cbc6-8d7f5580cb7c@oracle.com> <5B0FDB3D.9070705@oracle.com> Message-ID: <8f18cd33-cb29-0bce-42a5-5fb708acb4cb@oracle.com> Thanks, Erik. StefanK On 2018-05-31 13:23, Erik ?sterlund wrote: > Hi Stefan, > > Looks good and trivial. > > Thanks, > /Erik > > On 2018-05-31 13:09, Stefan Karlsson wrote: >> Hi all, >> >> Please review this trivial patch to add -XX:+UseBiasLocking to the >> BiasedLockingTest.java test. >> >> http://cr.openjdk.java.net/~stefank/8204160/webrev.01/ >> >> This is needed because some GCs (ZGC) turn off biased locking by default. >> >> Thanks, >> StefanK > From coleen.phillimore at oracle.com Thu May 31 12:07:37 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 31 May 2018 08:07:37 -0400 Subject: RFR(M): JDK-8203321: Windows: assert(current_query_index < process_query_set->size) failed: invariant In-Reply-To: References: <06e925c8-8d77-43c8-9ef2-ad9113034b98@default> <14f593dc-11e0-770d-4d70-155edfbc3be3@oracle.com> Message-ID: <4dea53d1-27d5-a648-4866-173ffa3bf27d@oracle.com> Markus, this looks good.? Thank you for fixing it. Coleen On 5/31/18 7:31 AM, Markus Gronlund wrote: > Thank you Robbin. > > Markus > > -----Original Message----- > From: Robbin Ehn > Sent: den 30 maj 2018 22:26 > To: Markus Gronlund ; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(M): JDK-8203321: Windows: assert(current_query_index < process_query_set->size) failed: invariant > > Hi Markus, > > I'm not familiar with PDH, but given your description and the code looks good. > > Thanks, Robbin > > On 2018-05-22 21:24, Markus Gronlund wrote: >> Greetings, >> >> Kindly asking for reviews for the following change: >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203321 >> Webrev: http://cr.openjdk.java.net/~mgronlun/8203321/webrev00/ >> >> Summary: >> >> For some context about what this is about, please see this (now) relatively old issue: https://bugs.openjdk.java.net/browse/JDK-8019921 >> >> The porting work that brought this code from closed to open were optimistic in that the following PDH query, "\Process(java#n)\ID Process", performed relatively stable on Windows 10. >> An invariant was added in that your ID Process query would never return an index that was lower than the index at construction. >> >> During testing, it was discovered that this invariant did not hold, especially when running on Windows Server 2012 R2 and there is a high churn rate with many processes with the same base name ("java#") starting and stopping (stressing PDH list of processes). >> >> We have to reinsert back the original code that handled the case where the PDH process list is not stable (that were originally put in place with JDK-8019921). >> The defensive logic is located at lines 418 - 422. >> >> I had to rework some related code to make some room for this as well as to keep track of the previous process index. >> >> Thanks >> Markus >> From david.holmes at oracle.com Thu May 31 12:47:36 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 31 May 2018 22:47:36 +1000 Subject: RFR: 8204160: BiasedLockingTest needs -XX:+UseBiasedLocking In-Reply-To: <9d26c61a-9bed-9ddb-cbc6-8d7f5580cb7c@oracle.com> References: <9d26c61a-9bed-9ddb-cbc6-8d7f5580cb7c@oracle.com> Message-ID: Hi Stefan, Seems reasonable. Thanks, David On 31/05/2018 9:09 PM, Stefan Karlsson wrote: > Hi all, > > Please review this trivial patch to add -XX:+UseBiasLocking to the > BiasedLockingTest.java test. > > http://cr.openjdk.java.net/~stefank/8204160/webrev.01/ > > This is needed because some GCs (ZGC) turn off biased locking by default. > > Thanks, > StefanK From stefan.karlsson at oracle.com Thu May 31 12:46:58 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 14:46:58 +0200 Subject: RFR: 8204160: BiasedLockingTest needs -XX:+UseBiasedLocking In-Reply-To: References: <9d26c61a-9bed-9ddb-cbc6-8d7f5580cb7c@oracle.com> Message-ID: Thanks, David. StefanK On 2018-05-31 14:47, David Holmes wrote: > Hi Stefan, > > Seems reasonable. > > Thanks, > David > > On 31/05/2018 9:09 PM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this trivial patch to add -XX:+UseBiasLocking to the >> BiasedLockingTest.java test. >> >> http://cr.openjdk.java.net/~stefank/8204160/webrev.01/ >> >> This is needed because some GCs (ZGC) turn off biased locking by default. >> >> Thanks, >> StefanK From harold.seigel at oracle.com Thu May 31 13:08:43 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Thu, 31 May 2018 09:08:43 -0400 Subject: RFR 8202913: loader constraint message for fields specifies incorrect referring class In-Reply-To: <1465e732076042298280084262f2aeba@sap.com> References: <9a209744d62a4e56a5ec81ddb77afb1f@sap.com> <7ed091c7498f44c1b6f2b8a4a5030c38@sap.com> <56a039e3-5b91-dbea-ecaa-be58b0646823@oracle.com> <1465e732076042298280084262f2aeba@sap.com> Message-ID: <0d795016-3c9d-ded1-cac4-b8e59a2a5011@oracle.com> Thanks! Harold On 5/30/2018 11:40 AM, Lindenmaier, Goetz wrote: > Hi Harold, > >> ... AppClassLoader) for the class|abstractclass|interface test.Parent defining > I meant you should print the proper identifier, not the text with '|' by using > "... %s ..." , ..., k->external_kind(), ... > As this replaces 'type' by one or two (abstract class) words I don't think > it makes the message more wordy, just more precise. See also patch below > (just edited, not tested). Feel free to ignore my proposal, and else I don't > need a new webrev in case you adapt this. > > The rest looks good. Great it's all with '.' and you moved the test! > > Best regards, > Goetz. > > diff -r 6b4a85ad6bea src/hotspot/share/interpreter/linkResolver.cpp > --- a/src/hotspot/share/interpreter/linkResolver.cpp Wed May 30 17:32:28 2018 +0200 > +++ b/src/hotspot/share/interpreter/linkResolver.cpp Wed May 30 17:37:04 2018 +0200 > @@ -690,19 +690,20 @@ > const char* msg = "loader constraint violation: when resolving field" > " \"%s\" of type %s, the class loader %s of the current class, " > "%s, and the class loader %s for the field's defining " > - "type, %s, have different Class objects for type %s"; > + "%s, %s, have different Class objects for type %s"; > const char* field_name = field->as_C_string(); > const char* loader1_name = java_lang_ClassLoader::describe_external(ref_loader()); > const char* sel = sel_klass->external_name(); > + const char* sel_type = sel_klass->external_kind(); > const char* loader2_name = java_lang_ClassLoader::describe_external(sel_loader()); > const char* failed_type_name = failed_type_symbol->as_klass_external_name(); > const char* curr_klass_name = current_klass->external_name(); > size_t buflen = strlen(msg) + strlen(field_name) + 2 * strlen(failed_type_name) + > strlen(loader1_name) + strlen(curr_klass_name) + > - strlen(loader2_name) + strlen(sel) + 1; > + strlen(loader2_name) + strlen(sel) + strlen(sel_type) + 1; > char* buf = NEW_RESOURCE_ARRAY_IN_THREAD(THREAD, char, buflen); > jio_snprintf(buf, buflen, msg, field_name, failed_type_name, loader1_name, > - curr_klass_name, loader2_name, sel, failed_type_name); > + curr_klass_name, loader2_name, sel_type, sel, failed_type_name); > THROW_MSG(vmSymbols::java_lang_LinkageError(), buf); > } > } > > > >> -----Original Message----- >> From: Harold David Seigel [mailto:harold.seigel at oracle.com] >> Sent: Mittwoch, 30. Mai 2018 15:07 >> To: Lindenmaier, Goetz ; hotspot-runtime- >> dev at openjdk.java.net >> Subject: Re: RFR 8202913: loader constraint message for fields specifies >> incorrect referring class >> >> Hi Goetz, >> >> Thanks for reviewing this change! >> >> >> Please review this updated webrev for this bug: >> >> http://cr.openjdk.java.net/~hseigel/bug_8202913.2/webrev/index.h >> tml >> >> The new exception message for the included test looks like this: >> >> loader constraint violation: when resolving field "_field1" of type >> pkg.Foo, the class loader "" (instance of >> pkg.ClassLoaderForChildGrandFoo, child of "app" >> jdk.internal.loader.ClassLoaders$AppClassLoader) of the current class, >> pkg.Child, and the class loader "" (instance of >> pkg.ClassLoaderForParentFoo, child of "app" >> jdk.internal.loader.ClassLoaders$AppClassLoader) for the field's defining >> type, pkg.Parent, have different Class objects for type pkg.Foo >> >> >> Please also see comments in-line. >> >> >> >> On 5/25/2018 10:23 AM, Lindenmaier, Goetz wrote: >> >> >> Hi, >> >> >> I think this would read better if you replace 'type' by what is >> returned by >> Klass::external_kind(). >> >> correcting myself, you can just say 'class' ... interfaces don't define >> fields ... >> >> Karen pointed out that interfaces can have public static fields. So, I'll keep it >> as 'type', not 'class'. >> >> >> >> >> Best regards, >> Goetz. >> >> >> >> -----Original Message----- >> From: Lindenmaier, Goetz >> Sent: Freitag, 25. Mai 2018 16:18 >> To: 'Harold David Seigel' >> ; hotspot-runtime- >> dev at openjdk.java.net >> Subject: RE: RFR 8202913: loader constraint message for fields >> specifies >> incorrect referring class >> >> Hi Harold, >> >> Thanks for adding this further improvement over my change. >> >> >> ... AppClassLoader) for the field's >> defining type, Parent, have different Class objects for >> type Foo >> >> I think this would read better if you replace 'type' by what is >> returned by >> Klass::external_kind(). >> >> Actually I would prefer to read >> ... AppClassLoader) for the class|abstractclass|interface >> test.Parent defining >> this field, >> have different Class objects for type Foo >> >> The message is already too wordy. I'd prefer to not add this additional text >> because it is not related to the reason for the exception. >> >> >> >> >> Could you please add the test files into some package so that >> you >> can assure class names are printed as test.Parent and not >> test/Parent? >> There is external_name() to get the proper names from >> classes, I don't >> know for symbols. >> Related tests are in runtime/LoaderConstraint. Please move >> your test >> there. >> >> Done. >> >> Thanks, Harold >> >> >> >> >> Best regards, >> Goetz. >> >> >> >> >> >> -----Original Message----- >> From: hotspot-runtime-dev [mailto:hotspot-runtime- >> dev- >> bounces at openjdk.java.net >> ] On Behalf Of Harold David Seigel >> Sent: Freitag, 25. Mai 2018 15:29 >> To: hotspot-runtime-dev at openjdk.java.net >> >> Subject: RFR 8202913: loader constraint message for >> fields specifies >> >> incorrect >> >> referring class >> >> Hi, >> >> Please review this change to correct and simplify the >> error message >> displayed when a loader constraint check fails when >> trying to access a >> field. >> >> The old message (for this test case): >> >> loader constraint violation: when resolving field >> "_field1" the >> class loader "" (instance of >> ClassLoaderForChildGrandFoo, >> child of "app" >> jdk.internal.loader.ClassLoaders$AppClassLoader) of >> the referring class, Parent, and the class loader >> "" >> (instance of ClassLoaderForParentFoo, child of >> "app" >> jdk.internal.loader.ClassLoaders$AppClassLoader) >> for the field's >> resolved type, Foo, have different Class objects for >> that type >> >> The new message: >> >> loader constraint violation: when resolving field >> "_field1" of type >> Foo, the class loader "" (instance of >> ClassLoaderForChildGrandFoo, child of "app" >> jdk.internal.loader.ClassLoaders$AppClassLoader) >> of the current >> class, Child, and the class loader "" >> (instance of >> ClassLoaderForParentFoo, child of "app" >> jdk.internal.loader.ClassLoaders$AppClassLoader) >> for the field's >> defining type, Parent, have different Class objects >> for type Foo >> >> Open Webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8202913/webrev/ >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK- >> 8202913 >> >> This fix was tested with Mach5 tiers 1 and 2 tests and >> builds on >> Linux-X64, Windows, Solaris Sparc, and Mac OS X, with >> tiers 3-5 tests on >> Linux-x64, and with JCK-11 Lang and VM tests. >> >> Thanks, Harold >> >> >> From leo.korinth at oracle.com Thu May 31 13:56:49 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 31 May 2018 15:56:49 +0200 Subject: RFR: runtime/8176717/TestInheritFD.java fails with java.lang.RuntimeException: could not match: VM RESULT => RETAINS FD Message-ID: <31737c2a-f4fc-242d-1f6b-e21d38594fb5@oracle.com> Hi! I am uppdating TestInheritFD.java to fix intermittent test failures. The reason is that /sys/fs/cgroup/memory/memory.limit_in_bytes sometimes is open. The new test does not count the number of open file descriptors but instead looks to see if the log file is still open; this is less fragile. This relies on a /proc file system, but that seems to be available on all non Windows ports except Mac. Mac testing is disabled. The Windows solution does not need to change. This test bug does cause intermittent test failures in tier1, so I would be grateful for reviews and also someone to sponsor the final hg export. Bug: https://bugs.openjdk.java.net/browse/JDK-8202740 Webrev: http://cr.openjdk.java.net/~lkorinth/8202740/ Testing: GNU/Linux, Solaris, Windows and Mac (Mac disabled). Thanks, Leo From gerard.ziemski at oracle.com Thu May 31 19:22:54 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Thu, 31 May 2018 14:22:54 -0500 Subject: RFR (XS) 8203939: JVMFlag::printError missing ATTRIBUTE_PRINTF Message-ID: Hi all, Please review this tiny fix, in which we add ATTRIBUTE_PRINTF to JVMFlag::printError. https://bugs.openjdk.java.net/browse/JDK-8203939 http://cr.openjdk.java.net/~gziemski/8203939_rev1 Testing via Mach5 hs-tier1,2,3 cheers From gerard.ziemski at oracle.com Thu May 31 19:35:11 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Thu, 31 May 2018 14:35:11 -0500 Subject: RFR(xxxs): 8204164: OOM-only logging in Metaspace In-Reply-To: References: Message-ID: <250EA1A8-54BB-4735-9509-C185A0C2E4C6@oracle.com> hi Thomas, Your enhancement looks useful to me. Removing ?freelist? tag from ?metaspace? seems worthy of a separate issue/discussion. cheers > On May 31, 2018, at 6:36 AM, Thomas St?fe wrote: > > Hi all, > > very tiny improvement. The intent is to be able to restrict metaspace > logging to OOM situations. > > CR: OOM-only logging in Metaspace > Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8204164-oom-only-logging-in-metaspace/webrev.00/webrev/ > > -- > > On a related note, would anyone be offended if I were to remove the > "freelist" logging tag from the metaspace coding? I am not sure that > is useful in any way. > > Thanks, Thomas From gerard.ziemski at oracle.com Thu May 31 20:06:25 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Thu, 31 May 2018 15:06:25 -0500 Subject: RFR(L): 8195097: Make it possible to process StringTable outside safepoint In-Reply-To: <0d09f2ad-47b9-2fca-e5ec-17547e923cbb@oracle.com> References: <0d09f2ad-47b9-2fca-e5ec-17547e923cbb@oracle.com> Message-ID: <4D8E57FA-AF13-4246-AE5B-6A310CB36C10@oracle.com> hi Robbin, I just started reviewing and need another daye for more in-depth review, but right now I wanted to start a conversation about 2 runtime flags that we seem to longer use in the new code: #1 VerifyStringTableAtExit I think we should be able to add the code back that verifies the table in this new implementation, right? (i.e StringTable::verify_and_compare_entries) #2 StringTableSize We either need to deprecate it, or keep, but change the behavior (to take a power of 2). If we change the behavior, then we need to document it in release notes. I personally favor keeping the option, despite the fact that the new table is capable to resize on its own. cheers > On May 28, 2018, at 8:19 AM, Robbin Ehn wrote: > > Hi all, please review. > > This implements the StringTable with the ConcurrentHashtable for managing the > strings using oopStorage for backing the actual oops via WeakHandles. > > The unlinking and freeing of hashtable nodes is moved outside the safepoint, > which means GC only needs to walk the oopStorage, either concurrently or in a > safepoint. Walking oopStorage is also faster so there is a good effect on all > safepoints visiting the oops. > > The unlinking and freeing happens during inserts when dead weak oops are > encountered in that bucket. In any normal workload the stringtable self-cleans > without needing any additional cleaning. Cleaning/unlinking can also be done > concurrently via the ServiceThread, it is started when we have a high ?dead > factor?. E.g. application have a lot of interned string removes the references > and never interns again. The ServiceThread also concurrently grows the table if > ?load factor? is high. Both the cleaning and growing take care to not prolonging > time to safepoint, at the cost of some speed. > > Kitchensink24h, multiple tier1-5 with no issue that I can relate to this > changeset, various benchmark such as JMH, specJBB2015. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8195097 > Webrev: http://cr.openjdk.java.net/~rehn/8195097/v0/webrev/ > > Thanks, Robbin From harold.seigel at oracle.com Thu May 31 20:19:51 2018 From: harold.seigel at oracle.com (Harold David Seigel) Date: Thu, 31 May 2018 16:19:51 -0400 Subject: RFR (XS) 8203939: JVMFlag::printError missing ATTRIBUTE_PRINTF In-Reply-To: References: Message-ID: <27e95046-7d03-32e0-8b3e-f687dfaff1fb@oracle.com> Hi Gerard, This looks good. Harold On 5/31/2018 3:22 PM, Gerard Ziemski wrote: > Hi all, > > Please review this tiny fix, in which we add ATTRIBUTE_PRINTF to JVMFlag::printError. > > https://bugs.openjdk.java.net/browse/JDK-8203939 > http://cr.openjdk.java.net/~gziemski/8203939_rev1 > > Testing via Mach5 hs-tier1,2,3 > > > cheers > From coleen.phillimore at oracle.com Thu May 31 20:37:03 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 31 May 2018 16:37:03 -0400 Subject: RFR(xxxs): 8204164: OOM-only logging in Metaspace In-Reply-To: <250EA1A8-54BB-4735-9509-C185A0C2E4C6@oracle.com> References: <250EA1A8-54BB-4735-9509-C185A0C2E4C6@oracle.com> Message-ID: <684f01ef-9e26-b4ae-42b4-8cb4f76ed77e@oracle.com> I agree, this seems good, and fine with me to remove freelist. Coleen On 5/31/18 3:35 PM, Gerard Ziemski wrote: > hi Thomas, > > Your enhancement looks useful to me. > > Removing ?freelist? tag from ?metaspace? seems worthy of a separate issue/discussion. > > > cheers > >> On May 31, 2018, at 6:36 AM, Thomas St?fe wrote: >> >> Hi all, >> >> very tiny improvement. The intent is to be able to restrict metaspace >> logging to OOM situations. >> >> CR: OOM-only logging in Metaspace >> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8204164-oom-only-logging-in-metaspace/webrev.00/webrev/ >> >> -- >> >> On a related note, would anyone be offended if I were to remove the >> "freelist" logging tag from the metaspace coding? I am not sure that >> is useful in any way. >> >> Thanks, Thomas From coleen.phillimore at oracle.com Thu May 31 21:02:48 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 31 May 2018 17:02:48 -0400 Subject: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details In-Reply-To: References: Message-ID: It seems that similar information can be found in the other patch that you sent: RFR(s): 8203219: VM.metaspace jcmd should optionally show loaded classes for loaders So you can do jcmd VM.metaspace with options to show all classes, VM.systemdictionary and show all classes (including initiated classes) and now VM.classloaders command.? This one seems to have nice output though.?? Maybe it's worth having another. I will comment on the code separately. On 5/28/18 1:59 AM, Thomas St?fe wrote: > Hi David, > > On Mon, May 28, 2018 at 7:23 AM, David Holmes wrote: >> Hi Thomas, >> >> I had a look at this and overall seems okay - the output looks good > thanks! > >> (though I'm not sure how useful the hex values are?). > That is mainly a developer option for us; CLD* and Klass* are useful > if one wishes to dig into core files; Loader oop too (I know that one > is volatile, which is were the jcmd-bundle-commands-at-safepoint idea > comes from. > > But yes, this is normally too much noise, therefore disabled by > default. You need to set "verbose" explicitly to see this. > >> Can't comment too much on the pretty-printing details - the proof is in the >> output there. (Though have to wonder whether there is any existing >> tree/graph printing logic somewhere in the OpenJDK code?) > None as good as mine :) Seriously, there is similar but not as evolved > printing for class hierarchy. But it does not really print a tree, > just a bunch of '|' dividers. > > If this patch gets in, I would in a follow up patch unify tree > printing for these two commands and any other tree-ish structures I > find. > >> Two queries: >> >> 1. Have we previously established whether a CSR request is needed for a new >> Dcmd? (My initial feeling is that it is.) > My feeling is no, since this adds a new command, so there can be no > backward compat issues. What is the general policy for new jcmd > commands, or for that matter anything new added to the outside facing > interface (new options, new Xlog tracing flags, changed output for > existing options)? Do these things require CSR? > > My problem with CSR is that it introduces a bottleneck, since it can > only be approved by three very busy people at Oracle - if I understand > the process right. Yes, we need a process to agree e.g. on syntax - > desperately so, since e.g. sub option syntax in jcmd is a mess - but > we seem to be strapped for reviewers even for normal code reviews, so > the effect of creating a CSR in my experience is just a stop-of-work. We haven't normally filed CSRs to add jcmd options.? These things seem to be hidden in the implementation and that seems appropriate. >> 2. Is ClassLoaderHierarchyVMOperation a safepoint VM-op? I would expect it >> needs to be to be able to walk the CLD hierarchy, unless that is already >> guaranteed to be safely walkable. Either way a comment clearly stating that >> would be useful I think. > According to Coleen, CLDG can be walked outside a safepoint, but I did > not want to risk it so I made it a safepoint operation (like other > commands walking the CLDG, e.g. VM.metaspace). Calling ClassLoaderDataGraph::cld_do() doesn't keep the ClassLoaderData alive while iterating since it's the GC interface to walking the CLDG, so this should be done in a safepoint.? For printing, the aliveness doesn't really matter so it could be done concurrently but I think it's better in a safepoint. > >> Related to #2, is it really possible to encounter a CLD in the process of >> being unloaded? Wouldn't that happen at a safepoint? > Not sure, I am not a GC expert. I see places where this may be called > concurrently, e.g. in the process of > -XX:+ClassUnloadingWithConcurrentMark? > > Since a diagnostic command should never endanger the VM it monitors, I > coded defensively. Thank you for checking for is_unloading.? I've recently changed the ClassLoaderData to not allow you to access the class_loader oop if the class is unloading.?? For the existing GCs, the CLDG will not have classes that are unloading in the list, but this may not always be the case. I'll comment on the code in a separate reply. thanks, Coleen >> Thanks, >> David >> > Thank you, > > Thomas > >> On 28/05/2018 2:50 PM, Thomas St?fe wrote: >>> All tests passed on jdk-submit. >>> >>> Anyone interested in a review? >>> >>> More output examples for jcmd VM.classloaders : >>> >>> Spring framework, basic tree: >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_short.txt >>> >>> Spring framework, including all classes: >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_long.txt >>> >>> ... Thomas >>> >>> On Wed, May 23, 2018 at 2:46 PM, Thomas St?fe >>> wrote: >>>> Dear all, >>>> >>>> (not sure if this would be a serviceability or runtime rfe, so sorry >>>> for crossposting) >>>> >>>> may I please have feedback/reviews for this small enhancement. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203682 >>>> Webrev: >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.00/webrev/ >>>> >>>> This adds a new command to jcmd, "VM.classloaders". It complements the >>>> existing command "VM.classloader_stats". >>>> >>>> This command, in its simplest form, prints the class loader tree. In >>>> addition to that, it optionally prints out loaded classes (both >>>> non-anonymous and anonymous) and various classloader specific >>>> information. >>>> >>>> Examples: >>>> >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example.txt >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-classes.txt >>>> >>>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-reflection-and-noinflation.txt >>>> >>>> >>>> Thanks and Best Regards, >>>> >>>> Thomas From gerard.ziemski at oracle.com Thu May 31 21:24:54 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Thu, 31 May 2018 16:24:54 -0500 Subject: RFR (XS) 8203939: JVMFlag::printError missing ATTRIBUTE_PRINTF In-Reply-To: <27e95046-7d03-32e0-8b3e-f687dfaff1fb@oracle.com> References: <27e95046-7d03-32e0-8b3e-f687dfaff1fb@oracle.com> Message-ID: <68F79BB9-CE89-4F40-A0BD-712EEE7359F8@oracle.com> Thank you Harold! > On May 31, 2018, at 3:19 PM, Harold David Seigel wrote: > > Hi Gerard, > > This looks good. > > Harold > > > On 5/31/2018 3:22 PM, Gerard Ziemski wrote: >> Hi all, >> >> Please review this tiny fix, in which we add ATTRIBUTE_PRINTF to JVMFlag::printError. >> >> https://bugs.openjdk.java.net/browse/JDK-8203939 >> http://cr.openjdk.java.net/~gziemski/8203939_rev1 >> >> Testing via Mach5 hs-tier1,2,3 >> >> >> cheers >> > From kim.barrett at oracle.com Thu May 31 22:06:56 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 31 May 2018 18:06:56 -0400 Subject: RFR (XS) 8203939: JVMFlag::printError missing ATTRIBUTE_PRINTF In-Reply-To: References: Message-ID: <30F78A72-9154-4785-AAAD-205D05E69ACA@oracle.com> > On May 31, 2018, at 3:22 PM, Gerard Ziemski wrote: > > Hi all, > > Please review this tiny fix, in which we add ATTRIBUTE_PRINTF to JVMFlag::printError. > > https://bugs.openjdk.java.net/browse/JDK-8203939 > http://cr.openjdk.java.net/~gziemski/8203939_rev1 > > Testing via Mach5 hs-tier1,2,3 > > > cheers Looks good. From coleen.phillimore at oracle.com Thu May 31 22:07:19 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 31 May 2018 18:07:19 -0400 Subject: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details In-Reply-To: References: Message-ID: <334dd99b-dd25-169a-1ce8-7662109dadce@oracle.com> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.00/webrev/src/hotspot/share/classfile/classLoaderHierarchyDCmd.cpp.html Can you make BranchTracker : public StackObj {} ? Are you going to move BranchTracker to utilities if you find other uses for it? I don't like the use of mtInternal, but there doesn't seem to be a better category (or one to add). 152 if (_loader_oop != NULL) { 153 loader_klass = _loader_oop->klass(); 154 if (loader_klass != NULL) { 155 loader_class_name = loader_klass->external_name(); 156 } 157 oop nameOop = java_lang_ClassLoader::name(_loader_oop); 158 if (nameOop != NULL) { 159 const char* s = java_lang_String::as_utf8_string(nameOop); 160 if (s != NULL) { 161 loader_name = s; 162 } 163 } These are available to the ClassLoaderData (_cld) as class_loader_name() and class_loader_klass().? I would rather see ClassLoaderData::loader_name() used - once fixed. Also, I don't like in the output.? It seems that most class loaders will be unnamed, so the output will be too noisy with that.? Also, there's an unnamed module so that makes it very confusing. Since this is in a safepoint, maybe you can add a ResourceMark at 472, remove it from 362 and make all classes ResourceObj allocated. Then you can remove the code to delete the types. Can you add assert that you are at a safepoint at the beginning of this, so you don't have to worry about LoaderTreeNode::_loader_oop getting collected. It seems that by saving _cld in LoaderTreeNode, that you don't need _loader_oop. This code looks like it's walking the CLDG in order to make a different Class loader graph for printing.? Is it necessary to show the hierarchy information? It seems like printing the address is better than , which is what I've looked at with the system dictionary logging. http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.00/webrev/test/hotspot/jtreg/serviceability/dcmd/vm/ClassLoaderHierarchyTest.java.html What is JMXExecutor? Is this something to get in the way of debugging this test if it fails? Sorry for the delay reviewing. thanks, Coleen On 5/28/18 5:33 AM, Thomas St?fe wrote: > Hi Bernd, > > definitely. I have two local patches here and corresponding RFEs open: > https://bugs.openjdk.java.net/browse/JDK-8203343 > "VM.{metaspace|classloaders} jcmd should show invocation targets for > Generated{Method|Constructor}AccessorImpl classes" > https://bugs.openjdk.java.net/browse/JDK-8203849 > "DelegatingClassLoader name could be used to indicate reflected class" > > Both complement each other and should make it possible to understand > core reflection class loader issues better. > > Nice idea about that folding, I can add this. > > I plan to get them upstream for 11, but that may be illusory seeing > how slow reviews are currently going. But certainly 12. > > Gruss, Thomas > > > On Mon, May 28, 2018 at 9:21 AM, Bernd Eckenfels wrote: >> Anything which can be done with the DelegatingClassLoaders (either add Info on their target or collabs them to a ?30 x unnamed DelegatingClassloader?. Even in Verbose mode they only reveal their type (constructor, method) but not much more. >> >> Gruss >> Bernd >> -- >> http://bernd.eckenfels.net >> ________________________________ >> From: serviceability-dev on behalf of Thomas St?fe >> Sent: Monday, May 28, 2018 6:50:34 AM >> To: serviceability-dev at openjdk.java.net serviceability-dev at openjdk.java.net; Hotspot dev runtime >> Subject: Re: RFR(s): 8203682: Add jcmd "VM.classloaders" command to print out class loader hierarchy, details >> >> All tests passed on jdk-submit. >> >> Anyone interested in a review? >> >> More output examples for jcmd VM.classloaders : >> >> Spring framework, basic tree: >> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_short.txt >> >> Spring framework, including all classes: >> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example_spring_long.txt >> >> ... Thomas >> >> On Wed, May 23, 2018 at 2:46 PM, Thomas St?fe wrote: >>> Dear all, >>> >>> (not sure if this would be a serviceability or runtime rfe, so sorry >>> for crossposting) >>> >>> may I please have feedback/reviews for this small enhancement. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203682 >>> Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/webrev.00/webrev/ >>> >>> This adds a new command to jcmd, "VM.classloaders". It complements the >>> existing command "VM.classloader_stats". >>> >>> This command, in its simplest form, prints the class loader tree. In >>> addition to that, it optionally prints out loaded classes (both >>> non-anonymous and anonymous) and various classloader specific >>> information. >>> >>> Examples: >>> >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example.txt >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-classes.txt >>> http://cr.openjdk.java.net/~stuefe/webrevs/8203682-jcmd-print-classloader-hierarchy/example-with-reflection-and-noinflation.txt >>> >>> >>> Thanks and Best Regards, >>> >>> Thomas From david.holmes at oracle.com Thu May 31 22:22:45 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 1 Jun 2018 08:22:45 +1000 Subject: RFR(xxxs): 8204164: OOM-only logging in Metaspace In-Reply-To: <684f01ef-9e26-b4ae-42b4-8cb4f76ed77e@oracle.com> References: <250EA1A8-54BB-4735-9509-C185A0C2E4C6@oracle.com> <684f01ef-9e26-b4ae-42b4-8cb4f76ed77e@oracle.com> Message-ID: <123ccd8a-f2f9-19b3-c5ef-9f00855041cb@oracle.com> On 1/06/2018 6:37 AM, coleen.phillimore at oracle.com wrote: > I agree, this seems good, and fine with me to remove freelist. Have we established any policy for lifecycle management of logging tags? If you just rip out a tag and someone has a special logging script they run occasionally if they need to gather specific information, then they suddenly get a failure because someone decided "naw don't need that tag". David ----- > Coleen > > > On 5/31/18 3:35 PM, Gerard Ziemski wrote: >> hi Thomas, >> >> Your enhancement looks useful to me. >> >> Removing ?freelist? tag from ?metaspace? seems worthy of a separate >> issue/discussion. >> >> >> cheers >> >>> On May 31, 2018, at 6:36 AM, Thomas St?fe >>> wrote: >>> >>> Hi all, >>> >>> very tiny improvement. The intent is to be able to restrict metaspace >>> logging to OOM situations. >>> >>> CR: OOM-only logging in Metaspace >>> Webrev: >>> http://cr.openjdk.java.net/~stuefe/webrevs/8204164-oom-only-logging-in-metaspace/webrev.00/webrev/ >>> >>> >>> -- >>> >>> On a related note, would anyone be offended if I were to remove the >>> "freelist" logging tag from the metaspace coding? I am not sure that >>> is useful in any way. >>> >>> Thanks, Thomas > From rkennke at redhat.com Thu May 31 22:25:21 2018 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 1 Jun 2018 00:25:21 +0200 Subject: RFR: JDK-8198285: More consistent Access API for arraycopy In-Reply-To: <5AE1F062.3020303@oracle.com> References: <5AE09F74.7050005@oracle.com> <5AE1F062.3020303@oracle.com> Message-ID: <3bb2415e-4d15-fbd3-dde2-73a25c7bd65b@redhat.com> Hi Erik, It took me a while to get back to this. I wrote a little wrapper around the extended arraycopy API that allows to basically write exactly what you suggested: > ArrayAccess::arraycopy_from_native(ptr, obj, index, > length); I agree that this seems the cleanest solution. The backend still sees both obj+off OR raw-ptr, but I think this is ok. Differential: http://cr.openjdk.java.net/~rkennke/JDK-8198285/webrev.02.diff/ Full: http://cr.openjdk.java.net/~rkennke/JDK-8198285/webrev.02/ What do you think? Roman > Each required parameter is clear when you read the API. > > But I am open to suggestions of course. > > Thanks, > /Erik > > On 2018-04-25 17:59, Roman Kennke wrote: >>> 1) Resolve the addresses as we do today, but you think you get better >>> Shenandoah performance if we either >>> 2) Have 3 different access calls, (from heap to native, from native to >>> heap, from heap to heap) for copying, or >>> 3) Have 1 access call that can encode the above 3 variants, but looks >>> ugly at the call sites. >> There's also the idea to pass Address arguments to arraycopy, one for >> src, one for dst, and have 2 different subclasses: one for obj+offset >> (heap access) and one with direct pointer (raw). Your comment gave me >> the idea to also provide arrayOop+idx. This would look clean on the >> caller side I think. >> >> It would also be useful on the GC side: BarrierSets would specialize >> only in the variants that they are interested in, for example, in case >> of Shenandoah: >> 1. arraycopy(HeapAddress,HeapAddress) Java->Java >> 2. arraycopy(HeapAddress,RawAddress)? Java->native >> 3. arraycopy(RawAddress,HeapAddress)? native->Java >> >> other barriersets can ignore the exact type and only call the args >> address->resolve() or so to get the actual raw address. >> >> This would *also* be beneficial for the other APIs: instead of having >> all the X() and X_at() variants, we can just use one X variant that >> either takes RawAddress or HeapAddress. >> >> I made a little (not yet compiling/working) prototype of this a while >> ago: >> >> http://cr.openjdk.java.net/~rkennke/JDK-8199801-2.patch >> >> >> What do you think? Would it make sense to go further down that road? >> >> Roman >> >>> You clearly went for 3, which leaves the callsites looking rather hard >>> to read. It is for example not obvious for me what is going on here >>> (javaClasses.cpp line 313): >>> >>> HeapAccess<>::arraycopy(NULL, 0, reinterpret_cast>> jbyte*>(utf8_str), value(h_obj()), >>> typeArrayOopDesc::element_offset(0), NULL, length); >>> >>> ...without looking very carefully at the long list of arguments encoding >>> what is actually going on (copy from native to the heap). What is worse >>> is that this will probably not compile without adding the template >>> keyword to the call (since you have a qualified template member function >>> behind a template class), like this: >>> HeapAccess<>::template arraycopy(NULL, 0, reinterpret_cast>> jbyte*>(utf8_str), value(h_obj()), >>> typeArrayOopDesc::element_offset(0), NULL, length); >>> >>> ...which as a public API leaves me feeling a bit like this:???? :C >>> >>> May I suggest adding an array access helper. The idea is to keep a >>> single call through Access, and a single backend point for array copy, >>> but let the helper provide the three different types of copying as >>> different functions, so that the call sites still look pretty and easy >>> to read and follow. >>> >>> This helper could additionally have load_at, and store_at use array >>> indices as opposed to offsets, and hence hide the offset calculations we >>> perform today (typically involving checking if we are using compressed >>> oops or not). >>> >>> I am thinking something along the lines of >>> ArrayAccess<>::arraycopy_to_native(readable_arguments), >>> ArrayAccess<>::arraycopy_from_native(readable_arguments), >>> ArrayAccess<>::arraycopy(readable_arguments), which translates to some >>> form of Access<>::arraycopy(unreadable_arguments). And for example >>> ArrayAccess<>::load_at(obj, index) would translate to some kind of >>> HeapAccess::load_at(obj, offset_for_index(index)) as a >>> bonus, making everyone using the API jump with happiness. >>> >>> What do you think about this idea? Good or bad? I guess the question is >>> whether this helper should be in access.hpp, or somewhere else (like in >>> arrayOop). Thoughts are welcome. >>> >>> Thanks, >>> /Erik >>> >>> On 2018-04-11 19:54, Roman Kennke wrote: >>>> ?? Currently, the arraycopy API in access.hpp gets the src and dst >>>> oops, >>>> plus the src and dst addresses. In order to be most useful to garbage >>>> collectors, it should receive the src and dst oops together with the >>>> src >>>> and dst offsets instead, and let the Access API / GC calculate the src >>>> and dst addresses. >>>> >>>> For example, Shenandoah needs to resolve the src and dst objects for >>>> arraycopy, and then apply the corresponding offsets. With the current >>>> API (obj+ptr) it would calculate the ptr-diff from obj to ptr, then >>>> resolve obj, then re-add the calculate ptr-diff. This is fragile >>>> because >>>> we also may resolve obj in the runtime before calculating ptr (e.g. via >>>> arrayOop::base()). If we then pass in the original obj and a ptr >>>> calculated from another copy of the same obj, the above resolution >>>> logic >>>> would not work. This is currently the case for obj-arraycopy. >>>> >>>> I propose to change the API to accept obj+offset, in addition to ptr >>>> for >>>> both src and dst. Only one or the other should be used. Heap accesses >>>> should use obj+offset and pass NULL for raw-ptr, off-heap accesses (or >>>> heap accesses that are already resolved.. use with care) should pass >>>> NULL+0 for obj+offset and the raw-ptr. Notice that this also allows the >>>> API to be used for Java<->native array bulk transfers. >>>> >>>> An alternative would be to break the API up into 4 variants: >>>> >>>> Java->Java transfer: >>>> arraycopy(oop src, size_t src_offs, oop dst, size_t dst_offs, size_t >>>> len) >>>> >>>> Java->Native transfer: >>>> arraycopy(oop src, size_t src_offs, D* raw_dst, size_t len) >>>> >>>> Native->Java transfer: >>>> arraycopy(S* src_raw, oop dst, size_t dst_offs, size_t len) >>>> >>>> 'Unsafe' transfer: >>>> arraycopy(S* src_raw, D* dst_raw, size_t len) >>>> >>>> >>>> But that seemed to be too much boilerplate copy+pasting for my taste. >>>> (See how having this overly complicated template layer hurts us?) >>>> >>>> Plus, I had a better idea: instead of accepting oop+offset OR T* for >>>> almost every Access API, we may want to abstract that and take an >>>> Address type argument, which would be either HeapAddress(obj, >>>> offset) or >>>> RawAddress(T* ptr). GCs may then just call addr->address() to get the >>>> actual address, or specialize for HeapAddress variants and resolve the >>>> objs and then resolve the address. This would also allow us to get rid >>>> of almost half of the API (all the *_at variants would go) and some >>>> other simplifications. However, this seemed to explode the scope of >>>> this >>>> RFE, and would be better handled in another RFE. >>>> >>>> This changes makes both typeArrayKlass and objArrayKlass use the >>>> changed >>>> API, plus I identified all (hopefully) places where we do bulk >>>> Java<->native array transfers and make them use the API too. Gets us >>>> rid >>>> of a bunch of memcpy calls :-) >>>> >>>> Please review the change: >>>> http://cr.openjdk.java.net/~rkennke/JDK-8198285/webrev.00/ >>>> >>>> Thanks, Roman >>>> >> >